Classification


This document describes how to use 'Classification' with SaigieVision.

If you've been following the 'Quick start guide', it shouldn't be hard to figure out how to do Classification.
The only thing added in the guide below is ClsInspectionOptions.

Below is sample code for 'Classification'.

//Initialize
SaigeVision.Core.SaigeVisionApp.Init();

//Prepare
IGpuService gpuService = new NvGpuService();
gpuService.DetectConnectedGPUs();
PhysicalGpu gpu = gpuService.CurrentDetectedGPUs[0];

SrImage image = new SrImage(@"..\..\000001.png");
Uri model = new Uri(@"..\..\model.srCls");

//Inspect
using (var engine = EngineLoader.LoadFrom<ClassificationEngine>(model))
{
    ClsInspectionOptions option = engine.ReadInspectionOptions();
    option.AdditionalScores[0] = 0;
    option.AdditionalScores[1] = 10;
    option.AdditionalScores[2] = 10;

    engine.AllocateNetwork(gpu, option);
    ClassificationReport report = engine.Inspect(image);

    ClassInfo prediction = report.Prediction;

    Console.WriteLine($"Inspection time : {report.InspectionTime}");
    Console.WriteLine($"Predicted : {prediction.Name}");
}

//Clean up (optional)
image.Dispose();

Initialize

In order to use SaigeVision.Core, you must initialize SaigeVision.Core with SaigeVisionApp.Init() once at runtime.

It is recommended that you do your initialization in your application's main()

If you're using a platform other than a Console App and have difficulty finding the main() function (WPF, WinForms, etc...), you can use any location, but preferably one that guarantees a "onc and only" call.

It should also be before using the EngineLoader

Prepare

This is the process of preparing all the data needed to use ClassificationEngine. In general, we require three things

  • GPU to use for the inference
  • Sample images to use for inference
  • A model (.srCls file) to use for inference

SaigeVision only supports GPUs made by NVIDIA

Also, SaigeVision is using all image objects through SrImage. SrImage has done a lot of work under the hood to optimize performance through .Net tools like ArrayPool<T>.

It supports a variety of constructors, so be sure to check out the API docs for SrImage.

Inspection

The 'Inspection' part starts by loading the ClassificationEngine via the EngineLoader.

For the 'Inspection' part, we'll break down the procedure as follows to make it easier to understand.


1. Load engine

This example uses the C# language's using statement for safety. If you don't think the using statement is appropriate in your scenario, you can ignore it. However, if you do not use the using statement, the ClassificationEngine must be managed and ClassificationEngine.Dispose() when it is finished using it.

2. Inspection Options

The sample code is loading an Engine and then loading ClsInspectionOptions through that Engine.

The ClassificationEngine can specify a number of options when performing an inspection. These options affect the actual execution of the inspection and are reflected in the report.

In this example, we are assigning an option value to 'Additional score' as shown in the code that follows.

The options created (ClsInspectionOptions) can be set via Engine's AllocateNetowrk().

  • The result of any 'Inspection' after setting is the value with that Option applied.


In certain scenarios, you may need to record these Option values in the model. LoadInspectionOptions() is a function that loads the most recent Option values recorded in the model.

If you want the options to be written to the model, use WirteInspectionOptions(). You can also revert to the default options with WriteInspectionOptionsAsDefault().

  • If you don't specify Options via AllocateNetowrk(), it will automatically use the Options recorded in the Model.

  • AllocateNetowrk() can take some time. This is usually enough to cause a thread to block.


3. Inspect

As you can see in the code, the main function that performs the 'Inspection' is engine.Inspect(). All kinds of Engines have this common Inspect() function. They are also nested in different forms for different uses. They can be optionally used in different scenarios, such as asynchronously or to inspect multiple images.

The ClassificationReport reports all the results returned by Inspect().


Clean up (Optional)

In this example, we're Dispose() of all the SrImage we've used for inspections because we no longer have a use for them.



Inspecting a large number of images at once

In the previous example, we looked at the scenario of inspecting a single image. But let's look at a scenario where you want to inspect a large number of images at once.

Let's fix this situation by modifying a few parts of the sample code.

First, prepare the "large number of images" data by making SrImage an array. It looks like this

SrImage[] data = new SrImage[]
{
    new SrImage(@"..\...\sample1.png"),
    new SrImage(@"..\...\sample2.png"),
    new SrImage(@"..\...\sample3.png"),
    new SrImage(@"..\...\sample4.png")
};

The last argument that was hidden in AllocateNetowrk() is then available.

    engine.AllocateNetwork(gpu, option, 4);

  • The last argument means "max images" and refers to the number of images that can be processed (in parallel) at once.

  • We recommend setting it to a power of 2 by default.

  • The larger the value set, the larger the GPU memory allocation, so set it to a smaller number if you're running low on memory.

  • If you are inspecting a larger number of images than the set value, we will perform multiple inspections internally.


Finally, modify after the line containing the Inspect() function.

ClassificationReport[] reports = engine.Inspect(data);

foreach (var report in reports)
{
    ClassInfo prediction = report.Prediction;

    Console.WriteLine($"Inspection time : {report.InspectionTime}");
    Console.WriteLine($"Predicted : {prediction.Name}");
}

You can run the code to inspect a "large number of images".

See also