Segmentation


This document describes how to use 'Segmentation' with SaigieVision.

Before looking at 'Segmentation', recommend reading about the 'Classification' and 'Detection' inspections.

Segmentation has a similar flow to the other inspections, but with some special elements. Also, if you don't have a good understanding of the Classification or Detection inspections, the SegInspectionOptions and SegmentationEngine.Inspect() used in the Segmentation inspection can be confusing.

Below is sample code for 'Segmentation'.

//Initialize
SaigeVision.Core.SaigeVisionApp.Init();

//Prepare
var gpuService = new NvGpuService();
gpuService.DetectConnectedGPUs();

SrImage sample = new SrImage("../../sample.jpg");
var modelUri = new Uri(@"../../../model.srSeg");

//Inspect
using (SegmentationEngine segEngine = EngineLoader.LoadFrom<SegmentationEngine>(modelUri))
{
    SegInspectionOptions option = segEngine.ReadInspectionOptions();
    option.TimeLimit = default;
    option.ContourOptions.SaveCroppedPatch = true;

    segEngine.AllocateNetwork(gpuService.CurrentDetectedGPUs[0], option);

    SegmentationReport<SegContourResult> report = segEngine.Inspect<SegContourResult>(sample);
}

//clean up (optional)
sample.DisposeAll();

Initialize

In order to use SaigeVision.Core, you must initialize SaigeVision.Core with SaigeVisionApp.Init() once at runtime.

It is recommended that you do your initialization in your application's main()

If you're using a platform other than a Console App and have difficulty finding the main() function (WPF, WinForms, etc...), you can use any location, but preferably one that guarantees a "onc and only" call.

It should also be before using the EngineLoader

Prepare

This is the process of preparing all the data needed to use SegmentationEngine. In general, we require three things

  • GPU to use for the inference
  • Sample images to use for inference
  • A model (.srSeg file) to use for inference

SaigeVision only supports GPUs made by NVIDIA

Also, SaigeVision is using all image objects through SrImage. SrImage has done a lot of work under the hood to optimize performance through .Net tools like ArrayPool<T>.

It supports a variety of constructors, so be sure to check out the API docs for SrImage.

Inspection

The 'Inspection' part starts by loading the SegmentationEngine via the EngineLoader.

For the 'Inspection' part, we'll break down the procedure as follows to make it easier to understand.


1. Load engine

This example uses the C# language's using statement for safety. If you don't think the using statement is appropriate in your scenario, you can ignore it. However, if you do not use the using statement, the SegmentationEngine must be managed and SegmentationEngine.Dispose() when it is finished using it.

2. Inspection Options

The sample code is loading an Engine and then loading SegInspectionOptions through that Engine.

SegInspectionOptions option = segEngine.ReadInspectionOptions();
option.TimeLimit = default;
option.ContourOptions.SaveCroppedPatch = true;

The SegmentataionEngine can specify a number of options when performing an inspection. These options affect the actual execution of the inspection and are reflected in the report.

In this example, we're specifying several inspection options, as shown in the code that follows. The Segmentation test is different from the other tests in that it contains many test options.

Of those options, the one we're going to look at now is ContourInspectionOptions.

The SegmentationEngine's Inspect() function is different from other Engines in that it allows you to select the type of result you want returned.
There are two result types to choose from: SegContourResult and SegScoremapResult.
In this example, we are using SegContourResult. (engine.Inspect<SegContourResult>(data))

Getting back to options, ContourInspectionOptions are the inspection options that are applied when the result is returned as a SegContourResult.

In this example, we have specified ContourOptions. ContourOptions.SaveCroppedPatch = true; to enable the ability to crop the image of the detected contour area when receiving a report of the results. When this option is enabled, the report returned by the Inspect() function will include the image values of the cropped areas (image objects).

These new inspection options can be set via Engine's AllocateNetwork().

  • The result of the 'Inspection' from now on is the value with the Option applied.

In certain scenarios, you may need to record these Option values in the model. LoadInspectionOptions() is a function that loads the most recent Option values recorded in the model.

If you want the options to be written to the model, use WirteInspectionOptions(). You can also revert to the default options with WriteInspectionOptionsAsDefault().

  • If you don't specify Options via AllocateNetowrk(), it will automatically use the Options recorded in the Model.

  • AllocateNetowrk() can take some time. This is usually enough to cause a thread to block.


3. Inspect

As mentioned earlier, the SegmentationEngine's Inspect(SrImage) function is a little different from the other Engines. 

    
        SegmentationReport<SegContourResult> report = segEngine.Inspect<
        SegContourResult>(sample);

The default function provided is Inspect<T>(SrImage), which takes the result type as Generic.

And here's the format you can specify for Generic T

You can specify which of the two types you want returned. If you need both in your scenario, use Inspect<T1, T2>(SrImage).

The SegmentationReport returned by Inspect<T>(SrImage) reports all the results after the inspection.

Clean up (Optional)

In this example, we're Dispose() of all the SrImage we've used for inspections because we no longer have a use for them.


Inspecting a large number of images at once

The 'Segmentation' feature is the same as the 'Classification' feature and provides the ability to examine a "large number of images" at once. Usage is similar to Classification.

See also