MultiThread.h

Go to the source code of this file.

Describes ExampleMultiThreadMultiModels.cpp example.

This example describes how to use multiple models in multiple threads. Multi-threading is a method used to execute multiple tasks simultaneously. In the case of the Runtime inspection function of SAIGE VISION, it is not always thread-safe. Therefore, multiple threads should be used in an appropriate way.

Note

Important: One thread should use one SAIGE VISION handle.

Inspection handles are not thread-safe. Using one handle in multiple threads can cause unexpected results. Therefore, each worker thread should use different inspection handles.

This example starts by creating a list of models to use in each handle. In this example, all handles use the same model file.

// ExampleMultiThreadMultiModels.cpp:45

std::string dataDir =
    get_env_var("SAIGE_HOME") + "/" + get_runtime_base() + "/engine/Examples/data/sfaw2023/images/test/";

// 사용할 모델 목록
std::vector<std::string> modelPathList = {dataDir + "../../models/seg_example.saigeseg",
                                          dataDir + "../../models/seg_example.saigeseg"};

Then declare the handle variables to use and create each handle. Here, each worker thread inspects one image at a time and uses one result handle per thread.

// ExampleMultiThreadMultiModels.cpp:75

std::vector<SaigeInferenceHandle> handles(modelPathList.size(), nullptr);
std::vector<SaigeInferenceResult> results(modelPathList.size(), nullptr);

/***********************************************************
 *    1. 모델 파일 및 검사 결과 핸들 생성,
 *       Create handles for model files and inference results
 ***********************************************************/
for (size_t i = 0; i < handles.size(); ++i) {
    SaigeHandleConfig config;
    config.inspectionSizeHeight = 2496;
    config.inspectionSizeWidth = 2496;
    unsigned int handleFlags = SAIGE_SET_INSPECTION_SIZE;
    if (optimize) {
        handleFlags |= SAIGE_OPTIMIZE_HANDLE;
    }

    retval = SaigeCreateInferenceHandle(&handles[i], kProjectTypeSEG, modelPathList[i].c_str(),
                                        deviceIndex, &config, handleFlags);

    if (retval < 0) {
        std::cerr << "SaigeCreateInferenceHandle failed - " << SaigeStrError(retval)
                  << std::endl;
        return retval;
    }

    retval = SaigeCreateInferenceResult(&results[i]);
    if (retval < 0) {
        std::cerr << "SaigeCreateInferenceResult failed - " << SaigeStrError(retval)
                  << std::endl;
        return retval;
    }
}

Warning

Caution when using the same model file: The following actions that affect the model file may fail when using the same model file. In the case of SAIGE_OPTIMIZE_HANDLE, if the same model is used, it may fail because the optimized model file is created. If multiple handles access the same model, the first handle that tries to access the file will create the file automatically, and the other handles will return a function error code. In the case of SaigeSetInferenceOption, if the SAIGE_WRITE_ON_MODEL_FILE flag is used, the inference option stored in the model file may be unintentionally changed, so it cannot be used.

In this example, we defined the job of each worker thread using std::thread and lambda functions available from C++11. The contents of the job are similar to other examples.

// ExampleMultiThreadMultiModels.cpp:302

std::vector<std::thread> threads{};

for (size_t i = 0; i < handles.size(); ++i) {
    threads.push_back(std::thread([verbosity, detailedTime, overlayAlpha, waitkeyMilliSecond,
                                   imagePaths, handle = handles[i], result = results[i],
                                   modelInfo = &modelInfos[i], modelNum = i] {

Join all threads to wait for all to run and exit.

// ExampleMultiThreadMultiModels.cpp:466

for (auto& th : threads) {
    th.join();
}