For normal inspection, the trained deep neural network inspects images one by one through GPU's computation.
If your GPU's computing power and memory size are large enough to process multiple images at once, you can utilize more GPU resources to reduce inspect time. As explained, the method of inspecting multiple images at once using more GPU resources is refered to Batch inspection mode in this document. Only Classification and Segmentation modules currently support this function.
There are only two things you need to do to use Batch inspection mode.
batchSize.batchSize is too large to allocated memory on GPU, Memory-allocation functions returns eErrorType::OUT_OF_GPU_MEMORY.batchSize as power of 2. ex) 2, 4, 8 ...batchSize set in step 1, the For loop is executed in units of batchSize until all images are inspected.