Skip to content

data

Module diagram

classDiagram
  class data {
  }
  class batch_sampler {
  }
  class base_sampler {
  }
  class builder {
  }
  class collate {
  }
  class crop {
  }
  class base_cropper {
  }
  class builder {
  }
  class edge_cropper {
  }
  class ocr_cropper {
  }
  class polygon_cropper {
  }
  class dataclass {
  }
  class box {
  }
  class segment {
  }
  class dataloader {
  }
  class builder {
  }
  class full2patch_loader {
  }
  class infinite_random_sampler {
  }
  class dataset {
  }
  class base_dataset {
  }
  class builder {
  }
  class platform_reader {
  }
  class saige_vision_reader {
  }
  class srproj_dataset {
  }
  class srproj_reader {
  }
  class process_config {
  }
  class transform {
  }
  class augmentation {
  }
  class api {
  }
  class augment_function {
  }
  class augment_transform {
  }
  class base_transform {
  }
  class builder {
  }
  class compose {
  }
  class base_compose {
  }
  class builder {
  }
  class compose {
  }
  class box_function {
  }
  class function_util {
  }
  class image_function {
  }
  class image_load {
  }
  class polygon_function {
  }
  class resize {
  }
  class roi {
  }
  class api {
  }
  class roi_calculator {
  }
  class roi_handler {
  }
  class transform {
  }
  class typevar {
  }
  batch_sampler --> base_sampler
  batch_sampler --> builder
  builder --> base_sampler
  crop --> builder
  builder --> base_cropper
  builder --> edge_cropper
  builder --> ocr_cropper
  builder --> polygon_cropper
  edge_cropper --> base_cropper
  polygon_cropper --> base_cropper
  dataloader --> builder
  builder --> full2patch_loader
  dataset --> builder
  builder --> base_dataset
  builder --> srproj_dataset
  srproj_dataset --> base_dataset
  srproj_dataset --> srproj_reader
  augmentation --> builder
  api --> augment_function
  augment_transform --> augment_function
  augment_transform --> base_transform
  builder --> augment_transform
  builder --> builder
  builder --> compose
  compose --> base_compose
  box_function --> polygon_function
  box_function --> typevar
  function_util --> typevar
  image_function --> typevar
  image_load --> image_function
  polygon_function --> augment_function
  polygon_function --> typevar
  resize --> box_function
  resize --> image_function
  resize --> polygon_function
  resize --> typevar
  roi --> api
  roi --> roi_handler
  api --> roi_handler
  roi_handler --> roi_calculator
  transform --> augmentation
  transform --> image_function
  transform --> image_load
  transform --> resize
  transform --> roi_handler

data

batch_sampler

base_sampler

BatchSampler 
BatchSampler(dataset: Dataset, batch_size: int = 1, repeat: bool = False, shuffle: bool = False, drop_last: bool = False)

Bases: Registerable, ABC

Baseclass for all batch_sampler classes.

Attributes:

  • dataset (Dataset) –

    dataset to sample from.

  • batch_size (int) –

    batch size.

  • repeat (bool) –

    repeat data when data is not enough for batch size.

  • shuffle (bool) –

    shuffle data.

  • drop_last (bool) –

    drop last data when data is not enough for batch size.

Note
  • batch_size, shuffle, and drop_last should be popped from dataloader config, then passed to the batch sampler config.
  • repeat: for SaigeVision2 api, should be set as True.
    • if number of data is enough for batch size, then repeat is neglected.
  • drop_last: for SaigeVision2 api, should be set as True.
    • to prevent the last batch from being smaller than the specified batch size.
    • if False, even when repeat is set as True, the last batch will be smaller than the specified batch size.
  • for Training, shuffle should be set as True.
    • drop_last and repeat are optional.
  • for Validation, repeat, shuffle, and drop_last should be set as False.

Examples:

>>> if dataloader_class.__name__ == "Full2PatchDataLoader":
...     cfg_sampler.update(
...         {
...             "shuffle": cfg_dataloader.get("shuffle", False),
...             "drop_last": cfg_dataloader.get("drop_last", False),
...         }
...     )
... else:
...     cfg_sampler.update(
...         {
...             "shuffle": cfg_dataloader.pop("shuffle", False),
...             "drop_last": cfg_dataloader.pop("drop_last", False),
...             "batch_size": cfg_dataloader.pop("batch_size", 1),
...         }
...     )
... batch_sampler = build_batch_sampler(dataset=dataset, **cfg_sampler)
... dataloader = build_dataloader(
...     dataset,
...     collate_fn=collate_fn,
...     batch_sampler=batch_sampler,
...     **cfg_dataloader,
... )
Source code in SaigeToolkit/data/batch_sampler/base_sampler.py 
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
def __init__(
    self,
    dataset: data.Dataset,
    batch_size: int = 1,
    repeat: bool = False,
    shuffle: bool = False,
    drop_last: bool = False,
) -> None:
    self.dataset = dataset

    self.batch_size = batch_size
    self.repeat = repeat

    self.shuffle = shuffle
    self.drop_last = drop_last

    self._n_repeat = None
    self._buffer_length = None
NaiveSampler 
NaiveSampler(dataset: Dataset, batch_size: int = 1, repeat: bool = False, shuffle: bool = False, drop_last: bool = False)

Bases: BatchSampler

Naive sampler implementation with buffer extension feature added.

Source code in SaigeToolkit/data/batch_sampler/base_sampler.py 
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
def __init__(
    self,
    dataset: data.Dataset,
    batch_size: int = 1,
    repeat: bool = False,
    shuffle: bool = False,
    drop_last: bool = False,
) -> None:
    self.dataset = dataset

    self.batch_size = batch_size
    self.repeat = repeat

    self.shuffle = shuffle
    self.drop_last = drop_last

    self._n_repeat = None
    self._buffer_length = None

builder

collate

images_to_4d_tensor 

images_to_4d_tensor(images: List[Union[ndarray, Image, List]], device: device = default_device, apply_contiguous_div255: bool = True) -> Tensor

Converts a list of images to a 4D tensor.

Parameters:

  • images (List[Union[ndarray, Image, List]]) –

    images to convert.

  • device (device, default: default_device ) –

    device to move the tensor to. Defaults to cpu.

  • apply_contiguous_div255 (bool, default: True ) –

    whether to apply contiguous_div255. Defaults to True.

Returns:

  • Tensor –

    torch.Tensor: 4D tensor of images.

Source code in SaigeToolkit/data/collate.py 
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
def images_to_4d_tensor(
    images: List[Union[np.ndarray, Image.Image, List]],
    device: torch.device = default_device,
    apply_contiguous_div255: bool = True,
) -> torch.Tensor:
    """Converts a list of images to a 4D tensor.

    Args:
        images (List[Union[np.ndarray, Image.Image, List]]): images to convert.
        device (torch.device, optional): device to move the tensor to. Defaults to cpu.
        apply_contiguous_div255 (bool, optional): whether to apply contiguous_div255. Defaults to True.

    Returns:
        torch.Tensor: 4D tensor of images.

    """

    # multipage인 경우 List[image] -> n channel image로 변환
    if isinstance(images[0], List):
        images = [_merge_multipage_to_n_channel_array(image) for image in images]

    # list -> np.ndarray (BHWC) -> BCHW
    if isinstance(images[0], np.ndarray) and len(images) == 1:
        data = images[0][np.newaxis]
    else:
        data = np.stack(images)

    if data.ndim == 3:  # gray images
        data = data[:, None]
    else:
        data = data.transpose(0, 3, 1, 2)

    # array -> tensor
    data = torch.from_numpy(data).to(device)

    if apply_contiguous_div255:
        data = _contiguous_div255(tensor=data)

    return data

crop

builder

get_crop_fn 
get_crop_fn(_target_: Optional[str] = None, **cfg_crop: dict) -> Optional[BaseCropper]

getting crop function

Parameters:

  • cfg_crop (dict, default: {} ) –

    config dict for building cropper

Returns:

  • Optional[BaseCropper] –

    Optional[BaseCropper]: saige cropper object. if cfg_crop is None, return None

Source code in SaigeToolkit/data/crop/builder.py 
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
def get_crop_fn(_target_: Optional[str] = None, **cfg_crop: dict) -> Optional[BaseCropper]:
    """getting crop function

    Args:
        cfg_crop (dict): config dict for building cropper

    Returns:
        Optional[BaseCropper]: saige cropper object.
            if cfg_crop is None, return None
    """
    if _target_ is None:
        return None
    cropper = get_crop_class(_target_)
    logger.info(f"[{'CROPPER'.center(9)}] {_target_} [params] {cfg_crop}")
    return cropper(**cfg_crop)
get_crop_class 
get_crop_class(cfg_crop_name: str) -> Type[BaseCropper]

getting crop class from name

Parameters:

  • cfg_crop_name (str) –

    cropper name

Returns:

  • Type[BaseCropper] –

    Type[BaseCropper]: cropper class

Source code in SaigeToolkit/data/crop/builder.py 
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
def get_crop_class(cfg_crop_name: str) -> Type[BaseCropper]:
    """getting crop class from name

    Args:
        cfg_crop_name (str): cropper name

    Returns:
        Type[BaseCropper]: cropper class
    """

    try:
        return CROPPER_IMPLEMENTATION[cfg_crop_name]
    except KeyError as e:
        raise RuntimeError(
            f"Cropper {cfg_crop_name} not implemented. Supported cropper list: {CROPPER_IMPLEMENTATION.keys()}"
        ) from e
    except Exception as e:
        logger.error(f"Error: {e}")
        raise e

edge_cropper

Crop image patches along the contour of mask (or polygon)

Imported from DefectGeneration repository. generation/data/crop/edge_cropper.py

WindowState

Bases: Enum

relation state enum between center point and target point

WindowManager 
WindowManager(width: int = 256, height: int = 256, strict_inner_patch: bool = True)

Manager for cropping window size, relation and window coordinates.

Attributes:

  • width (int) –

    window width for image patch.

  • half_width (int) –

    half of window width for image patch.

  • height (int) –

    window height for image patch.

  • half_height (int) –

    half of window height for image patch.

  • strict_inner_patch (bool) –

    whether not allowing outer area of image.

Source code in SaigeToolkit/data/crop/edge_cropper.py 
45
46
47
48
49
50
51
52
53
54
55
def __init__(
    self,
    width: int = 256,
    height: int = 256,
    strict_inner_patch: bool = True,
):
    self.width = width
    self.half_width = width // 2
    self.height = height
    self.half_height = height // 2
    self.strict_inner_patch = strict_inner_patch
check_window 
check_window(point: Iterable[int], center: Iterable[int]) -> WindowState

check whether target point is inside of window, returns WindowState

Parameters:

  • point (Iterable[int]) –

    target point to be checked

  • center (Iterable[int]) –

    reference center point

Returns:

Source code in SaigeToolkit/data/crop/edge_cropper.py 
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
def check_window(
    self, point: Iterable[int], center: Iterable[int]
) -> WindowState:  # XXX: is this ok?
    """check whether target point is inside of window, returns WindowState

    Args:
        point (Iterable[int]): target point to be checked
        center (Iterable[int]): reference center point

    Returns:
        WindowState: relation state enum
    """
    width_distance = point[0] - center[0]
    abs_width_distance = abs(width_distance)
    height_distance = point[1] - center[1]
    abs_height_distance = abs(height_distance)

    if abs_width_distance <= self.half_width and abs_height_distance <= self.half_height:
        return WindowState.IN

    if abs_width_distance > abs_height_distance:
        if width_distance > 0:
            return WindowState.WIDTH_POSITIVE_OUT
        else:
            return WindowState.WIDTH_NEGATIVE_OUT
    else:
        if height_distance > 0:
            return WindowState.HEIGHT_POSITIVE_OUT
        else:
            return WindowState.HEIGHT_NEGATIVE_OUT
is_in_window 
is_in_window(point: Iterable[int], center: Iterable[int]) -> bool

is target point inside of window

Parameters:

  • point (Iterable[int]) –

    target point to be checked

  • center (Iterable[int]) –

    reference center point

Returns:

  • bool ( bool ) –

    boolean result

Source code in SaigeToolkit/data/crop/edge_cropper.py 
88
89
90
91
92
93
94
95
96
97
98
99
def is_in_window(self, point: Iterable[int], center: Iterable[int]) -> bool:
    """is target point inside of window

    Args:
        point (Iterable[int]): target point to be checked
        center (Iterable[int]): reference center point

    Returns:
        bool: boolean result
    """
    window_state = self.check_window(point, center)
    return window_state == WindowState.IN
is_in_any_window 
is_in_any_window(point: Iterable[int], centers: Iterable[Iterable[int]]) -> bool

is target point inside of any of windows

Parameters:

  • point (Iterable[int]) –

    target point to be checked

  • center (Iterable[int]) –

    list of reference center points

Returns:

  • bool ( bool ) –

    boolean result

Source code in SaigeToolkit/data/crop/edge_cropper.py 
101
102
103
104
105
106
107
108
109
110
111
def is_in_any_window(self, point: Iterable[int], centers: Iterable[Iterable[int]]) -> bool:
    """is target point inside of any of windows

    Args:
        point (Iterable[int]): target point to be checked
        center (Iterable[int]): list of reference center points

    Returns:
        bool: boolean result
    """
    return any(self.is_in_window(point, c) for c in centers)
get_coordinates_from_center 
get_coordinates_from_center(h_center: int, w_center: int, h: int, w: int) -> Tuple[int, int, int, int]

get left/right top/bottom coordinates from picked center point.

Parameters:

  • h_center (int) –

    picked center point h-coordinate

  • w_center (int) –

    picked center point w-coordinate

  • h (int) –

    image size height

  • w (int) –

    image size width

Returns:

  • Tuple[int, int, int, int] –

    Tuple[int, int, int, int]: (crop_left, crop_top, crop_right, crop_bottom)

Source code in SaigeToolkit/data/crop/edge_cropper.py 
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
def get_coordinates_from_center(
    self, h_center: int, w_center: int, h: int, w: int
) -> Tuple[int, int, int, int]:
    """get left/right top/bottom coordinates from picked center point.

    Args:
        h_center (int): picked center point h-coordinate
        w_center (int): picked center point w-coordinate
        h (int): image size height
        w (int): image size width

    Returns:
        Tuple[int, int, int, int]: (crop_left, crop_top, crop_right, crop_bottom)
    """
    if self.strict_inner_patch:
        w_center = np.clip(w_center, self.half_width, w - self.width + self.half_width)
        h_center = np.clip(h_center, self.half_height, h - self.height + self.half_height)
    crop_left = w_center - self.half_width
    crop_right = crop_left + self.width
    crop_top = h_center - self.half_height
    crop_bottom = crop_top + self.height
    return (crop_left, crop_top, crop_right, crop_bottom)
CenterWithSatellite dataclass 
CenterWithSatellite(point: Iterable[int], window_manager: WindowManager, satellite: Optional[Iterable[int]] = None)

Center point with 'satellites'. 'Satellites' is a point within center point's window, but not the center point. Also intra distance between 'satellites' should shorter than window size.

Attributes:

  • coordinate (Iterable[int]) –

    center point coordinate

  • window_manager (WindowManager) –

    WindowManager

  • max_width_distance (int) –

    max width-wise positive distance of satellites

  • max_height_distance (int) –

    max height-wise positive distance of satellites

  • min_width_distance (int) –

    max width-wise negative distance of satellites

  • min_height_distance (int) –

    max height-wise negative distance of satellites

  • satellite_points (List[Dict[str, any]]) –

    satellite_points data

Source code in SaigeToolkit/data/crop/edge_cropper.py 
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
def __init__(
    self,
    point: Iterable[int],
    window_manager: WindowManager,
    satellite: Optional[Iterable[int]] = None,
):
    self.coordinate = point
    self.window_manager = window_manager

    self.max_width_distance = 0
    self.max_height_distance = 0
    self.min_width_distance = 0
    self.min_height_distance = 0

    self.satellite_points = {}
    if satellite is not None:
        self.calculate_relation_as_satellite(satellite)
calculate_relation_as_satellite 
calculate_relation_as_satellite(point: Iterable[int]) -> Union[Dict[str, Any], List[Dict[str, Any]]]

calculate relation between target point and center point. case 1: If target point is outside of center point's window, target point should be considered as next center point candidate.

case 2

Even when target point is in the window, check whether distance between satellites is larger than window size. If so, target point should be considered as next center point candidate.

Parameters:

  • point (Iterable[int]) –

    target point to be checked

Returns:

  • Union[Dict[str, Any], List[Dict[str, Any]]] –

    Union[Dict[str, Any], List[Dict[str, Any]]]: Dict[str, Any]: simple relation between center point and target point List[Dict[str, Any]]: reltations between satellite points and target point

Source code in SaigeToolkit/data/crop/edge_cropper.py 
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
def calculate_relation_as_satellite(
    self, point: Iterable[int]
) -> Union[Dict[str, Any], List[Dict[str, Any]]]:
    """calculate relation between target point and center point.
    case 1:
        If target point is outside of center point's window,
        target point should be considered as next center point candidate.

    case 2:
        Even when target point is in the window,
        check whether distance between satellites is larger than window size.
        If so, target point should be considered as next center point candidate.

    Args:
        point (Iterable[int]): target point to be checked

    Returns:
        Union[Dict[str, Any], List[Dict[str, Any]]]:
            Dict[str, Any]: simple relation between center point and target point
            List[Dict[str, Any]]: reltations between satellite points and target point
    """
    width_distance = point[0] - self.coordinate[0]
    height_distance = point[1] - self.coordinate[1]
    relation = self.window_manager.check_window(point, self.coordinate)

    # target point is outside of center point's window.
    # Do not update satellite and return the relationship.
    if relation != WindowState.IN:
        return {"coordinate": self.coordinate, "relation": relation}

    satellites_intra_distance_errors = []
    if width_distance > self.max_width_distance:
        if abs(width_distance) + abs(self.min_width_distance) > self.window_manager.half_width:
            self._compare_point_and_calculate_error(
                "min_width", point, satellites_intra_distance_errors
            )
        else:
            # else, update as satellite point.
            self.max_width_distance = width_distance
            self.satellite_points.update({"max_width": {"coordinate": point, "relation": relation}})

    if height_distance > self.max_height_distance:
        if abs(height_distance) + abs(self.min_height_distance) > self.window_manager.half_height:
            self._compare_point_and_calculate_error(
                "min_height", point, satellites_intra_distance_errors
            )
        else:
            # else, update as satellite point.
            self.max_height_distance = height_distance
            self.satellite_points.update({"max_height": {"coordinate": point, "relation": relation}})

    if width_distance < self.min_width_distance:
        if abs(width_distance) + abs(self.max_width_distance) > self.window_manager.half_width:
            self._compare_point_and_calculate_error(
                "max_width", point, satellites_intra_distance_errors
            )
        else:
            # else, update as satellite point.
            self.min_width_distance = width_distance
            self.satellite_points.update({"min_width": {"coordinate": point, "relation": relation}})

    if height_distance < self.min_height_distance:
        if abs(height_distance) + abs(self.max_height_distance) > self.window_manager.half_height:
            self._compare_point_and_calculate_error(
                "max_height", point, satellites_intra_distance_errors
            )
        else:
            # else, update as satellite point.
            self.min_height_distance = height_distance
            self.satellite_points.update({"min_height": {"coordinate": point, "relation": relation}})

    if satellites_intra_distance_errors:
        # any distance between satellites larger than window
        return satellites_intra_distance_errors

    return {"coordinate": self.coordinate, "relation": relation}
_compare_point_and_calculate_error 
_compare_point_and_calculate_error(key, point, satellites_intra_distance_errors)

Refactored from the original code from Defect Generation (Function extracted).

Source code in SaigeToolkit/data/crop/edge_cropper.py 
248
249
250
251
252
253
254
255
def _compare_point_and_calculate_error(self, key, point, satellites_intra_distance_errors):
    """Refactored from the original code from Defect Generation (Function extracted)."""
    # distance between satellites is larger than window size.
    compare_point = (
        self.satellite_points[key]["coordinate"] if key in self.satellite_points else self.coordinate
    )
    _relation = self.window_manager.check_window(point, compare_point)
    satellites_intra_distance_errors.append({"coordinate": compare_point, "relation": _relation})
EdgeCropper 
EdgeCropper(crop_w: int = 512, crop_h: int = 512, random_sample: bool = False, patch_per_polygon: int = 1, strict_inner_patch: bool = True, ok_patch_prob: float = 0.01)

Bases: BaseCropper

Crop image patches along the contour of mask (or polygon)

Attributes:

  • window_manager (WindowManager) –

    WindowManager

  • random_sample (bool) –

    whether randomly select the first point of each polygon

  • patch_per_polygon (bool) –

    number of repeat for cropping patches around each polygon.

Source code in SaigeToolkit/data/crop/edge_cropper.py 
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
def __init__(
    self,
    crop_w: int = 512,
    crop_h: int = 512,
    random_sample: bool = False,
    patch_per_polygon: int = 1,
    strict_inner_patch: bool = True,
    ok_patch_prob: float = 0.01,
) -> None:
    self.crop_w = crop_w
    self.crop_h = crop_h
    self.window_manager = WindowManager(crop_w, crop_h, strict_inner_patch)
    self.random_sample = random_sample
    self.patch_per_polygon = patch_per_polygon
    self.ok_patch_prob = ok_patch_prob
    self.n_defect_patch_made = 0

    self.init_centers()
get_n_patch 
get_n_patch(polygons: List[ndarray]) -> int

PolygonCropper crops all polygon areas {self.patch_per_polygon} times per polygons, and also crops random position of image {self.random_patch_per_img} times per images. Therefore, resultant number of patches is weighted-sum result as coded below.

Parameters:

  • polygons (List[ndarray]) –

    polygons of single image in dataset.

Returns:

  • int ( int ) –

    number of patches to be cropped in single image

Source code in SaigeToolkit/data/crop/edge_cropper.py 
289
290
291
292
293
294
295
296
297
298
299
300
def get_n_patch(self, polygons: List[np.ndarray]) -> int:
    """PolygonCropper crops all polygon areas {self.patch_per_polygon} times per polygons,
    and also crops random position of image {self.random_patch_per_img} times per images.
    Therefore, resultant number of patches is weighted-sum result as coded below.

    Args:
        polygons (List[np.ndarray]): polygons of single image in dataset.

    Returns:
        int: number of patches to be cropped in single image
    """
    raise NotImplementedError
__call__ 
__call__(image: Union[Image, ndarray], mask: Union[Image, ndarray], polygons: Optional[List[ndarray]] = None, **kwargs) -> List[dict]

crop image into image patches.

Parameters:

  • image (Union[Image, ndarray]) –

    original image

  • polygons (List[ndarray], default: None ) –

    polygon data (# of polygons, (4, 2)) polygon should have 4 points

  • mask (Union[Image, ndarray]) –

    segmentation mask image

Returns:

  • List[dict] –

    List[dict]: list of cropped image data dict

Source code in SaigeToolkit/data/crop/edge_cropper.py 
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
def __call__(
    self,
    image: Union[Image.Image, np.ndarray],
    mask: Union[Image.Image, np.ndarray],
    polygons: Optional[List[np.ndarray]] = None,
    **kwargs,
) -> List[dict]:
    """crop image into image patches.

    Args:
        image (Union[Image.Image, np.ndarray]): original image
        polygons (List[np.ndarray]): polygon data
            (# of polygons, (4, 2)) polygon should have 4 points
        mask (Union[Image.Image, np.ndarray]): segmentation mask image

    Returns:
        List[dict]: list of cropped image data dict
    """

    w, h = read_image_size(image)

    if polygons is None:
        polygons, _ = cv2.findContours(np.asarray(mask), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
        polygons = [polygon[:, 0] for polygon in polygons]  # TODO: check..

    l_data = []
    self.init_centers()

    # crop defect polygon
    _polygons = random.sample(polygons, len(polygons)) if self.random_sample else polygons
    for polygon in _polygons:
        polygon = subdivide_points_into_min_resolution(list(polygon), 32, 32)
        for _ in range(self.patch_per_polygon):
            max_iter_per_polygon = 3 if self.random_sample else 1  # TODO: as class variable?
            min_num_point_not_included = None
            for _ in range(max_iter_per_polygon):
                new_centers = self.pick_centers_from_polygon(polygon)

                num_point_not_included = sum(
                    not self.window_manager.is_in_any_window(p, [*self.centers, *new_centers])
                    for p in polygon
                )
                if num_point_not_included == 0:
                    self.centers.extend(new_centers)
                    break

                if min_num_point_not_included is None:
                    min_num_point_not_included = num_point_not_included
                    new_centers_candidate = new_centers

                if num_point_not_included < min_num_point_not_included:
                    min_num_point_not_included = num_point_not_included
                    new_centers_candidate = new_centers

            else:
                # If fail to get `num_point_not_included==0` sample,
                # use min `num_point_not_included` sample.
                new_centers = new_centers_candidate
                self.centers.extend(new_centers)

            for center in new_centers:
                single_defect_patch_data = self.get_data_dict_from_center_point(
                    image, mask, center, **kwargs
                )
                l_data.append(single_defect_patch_data)

    self.n_defect_patch_made += len(l_data)

    # crop ok region
    # XXX: OK patch portion can be larger w.r.t num of ok images.
    probability_multiplier = max(self.n_defect_patch_made, 1)
    if random.random() < self.ok_patch_prob * probability_multiplier:
        num_to_make_ok_patch = ceil(self.ok_patch_prob * probability_multiplier)
        max_iteration = 5
        for _ in range(num_to_make_ok_patch):
            for _ in range(max_iteration):
                if self.window_manager.strict_inner_patch:
                    window_half_w = self.window_manager.half_width
                    window_half_h = self.window_manager.half_height

                    # Cannot sample random window center.
                    if w - window_half_w < window_half_w or h - window_half_h < window_half_h:
                        logger.warning(f"Crop size is larger than image size: ({w}, {h}).")
                        continue

                    random_center = (
                        random.randint(window_half_w, w - window_half_w),
                        random.randint(window_half_h, h - window_half_h),
                    )
                else:
                    random_center = random.randint(0, w), random.randint(0, h)
                if not self.window_manager.is_in_any_window(random_center, self.centers):
                    self.centers.append(random_center)
                    self.n_defect_patch_made -= 1 / self.ok_patch_prob
                    break
            else:
                # Max trial ended with no ok patch.
                return l_data

            single_ok_patch_data = self.get_data_dict_from_center_point(
                image, mask, random_center, **kwargs
            )
            l_data.append(single_ok_patch_data)

    return l_data
get_proper_point 
get_proper_point(state_as_satellite: Union[Dict[str, Any], List[Dict[str, Any]]], must_contain_coordinate: List[Iterable[int]], point_in: Iterable[int], point_out: Iterable[int]) -> Optional[Iterable[int]]

get proper next center point that contains every must_contain_coordinate

Parameters:

  • state_as_satellite (Union[Dict[str, Any], List[Dict[str, Any]]]) –

    relation between center point(s) and target point.

  • must_contain_coordinate (List[Iterable[int]]) –

    points should be in next window

  • point_in (Iterable[int]) –

    point inside window from previous boundary

  • point_out (Iterable[int]) –

    point outside window from previous boundary

Returns:

  • Optional[Iterable[int]] –

    Optional[Iterable[int]]: proper next center (or None)

Source code in SaigeToolkit/data/crop/edge_cropper.py 
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
def get_proper_point(
    self,
    state_as_satellite: Union[Dict[str, Any], List[Dict[str, Any]]],
    must_contain_coordinate: List[Iterable[int]],
    point_in: Iterable[int],
    point_out: Iterable[int],
) -> Optional[Iterable[int]]:
    """get proper next center point that contains every `must_contain_coordinate`

    Args:
        state_as_satellite (Union[Dict[str, Any], List[Dict[str, Any]]]):
            relation between center point(s) and target point.
        must_contain_coordinate (List[Iterable[int]]): points should be in next window
        point_in (Iterable[int]): point inside window from previous boundary
        point_out (Iterable[int]): point outside window from previous boundary

    Returns:
        Optional[Iterable[int]]: proper next center (or None)
    """

    if isinstance(state_as_satellite, list):
        for u_r in state_as_satellite:
            p_dst = self.get_proper_point(u_r, must_contain_coordinate, point_in, point_out)
            if p_dst is not None:
                break

        else:
            return None

    else:
        relation, inner_point = state_as_satellite["relation"], state_as_satellite["coordinate"]
        p_dst = self.get_destination_point(relation, inner_point, point_in, point_out)

        if p_dst is None:
            return None

        for _p in must_contain_coordinate:
            if not self.window_manager.is_in_window(_p, p_dst):
                return None

    return p_dst
get_destination_point 
get_destination_point(relation: WindowState, point_ref: Iterable[int], point_in: Iterable[int], point_out: Iterable[int]) -> ndarray

calculate point a-window-away from reference point

Parameters:

  • relation (WindowState) –

    relation between reference point and next center

  • point_ref (Iterable[int]) –

    reference point to calculate next center

  • point_in (Iterable[int]) –

    point inside window from previous boundary

  • point_out (Iterable[int]) –

    point outside window from previous boundary

Returns:

  • ndarray –

    np.ndarray: result destination point

Source code in SaigeToolkit/data/crop/edge_cropper.py 
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
def get_destination_point(
    self,
    relation: WindowState,
    point_ref: Iterable[int],
    point_in: Iterable[int],
    point_out: Iterable[int],
) -> np.ndarray:
    """calculate point a-window-away from reference point

    Args:
        relation (WindowState): relation between reference point and next center
        point_ref (Iterable[int]): reference point to calculate next center
        point_in (Iterable[int]): point inside window from previous boundary
        point_out (Iterable[int]): point outside window from previous boundary

    Returns:
        np.ndarray: result destination point
    """
    w_in, h_in = point_in
    w_out, h_out = point_out

    def _get_interior_point(a_dst, a_in, a_out, b_in, b_out):
        if (a_out != a_in) and (a_in <= a_dst <= a_out or a_out <= a_dst <= a_in):
            rate = (a_dst - a_in) / (a_out - a_in)
        else:
            # if a_dst is not interior between two points, update a_dst to the middle point.
            a_dst = int(round((a_out + a_in) / 2))
            rate = 0.5

        b_dst = b_in + int(round(rate * (b_out - b_in)))
        return a_dst, b_dst

    if relation == WindowState.WIDTH_POSITIVE_OUT:
        w_dst = point_ref[0] + self.window_manager.half_width
        w_dst, h_dst = _get_interior_point(w_dst, w_in, w_out, h_in, h_out)
    elif relation == WindowState.WIDTH_NEGATIVE_OUT:
        w_dst = point_ref[0] - self.window_manager.half_width
        w_dst, h_dst = _get_interior_point(w_dst, w_in, w_out, h_in, h_out)
    elif relation == WindowState.HEIGHT_POSITIVE_OUT:
        h_dst = point_ref[1] + self.window_manager.half_height
        h_dst, w_dst = _get_interior_point(h_dst, h_in, h_out, w_in, w_out)
    elif relation == WindowState.HEIGHT_NEGATIVE_OUT:
        h_dst = point_ref[1] - self.window_manager.half_height
        h_dst, w_dst = _get_interior_point(h_dst, h_in, h_out, w_in, w_out)

    else:
        # if relation is `WindowState.In`, find farthest point and set as target point.
        width_distance = w_out - point_ref[0]
        height_distance = h_out - point_ref[1]
        if abs(width_distance) > abs(height_distance):
            if width_distance > 0:
                relation = WindowState.WIDTH_POSITIVE_OUT
            else:
                relation = WindowState.WIDTH_NEGATIVE_OUT
        else:
            if height_distance > 0:
                relation = WindowState.HEIGHT_POSITIVE_OUT
            else:
                relation = WindowState.HEIGHT_NEGATIVE_OUT

        dst = self.get_destination_point(relation, point_ref, point_in, point_out)

        return dst

    return np.array([w_dst, h_dst])
remove_inner_points_and_get_outer_points 
remove_inner_points_and_get_outer_points(center: Iterable[int], points: List[Iterable[int]]) -> Tuple[List[Iterable[int]], Iterable[int], Iterable[int]]

once center decided, check other points and remove insiders.

Parameters:

  • center (Iterable[int]) –

    next center coordinate

  • points (List[Iterable[int]]) –

    current points on polygon

Returns:

  • Tuple[List[Iterable[int]], Iterable[int], Iterable[int]] –

    Tuple[List[Iterable[int]], Iterable[int], Iterable[int]]: - List[Iterable[int]]: clean-up'd polygon points - Iterable[int]: boundary point (positive direction) - Iterable[int]: boundary point (negative direction)

Source code in SaigeToolkit/data/crop/edge_cropper.py 
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
def remove_inner_points_and_get_outer_points(
    self, center: Iterable[int], points: List[Iterable[int]]
) -> Tuple[List[Iterable[int]], Iterable[int], Iterable[int]]:
    """once center decided, check other points and remove insiders.

    Args:
        center (Iterable[int]): next center coordinate
        points (List[Iterable[int]]): current points on polygon

    Returns:
        Tuple[List[Iterable[int]], Iterable[int], Iterable[int]]:
            - List[Iterable[int]]: clean-up'd polygon points
            - Iterable[int]: boundary point (positive direction)
            - Iterable[int]: boundary point (negative direction)
    """
    idx_positive_direction, idx_negative_direction = 0, 0
    p_dst, n_dst = None, None

    for idx, p in enumerate(points):
        relation = self.window_manager.check_window(p, center)
        if relation == WindowState.IN:
            continue

        idx_positive_direction = idx - 1
        p_dst = self.get_destination_point(
            relation,
            center,
            points[idx_positive_direction],
            points[idx_positive_direction + 1],
        )
        break

    for idx, p in enumerate(points[::-1]):
        relation = self.window_manager.check_window(p, center)
        if relation == WindowState.IN:
            continue

        idx_negative_direction = -idx
        n_dst = self.get_destination_point(
            relation,
            center,
            points[idx_negative_direction],
            points[idx_negative_direction - 1],
        )
        break

    if p_dst is None:
        return [], p_dst, n_dst

    points = (
        points[idx_positive_direction + 1 : idx_negative_direction]
        if idx_negative_direction < 0
        else points[idx_positive_direction + 1 :]
    )

    return points, p_dst, n_dst
subdivide_points_into_min_resolution 
subdivide_points_into_min_resolution(points: List[Iterable[int]], width_resolution: int = 32, height_resolution: int = 32) -> List[Iterable[int]]

refine cv2.findContours result. long straight edge in mask results in long distance between two points.

Parameters:

  • points (List[Iterable[int]]) –

    cv2.findContours result

  • width_resolution (int, default: 32 ) –

    maximum distance width-wise. Defaults to 32.

  • height_resolution (int, default: 32 ) –

    maximum distance height-wise. Defaults to 32.

Returns:

  • List[Iterable[int]] –

    List[Iterable[int]]: refined polygon points

Source code in SaigeToolkit/data/crop/edge_cropper.py 
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
def subdivide_points_into_min_resolution(
    points: List[Iterable[int]], width_resolution: int = 32, height_resolution: int = 32
) -> List[Iterable[int]]:
    """refine cv2.findContours result.
    long straight edge in mask results in long distance between two points.

    Args:
        points (List[Iterable[int]]): cv2.findContours result
        width_resolution (int, optional): maximum distance width-wise. Defaults to 32.
        height_resolution (int, optional): maximum distance height-wise. Defaults to 32.

    Returns:
        List[Iterable[int]]: refined polygon points
    """

    idx_inspected = 0

    while idx_inspected < len(points):
        for idx in range(idx_inspected, len(points)):
            point_prev = points[idx]
            point_next = points[idx + 1] if idx + 1 < len(points) else points[0]

            w_length = abs(point_next[0] - point_prev[0])
            if w_length > width_resolution:
                num_insert = ceil(w_length / width_resolution) - 1
                break

            h_length = abs(point_next[1] - point_prev[1])
            if h_length > height_resolution:
                num_insert = ceil(h_length / height_resolution) - 1
                break
        else:
            break

        for n in range(num_insert - 1, -1, -1):
            rate = (n + 1) / (num_insert + 1)
            point_prev = points[idx]
            point_next = points[idx + 1] if idx + 1 < len(points) else points[0]
            points.insert(
                idx + 1, point_prev + np.round(rate * (point_next - point_prev)).astype(np.int32)
            )

        idx_inspected = idx + n

    return points
get_mid_point 
get_mid_point(points: List[Iterable[int]]) -> Iterable[int]

In case failed to get next center point, get mid point from remaining polygon points.

Parameters:

  • points (List[Iterable[int]]) –

    remaining polygon points

Returns:

  • Iterable[int] –

    Iterable[int]: next center point (mid)

Source code in SaigeToolkit/data/crop/edge_cropper.py 
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
def get_mid_point(points: List[Iterable[int]]) -> Iterable[int]:
    """In case failed to get next center point,
    get mid point from remaining polygon points.

    Args:
        points (List[Iterable[int]]): remaining polygon points

    Returns:
        Iterable[int]: next center point (mid)
    """
    if len(points) == 1:
        return points[0]

    np_points = np.array(points)
    length_between_points = np.linalg.norm(np_points[1:] - np_points[:-1], axis=1)
    length_half = np.sum(length_between_points) / 2
    for idx in range(len(length_between_points)):
        if np.sum(length_between_points[:idx]) >= length_half:
            break

    length_desired = length_half - np.sum(length_between_points[: idx - 1])
    length_entire = length_between_points[idx - 1]
    rate = length_desired / length_entire

    return points[idx] + np.round(rate * (points[idx + 1] - points[idx])).astype(np.int32)

ocr_cropper

OcrCropper 
OcrCropper(chr_height: int = 32, crop_jitter: Optional[float] = None, crop_jitter_rate: Optional[List[float]] = None, save_polygons: bool = False)

Cropper class for OCR task.

Attributes:

  • chr_height (int) –

    cropped image patch is resized to be the same height as chr_height.

  • crop_jitter (Optional[float]) –

    if not None, cropping box is jittered horizontally with ratio crop_jitter.

  • crop_jitter_rate (Optional[List[float]]) –

    if not None, cropping box points are jittered to any direction with ratio crop_jitter_rate.

  • save_polygons (bool) –

    whether polygon label is preserved after cropping.

initializing OcrCropper. all initializing input is set to class attribute.

Parameters:

  • chr_height (int, default: 32 ) –

    Defaults to 32.

  • crop_jitter (Optional[float], default: None ) –

    Defaults to None.

  • crop_jitter_rate (Optional[List[float]], default: None ) –

    Defaults to None.

  • save_polygons (bool, default: False ) –

    Defaults to False.

Source code in SaigeToolkit/data/crop/ocr_cropper.py 
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
def __init__(
    self,
    chr_height: int = 32,
    crop_jitter: Optional[float] = None,
    crop_jitter_rate: Optional[List[float]] = None,
    save_polygons: bool = False,
) -> None:
    """initializing OcrCropper.
    all initializing input is set to class attribute.

    Args:
        chr_height (int): Defaults to 32.
        crop_jitter (Optional[float], optional): Defaults to None.
        crop_jitter_rate (Optional[List[float]], optional): Defaults to None.
        save_polygons (bool): Defaults to False.
    """
    self.chr_height = chr_height
    self.crop_jitter = crop_jitter
    self.crop_jitter_rate = crop_jitter_rate
    self.save_polygons = save_polygons
get_n_patch 
get_n_patch(polygons: List[List[ndarray]]) -> int

OcrCropper only crops all polygon areas, unlike SegCropper. Therefore, resultant number of patches is simply equals to number of polygons.

Parameters:

  • polygons (List[List[ndarray]]) –

    polygons of all images in dataset.

Returns:

  • int ( int ) –

    total number of patches to be cropped

Source code in SaigeToolkit/data/crop/ocr_cropper.py 
47
48
49
50
51
52
53
54
55
56
57
def get_n_patch(self, polygons: List[List[np.ndarray]]) -> int:
    """OcrCropper only crops all polygon areas, unlike SegCropper.
    Therefore, resultant number of patches is simply equals to number of polygons.

    Args:
        polygons (List[List[np.ndarray]]): polygons of all images in dataset.

    Returns:
        int: total number of patches to be cropped
    """
    return len(polygons)
__call__ 
__call__(image: Union[Image, ndarray], polygons: List[ndarray], strings: List[str], ignore: List[bool], is_vertical: Optional[List[bool]] = None, **kwargs) -> List[dict]

crop image into image patches. polygon, string, ignore data should be same length.

Parameters:

  • image (Union[Image, ndarray]) –

    original image

  • polygons (List[ndarray]) –

    polygon data (# of polygons, (4, 2)) polygon should have 4 points

  • strings (List[str]) –

    string data

  • ignore (List[bool]) –

    ignore notation data

  • is_vertical (List[bool], default: None ) –

    is_vertical notation data

Returns:

  • List[dict] –

    List[dict]: list of cropped image data dict

Source code in SaigeToolkit/data/crop/ocr_cropper.py 
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
def __call__(
    self,
    image: Union[Image.Image, np.ndarray],
    polygons: List[np.ndarray],
    strings: List[str],
    ignore: List[bool],
    is_vertical: Optional[List[bool]] = None,
    **kwargs,
) -> List[dict]:
    """crop image into image patches.
    polygon, string, ignore data should be same length.

    Args:
        image (Union[Image.Image, np.ndarray]): original image
        polygons (List[np.ndarray]): polygon data
            (# of polygons, (4, 2)) polygon should have 4 points
        strings (List[str]): string data
        ignore (List[bool]): ignore notation data
        is_vertical (List[bool]): is_vertical notation data

    Returns:
        List[dict]: list of cropped image data dict
    """
    image = np.asarray(image)

    if is_vertical is None:
        is_vertical = [False] * len(polygons)

    l_data = []
    for pol, string, ign, is_vert in zip(polygons, strings, ignore, is_vertical):
        if ign:
            continue

        max_trial = 10
        trial = 0
        while trial < max_trial:
            trial += 1

            tops, bots = self.jitter_polygon(pol)
            widths, heights = self.calculate_width_height(tops, bots)

            if all(np.array(widths + heights) >= 1):
                break

            if trial == max_trial:
                logger.info("Max trial exeeded.. no jitter applied.")
                tops = pol[: len(pol) // 2]
                bots = pol[: len(pol) // 2 - 1 : -1]
                widths, heights = self.calculate_width_height(tops, bots)

        if any(np.array(widths + heights) < 1):
            logger.warning(f"Invalid patch shape (jittered) encounter.. string: {string}")
            continue

        text_patch_list = []
        for i, (w, h) in enumerate(zip(widths, heights)):
            width = int(np.round((w / h * self.chr_height)))
            if width == 0:
                continue

            startpoints = np.array([tops[i], tops[i + 1], bots[i + 1], bots[i]], np.float32)
            endpoints = np.array(
                [
                    [0, 0],
                    [width - 1, 0],
                    [width - 1, self.chr_height - 1],
                    [0, self.chr_height - 1],
                ],
                np.float32,
            )

            affine_matrix = cv2.getPerspectiveTransform(startpoints, endpoints)
            text_patch = cv2.warpPerspective(
                image, affine_matrix, (width, self.chr_height), borderValue=0
            )
            if i != 0:
                # remove duplicated leftmost column
                # border area is duplicated in adjacent patches.
                text_patch = text_patch[:, 1:, :]
            text_patch_list.append(text_patch)

        text_patch = np.concatenate(text_patch_list, axis=1)

        if int(text_patch.shape[1] / (self.chr_height / 8)) + 1 >= len(string):
            patch_data = {
                "image": Image.fromarray(text_patch),
                "strings": string,
                "is_vertical": is_vert,
            }

            if self.save_polygons:
                patch_data.update(polygons=pol)

            l_data.append(patch_data)
        else:
            logger.warning(
                f"Invalid patch length! string: {string} patch width: {text_patch.shape[1]}"
            )

    return l_data
calculate_width_height 
calculate_width_height(tops: ndarray, bots: ndarray) -> Tuple[List[ndarray], List[ndarray]]

calculate width and heights for every four polygon points (2 tops and 2 bots). Assuming that polygon is composed of top-points and bot-points, and every top and bot points are paired.

Parameters:

  • tops (ndarray) –

    top points of polygon

  • bots (ndarray) –

    bot points of polygon

Returns:

  • Tuple[List[ndarray], List[ndarray]] –

    Tuple[List[np.ndarray]]: width and height list for every four points.

Source code in SaigeToolkit/data/crop/ocr_cropper.py 
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
def calculate_width_height(
    self, tops: np.ndarray, bots: np.ndarray
) -> Tuple[List[np.ndarray], List[np.ndarray]]:
    """calculate width and heights for every four polygon points (2 tops and 2 bots).
    Assuming that polygon is composed of top-points and bot-points,
    and every top and bot points are paired.

    Args:
        tops (np.ndarray): top points of polygon
        bots (np.ndarray): bot points of polygon

    Returns:
        Tuple[List[np.ndarray]]: width and height list for every four points.
    """

    widths = []
    for i in range(len(tops) - 1):
        widths.append(
            np.mean(
                (
                    np.linalg.norm(tops[i + 1] - tops[i]),
                    np.linalg.norm(bots[i + 1] - bots[i]),
                )
            )
        )

    heights = []
    for t, b in zip(tops, bots):
        heights.append(np.linalg.norm(b - t))
    heights = [np.mean([heights[i + 1], heights[i]]) for i in range(len(heights) - 1)]

    return widths, heights
jitter_polygon 
jitter_polygon(polygon: ndarray) -> Tuple[ndarray, ndarray]

jittering polygon points and split into tops and bots points

Parameters:

  • polygon (ndarray) –

    original polygon data

Returns:

  • Tuple[ndarray, ndarray] –

    Tuple[np.ndarray, np.ndarray]: jittered tops, bots points

Source code in SaigeToolkit/data/crop/ocr_cropper.py 
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
def jitter_polygon(self, polygon: np.ndarray) -> Tuple[np.ndarray, np.ndarray]:
    """jittering polygon points and split into tops and bots points

    Args:
        polygon (np.ndarray): original polygon data

    Returns:
        Tuple[np.ndarray, np.ndarray]: jittered tops, bots points
    """
    tops = deepcopy(polygon[: len(polygon) // 2])
    bots = deepcopy(polygon[: len(polygon) // 2 - 1 : -1])

    if self.crop_jitter is not None:

        heights = []
        for i, (t, b) in enumerate(zip(tops, bots)):
            heights.append(np.linalg.norm(b - t))

            crop_jitter = self.crop_jitter * heights[-1]

            x_jitter = random.uniform(-crop_jitter, crop_jitter)
            tops[i, 0] += x_jitter
            bots[i, 0] += x_jitter

        height = np.mean(heights)
        y_jitter = self.crop_jitter * height

        tops[:, 1] += random.uniform(-y_jitter, y_jitter)
        bots[:, 1] += random.uniform(-y_jitter, y_jitter)

    elif self.crop_jitter_rate is not None:

        for i, (t, b) in enumerate(zip(tops, bots)):
            height = np.linalg.norm(b - t)

            crop_jitter_short = self.crop_jitter_rate[0] * height
            crop_jitter_long = self.crop_jitter_rate[1] * height

            top_y_jitter = random.uniform(-crop_jitter_long, crop_jitter_short)
            bot_y_jitter = random.uniform(-crop_jitter_short, crop_jitter_long)
            if i == 0:
                x_jitter = random.uniform(-crop_jitter_long, crop_jitter_short)
            elif i == len(tops) - 1:
                x_jitter = random.uniform(-crop_jitter_short, crop_jitter_long)
            else:
                x_jitter = random.uniform(-crop_jitter_short, crop_jitter_short)

            tops[i, :] += (x_jitter, top_y_jitter)
            bots[i, :] += (x_jitter, bot_y_jitter)

    # polygon_out = np.concatenate((tops, bots[::-1]), axis=0)

    return tops, bots

polygon_cropper

PolygonCropper 
PolygonCropper(mode: str = 'center', crop_w: int = 512, crop_h: int = 512, patch_per_polygon: int = 1, random_patch_per_img: int = 1, max_patch_per_img: Optional[int] = None, force_random_patch_normal: bool = False, strict_inner_patch: bool = False, fixed_polygon_order: bool = False)

Bases: BaseCropper

Crop image patches using polygons

Attributes:

  • mode (str) –

    cropping mode. ["center", "defectrandom"] available

  • crop_w (int) –

    cropping width for image patch.

  • crop_h (int) –

    cropping height for image patch.

  • half_w (int) –

    half of cropping width.

  • half_h (int) –

    half of cropping height.

  • patch_per_polygon (int) –

    number of repeat for cropping patch around each polygon.

  • random_patch_per_img (int) –

    number of random patch cropping per image.

  • force_random_patch_normal (bool) –

    whether random patch should not contain polygon area.

  • strict_inner_patch (bool) –

    whether not allowing outer area of image.

  • fixed_polygon_order (bool) –

    whether order of polygons is fixed during cropping.

All input is directly set to class attribute.

Parameters:

  • mode (str, default: 'center' ) –

    Defaults to "center".

  • crop_w (int, default: 512 ) –

    Defaults to 512.

  • crop_h (int, default: 512 ) –

    Defaults to 512.

  • patch_per_polygon (int, default: 1 ) –

    Defaults to 1.

  • random_patch_per_img (int, default: 1 ) –

    Defaults to 1.

  • force_random_patch_normal (bool, default: False ) –

    Defaults to False.

  • strict_inner_patch (bool, default: False ) –

    Defaults to False.

  • fixed_polygon_order (bool, default: False ) –

    Defaults to False.

Source code in SaigeToolkit/data/crop/polygon_cropper.py 
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
def __init__(
    self,
    mode: str = "center",
    crop_w: int = 512,
    crop_h: int = 512,
    patch_per_polygon: int = 1,
    random_patch_per_img: int = 1,
    max_patch_per_img: Optional[int] = None,
    force_random_patch_normal: bool = False,
    strict_inner_patch: bool = False,
    fixed_polygon_order: bool = False,
) -> None:
    """All input is directly set to class attribute.

    Args:
        mode (str, optional): Defaults to "center".
        crop_w (int, optional): Defaults to 512.
        crop_h (int, optional): Defaults to 512.
        patch_per_polygon (int, optional): Defaults to 1.
        random_patch_per_img (int, optional): Defaults to 1.
        force_random_patch_normal (bool, optional): Defaults to False.
        strict_inner_patch (bool, optional): Defaults to False.
        fixed_polygon_order (bool, optional): Defaults to False.
    """
    self.mode = mode
    self.crop_w = crop_w
    self.crop_h = crop_h
    self.patch_per_polygon = patch_per_polygon
    self.random_patch_per_img = random_patch_per_img
    self.max_patch_per_img = max_patch_per_img
    self.force_random_patch_normal = force_random_patch_normal
    self.strict_inner_patch = strict_inner_patch
    self.fixed_polygon_order = fixed_polygon_order
get_n_patch 
get_n_patch(polygons: List[ndarray]) -> int

PolygonCropper crops all polygon areas {self.patch_per_polygon} times per polygons, and also crops random position of image {self.random_patch_per_img} times per images. Therefore, resultant number of patches is weighted-sum result as coded below.

Parameters:

  • polygons (List[ndarray]) –

    polygons of single image in dataset.

Returns:

  • int ( int ) –

    number of patches to be cropped in single image

Source code in SaigeToolkit/data/crop/polygon_cropper.py 
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
def get_n_patch(self, polygons: List[np.ndarray]) -> int:
    """PolygonCropper crops all polygon areas {self.patch_per_polygon} times per polygons,
    and also crops random position of image {self.random_patch_per_img} times per images.
    Therefore, resultant number of patches is weighted-sum result as coded below.

    Args:
        polygons (List[np.ndarray]): polygons of single image in dataset.

    Returns:
        int: number of patches to be cropped in single image
    """
    n_patch = len(polygons) * self.patch_per_polygon + self.random_patch_per_img
    if self.max_patch_per_img is not None:
        n_patch = min(n_patch, self.max_patch_per_img)

    return n_patch
pick_point 
pick_point(polygon: ndarray) -> Tuple[int, int]

picking cropping center point.

Parameters:

  • polygon (ndarray) –

    original polygon data

Raises:

  • NotImplementedError –

    only two modes ["center", "defectrandom"] available

Returns:

  • Tuple[int, int] –

    Tuple[int, int]: picked point, (h_center, w_center)

Source code in SaigeToolkit/data/crop/polygon_cropper.py 
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
def pick_point(self, polygon: np.ndarray) -> Tuple[int, int]:
    """picking cropping center point.

    Args:
        polygon (np.ndarray): original polygon data

    Raises:
        NotImplementedError: only two modes ["center", "defectrandom"] available

    Returns:
        Tuple[int, int]: picked point, (h_center, w_center)
    """
    plg = list(zip(*polygon))
    pts_w, pts_h = np.array(plg[0]), np.array(plg[1])
    h_min, h_max = np.min(pts_h), np.max(pts_h)
    w_min, w_max = np.min(pts_w), np.max(pts_w)

    if self.mode == "center":
        h_center = int(np.round((h_min + h_max) / 2.0))
        w_center = int(np.round((w_min + w_max) / 2.0))

    elif self.mode == "defectrandom":
        count = 0
        max_count = 10
        while True:
            h_center = np.random.randint(h_min, h_max + 1)
            w_center = np.random.randint(w_min, w_max + 1)

            if cv2.pointPolygonTest(polygon, (w_center, h_center), False) >= 0:
                break

            if count > max_count:
                h_center = int(np.round((h_min + h_max) / 2.0))
                w_center = int(np.round((w_min + w_max) / 2.0))
                break
            count += 1

    else:
        raise NotImplementedError

    return h_center, w_center
get_coordinates_from_center 
get_coordinates_from_center(h_center: int, w_center: int, h: int, w: int) -> Tuple[int, int, int, int]

get left/right top/bottom coordinates from picked center point.

Parameters:

  • h_center (int) –

    picked center point h-coordinate

  • w_center (int) –

    picked center point w-coordinate

  • h (int) –

    image size height

  • w (int) –

    image size width

Returns:

  • Tuple[int, int, int, int] –

    Tuple[int, int, int, int]: (crop_left, crop_top, crop_right, crop_bottom)

Source code in SaigeToolkit/data/crop/polygon_cropper.py 
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
def get_coordinates_from_center(
    self, h_center: int, w_center: int, h: int, w: int
) -> Tuple[int, int, int, int]:
    """get left/right top/bottom coordinates from picked center point.

    Args:
        h_center (int): picked center point h-coordinate
        w_center (int): picked center point w-coordinate
        h (int): image size height
        w (int): image size width

    Returns:
        Tuple[int, int, int, int]: (crop_left, crop_top, crop_right, crop_bottom)
    """
    if self.strict_inner_patch:
        w_center = np.clip(w_center, self.half_w, w - self.crop_w + self.half_w)
        h_center = np.clip(h_center, self.half_h, h - self.crop_h + self.half_h)
    crop_left = w_center - self.half_w
    crop_right = crop_left + self.crop_w
    crop_top = h_center - self.half_h
    crop_bottom = crop_top + self.crop_h
    return (crop_left, crop_top, crop_right, crop_bottom)
__call__ 
__call__(image: Union[Image, ndarray], mask: Union[Image, ndarray], polygons: Optional[List[ndarray]] = None, **kwargs) -> List[dict]

crop image into image patches.

Parameters:

  • image (Union[Image, ndarray]) –

    original image

  • polygons (List[ndarray], default: None ) –

    polygon data (# of polygons, (4, 2)) polygon should have 4 points

  • mask (Union[Image, ndarray]) –

    segmentation mask image

Returns:

  • List[dict] –

    List[dict]: list of cropped image data dict

Source code in SaigeToolkit/data/crop/polygon_cropper.py 
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
def __call__(
    self,
    image: Union[Image.Image, np.ndarray],
    mask: Union[Image.Image, np.ndarray],
    polygons: Optional[List[np.ndarray]] = None,
    **kwargs,
) -> List[dict]:
    """crop image into image patches.

    Args:
        image (Union[Image.Image, np.ndarray]): original image
        polygons (List[np.ndarray]): polygon data
            (# of polygons, (4, 2)) polygon should have 4 points
        mask (Union[Image.Image, np.ndarray]): segmentation mask image

    Returns:
        List[dict]: list of cropped image data dict
    """

    w, h = read_image_size(image)

    if polygons is None:
        polygons, _ = cv2.findContours(np.array(mask), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
        polygons = [polygon[:, 0] for polygon in polygons]  # TODO: check..

    l_data = []

    # crop defect polygon
    _polygons = polygons if self.fixed_polygon_order else random.sample(polygons, len(polygons))
    for polygon in _polygons:
        for _ in range(self.patch_per_polygon):
            h_center, w_center = self.pick_point(polygon)
            crop_coordinates = self.get_coordinates_from_center(h_center, w_center, h, w)
            l_data.append(
                {
                    "image": crop(image, crop_coordinates),
                    "mask": crop(mask, crop_coordinates),
                    **kwargs,
                }
            )
            if (
                self.max_patch_per_img is not None
                and len(l_data) + self.random_patch_per_img >= self.max_patch_per_img
            ):
                break
        else:
            continue
        break

    # crop undefect patches
    for _ in range(self.random_patch_per_img):
        while True:
            crop_coordinates = self.get_coordinates_from_center(
                np.random.randint(h), np.random.randint(w), h, w
            )

            patch_lbl = crop(mask, crop_coordinates)
            if (np.array(patch_lbl).sum() == 0) or (not self.force_random_patch_normal):
                break

        l_data.append(
            {
                "image": crop(image, crop_coordinates),
                "mask": patch_lbl,
                **kwargs,
            }
        )

    return l_data

dataclass

segment

Segment 
Segment(bounding_box: Optional[SegmentBox] = None, bitmap: Optional[ndarray] = None, contours: Optional[Contours] = None, class_index: Optional[int] = None)

Segment 타입을 정의합니다. (= SegmentedObject) Segmentation 라벨링 혹은 모델의 예측 결과로 나오는 연결된 픽셀 덩어리이며, 1개의 outer polygon과 여러개의 inner polygon (도넛 형태인 경우) 으로 구성된 오브젝트입니다.

해당 오브젝트를 표현하는 방식은 2가지가 존재하며, segment를 이용해 어떤 연산을 수행하는가에 따라 다른 표현 방식이 필요합니다. 1. bounding_box & bitmap: SegmentBox & np.ndarray 2. contours: Sequence[np.ndarray]

Segment 오브젝트를 생성하기 위해서는 1가지 표현의 데이터만 필요하고, 다른 표현이 필요한 연산의 경우 lazy 한 방식으로 해당 표현을 계산합니다.

Segment 오브젝트 생성 시 bounding_box, bitmap 과 contours가 모두 주어진 경우, 서로 일치하는지 확인하지 않습니다.

Segment 오브젝트 생성 시 bounding_box, bitmap 표현이 주어진 경우 모든 픽셀이 연결되어 있는지 확인하지 않습니다.

Parameters:

  • bounding_box (Optional[SegmentBox], default: None ) –

    description. Defaults to None.

  • bitmap (Optional[Bitmap], default: None ) –

    description. Defaults to None.

  • contours (Optional[Contours], default: None ) –

    description. Defaults to None.

  • class_index (Optional[int], default: None ) –

    description. Defaults to None.

Source code in SaigeToolkit/data/dataclass/segment.py 
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
def __init__(
    self,
    bounding_box: Optional[SegmentBox] = None,
    bitmap: Optional[np.ndarray] = None,
    contours: Optional[Contours] = None,
    class_index: Optional[int] = None,
) -> None:
    if contours is None:
        assert (
            bitmap is not None and bounding_box is not None
        ), "Either `bitmap` + `bounding_box` or `contours` must be given"
    self._bounding_box = bounding_box
    self._bitmap = bitmap
    self._contours = contours
    self.class_index = class_index
get_bounding_box_from_contours 
get_bounding_box_from_contours(contours: Contours) -> SegmentBox

contours의 bounding box를 구합니다.

Source code in SaigeToolkit/data/dataclass/segment.py 
130
131
132
133
def get_bounding_box_from_contours(contours: Contours) -> SegmentBox:
    """contours의 bounding box를 구합니다."""
    left, top, width, height = cv2.boundingRect(contours[0])
    return SegmentBox.from_xywh(left, top, width, height)
convert_contours_to_box_and_bitmap 
convert_contours_to_box_and_bitmap(contours: Contours, bounding_box: Optional[SegmentBox] = None) -> Tuple[SegmentBox, Bitmap]

contours를 bounding_box 와 bitmap representation으로 변환합니다

Source code in SaigeToolkit/data/dataclass/segment.py 
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
def convert_contours_to_box_and_bitmap(
    contours: Contours,
    bounding_box: Optional[SegmentBox] = None,
) -> Tuple[SegmentBox, Bitmap]:
    """contours를 bounding_box 와 bitmap representation으로 변환합니다"""
    if bounding_box is None:
        bounding_box = get_bounding_box_from_contours(contours)
    bitmap = cv2.drawContours(
        np.zeros((bounding_box.height, bounding_box.width), dtype=np.uint8),
        contours=contours,
        contourIdx=-1,
        color=BITMAP_PIXEL_VALUE,
        thickness=-1,
        offset=[-bounding_box.left, -bounding_box.top],
    )
    return bounding_box, bitmap
convert_box_and_bitmap_to_contours 
convert_box_and_bitmap_to_contours(bitmap: Bitmap, bounding_box: Optional[SegmentBox] = None) -> Contours

bounding_box 와 bitmap을 contours representation으로 변환합니다

Source code in SaigeToolkit/data/dataclass/segment.py 
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
def convert_box_and_bitmap_to_contours(
    bitmap: Bitmap,
    bounding_box: Optional[SegmentBox] = None,
) -> Contours:
    """bounding_box 와 bitmap을 contours representation으로 변환합니다"""
    polygons, hierarchy = cv2.findContours(bitmap, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE)
    if not polygons:
        raise SegmentValueError
    parents = hierarchy[0, :, -1]
    outers = np.where(parents == -1)[0]
    if len(outers) != 1:
        raise SegmentValueError
    outer_idx = outers[0]
    contours = [polygons[outer_idx]]
    for idx_inner in np.where(parents == outer_idx)[0]:
        contours.append(polygons[idx_inner])
    if bounding_box is not None:
        contours = [contour + (bounding_box.left, bounding_box.top) for contour in contours]
    return contours
compute_box_intersection 
compute_box_intersection(box1: SegmentBox, box2: SegmentBox) -> Optional[SegmentBox]

intersecting box를 계산합니다. 겹치지 않는 경우 None

Source code in SaigeToolkit/data/dataclass/segment.py 
175
176
177
178
179
180
181
182
183
184
185
def compute_box_intersection(box1: SegmentBox, box2: SegmentBox) -> Optional[SegmentBox]:
    """intersecting box를 계산합니다. 겹치지 않는 경우 None"""
    left = max(box1.left, box2.left)
    top = max(box1.top, box2.top)
    right = min(box1.right, box2.right)
    bottom = min(box1.bottom, box2.bottom)
    intersection = SegmentBox.from_xyxy(left, top, right, bottom)
    if intersection.width <= 0 or intersection.height <= 0:
        return None
    else:
        return intersection
to_segments 
to_segments(segments: List[Union[Segment, Dict]]) -> List[Segment]

List[Union[Segment, Dict]]를을 List[Segment]들로 변환합니다

Source code in SaigeToolkit/data/dataclass/segment.py 
188
189
190
def to_segments(segments: List[Union[Segment, Dict]]) -> List[Segment]:
    """List[Union[Segment, Dict]]를을 List[Segment]들로 변환합니다"""
    return [Segment.from_any(segment) for segment in segments]
merge_segments 
merge_segments(segments: List[Segment]) -> Segment

여러 Segment들을 하나의 Segment로 합칩니다. (union)

Source code in SaigeToolkit/data/dataclass/segment.py 
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
def merge_segments(segments: List[Segment]) -> Segment:
    """여러 Segment들을 하나의 Segment로 합칩니다. (union)"""
    if not segments:
        raise ValueError
    left = min(segment.bounding_box.left for segment in segments)
    top = min(segment.bounding_box.top for segment in segments)
    right = max(segment.bounding_box.right for segment in segments)
    bottom = max(segment.bounding_box.bottom for segment in segments)
    box = SegmentBox.from_xyxy(left, top, right, bottom)
    bitmap = np.zeros((box.height, box.width), dtype=np.uint8)
    for segment in segments:
        offset_x = segment.bounding_box.left - left
        offset_y = segment.bounding_box.top - top
        bitmap[
            offset_y : offset_y + segment.bounding_box.height,
            offset_x : offset_x + segment.bounding_box.width,
        ][segment.bitmap > 0] = BITMAP_PIXEL_VALUE
    return Segment(bounding_box=box, bitmap=bitmap, class_index=segments[0].class_index)
compute_contours_area_exact 
compute_contours_area_exact(contours: Contours) -> int

bounding box 크기의 이미지에 contour를 그린 뒤 픽셀 개수 카운트

Source code in SaigeToolkit/data/dataclass/segment.py 
213
214
215
216
def compute_contours_area_exact(contours: Contours) -> int:
    """bounding box 크기의 이미지에 contour를 그린 뒤 픽셀 개수 카운트"""
    _, bitmap = convert_contours_to_box_and_bitmap(contours=contours)
    return int(np.sum(bitmap) / BITMAP_PIXEL_VALUE)
compute_contours_area_fast 
compute_contours_area_fast(contours: Contours) -> int

cv2.contourArea() 로 contour 면적 계산: contour 테두리를 0.5 픽셀 제외하고 계산되는 듯. average error: 17.08%

Source code in SaigeToolkit/data/dataclass/segment.py 
219
220
221
222
223
224
225
226
227
def compute_contours_area_fast(contours: Contours) -> int:
    """cv2.contourArea() 로 contour 면적 계산: contour 테두리를 0.5 픽셀 제외하고 계산되는 듯.
    average error: 17.08%
    """
    outer = contours[0]
    area = cv2.contourArea(outer)
    for inner in contours[1:]:
        area -= cv2.contourArea(inner)
    return max(int(area), 1)
compute_contours_area_fast_plus 
compute_contours_area_fast_plus(contours: Contours) -> int

cv2.contourArea() 로 contour 면적 계산, outer의 경우 cv2.arcLength() 더해줌 average error: 1.50%

Source code in SaigeToolkit/data/dataclass/segment.py 
230
231
232
233
234
235
236
237
238
239
def compute_contours_area_fast_plus(contours: Contours) -> int:
    """cv2.contourArea() 로 contour 면적 계산, outer의 경우 cv2.arcLength() 더해줌
    average error: 1.50%

    """
    outer = contours[0]
    area = cv2.contourArea(outer) + 0.5 * cv2.arcLength(outer, closed=True)
    for inner in contours[1:]:
        area -= cv2.contourArea(inner)
    return max(int(area), 1)
compute_contours_area_fast_plusplus 
compute_contours_area_fast_plusplus(contours: Contours) -> int

cv2.contourArea() + cv2.arcLength() 로 contour 면적 계산 average error: 1.58%

Source code in SaigeToolkit/data/dataclass/segment.py 
242
243
244
245
246
247
248
249
250
def compute_contours_area_fast_plusplus(contours: Contours) -> int:
    """cv2.contourArea() + cv2.arcLength() 로 contour 면적 계산
    average error: 1.58%
    """
    outer = contours[0]
    area = cv2.contourArea(outer) + 0.5 * cv2.arcLength(outer, closed=True)
    for inner in contours[1:]:
        area -= cv2.contourArea(inner) + 0.5 * cv2.arcLength(inner, closed=True)
    return max(int(area), 1)
compute_contours_area 
compute_contours_area(contours: Contours, method: str) -> int

contours의 면적을 계산합니다. 참고: https://www.notion.so/65e5a56b2b8744bea087b1a0d2f9bfbf

Source code in SaigeToolkit/data/dataclass/segment.py 
261
262
263
264
265
266
267
def compute_contours_area(contours: Contours, method: str) -> int:
    """contours의 면적을 계산합니다.
    참고: https://www.notion.so/65e5a56b2b8744bea087b1a0d2f9bfbf
    """
    if method not in contours_area_methods:
        raise KeyError
    return contours_area_methods[method](contours)
compute_segment_intersection_area 
compute_segment_intersection_area(segment1: Segment, segment2: Segment) -> int

두 Segment 사이의 겹치는 영역 넓이를 계산합니다. bounding_box를 이용해 겹치는 박스를 먼저 계산한 뒤 해당 박스만 잘라서 겹치는 픽셀 수를 셉니다.

Source code in SaigeToolkit/data/dataclass/segment.py 
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
def compute_segment_intersection_area(segment1: Segment, segment2: Segment) -> int:
    """두 Segment 사이의 겹치는 영역 넓이를 계산합니다.
    bounding_box를 이용해 겹치는 박스를 먼저 계산한 뒤 해당 박스만 잘라서 겹치는 픽셀 수를 셉니다.
    """
    intersection_box = compute_box_intersection(segment1.bounding_box, segment2.bounding_box)
    if intersection_box is None:
        return 0

    left = intersection_box.left - segment1.bounding_box.left
    top = intersection_box.top - segment1.bounding_box.top
    bitmap1_intersection = segment1.bitmap[
        top : top + intersection_box.height,
        left : left + intersection_box.width,
    ]

    left = intersection_box.left - segment2.bounding_box.left
    top = intersection_box.top - segment2.bounding_box.top
    bitmap2_intersection = segment2.bitmap[
        top : top + intersection_box.height,
        left : left + intersection_box.width,
    ]
    return int(np.sum(bitmap1_intersection * bitmap2_intersection))
compute_segment_score 
compute_segment_score(segment: Segment, scoremap: ndarray, method: str = 'mean', return_float: bool = False) -> Union[float, int]

scoremap 중 segment 영역의 score를 대표하는 값을 계산합니다.

Source code in SaigeToolkit/data/dataclass/segment.py 
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
def compute_segment_score(
    segment: Segment,
    scoremap: np.ndarray,
    method: str = "mean",
    return_float: bool = False,
) -> Union[float, int]:
    """scoremap 중 segment 영역의 score를 대표하는 값을 계산합니다."""
    bbox = segment.bounding_box
    scoremap_crop = scoremap[bbox.top : bbox.top + bbox.height, bbox.left : bbox.left + bbox.width]
    scoremap_masked = scoremap_crop[segment.bitmap > 0]

    if method == "mean":
        score = scoremap_masked.mean()
    elif method == "max":
        score = scoremap_masked.max()
    else:
        raise ValueError

    if (not return_float) and np.issubdtype(scoremap.dtype, np.integer):
        return int(score)
    else:
        return float(score)
compute_segment_properties_and_apply_threshold 
compute_segment_properties_and_apply_threshold(segment: Segment, calc_area_and_apply_threshold: bool = False, area_method: str = 'fast_plus', area_threshold: int = 0, calc_score_and_apply_threshold: bool = False, scoremap: Optional[ndarray] = None, score_method: str = 'mean', score_threshold: Union[float, int] = 0, score_as_float: bool = False) -> Optional[Dict]

API 계산 결과로 필요한 Segment의 property 들을 계산하고 threshold를 적용합니다. threshold에 걸리지 않는 경우 각 property들의 Dict를, threshold 에 걸리는 경우 None을 리턴합니다. 계산 로직은 다음과 순서로 적용됩니다.

  1. calc_object_area_and_apply_threshold=True 인 경우
    1. area 계산
    2. area_threshold 적용
  2. calc_object_score_and_apply_threshold=True 인 경우
    1. score 계산
    2. score_threshold 적용
  3. 아래 항목들 계산
    1. bounding_box
    2. bounding_rotated_box
    3. fitted_ellipse

Parameters:

Returns:

  • Optional[Dict] –

    Optional[Dict]: segment property dictionary. threshold에 걸려서 필터링 된 경우 None

Source code in SaigeToolkit/data/dataclass/segment.py 
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
def compute_segment_properties_and_apply_threshold(
    segment: Segment,
    calc_area_and_apply_threshold: bool = False,
    area_method: str = "fast_plus",
    area_threshold: int = 0,
    calc_score_and_apply_threshold: bool = False,
    scoremap: Optional[np.ndarray] = None,
    score_method: str = "mean",
    score_threshold: Union[float, int] = 0,
    score_as_float: bool = False,
) -> Optional[Dict]:
    """API 계산 결과로 필요한 Segment의 property 들을 계산하고 threshold를 적용합니다.
    threshold에 걸리지 않는 경우 각 property들의 Dict를, threshold 에 걸리는 경우 None을 리턴합니다.
    계산 로직은 다음과 순서로 적용됩니다.

    1. `calc_object_area_and_apply_threshold=True` 인 경우
        1. area 계산
        2. area_threshold 적용
    2. `calc_object_score_and_apply_threshold=True` 인 경우
        1. score 계산
        2. score_threshold 적용
    3. 아래 항목들 계산
        1. bounding_box
        2. bounding_rotated_box
        3. fitted_ellipse

    Args:
        segment (Segment): segment

    Returns:
        Optional[Dict]: segment property dictionary. threshold에 걸려서 필터링 된 경우 None
    """
    properties = {}

    # area
    if calc_area_and_apply_threshold:
        area = compute_contours_area(contours=segment.contours, method=area_method)
        if area < area_threshold:
            return None
        properties["area"] = area

    # score
    if calc_score_and_apply_threshold:
        assert scoremap is not None
        score = compute_segment_score(
            segment=segment,
            scoremap=scoremap,
            method=score_method,
            return_float=score_as_float,
        )
        if score < score_threshold:
            return None
        properties["score"] = score

    # default properties
    if segment.class_index is not None:
        properties["class_index"] = segment.class_index
    properties["contours"] = segment.contours
    outer = segment.contours[0]

    # bounding_box: v1 legacy logic
    bbox = segment.bounding_box
    properties["bounding_box"] = [bbox.left, bbox.top, bbox.width, bbox.height]

    # bounding_rot_box: v1 legacy logic
    ((center_x, center_y), (width, height), angle) = cv2.minAreaRect(outer)
    rot_box_points = cv2.boxPoints(((center_x, center_y), (width, height), angle)).tolist()
    bounding_rot_box = [[center_x, center_y], [width, height], angle, rot_box_points]
    properties["bounding_rot_box"] = bounding_rot_box

    # fitted_ellipse: v1 legacy logic
    fitted_ellipse = bounding_rot_box[:3]
    if len(outer) >= 5:
        ((center_x, center_y), (width, height), angle) = cv2.fitEllipse(outer)
        if width > 1e-5 and height > 1e-5:
            fitted_ellipse = [[center_x, center_y], [width, height], angle]
    properties["fitted_ellipse"] = fitted_ellipse

    return properties

dataloader

builder

get_dataloader_class 
get_dataloader_class(cfg_dataloader_name: Optional[str] = None, crop_fn: Optional[Callable[[dict], List[dict]]] = None) -> Type[SaigeDataLoader]

getting dataloader class

Parameters:

  • cfg_dataloader_name (Optional[str], default: None ) –

    name of dataloader class. If None, choose appropriate class considering crop_fn. Defaults to None.

  • crop_fn (Optional[Callable[[dict], List[dict]]], default: None ) –

    image crop function. Used for segmentation/OCR patch-wise training. Defaults to None.

Returns:

Source code in SaigeToolkit/data/dataloader/builder.py 
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
def get_dataloader_class(
    cfg_dataloader_name: Optional[str] = None, crop_fn: Optional[Callable[[dict], List[dict]]] = None
) -> Type[SaigeDataLoader]:
    """getting dataloader class

    Args:
        cfg_dataloader_name (Optional[str], optional): name of dataloader class.
            If None, choose appropriate class considering `crop_fn`. Defaults to None.
        crop_fn (Optional[Callable[[dict], List[dict]]], optional): image crop function.
            Used for segmentation/OCR patch-wise training. Defaults to None.

    Returns:
        Type[SaigeDataLoader]: dataloader class
    """
    if cfg_dataloader_name:
        try:
            return {
                "torch": data.DataLoader,
                "full2patch": Full2PatchDataLoader,
            }[cfg_dataloader_name]
        except:
            raise (f"Dataset {cfg_dataloader_name} not available")

    if crop_fn:
        return Full2PatchDataLoader

    # monkey patch
    def _stop(self):
        pass

    data.DataLoader.stop = _stop

    return data.DataLoader

full2patch_loader

Full2PatchDataLoader 
Full2PatchDataLoader(dataset: Dataset, collate_fn: Optional[Callable[[List[dict]], dict]] = None, batch_sampler: Optional[Iterable[int]] = None, batch_size: int = 1, shuffle: bool = False, drop_last: bool = True, **cfg_dataloader: dict)

DataLoader class for Segmentation and SceneTextRecognition tasks. Whole image is divided into multiple image patches, based on polygon labels. Full2PatchDataLoader has dataloader as class attribute, which returns croppped image patches. Image patches are stocked in buffer attribute, then passed to deep network in batch_size.

Attributes:

  • dataset (Dataset) –

    base dataset

  • batch_size (int) –

    batch size

  • drop_last (bool) –

    whether drop remaining data items less than batch_size at last iter.

  • buffer_multiple (int) –

    maximum size of buffer is decided by {buffer_multiple} x {batch_size}.

  • loader (DataLoader) –

    DataLoader that provides cropped image patches.

  • loader_iter (Iterable[list]) –

    iterator of loader attribute.

  • buffer (List[dict]) –

    buffer where image patches are stacking.

  • collate_function (Callable[[List[dict]], dict]) –

    function collating image patches into a batch.

initializing Full2PatchLoader

Parameters:

  • dataset (Dataset) –

    base dataset to be loaded.

  • collate_fn (Optional[Callable[[List[dict]], dict]], default: None ) –

    function collating image patches into a batch. Defaults to None.

  • sampler (Optional[Iterable[int]]) –

    specific sampler such as balanced sampler. Defaults to None.

  • batch_size (int, default: 1 ) –

    size of a batch for one iteration. Defaults to 1.

  • drop_last (bool, default: True ) –

    drop_last boolean. Defaults to True.

Source code in SaigeToolkit/data/dataloader/full2patch_loader.py 
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
def __init__(
    self,
    dataset: data.Dataset,
    collate_fn: Optional[Callable[[List[dict]], dict]] = None,
    batch_sampler: Optional[Iterable[int]] = None,
    batch_size: int = 1,
    shuffle: bool = False,
    drop_last: bool = True,
    **cfg_dataloader: dict,
):
    """initializing Full2PatchLoader

    Args:
        dataset (data.Dataset): base dataset to be loaded.
        collate_fn (Optional[Callable[[List[dict]], dict]], optional):
            function collating image patches into a batch. Defaults to None.
        sampler (Optional[Iterable[int]], optional):
            specific sampler such as balanced sampler. Defaults to None.
        batch_size (int, optional): size of a batch for one iteration. Defaults to 1.
        drop_last (bool, optional): drop_last boolean. Defaults to True.
    """

    self.dataset = dataset
    self.batch_size = batch_size
    self.shuffle = shuffle
    self.drop_last = drop_last

    self.buffer_multiple = 10
    self.loader = data.DataLoader(
        dataset,
        collate_fn=self.collate_patch,
        batch_sampler=batch_sampler,
        **cfg_dataloader,
    )
    self.loader_iter = None
    self.buffer = []

    self.collate_function = collate_fn if collate_fn is not None else self.collate_batch
collate_patch 
collate_patch(batch: List[List[dict]]) -> List[dict]

dataset with crop_fn returns List[List[dict]] type. Since batch_size of self.loader is hard-defined as 1, simply returning first item of input is enough.

Parameters:

  • batch (List[List[dict]]) –

    batch (1) of cropped patches (n)

Returns:

  • List[dict] –

    List[dict]: cropped patches (n)

Source code in SaigeToolkit/data/dataloader/full2patch_loader.py 
68
69
70
71
72
73
74
75
76
77
78
79
def collate_patch(self, batch: List[List[dict]]) -> List[dict]:
    """dataset with crop_fn returns List[List[dict]] type.
    Since batch_size of self.loader is hard-defined as 1,
    simply returning first item of input is enough.

    Args:
        batch (List[List[dict]]): batch (1) of cropped patches (n)

    Returns:
        List[dict]: cropped patches (n)
    """
    return batch[0]
collate_batch 
collate_batch(batch: List[dict]) -> dict

collating image patches into a batch

Parameters:

  • batch (List[dict]) –

    list cropped patch data dicts (n)

Returns:

  • dict ( dict ) –

    a batch dict

Source code in SaigeToolkit/data/dataloader/full2patch_loader.py 
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
def collate_batch(self, batch: List[dict]) -> dict:
    """collating image patches into a batch

    Args:
        batch (List[dict]): list cropped patch data dicts (n)

    Returns:
        dict: a batch dict
    """

    data_out = {k: [] for k in batch[0].keys()}
    for data in batch:
        for key, value in data.items():
            if torch.is_tensor(value):
                value = value[None]
            data_out[key].append(value)

    for key, value in data_out.items():
        if torch.is_tensor(value[0]):
            try:
                data_out[key] = torch.cat(value)
            except:
                print(f'[{"WARNING".center(9)}] {key} is not concatenated')

    return data_out
__next__ 
__next__() -> dict

generating a batch data, extracting from buffer stack.

Raises:

  • StopIteration –

    self.get_buffer() called due to (len(self.buffer) < self.batch_size) and (len(self.buffer) == 0) even after self.get_buffer() : raise StopIteration.

  • StopIteration –

    self.get_buffer() called due to (len(self.buffer) < self.batch_size) and (len(self.buffer) < self.batch_size) even after self.get_buffer() and (drop_last == True) : raise StopIteration.

Returns:

  • dict ( dict ) –

    a batch dict

Source code in SaigeToolkit/data/dataloader/full2patch_loader.py 
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
def __next__(self) -> dict:
    """generating a batch data, extracting from buffer stack.

    Raises:
        StopIteration:
            self.get_buffer() called due to (len(self.buffer) < self.batch_size)
            and (len(self.buffer) == 0) even after self.get_buffer()
            : raise StopIteration.
        StopIteration:
            self.get_buffer() called due to (len(self.buffer) < self.batch_size)
            and (len(self.buffer) < self.batch_size) even after self.get_buffer()
            and (drop_last == True)
            : raise StopIteration.

    Returns:
        dict: a batch dict
    """

    batch_size = self.batch_size
    if len(self.buffer) < self.batch_size:
        self.get_buffer()

        if len(self.buffer) == 0:
            raise StopIteration

        if len(self.buffer) < self.batch_size:
            if self.drop_last:
                raise StopIteration
            batch_size = len(self.buffer)

    if self.shuffle:
        idx = np.sort(np.random.choice(range(len(self.buffer)), size=batch_size, replace=False))
    else:
        idx = np.array([0 for _ in range(batch_size)])

    batched_buffer = [self.buffer.pop(idx[-i - 1]) for i in range(len(idx))]

    return self.collate_function(batched_buffer)

infinite_random_sampler

InfiniteRandomSampler

Bases: RandomSampler

Infinitely samples elements randomly. If without replacement, then sample from a shuffled dataset. If with replacement, then user can specify :attr:num_samples to draw.

Parameters:

  • data_source (Dataset) –

    dataset to sample from

  • replacement (bool) –

    samples are drawn on-demand with replacement if True, default=False

  • num_samples (int) –

    number of samples to draw, default=len(dataset).

  • generator (Generator) –

    Generator used in sampling.

Assume that this DataLoader is used for training with shuffle=True.

We use InfiniteRandomSampler to make the DataLoader to infinitely sample the dataset. This prevents the "drop_last" and the "prefeching" problem across epochs. Be careful that you should stop the training loop by counting the number of steps manually. The DataLoader will never stop by itself.

NOTE: Note that this sampler has infinite length and thus you should be careful when calculating the current epoch. We recommend you to use step//steps_per_epoch to calculate the current epoch.

dataset

base_dataset

ABCMeta

Bases: ABCMeta

"abstract_attribute" from:\ https://stackoverflow.com/questions/23831510/abstract-attribute-not-property/50381071#50381071

SaigeDataset 
SaigeDataset(crop: Optional[dict] = None, transform: Optional[dict] = None, collate: Optional[Callable[[List[dict]], dict]] = None, device: device = torch.device('cpu'), **neglect: dict)

Bases: Dataset

Saige's basic dataset class.

Attributes:

  • crop_fn (Optional[Callable[[dict], List[dict]]) –

    function cropping image data into image patch data.

  • transform (Type[Transform]) –

    transform class for transforming to a single image data.

  • to_tensor (Callable[[Union[Image.Image, np.ndarray], torch.Tensor]) –

    data returning as Tensor type

  • n_patch (int) –

    number of patches when crop_fn is used.

  • collate_function (Callable[[List[dict]], dict]) –

    function list of data collating into one batch data

initializing Saige base dataset.

Parameters:

  • crop (Optional[dict], default: None ) –

    config for crop function. Defaults to None.

  • transform (Optional[dict], default: None ) –

    config for transforming to a single raw data

  • collate (Optional[Callable[[List[dict]], dict]], default: None ) –

    list of data collating into one batch data. if None, torch.utils basic collate function would be used. Defaults to None.

  • device (device, default: device('cpu') ) –

    gpu device

Source code in SaigeToolkit/data/dataset/base_dataset.py 
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
def __init__(
    self,
    crop: Optional[dict] = None,
    transform: Optional[dict] = None,
    collate: Optional[Callable[[List[dict]], dict]] = None,
    device: torch.device = torch.device("cpu"),
    **neglect: dict,
) -> None:
    """initializing Saige base dataset.

    Args:
        crop (Optional[dict], optional):
            config for crop function. Defaults to None.
        transform (Optional[dict], optional):
            config for transforming to a single raw data
        collate (Optional[Callable[[List[dict]], dict]], optional):
            list of data collating into one batch data.
            if None, torch.utils basic collate function would be used. Defaults to None.
        device (torch.device, optional):
            gpu device
    """
    if neglect:
        logger.warning(f"Neglected arguments: {neglect}")

    # Config options
    self.crop_fn = get_crop_fn(**(crop or {}))
    self.has_crop_fn = crop is not None

    self.transform = Transform(**(transform or {}))

    self.device = device

    if collate is not None:
        assert collate in ["resize", "padding"]
        self.collate_function = {
            "resize": self.resize_collate,
            "padding": self.padding_collate,
        }[collate]
        logger.info(f'[{"DATA".center(9)}] [Base dataset collate] {collate}')
data_on_memory 
data_on_memory()

To store data in RAM memory (faster data loading), put all loaded data ( use self.load_raw_data() ) in list self.data_on_memory. Or define self.data_on_memory as empty list to load data from filesystem everytime.

Source code in SaigeToolkit/data/dataset/base_dataset.py 
118
119
120
121
122
123
124
125
@abstract_attribute
def data_on_memory(self):
    """
    To store data in RAM memory (faster data loading),
    put all loaded data ( use self.load_raw_data() ) in list `self.data_on_memory`.
    Or define self.data_on_memory as empty list to load data from filesystem everytime.
    """
    pass
stack_data_on_memory 
stack_data_on_memory() -> List[dict]

Stack all data on RAM memory for quick-loading purpose. output would be saved on class attribute.

Returns:

  • List[dict] –

    List[dict]: Entire data list-dict.

Source code in SaigeToolkit/data/dataset/base_dataset.py 
127
128
129
130
131
132
133
134
135
136
137
138
139
140
def stack_data_on_memory(self) -> List[dict]:
    """Stack all data on RAM memory for quick-loading purpose.
    output would be saved on class attribute.

    Returns:
        List[dict]: Entire data list-dict.
    """
    data_on_memory = []
    for idx in range(len(self)):
        data_on_memory.append(self.load_raw_data(idx))

    logger.info(f'[{"DATA".center(9)}] [DATA_ON_MEMORY] activated')

    return data_on_memory
__getitem__ 
__getitem__(index: int) -> Union[dict, List[dict]]

dataset default getitem function. data is first loaded, then processed as following order: resize, augmentation, transform to torch.Tensor. data could be loaded from file_system (load_raw_data) or RAM memory (data_on_memory).

Parameters:

  • index (int) –

    data item index.

Returns:

  • Union[dict, List[dict]] –

    Union[dict, List[dict]]: single data, or cropped data list when crop_fn exists.

Source code in SaigeToolkit/data/dataset/base_dataset.py 
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
def __getitem__(self, index: int) -> Union[dict, List[dict]]:
    """dataset default __getitem__ function.
    data is first loaded, then processed as following order:
        resize, augmentation, transform to torch.Tensor.
    data could be loaded from file_system (load_raw_data) or RAM memory (data_on_memory).

    Args:
        index (int): data item index.

    Returns:
        Union[dict, List[dict]]:
            single data, or cropped data list when crop_fn exists.
    """

    # Load Images and Text or Image Labels
    if self.data_on_memory:
        data = self.data_on_memory[index]
    else:
        data = self.load_raw_data(index)

    # ROI
    if self.transform.roi_handler is not None:
        if self.has_crop_fn:
            data = self.transform.roi_handler(**data)
        else:
            data = self.transform.roi_handler.apply_crop(**data)

    assert "image" in data, "SaigeDataset currently only supports image dataset"

    if not self.has_crop_fn:
        data = self.process_data(data)

        return data

    else:
        l_data_raw = self.crop_fn(**data)

        batch = []
        for data_patch in l_data_raw:
            batch.append(self.process_data(data_patch))

        return batch
process_data 
process_data(data: dict) -> dict

data processing after raw loading.

- resize: Resize Image and Label
- btw_resize_aug: Dummy function for user customization
- augmentation: Process augmentation defined in util/augmentation
- btw_aug_transform: Dummy function for user customization
- to_tensor: Transform data to torch tensor

Parameters:

  • data (dict) –

    raw data dict

Returns:

  • dict ( dict ) –

    processed data dict

Source code in SaigeToolkit/data/dataset/base_dataset.py 
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
def process_data(self, data: dict) -> dict:
    """data processing after raw loading.

        - resize: Resize Image and Label
        - btw_resize_aug: Dummy function for user customization
        - augmentation: Process augmentation defined in util/augmentation
        - btw_aug_transform: Dummy function for user customization
        - to_tensor: Transform data to torch tensor

    Args:
        data (dict): raw data dict

    Returns:
        dict: processed data dict
    """
    if self.transform.resizer is not None:
        data = self.transform.resizer(**data)

    if self.transform.roi_handler is not None and not self.has_crop_fn:
        data = self.transform.roi_handler.apply_mask(**data)

    if self.transform.augmentation is not None:
        data = self.transform.augmentation(**data)

    data = self.to_tensor(**data)

    return data
to_tensor 
to_tensor(**data: dict) -> dict

to_tensor items in data into torch.Tensor (if convertible)

Parameters:

  • data (dict, default: {} ) –

    raw data dict

Returns:

  • dict ( dict ) –

    tensorized data dict

Source code in SaigeToolkit/data/dataset/base_dataset.py 
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
def to_tensor(self, **data: dict) -> dict:
    """to_tensor items in data into torch.Tensor (if convertible)

    Args:
        data (dict): raw data dict

    Returns:
        dict: tensorized data dict
    """

    def _to_tensor_from_numpy(image: np.ndarray):
        return torch.from_numpy(image).to(self.device).contiguous()

    def _to_tensor_from_pil_image(image: Image.Image):
        return _to_tensor_from_numpy(np.array(image))

    for k, v in data.items():
        if k == "org_image":
            continue

        if isinstance(v, Image.Image):
            data[k] = _to_tensor_from_pil_image(v)

        elif isinstance(v, np.ndarray):
            data[k] = _to_tensor_from_numpy(v)

        elif isinstance(v, int):
            data[k] = torch.tensor(v).to(self.device).contiguous()

        else:
            continue

        if k == "image":
            data[k] = data[k].permute((2, 0, 1)).div(255)
        else:
            data[k] = data[k].long()

    return data
resize_collate 
resize_collate(batch: List[dict]) -> dict

Example collate function with resizing items.

Parameters:

  • batch (List[dict]) –

    list of data dicts

Returns:

  • dict ( dict ) –

    resized-and-collated data dict

Source code in SaigeToolkit/data/dataset/base_dataset.py 
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
def resize_collate(self, batch: List[dict]) -> dict:
    """Example collate function with resizing items.

    Args:
        batch (List[dict]): list of data dicts

    Returns:
        dict: resized-and-collated data dict
    """

    img_size = (
        int(np.mean([data["image"].shape[-2] for data in batch])),
        int(np.mean([data["image"].shape[-1] for data in batch])),
    )

    # 이미 tensor 형태로 들어온 상태기 때문에 F.interpolate 함수 사용해보자
    data_out = {key: [] for key in batch[0].keys()}
    for data in batch:
        data_out["image"].append(F.interpolate(data["image"][None], (img_size[0], img_size[1])))

        for k, v in data.items():
            if k == "class":
                data_out[k].append(v.unsqueeze(0))
            elif k == "boxes":
                data_out[k].append(v)
            elif k == "mask":
                data_out[k].append(
                    F.interpolate(v[None, None].float(), (img_size[0], img_size[1]))
                    .long()
                    .squeeze(1)
                )

    for key, value in data_out.items():
        data_out[key] = torch.cat(value)

    return data_out
padding_collate 
padding_collate(batch: List[dict]) -> dict

Example collate function with padding items.

Parameters:

  • batch (List[dict]) –

    list of data dicts

Returns:

  • dict ( dict ) –

    padded-and-collated data dict

Source code in SaigeToolkit/data/dataset/base_dataset.py 
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
def padding_collate(self, batch: List[dict]) -> dict:
    """Example collate function with padding items.

    Args:
        batch (List[dict]): list of data dicts

    Returns:
        dict: padded-and-collated data dict
    """

    max_h = np.max([d["image"].shape[-2] for d in batch])
    max_w = np.max([d["image"].shape[-1] for d in batch])

    data_out = {key: [] for key in batch[0].keys()}
    for data in batch:
        data_out["image"].append(
            F.pad(
                data["image"][None],
                (0, max_w - data["image"].shape[-1], 0, max_h - data["image"].shape[-2]),
            )
        )

        for k, v in data.items():
            if k == "class":
                data_out[k].append(v.unsqueeze(0))
            elif k == "boxes":
                data_out[k].append(v)
            elif k == "mask":
                data_out[k].append(F.pad(v[None], (0, max_w - v.shape[-1], 0, max_h - v.shape[-2])))

    for key, value in data_out.items():
        data_out[key] = torch.cat(value)

    return data_out
calculate_patch_number 
calculate_patch_number(files: Optional[List[dict]] = None) -> int

When crop_fn is activated, calculate number of text patches.

Parameters:

  • files (Optional[List[dict]], default: None ) –

    list of meta data dict. if None, calculate patch_number from self.files attribute. Defaults to None.

Returns:

  • int ( int ) –

    number of patches in meta data list files

Source code in SaigeToolkit/data/dataset/base_dataset.py 
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
def calculate_patch_number(self, files: Optional[List[dict]] = None) -> int:
    """When crop_fn is activated, calculate number of text patches.

    Args:
        files (Optional[List[dict]], optional): list of meta data dict.
            if None, calculate patch_number from self.files attribute. Defaults to None.

    Returns:
        int: number of patches in meta data list `files`
    """
    if not self.has_crop_fn:
        return len(self)

    if files is None:
        files = self.files

    n_patch = 0
    for file in files:
        labels = self.load_label(file)
        if "polygons" in labels:
            n_patch += self.crop_fn.get_n_patch(labels["polygons"])
        elif "labels" in labels:
            n_patch += len(labels["labels"])
        elif "labels_count" in labels:
            n_patch += labels["labels_count"]
        else:
            n_patch += 1

    logger.info(f"total patch number in files: {n_patch}")

    return n_patch

builder

build_dataset 
build_dataset(_target_, **cfg_dataset: dict) -> SaigeDataset

build dataset from config["dataset"]

Parameters:

  • cfg_dataset (dict, default: {} ) –

    config dict for building dataset

Returns:

Source code in SaigeToolkit/data/dataset/builder.py 
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
def build_dataset(_target_, **cfg_dataset: dict) -> SaigeDataset:
    """build dataset from config["dataset"]

    Args:
        cfg_dataset (dict): config dict for building dataset

    Returns:
        SaigeDataset: dataset object
    """

    # Setup Dataset
    dataset_class = DATASET_IMPLEMENTED[_target_]
    dataset: SaigeDataset = dataset_class(**cfg_dataset)

    return dataset

platform_reader

request_vision_projects 
request_vision_projects(ip: str, port: int, project_ids: List[int]) -> List[dict]

Data platform으로부터 project 정보들을 가져옵니다.

Parameters:

  • ip (str) –

    API서버에 접근하기 위한 IP 주소입니다.

  • port (int) –

    API서버에 접근하기 위한 PORT 번호입니다.

  • project_ids (list) –

    project 정보를 가져오기 위한 project_id의 list입니다.

Returns:

  • List[dict] –

    List[dict]: project 정보를 dict형태로 저장한 list를 반환합니다. 

    [
    {
        "project_id": int,
        "class_info": List[str],
        "dataset": {
            "train_images": {
                "{image_id}": {
                    "path": str,
                    "width": int,
                    "height": int,
                    "client": str,    # Dataset metadata
                    "end_user": str,  # Dataset metadata
                    "domain": str,    # Dataset metadata
                    "class_index": int,  # CLS only have this key
                    "labels" : [         # DET, SEG only have this key
                        {
                            "class_index": int,
                            "bounding_box": List[int],   # DET only have this key
                            "contours": List[List[int]]  # SEG only have this key
                        },
                        ..., # times number of labels
                    ]
                },
                ..., # times number of images
            },
            "validation_images": {...},
            "Not split: {...}
        }
    },
    ...,    # times number of projects
]

Usage

projects = request_vision_projects("platform.saige.in", 8502, project_ids)

Source code in SaigeToolkit/data/dataset/platform_reader.py 
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
def request_vision_projects(ip: str, port: int, project_ids: List[int]) -> List[dict]:
    """Data platform으로부터 project 정보들을 가져옵니다.

    Args:
        ip (str): API서버에 접근하기 위한 IP 주소입니다.
        port (int): API서버에 접근하기 위한 PORT 번호입니다.
        project_ids (list): project 정보를 가져오기 위한 project_id의 list입니다.

    Returns:
        List[dict]: project 정보를 dict형태로 저장한 list를 반환합니다.
            ```python
            [
                {
                    "project_id": int,
                    "class_info": List[str],
                    "dataset": {
                        "train_images": {
                            "{image_id}": {
                                "path": str,
                                "width": int,
                                "height": int,
                                "client": str,    # Dataset metadata
                                "end_user": str,  # Dataset metadata
                                "domain": str,    # Dataset metadata
                                "class_index": int,  # CLS only have this key
                                "labels" : [         # DET, SEG only have this key
                                    {
                                        "class_index": int,
                                        "bounding_box": List[int],   # DET only have this key
                                        "contours": List[List[int]]  # SEG only have this key
                                    },
                                    ..., # times number of labels
                                ]
                            },
                            ..., # times number of images
                        },
                        "validation_images": {...},
                        "Not split: {...}
                    }
                },
                ...,    # times number of projects
            ]
            ```

    Usage:
        >>> projects = request_vision_projects("platform.saige.in", 8502, project_ids)
    """
    projects = list()
    for project_id in project_ids:
        response = requests.get(f"http://{ip}:{port}/get_project/{project_id}")
        if response.status_code == 200:
            exported_project = response.json()
            projects.append(exported_project)
        else:
            print(response.text)

    return projects

saige_vision_reader

load_label_det 
load_label_det(file: Dict) -> Dict

박스 별 dict를 concat 되어있는 bboxes, labels, scores로 변환합니다.

Source code in SaigeToolkit/data/dataset/saige_vision_reader.py 
32
33
34
35
36
37
38
39
40
41
42
43
def load_label_det(file: Dict) -> Dict:
    """박스 별 dict를 concat 되어있는 bboxes, labels, scores로 변환합니다."""
    bboxes = file["labels"]

    label = {
        "bboxes": [bbox["bounding_box"] for bbox in bboxes],
        "labels": [bbox["class_index"] for bbox in bboxes],
    }
    if bboxes and "score" in bboxes[0]:
        label["scores"] = [bbox["score"] for bbox in bboxes]

    return label

srproj_dataset

SrprojDataset 
SrprojDataset(code: Union[str, List[Union[str, dict]]], mode: Optional[str] = None, two_class: bool = False, split: str = 'Training', preload_data_on_memory: bool = False, srproj_params: Optional[dict] = None, **cfg_dataset: dict)

Bases: SaigeDataset

Dataset class with building data from srproj file. Basic structure of dataset class is defined in SaigeDataset. SrprojDataset class only defines data loading from srproj file.

Attributes:

  • files (List[dict]) –

    meta_data dict list from srproj file.

  • n_classes (int) –

    number of classes of data. fix to 2 if two_class mode is on.

  • classes (List[str]) –

    list of class names for each class index. meta_data: only contains information about how to load data

  • n_patch (int) –

    number of patches when crop_fn is used.

  • data_on_memory (List[dict]) –

    list of loaded data dicts if preload_data_on_memory is activated

initializing SrprojDataset

Parameters:

  • code (Union[str, List[Union[str, dict]]]) –

    dataset code(s) with srproj params. 여러 srproj를 사용하는 경우, 각 srproj_param을 dictionary 형태로 추가할 수 있습니다. 이 경우, "srproj_params" 파라미터를 각자의 srproj_param으로 업데이트 하여 사용합니다.

  • mode (Optional[str], default: None ) –

    data label type. Defaults to None.

  • two_class (bool, default: False ) –

    two_class mode selector. Defaults to False.

  • split (str, default: 'Training' ) –

    ["Training", "Validation"]. Defaults to "Training".

  • preload_data_on_memory (bool, default: False ) –

    whether pre-load all data and save on RAM memory. Defaults to False.

  • srproj_params (dict, default: None ) –

    read_srproj 함수에 전달하는 추가적인 파라미터들 입니다. 현재는 이미지 경로 핸들링을 위한 파라미터들이 있으며, 지속적으로 추가될 수 있습니다.

  • cfg_dataset (dict, default: {} ) –

    config dicts for mother class

Source code in SaigeToolkit/data/dataset/srproj_dataset.py 
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
def __init__(
    self,
    code: Union[str, List[Union[str, dict]]],
    mode: Optional[str] = None,
    two_class: bool = False,
    split: str = "Training",
    preload_data_on_memory: bool = False,
    srproj_params: Optional[dict] = None,
    **cfg_dataset: dict,
) -> None:
    """initializing SrprojDataset

    Args:
        code (Union[str, List[Union[str, dict]]]): dataset code(s) with srproj params.
                                                여러 srproj를 사용하는 경우, 각 srproj_param을 dictionary 형태로 추가할 수 있습니다.
                                                이 경우, "srproj_params" 파라미터를 각자의 srproj_param으로 업데이트 하여 사용합니다.
        mode (Optional[str], optional): data label type. Defaults to None.
        two_class (bool, optional): two_class mode selector. Defaults to False.
        split (str, optional): ["Training", "Validation"]. Defaults to "Training".
        preload_data_on_memory (bool, optional):
            whether pre-load all data and save on RAM memory. Defaults to False.
        srproj_params (dict, optional): `read_srproj` 함수에 전달하는 추가적인 파라미터들 입니다.
                                        현재는 이미지 경로 핸들링을 위한 파라미터들이 있으며, 지속적으로 추가될 수 있습니다.
        cfg_dataset (dict): config dicts for mother class
    """
    super().__init__(**cfg_dataset)

    if srproj_params is None:
        srproj_params = {}

    if isinstance(code, list):
        self.files = []
        self.classes = []
        for c in code:
            if isinstance(c, dict):
                _c = c.pop("code")
                _srproj_params = deepcopy(srproj_params)
                _srproj_params.update(c)
                files, _, classes = read_srproj(_c, split, two_class, mode, **_srproj_params)
            elif isinstance(c, str):
                files, _, classes = read_srproj(c, split, two_class, mode, **srproj_params)
            else:
                raise NotImplementedError

            self.files.extend(files)
            self.classes = merge_srproj_classes(self.classes, classes)
        self.n_classes = len(self.classes)
    else:
        self.files, self.n_classes, self.classes = read_srproj(
            code, split, two_class, mode, **srproj_params
        )

    self.data_on_memory = self.stack_data_on_memory() if preload_data_on_memory else []

    logger.info(
        f'[{"DATA".center(9)}] [count {split}] {len(self)} [number of classes] {self.n_classes}'
    )
load_label 
load_label(file: dict) -> dict

load label data from file-system

Parameters:

  • file (dict) –

    meta data dict

Returns:

  • dict ( dict ) –

    label data as dict

Source code in SaigeToolkit/data/dataset/srproj_dataset.py 
84
85
86
87
88
89
90
91
92
93
94
95
96
def load_label(self, file: dict) -> dict:
    """load label data from file-system

    Args:
        file (dict): meta data dict

    Returns:
        dict: label data as dict
    """
    if self.crop_fn is not None:
        return load_srproj_label(file, use_crop_fn=True)
    else:
        return load_srproj_label(file, use_crop_fn=False)

srproj_reader

path_interpreter 
path_interpreter(code: str) -> Tuple[str, str, str, str]

data path interpreter

Parameters:

  • code (str) –

    dataset code (rule: {domain}-{source}-{category}) (ex) sample dataset for cls: "test_directory-sample-cls")

Raises:

  • RuntimeError –

    invalid classifier keys (code) for srproj code.

  • Exception –

    wrong type of input code

Returns:

  • Tuple[str, str, str, str] –

    Tuple[str, str, str, str]: absolute path for dataset, domain, source, category

Source code in SaigeToolkit/data/dataset/srproj_reader.py 
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
def path_interpreter(code: str) -> Tuple[str, str, str, str]:
    """data path interpreter

    Args:
        code (str): dataset code (rule: {domain}-{source}-{category})
            (ex) sample dataset for cls: "test_directory-sample-cls")

    Raises:
        RuntimeError: invalid classifier keys (code) for srproj code.
        Exception: wrong type of input code

    Returns:
        Tuple[str, str, str, str]: absolute path for dataset, domain, source, category
    """

    if os.path.isdir(code) or os.path.isfile(code):
        # correct directory/file
        path = code
        keys = os.path.dirname(os.path.dirname(path)).split(os.sep)
        keys = keys + ["dummy" for _ in range(3 - len(keys))]
        domain, source, category = keys[-3:]
    else:
        # wrong directory/file/code
        raise Exception(f"Wrong Directory / File / Code: {code}")

    return path, domain, source, category
read_srproj 
read_srproj(code: str, split: str, two_class: bool = False, mode: Optional[str] = None, use_absolute_path: bool = False, srproj_image_directory: Optional[str] = None, system_image_directory: Optional[str] = None, get_class_colors: bool = False, skip_problematic_files: bool = False) -> Tuple[List[Dict], int, Union[List[str], List[Dict]]]

read srproj dataset from file-system.

Parameters:

  • code (str) –

    dataset code.

  • split (str) –

    split dataset, "Training" or "Validation".

  • two_class (bool, default: False ) –

    load data as two class mode (ex) normal <-> abnormal)

  • mode (Optional[str], default: None ) –

    select data label type.

  • use_absolute_path (bool, default: False ) –

    use the image path in srproj as is.

  • srproj_image_directory (Optional[str], default: None ) –

    srproj 파일에 작성된 이미지 경로를 현재 시스템 상의 이미지 경로로 변환하기 위해 치환해야 하는 이미지 폴더 경로. None이면 SaigeDatabase 규칙 사용.

  • system_image_directory (Optional[str], default: None ) –

    코드가 구동되는 시스템 상의 이미지 폴더 경로. None이면 SaigeDatabase 규칙 사용.

  • get_class_colors(bool) –

    srproj에 정의된 각 클래스의 color를 받을지 여부. True이면

  • skip_problematic_files(bool) –

    srproj 파일에 작성된 이미지 경로로부터 이미지들을 읽을 때, 해당 파라미터가 True 라면, 파일이 실제 존재하지 않거나, 읽다가 문제가 발생할 경우 해당 파일을 제외하고 나머지 파일을 계속 읽음. 만약, False라면 Error를 raise 함.

Returns:

  • Tuple[List[Dict], int, Union[List[str], List[Dict]]] –

    Tuple[List[Dict], int, Union[List[str], List[Dict]]]: list of meta data dicts, number of classes, and list of class names or dict

Note

.sproj 파일에는 각 이미지들의 경로와 라벨 정보 등이 포함되어 있습니다. 이때 이미지 경로는 해당 파일을 생성한 PC 기준의 절대 경로로 작성되어 있기 때문에, 현재 이 코드가 실행되는 시스템에서의 경로로 변환해 주어야 합니다. 현재 이미지 경로 변환 옵션은 3가지가 존재합니다. 1. sproj를 생성한 PC에서 코드를 실행하는 경우, use_absolute_path= True로 세팅하면 srproj에 작성된 이미지 경로를 그대로 사용합니다. 2. 이미지와 srproj가 SaigeDatabase 룰에 맞게 구성된 경우, 이미지는 images 폴더에 하위에 있고 (root/images/.../xxx.png) srproj는 images 폴더보다 한 단계 아래 경로에 있어야 합니다 (root/projects/xxx.srproj) 이때는srproj_image_directory= None,system_image_directory= None 으로 설정합니다. 3. 이미지와 srproj가 SaigeDatabase 룰을 따르지 않는 경우,srproj_image_directory와system_image_directory`를 설정하면 srproj에 작성된 이미지 경로를 다음 규칙으로 변환합니다: {srproj_image_directory}/.../xxx.png -> {system_image_directory}/.../xxx.png

Source code in SaigeToolkit/data/dataset/srproj_reader.py 
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
def read_srproj(
    code: str,
    split: str,
    two_class: bool = False,
    mode: Optional[str] = None,
    use_absolute_path: bool = False,
    srproj_image_directory: Optional[str] = None,
    system_image_directory: Optional[str] = None,
    get_class_colors: bool = False,
    skip_problematic_files: bool = False,
) -> Tuple[List[Dict], int, Union[List[str], List[Dict]]]:
    """read srproj dataset from file-system.

    Args:
        code (str): dataset code.
        split (str): split dataset, "Training" or "Validation".
        two_class (bool): load data as two class mode (ex) normal <-> abnormal)
        mode (Optional[str]): select data label type.
        use_absolute_path (bool): use the image path in srproj as is.
        srproj_image_directory (Optional[str]): srproj 파일에 작성된 이미지 경로를 현재 시스템 상의 이미지 경로로 변환하기 위해
                                                치환해야 하는 이미지 폴더 경로. None이면 SaigeDatabase 규칙 사용.
        system_image_directory (Optional[str]): 코드가 구동되는 시스템 상의 이미지 폴더 경로. None이면 SaigeDatabase 규칙 사용.
        get_class_colors(bool): srproj에 정의된 각 클래스의 color를 받을지 여부. True이면
        skip_problematic_files(bool): srproj 파일에 작성된 이미지 경로로부터 이미지들을 읽을 때, 해당 파라미터가 True 라면,
                                      파일이 실제 존재하지 않거나, 읽다가 문제가 발생할 경우 해당 파일을 제외하고 나머지 파일을 계속 읽음.
                                      만약, False라면 Error를 raise 함.

    Returns:
        Tuple[List[Dict], int, Union[List[str], List[Dict]]]:
            list of meta data dicts, number of classes, and list of class names or dict

    Note:
        .sproj 파일에는 각 이미지들의 경로와 라벨 정보 등이 포함되어 있습니다.
        이때 이미지 경로는 해당 파일을 생성한 PC 기준의 절대 경로로 작성되어 있기 때문에, 현재 이 코드가 실행되는 시스템에서의 경로로 변환해 주어야 합니다.
        현재 이미지 경로 변환 옵션은 3가지가 존재합니다.
        1. sproj를 생성한 PC에서 코드를 실행하는 경우,
            use_absolute_path` = True로 세팅하면 srproj에 작성된 이미지 경로를 그대로 사용합니다.
        2. 이미지와 srproj가 SaigeDatabase 룰에 맞게 구성된 경우,
            이미지는 images 폴더에 하위에 있고 (root/images/.../xxx.png)
            srproj는 images 폴더보다 한 단계 아래 경로에 있어야 합니다 (root/projects/xxx.srproj)
            이때는 `srproj_image_directory` = None, `system_image_directory` = None 으로 설정합니다.
        3. 이미지와 srproj가 SaigeDatabase 룰을 따르지 않는 경우,
            `srproj_image_directory`와 `system_image_directory`를 설정하면
            srproj에 작성된 이미지 경로를 다음 규칙으로 변환합니다:
            {srproj_image_directory}/.../xxx.png -> {system_image_directory}/.../xxx.png

    """

    path, domain, source, category = path_interpreter(code)

    # Check if "path" is directory
    if os.path.isdir(path):
        srproj_list = [os.path.join(path, f) for f in os.listdir(path) if f.endswith(".srproj")]
        path = np.random.choice(srproj_list)

    # Read srproj
    with open(path, encoding="utf-8") as file:
        txt = file.read()
    srproj = xmltodict.parse(txt, dict_constructor=dict)["Project"]

    # Config mode
    if mode is None:
        mode = srproj["Type"]

    # Return empty list when dummy.srproj
    if srproj["ImageGroup"]["Image"] is None:
        return [], 0, []

    # Config files
    image_list = (
        srproj["ImageGroup"]["Image"]
        if isinstance(srproj["ImageGroup"]["Image"], list)
        else [srproj["ImageGroup"]["Image"]]
    )
    assert split in [
        "Training",
        "Validation",
        "All",
    ], "Split for *.srproj Should be 'Training' or 'Validation'"
    if split == "All":
        files = [l for l in image_list]
    else:
        files = [l for l in image_list if l["SplitState"] == split]
    files = check_label(files, mode)

    n_classes = 0

    image_path_converter = lambda srproj_image_path: srproj_image_path
    if not use_absolute_path:
        if srproj_image_directory is None:
            srproj_image_path = (
                files[0]["Path"] if "Path" in files[0] else files[0]["PathGroup"]["Path"][0]
            )
            srproj_image_path = srproj_image_path.replace("\\", os.sep)
            srproj_image_directory = srproj_image_path.split(os.sep + "images" + os.sep)[0]
        srproj_image_directory = srproj_image_directory.replace("\\", os.sep)

        if system_image_directory is None:
            system_image_directory = os.path.dirname(os.path.dirname(path))

        assert os.path.isdir(
            system_image_directory
        ), f"Image data should be in following directory: {system_image_directory}"

        image_path_converter = lambda srproj_image_path: os.path.join(
            system_image_directory,
            os.path.relpath(srproj_image_path.replace("\\", os.sep), srproj_image_directory),
        )

    def _update_file_information(file: Dict) -> bool:
        image_path = file["image_path"]
        if isinstance(image_path, List):
            image_path = image_path[0]

        if os.path.isfile(image_path):
            try:
                with Image.open(image_path) as image_file:
                    # NOTE: https://github.com/python-pillow/Pillow/blob/main/src/PIL/ImageOps.py#LL579C2-L579C2
                    orientation = image_file.getexif().get(0x0112)
                    if orientation in range(2, 9):
                        img_size = list(ImageOps.exif_transpose(image_file).size)
                    else:
                        img_size = list(image_file.size)
            except (IOError, OSError):
                if skip_problematic_files:
                    return False
                raise error.InvalidImageFileError
        else:
            if skip_problematic_files:
                print(f"image not found: {file['image_path']}")
                return False
            raise error.FileNotFoundError

        file.update(
            {
                "domain": domain,
                "source": source,
                "category": category,
                "image_size": img_size,
                "label_extension": "srproj",
                "mode": mode,
            }
        )

        if mode == "Classification":
            lbl = int(file["ClassIndexOfLabel"])
            if two_class:
                lbl = 1 if (lbl > 0) else lbl

            file["ClassIndexOfLabel"] = lbl + n_classes

        elif mode == "MultiLabelClassification":
            lbl = list(file["ClassIndexOfLabel"])
            file["ClassIndexOfLabel"] = [int(l) + n_classes for l in lbl]

        elif mode == "Segmentation":
            if str(file["LabelGroup"]["IsNormal"]) in ["False", "false", "FALSE"]:
                lbl_list = (
                    file["LabelGroup"]["Label"]
                    if isinstance(file["LabelGroup"]["Label"], list)
                    else [file["LabelGroup"]["Label"]]
                )
                for l in lbl_list:
                    lbl = 1 if two_class else int(l["ClassIndex"]) + 1
                    l["ClassIndex"] = lbl + n_classes

        return True

    cleaned_files = []
    for f in files:
        if "Path" in f:
            f["image_path"] = image_path_converter(f["Path"])
        else:
            f["image_path"] = [image_path_converter(_path) for _path in f["PathGroup"]["Path"]]

        is_updated = _update_file_information(f)
        if is_updated:
            cleaned_files.append(f)

    # Config n_classes
    classes = []
    if "ClassGroup" in srproj:
        n_classes = int(srproj["ClassGroup"]["NumberOfClasses"])
        class_list = (
            srproj["ClassGroup"]["Class"]
            if isinstance(srproj["ClassGroup"]["Class"], list)
            else [srproj["ClassGroup"]["Class"]]
        )
        if get_class_colors:
            classes = [
                {
                    "name": class_["Name"],
                    "color": convert_srproj_class_color_to_rgb(int(class_["Color"])),
                }
                for class_ in class_list
            ]
        else:
            classes = [class_["Name"] for class_ in class_list]
    elif "CharacterGroup" in srproj:
        classes = srproj["CharacterGroup"]["Included"]["@xmlns"]
        n_classes = len(classes) + 1

    if mode in ["Segmentation"]:
        if isinstance(classes[0], dict):
            classes.insert(0, {"name": "background", "color": (0, 0, 0)})
        else:
            classes.insert(0, "background")
        n_classes = len(classes)

    if two_class:
        classes = ["Normal", "Defect"]
        n_classes = len(classes)

    return cleaned_files, n_classes, classes
check_label 
check_label(files: List[Dict], mode: str) -> List[Dict]

check all data label is valid. If any data is corrupted, pop out from meta data list.

Parameters:

  • files (List[Dict]) –

    meta data list to be checked

  • mode (str) –

    label data type

Returns:

  • List[Dict] –

    List[Dict]: cleaned meta data list

Source code in SaigeToolkit/data/dataset/srproj_reader.py 
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
def check_label(files: List[Dict], mode: str) -> List[Dict]:
    """check all data label is valid.
    If any data is corrupted, pop out from meta data list.

    Args:
        files (List[Dict]): meta data list to be checked
        mode (str): label data type

    Returns:
        List[Dict]: cleaned meta data list
    """
    for i in range(len(files) - 1, -1, -1):
        if mode == "Classification":
            if int(files[i]["ClassIndexOfLabel"]) == -1:
                files.pop(i)
        elif mode == "MultiLabelClassification":
            if not isinstance(files[i]["ClassIndexOfLabel"], list):
                files[i]["ClassIndexOfLabel"] = [files[i]["ClassIndexOfLabel"]]

        elif mode == "OpticalCharacterRecognition":
            if int(files[i]["LabelGroup"].get("NumberOfLabels", 0)) == 0:
                continue
            labels = (
                files[i]["LabelGroup"]["Label"]
                if isinstance(files[i]["LabelGroup"]["Label"], list)
                else [files[i]["LabelGroup"]["Label"]]
            )
            for idx in range(len(labels) - 1, -1, -1):
                # XXX: Temporary implementation for memory issue
                for key, value in labels[idx]["Coordinate"].items():
                    labels[idx]["Coordinate"][key] = float(value)

                if labels[idx]["Characters"] == '"value4"':
                    labels.pop(idx)
                elif not labels[idx]["Characters"]:
                    labels.pop(idx)
                else:
                    continue

                files[i]["LabelGroup"]["Label"] = labels
                files[i]["LabelGroup"]["NumberOfLabels"] = len(labels)

        else:
            if str(files[i]["LabelGroup"]["IsNormal"]) in [
                "False",
                "false",
                "FALSE",
            ]:
                if int(files[i]["LabelGroup"].get("NumberOfLabels", 0)) == 0:
                    files.pop(i)
    return files
load_srproj_label 
load_srproj_label(file: Dict, use_crop_fn: Optional[bool] = False, **kwargs) -> Dict

load single data dict from srproj meta data.

Parameters:

  • file (Dict) –

    single meta data dict

  • use_crop_fn (bool, default: False ) –

    Whether to use the crop function. Defaults to False.

Raises:

  • NotImplementedError –

    only four modes are available. ["Classification", "Detection", "Segmentation", "OpticalCharacterRecognition"]

Returns:

  • Dict ( Dict ) –

    label data as dict

Source code in SaigeToolkit/data/dataset/srproj_reader.py 
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
def load_srproj_label(file: Dict, use_crop_fn: Optional[bool] = False, **kwargs) -> Dict:
    """load single data dict from srproj meta data.

    Args:
        file (Dict): single meta data dict
        use_crop_fn (bool, optional): Whether to use the crop function.
            Defaults to False.

    Raises:
        NotImplementedError: only four modes are available.
            ["Classification", "Detection", "Segmentation", "OpticalCharacterRecognition"]

    Returns:
        Dict: label data as dict
    """
    mode = file["mode"]

    # make label for the right mode
    if mode == "Classification":
        label = load_label_cls(file, **kwargs)

    elif mode == "Detection":
        label = load_label_det(file, **kwargs)

    elif mode == "Segmentation":
        # In Segmentation task, need polygon data to use the crop function.
        label = load_label_seg(file, return_polygon=use_crop_fn, **kwargs)

    elif mode == "OpticalCharacterRecognition":
        label = load_label_ocr(file, **kwargs)

    else:
        raise NotImplementedError(f"Mode {mode} not implemented")

    return label
load_label_det 
load_label_det(file: Dict) -> Dict

load detection label

Parameters:

  • file (Dict) –

    meta data for detection label

Returns:

  • Dict ( Dict ) –

    detection label data dict 

    {
    "bboxes": [
        [x, y, w, h],
        ...,
    ],
    "labels": List[int],
}

Source code in SaigeToolkit/data/dataset/srproj_reader.py 
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
def load_label_det(file: Dict) -> Dict:
    """load detection label

    Args:
        file (Dict): meta data for detection label

    Returns:
        Dict: detection label data dict
            ```python
            {
                "bboxes": [
                    [x, y, w, h],
                    ...,
                ],
                "labels": List[int],
            }
            ```
    """
    srproj_label_group = file["LabelGroup"]
    label = {"bboxes": [], "labels": []}

    if str(srproj_label_group["IsNormal"]).lower() == "false":
        srproj_labels = srproj_label_group["Label"]
        if not isinstance(srproj_labels, list):
            srproj_labels = [srproj_labels]
        for srproj_label in srproj_labels:
            x = int(srproj_label["Coordinate"]["@X"])
            y = int(srproj_label["Coordinate"]["@Y"])
            w = int(srproj_label["Coordinate"]["@Width"])
            h = int(srproj_label["Coordinate"]["@Height"])
            label["bboxes"].append([x, y, w, h])
            label["labels"].append(int(srproj_label["ClassIndex"]))

    return label
load_label_seg 
load_label_seg(file: Dict, return_polygon: bool = True, mask_to_pil: bool = True) -> Dict

load segmentation label

Parameters:

  • file (Dict) –

    meta data for segmentation label

  • return_polygon (bool, default: True ) –

    whether return polygon data. Defaults to True.

Returns:

  • Dict ( Dict ) –

    segmentation label data dict

Source code in SaigeToolkit/data/dataset/srproj_reader.py 
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
def load_label_seg(file: Dict, return_polygon: bool = True, mask_to_pil: bool = True) -> Dict:
    """load segmentation label

    Args:
        file (Dict): meta data for segmentation label
        return_polygon (bool, optional): whether return polygon data. Defaults to True.

    Returns:
        Dict: segmentation label data dict
    """
    w, h = file["image_size"]
    lbl = np.zeros((h, w), dtype=np.uint8)

    lbl_list = file["LabelGroup"]

    if str(lbl_list["IsNormal"]) in ["False", "false", "FALSE"]:
        lbl_list = lbl_list["Label"] if isinstance(lbl_list["Label"], list) else [lbl_list["Label"]]
        contour = []
        for l in lbl_list:
            canvas = np.zeros((h, w), dtype=np.uint8)
            color = l["ClassIndex"]

            contour_list = (
                l["ContourGroup"]["Contour"]
                if isinstance(l["ContourGroup"]["Contour"], list)
                else [l["ContourGroup"]["Contour"]]
            )

            # Contour functions in cv2 gets list type contours
            contours = {"Outer": [], "Inner": []}
            for c_l in contour_list:
                contours[c_l["@Type"]].append(
                    np.array([[int(p["@X"]), int(p["@Y"])] for p in c_l["Point"]])
                )

            # One Outer per One Label
            cv2.fillPoly(canvas, pts=contours["Outer"], color=color)

            # Many Inner could exist in One Label
            if contours["Inner"]:
                cv2.fillPoly(canvas, pts=contours["Inner"], color=0)

            nonzeros = canvas > 0
            lbl[nonzeros] = canvas[nonzeros]
            contour.append(contours["Outer"][0])
    else:
        contour = []

    if mask_to_pil:
        lbl = Image.fromarray(lbl)

    lbl = {"mask": lbl}

    if return_polygon:
        lbl.update({"polygons": contour})

    return lbl
load_label_ocr 
load_label_ocr(file: Dict) -> Dict

load ocr label

Parameters:

  • file (Dict) –

    meta data for ocr label

Returns:

  • Dict ( Dict ) –

    ocr label data dict

Source code in SaigeToolkit/data/dataset/srproj_reader.py 
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
def load_label_ocr(file: Dict) -> Dict:
    """load ocr label

    Args:
        file (Dict): meta data for ocr label

    Returns:
        Dict: ocr label data dict
    """
    lbl_list = file["LabelGroup"]

    if int(lbl_list.get("NumberOfLabels", 0)) == 0:
        return {"polygons": [], "strings": [], "ignore": []}

    pols, strs = [], []

    lbl_list = lbl_list["Label"] if isinstance(lbl_list["Label"], list) else [lbl_list["Label"]]
    for l in lbl_list:
        if not l["Characters"]:
            continue
        # if self.ignore_whitespace:
        #     l["Characters"] = l["Characters"].replace(" ", "")
        strs.append(l["Characters"])
        n_point = max(
            [
                int(key.replace("@Vertex", "").replace("X", "").replace("Y", ""))
                for key in l["Coordinate"].keys()
                if "@Vertex" in key
            ]
        )
        pols.append(
            np.array(
                [
                    [
                        float(l["Coordinate"][f"@Vertex{idx + 1}X"]),
                        float(l["Coordinate"][f"@Vertex{idx + 1}Y"]),
                    ]
                    for idx in range(n_point)
                ]
            )
        )

    strings, kie_labels = [], []
    for string in strs:
        string, kie_label = _refine_kie_labels(string)
        strings.append(string)
        kie_labels.append(kie_label)

    ignore = [s is None for s in strs]

    lbl = {"polygons": pols, "strings": strings, "kie_labels": kie_labels, "ignore": ignore}

    return lbl
convert_srproj_class_color_to_rgb 
convert_srproj_class_color_to_rgb(argb: int) -> Tuple[int, int, int]

srproj의 각 클래스 색 코드를 rgb 값으로 변환합니다.

Source code in SaigeToolkit/data/dataset/srproj_reader.py 
536
537
538
539
540
541
542
def convert_srproj_class_color_to_rgb(argb: int) -> Tuple[int, int, int]:
    """srproj의 각 클래스 색 코드를 rgb 값으로 변환합니다."""
    a = (argb >> 24) & 255
    r = (argb >> 16) & 255
    g = (argb >> 8) & 255
    b = argb & 255
    return (r, g, b)

process_config

process_data_config 

process_data_config(cfg_data: dict) -> Tuple[dict, dict]

split training/validation configs from entier data config dict. you can edit cfg by split to override some configs in cfg_data entire config example) data: code: A.srproj validation: <- code: B.srproj <- this will override base config {"code": "A.srproj"}

Parameters:

  • cfg_data (dict) –

    entire config

Returns:

  • Tuple[dict, dict] –

    Tuple[dict, dict]: training and validation configs

Source code in SaigeToolkit/data/process_config.py 
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
def process_data_config(cfg_data: dict) -> Tuple[dict, dict]:
    """split training/validation configs from entier data config dict.
    you can edit cfg by split to override some configs in cfg_data
    entire config example)
        data:
            code: A.srproj
            validation: <-
                code: B.srproj <- this will override base config {"code": "A.srproj"}

    Args:
        cfg_data (dict): entire config

    Returns:
        Tuple[dict, dict]: training and validation configs
    """
    cfg_data_copy = cfg_data.copy()
    cfg_training = cfg_data_copy.pop("training", {})
    cfg_validation = cfg_data_copy.pop("validation", {})

    # update config for training
    if cfg_training != "NO_TRAINING":
        assert isinstance(
            cfg_training, dict
        ), f"config for training data should be Dict, not {type(cfg_training)}"

        cfg_training = override_dict(deepcopy(cfg_data_copy), cfg_training)  # TODO:
        if "split" not in cfg_training["dataset"]:
            cfg_training["dataset"].update({"split": srproj_split["training"]})

    # update config for validation
    if cfg_validation != "NO_VALIDATION":
        assert isinstance(
            cfg_validation, dict
        ), f"config for validation data should be Dict, not {type(cfg_validation)}"

        cfg_validation = override_dict(deepcopy(cfg_data_copy), cfg_validation)  # TODO:
        if "split" not in cfg_validation["dataset"]:
            cfg_validation["dataset"].update({"split": srproj_split["validation"]})

    return cfg_training, cfg_validation

transform

augmentation

api

Image Augmentation Preview를 위한 API를 제공합니다.

ImageProcessor 클래스를 통해 각 augmentation들이 특정 파라미터 값에 대해 이미지를 어떻게 변형 시키는지 확인할 수 있습니다.

_APIDecorator

ImageProcessor에서 정의된 함수들을 decorate 해주는 헬퍼입니다.

staticmethod 와 decorator를 함께 사용할 경우 Cythonize시 제대로 동작하지 않는 이슈를 해결하기 위한 패치입니다. 참고: https://github.com/cython/cython/issues/1434

ImageProcessor의 각 함수에 다음과 같은 decorator를 씌우는 것과 동일한 역할을 합니다. 

@staticmethod
@error_handler
@support_multi_image
@support_3dim_gray_image
def image_processor_function(image: np.ndarray, ...):
    ...

ImageProcessor

Bases: _APIDecorator

Image Augmentation Preview를 위한 API 입니다. 각 augmentation들이 특정 파라미터 값에 대해 이미지를 어떻게 변형 시키는지 확인할 수 있습니다.

Note1

일반적으로 학습 시에는 파라미터를 특정 값이 아닌 범위로 설정하여 해당 범위에서 매번 랜덤한 값을 선택해 이미지에 적용합니다. 따라서 preview API의 입력 파라미터와 학습 시 넘겨주는 파라미터는 대부분 값 vs 범위의 차이를 가지게 됩니다. 예를 들어 preview API에서 rotate의 경우 angle (float) 값을 받지만, 학습 config에서는 angle_limit (List[float]) 범위를 받게됩니다. 각 augmentation을 학습에 사용시 필요한 config는 각 함수 설명의 Trainer Config 섹션을 참고하세요.

Note2

API 기획상, augmentation의 실제 자유도보다, 유저가 설정할 수 있는 파라미터가 적은 경우가 있습니다. (각 변 혹은 꼭짓점 마다 독립적으로 적용되는 ratio_jitter나 perspective_transform의 경우) 이러한 augmentation들은 api 호출 시, '값'을 입력 받아서, 이 '값'으로 부터 정의된 '범위'에서 필요한 값들을 랜덤하게 샘플링하게 됩니다. 이러한 augmentation들의 preview 함수를 정의할 때는, 인풋에 fixed_aug_params를 받을 수 있도록 해주어야합니다. (ImageProcessor.ratio_jitter 참고)

Usage

rotate augmentation preview 예제입니다. 상세 설명은 각 함수 설명 참고. 

image = np.zeros((100, 100, 3), dtype=np.unit8)
error, augmented_result = ImageProcessor.rotate(image=image, angle=15)
augmented_image, augmentation_parameters = augmented_result

# In the case of 'rotate' augmentation, `augmentation_parameters` is None

vertical_flip 
vertical_flip(image: ndarray) -> Tuple[ndarray, None]

image를 상하로 뒤집습니다.

Parameters:

  • image (ndarray) –

    augmentation을 적용할 image 입니다. 

    data type: uint8
channel: H x W / H x W x 1  - Gray
         H x W x 3 - RGB
         H x W x 4 - RGBA

Returns:

  • ndarray –

    np.ndarray: augmentation이 적용된 image 입니다.

  • NoneType ( None ) –

    None

Example 
error, augmented_image = ImageProcessor.vertical_flip(image=image)
Trainer Config 
{
    "_target_": "vertical_flip",
}
Source code in SaigeToolkit/data/transform/augmentation/api.py 
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
def vertical_flip(image: np.ndarray) -> Tuple[np.ndarray, None]:
    """image를 상하로 뒤집습니다.

    Args:
        image (np.ndarray): augmentation을 적용할 image 입니다.
            ```
            data type: uint8
            channel: H x W / H x W x 1  - Gray
                     H x W x 3 - RGB
                     H x W x 4 - RGBA
            ```

    Returns:
        np.ndarray: augmentation이 적용된 image 입니다.
        NoneType: None

    Example:
        ```python
        error, augmented_image = ImageProcessor.vertical_flip(image=image)
        ```

    Trainer Config:
        ```python
        {
            "_target_": "vertical_flip",
        }
        ```
    """
    return vertical_flip(image=image), None
horizontal_flip 
horizontal_flip(image: ndarray) -> Tuple[ndarray, None]

image를 좌우로 뒤집습니다.

Parameters:

  • image (ndarray) –

    augmentation을 적용할 image 입니다. 

    data type: uint8
channel: H x W / H x W x 1  - Gray
         H x W x 3 - RGB
         H x W x 4 - RGBA

Returns:

  • ndarray –

    np.ndarray: augmentation이 적용된 image 입니다.

  • NoneType ( None ) –

    None

Example 
error, augmented_image = ImageProcessor.horizontal_flip(image=image)
Trainer Config 
{
    "_target_": "horizontal_flip",
}
Source code in SaigeToolkit/data/transform/augmentation/api.py 
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
def horizontal_flip(image: np.ndarray) -> Tuple[np.ndarray, None]:
    """image를 좌우로 뒤집습니다.

    Args:
        image (np.ndarray): augmentation을 적용할 image 입니다.
            ```
            data type: uint8
            channel: H x W / H x W x 1  - Gray
                     H x W x 3 - RGB
                     H x W x 4 - RGBA
            ```

    Returns:
        np.ndarray: augmentation이 적용된 image 입니다.
        NoneType: None

    Example:
        ```python
        error, augmented_image = ImageProcessor.horizontal_flip(image=image)
        ```

    Trainer Config:
        ```python
        {
            "_target_": "horizontal_flip",
        }
        ```
    """
    return horizontal_flip(image=image), None
rotate 
rotate(image: ndarray, angle: float = 0.0) -> Tuple[ndarray, None]

image를 angle만큼 회전시킵니다.

Parameters:

  • image (ndarray) –

    augmentation을 적용할 image 입니다. 

    data type: uint8
channel: H x W / H x W x 1  - Gray
         H x W x 3 - RGB
         H x W x 4 - RGBA

  • angle (float, default: 0.0 ) –

    회전하는 각도 입니다. 유효 범위는 다음과 같습니다. [-360.0, 360.0]. Defaults to 0.0.

Returns:

  • ndarray –

    np.ndarray: augmentation이 적용된 image 입니다.

  • NoneType ( None ) –

    None

Example 
error, augmented_image = ImageProcessor.rotate(image=image, angle=60.0)
Trainer Config 
{
    "_target_": "rotate",
    "angle_limit": List[float],  # angle 최소 최대 범위
}
Source code in SaigeToolkit/data/transform/augmentation/api.py 
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
def rotate(image: np.ndarray, angle: float = 0.0) -> Tuple[np.ndarray, None]:
    """image를 angle만큼 회전시킵니다.

    Args:
        image (np.ndarray): augmentation을 적용할 image 입니다.
            ```
            data type: uint8
            channel: H x W / H x W x 1  - Gray
                     H x W x 3 - RGB
                     H x W x 4 - RGBA
            ```
        angle (float, optional): 회전하는 각도 입니다. 유효 범위는 다음과 같습니다. [-360.0, 360.0]. Defaults to 0.0.

    Returns:
        np.ndarray: augmentation이 적용된 image 입니다.
        NoneType: None

    Example:
        ```python
        error, augmented_image = ImageProcessor.rotate(image=image, angle=60.0)
        ```

    Trainer Config:
        ```python
        {
            "_target_": "rotate",
            "angle_limit": List[float],  # angle 최소 최대 범위
        }
        ```
    """
    return (
        rotate(
            image=image,
            angle=angle,
            interpolation=cv2.INTER_LINEAR,
            border_mode=cv2.BORDER_REFLECT_101,
            value=0,
            crop_border=False,
        ),
        None,
    )
random_rotate90 
random_rotate90(image: ndarray, factor: int = 0) -> Tuple[ndarray, None]

image를 [0, 90, 180, 270] 중 랜덤한 각도만큼 회전시킵니다.

NOTE
  • 모든 이미지 픽셀은 유지되며, 회전 후 이미지 크기가 변경될 수 있습니다.
  • 예시: factor가 1일 경우 90도 회전하며, 이미지 사이즈는 (W, H) -> (H, W)로 변경됩니다.

Parameters:

  • image (ndarray) –

    augmentation을 적용할 image 입니다. 

    data type: uint8
channel: H x W / H x W x 1  - Gray
         H x W x 3 - RGB
         H x W x 4 - RGBA

  • factor (int, default: 0 ) –

    회전하는 각도 입니다. factor에 90을 곱한 값만큼 회전합니다. (ex. factor가 1일 경우 90도) 유효범위는 다음과 같습니다 [0, 3]. Defaults to 0.

Returns:

  • ndarray –

    np.ndarray: augmentation이 적용된 image 입니다.

  • NoneType ( None ) –

    None

Example 
error, augmented_image = ImageProcessor.random_rotate90(image=image, factor=1)
Trainer Config 
{
    "_target_": "random_rotate90",
}
Source code in SaigeToolkit/data/transform/augmentation/api.py 
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
def random_rotate90(image: np.ndarray, factor: int = 0) -> Tuple[np.ndarray, None]:
    """image를 [0, 90, 180, 270] 중 랜덤한 각도만큼 회전시킵니다.

    NOTE:
        - 모든 이미지 픽셀은 유지되며, 회전 후 이미지 크기가 변경될 수 있습니다.
        - 예시: factor가 1일 경우 90도 회전하며, 이미지 사이즈는 (W, H) -> (H, W)로 변경됩니다.

    Args:
        image (np.ndarray): augmentation을 적용할 image 입니다.
            ```
            data type: uint8
            channel: H x W / H x W x 1  - Gray
                     H x W x 3 - RGB
                     H x W x 4 - RGBA
            ```
        factor (int, optional): 회전하는 각도 입니다. factor에 90을 곱한 값만큼 회전합니다. (ex. factor가 1일 경우 90도)
                                유효범위는 다음과 같습니다 [0, 3]. Defaults to 0.

    Returns:
        np.ndarray: augmentation이 적용된 image 입니다.
        NoneType: None

    Example:
        ```python
        error, augmented_image = ImageProcessor.random_rotate90(image=image, factor=1)
        ```

    Trainer Config:
        ```python
        {
            "_target_": "random_rotate90",
        }
        ```
    """
    return random_rotate90(image=image, factor=factor), None
color_jitter 
color_jitter(image: ndarray, brightness: float = 1.0, contrast: float = 1.0, saturation: float = 1.0, hue: float = 0.0) -> Tuple[ndarray, None]

image의 밝기 (brightness), 대비 (contrast), 채도 (saturation), 색상 (hue)을 변경합니다.

Parameters:

  • image (ndarray) –

    augmentation을 적용할 image 입니다. 

    data type: uint8
channel: H x W / H x W x 1  - Gray
         H x W x 3 - RGB
         H x W x 4 - RGBA

  • brightness (float, default: 1.0 ) –

    밝기를 담당하는 요소입니다. 유효 범위는 다음과 같습니다. [0.01, 10.00]. Defaults to 1.0.

  • contrast (float, default: 1.0 ) –

    대비를 담당하는 요소입니다. 유효 범위는 다음과 같습니다. [0.01, 10.00]. Defaults to 1.0.

  • saturation (float, default: 1.0 ) –

    채도를 담당하는 요소입니다. 유효 범위는 다음과 같습니다. [0.01, 10.00]. Defaults to 1.0.

  • hue (float, default: 0.0 ) –

    색상을 담당하는 요소입니다. 유효 범위는 다음과 같습니다. [-0.50, 0.50]. Defaults to 0.0.

Returns:

  • ndarray –

    np.ndarray: augmentation이 적용된 image 입니다.

  • NoneType ( None ) –

    None

Example 
error, augmented_image = ImageProcessor.color_jitter(image=image,
                                                     brightness=0.5,
                                                     contrast=0.5,
                                                     saturation=0.5,
                                                     hue=0.1)  #  augmentation 적용
Trainer Config 
{
    "_target_": "color_jitter",
    "brightness_limit": List[float],  # brightness 최소 최대 범위
    "contrast_limit": List[float],  # contrast 최소 최대 범위
    "saturation_limit": List[float],  # saturation 최소 최대 범위
    "hue_limit": List[float],  # hue 최소 최대 범위
}
Source code in SaigeToolkit/data/transform/augmentation/api.py 
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
def color_jitter(
    image: np.ndarray,
    brightness: float = 1.0,
    contrast: float = 1.0,
    saturation: float = 1.0,
    hue: float = 0.0,
) -> Tuple[np.ndarray, None]:
    """image의 밝기 (brightness), 대비 (contrast), 채도 (saturation), 색상 (hue)을 변경합니다.

    Args:
        image (np.ndarray): augmentation을 적용할 image 입니다.
            ```
            data type: uint8
            channel: H x W / H x W x 1  - Gray
                     H x W x 3 - RGB
                     H x W x 4 - RGBA
            ```
        brightness (float, optional): 밝기를 담당하는 요소입니다. 유효 범위는 다음과 같습니다. [0.01, 10.00]. Defaults to 1.0.
        contrast (float, optional): 대비를 담당하는 요소입니다. 유효 범위는 다음과 같습니다. [0.01, 10.00]. Defaults to 1.0.
        saturation (float, optional): 채도를 담당하는 요소입니다. 유효 범위는 다음과 같습니다. [0.01, 10.00]. Defaults to 1.0.
        hue (float, optional): 색상을 담당하는 요소입니다. 유효 범위는 다음과 같습니다. [-0.50, 0.50]. Defaults to 0.0.

    Returns:
        np.ndarray: augmentation이 적용된 image 입니다.
        NoneType: None

    Example:
        ```python
        error, augmented_image = ImageProcessor.color_jitter(image=image,
                                                             brightness=0.5,
                                                             contrast=0.5,
                                                             saturation=0.5,
                                                             hue=0.1)  #  augmentation 적용
        ```

    Trainer Config:
        ```python
        {
            "_target_": "color_jitter",
            "brightness_limit": List[float],  # brightness 최소 최대 범위
            "contrast_limit": List[float],  # contrast 최소 최대 범위
            "saturation_limit": List[float],  # saturation 최소 최대 범위
            "hue_limit": List[float],  # hue 최소 최대 범위
        }
        ```
    """
    return (
        color_jitter(
            image=image,
            brightness=brightness,
            contrast=contrast,
            saturation=saturation,
            hue=hue,
            order=[0, 1, 2, 3],
        ),
        None,
    )
blur 
blur(image: ndarray, ksize: int = 1) -> Tuple[ndarray, None]

image에 averaging blur를 적용합니다.

Parameters:

  • image (ndarray) –

    augmentation을 적용할 image 입니다. 

    data type: uint8
channel: H x W / H x W x 1  - Gray
         H x W x 3 - RGB
         H x W x 4 - RGBA

  • ksize (int, default: 1 ) –

    blur kernel의 크기 입니다. 유효 범위는 다음과 같습니다. [1, 100]. Defaults to 1.

Returns:

  • ndarray –

    np.ndarray: augmentation이 적용된 image 입니다.

  • NoneType ( None ) –

    None

Example 
error, augmented_image = ImageProcessor.blur(image=image, ksize=3)
Trainer Config 
{
    "_target_": "blur",
    "ksize_limit": List[int],  # ksize 최소 최대 범위
}
Source code in SaigeToolkit/data/transform/augmentation/api.py 
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
def blur(image: np.ndarray, ksize: int = 1) -> Tuple[np.ndarray, None]:
    """image에 averaging blur를 적용합니다.

    Args:
        image (np.ndarray): augmentation을 적용할 image 입니다.
            ```
            data type: uint8
            channel: H x W / H x W x 1  - Gray
                     H x W x 3 - RGB
                     H x W x 4 - RGBA
            ```
        ksize (int, optional): blur kernel의 크기 입니다. 유효 범위는 다음과 같습니다. [1, 100]. Defaults to 1.

    Returns:
        np.ndarray: augmentation이 적용된 image 입니다.
        NoneType: None

    Example:
        ```python
        error, augmented_image = ImageProcessor.blur(image=image, ksize=3)
        ```

    Trainer Config:
        ```python
        {
            "_target_": "blur",
            "ksize_limit": List[int],  # ksize 최소 최대 범위
        }
        ```
    """
    return blur(image=image, ksize=ksize), None
gaussian_blur 
gaussian_blur(image: ndarray, ksize: int = 1) -> Tuple[ndarray, None]

image에 gaussian blur를 적용합니다.

Parameters:

  • image (ndarray) –

    augmentation을 적용할 image 입니다. 

    data type: uint8
channel: H x W / H x W x 1  - Gray
         H x W x 3 - RGB
         H x W x 4 - RGBA

  • ksize (int, default: 1 ) –

    blur kernel의 크기 입니다. 유효 범위는 다음과 같습니다. [1, 100]. Defaults to 1.

Returns:

  • ndarray –

    np.ndarray: augmentation이 적용된 image 입니다.

  • NoneType ( None ) –

    None

Example 
error, augmented_image = ImageProcessor.gaussian_blur(image=image, ksize=3)
Trainer Config 
{
    "_target_": "gaussian_blur",
    "ksize_limit": List[int],  # ksize 최소 최대 범위
}
Source code in SaigeToolkit/data/transform/augmentation/api.py 
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
def gaussian_blur(image: np.ndarray, ksize: int = 1) -> Tuple[np.ndarray, None]:
    """image에 gaussian blur를 적용합니다.

    Args:
        image (np.ndarray): augmentation을 적용할 image 입니다.
            ```
            data type: uint8
            channel: H x W / H x W x 1  - Gray
                     H x W x 3 - RGB
                     H x W x 4 - RGBA
            ```
        ksize (int, optional): blur kernel의 크기 입니다. 유효 범위는 다음과 같습니다. [1, 100]. Defaults to 1.

    Returns:
        np.ndarray: augmentation이 적용된 image 입니다.
        NoneType: None

    Example:
        ```python
        error, augmented_image = ImageProcessor.gaussian_blur(image=image, ksize=3)
        ```

    Trainer Config:
        ```python
        {
            "_target_": "gaussian_blur",
            "ksize_limit": List[int],  # ksize 최소 최대 범위
        }
        ```
    """
    return gaussian_blur(image=image, ksize=ksize, sigma=0.0), None
adjust_brightness 
adjust_brightness(image: ndarray, brightness: float = 0.0) -> Tuple[ndarray, None]

image의 밝기 (brightness)를 변경합니다.

Parameters:

  • image (ndarray) –

    augmentation을 적용할 image 입니다. 

    data type: uint8
channel: H x W / H x W x 1  - Gray
         H x W x 3 - RGB
         H x W x 4 - RGBA

  • brightness (float, default: 0.0 ) –

    밝기를 변화 강도입니다. 유효 범위는 다음과 같습니다. [-1.00, 1.00]. Defaults to 0.0.

Returns:

  • ndarray –

    np.ndarray: augmentation이 적용된 image 입니다.

  • NoneType ( None ) –

    None

Example 
error, augmented_image = ImageProcessor.adjust_brightness(image=image, brightness=0.3)
Trainer Config 
{
    "_target_": "adjust_brightness",
    "brightness_limit": List[float],  # brightness 최소 최대 범위
}
Source code in SaigeToolkit/data/transform/augmentation/api.py 
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
def adjust_brightness(image: np.ndarray, brightness: float = 0.0) -> Tuple[np.ndarray, None]:
    """image의 밝기 (brightness)를 변경합니다.

    Args:
        image (np.ndarray): augmentation을 적용할 image 입니다.
            ```
            data type: uint8
            channel: H x W / H x W x 1  - Gray
                     H x W x 3 - RGB
                     H x W x 4 - RGBA
            ```
        brightness (float, optional): 밝기를 변화 강도입니다. 유효 범위는 다음과 같습니다. [-1.00, 1.00]. Defaults to 0.0.

    Returns:
        np.ndarray: augmentation이 적용된 image 입니다.
        NoneType: None

    Example:
        ```python
        error, augmented_image = ImageProcessor.adjust_brightness(image=image, brightness=0.3)
        ```

    Trainer Config:
        ```python
        {
            "_target_": "adjust_brightness",
            "brightness_limit": List[float],  # brightness 최소 최대 범위
        }
        ```
    """
    return (
        adjust_brightness(image=image, brightness=brightness, brightness_by_max=True),
        None,
    )
adjust_contrast 
adjust_contrast(image: ndarray, contrast: float = 0.0) -> Tuple[ndarray, None]

image의 대비 (contrast)를 변경합니다.

Parameters:

  • image (ndarray) –

    augmentation을 적용할 image 입니다. 

    data type: uint8
channel: H x W / H x W x 1  - Gray
         H x W x 3 - RGB
         H x W x 4 - RGBA

  • contrast (float, default: 0.0 ) –

    대비 변화 강도입니다. 유효 범위는 다음과 같습니다. [-1.00, 1.00]. Defaults to 0.0.

Returns:

  • ndarray –

    np.ndarray: augmentation이 적용된 image 입니다.

  • NoneType ( None ) –

    None

Example 
error, augmented_image = ImageProcessor.adjust_contrast(image=image, contrast=0.7)
Trainer Config 
{
    "_target_": "adjust_contrast",
    "contrast_limit": List[float],  # contrast 최소 최대 범위
}
Source code in SaigeToolkit/data/transform/augmentation/api.py 
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
def adjust_contrast(image: np.ndarray, contrast: float = 0.0) -> Tuple[np.ndarray, None]:
    """image의 대비 (contrast)를 변경합니다.

    Args:
        image (np.ndarray): augmentation을 적용할 image 입니다.
            ```
            data type: uint8
            channel: H x W / H x W x 1  - Gray
                     H x W x 3 - RGB
                     H x W x 4 - RGBA
            ```
        contrast (float, optional): 대비 변화 강도입니다. 유효 범위는 다음과 같습니다. [-1.00, 1.00]. Defaults to 0.0.

    Returns:
        np.ndarray: augmentation이 적용된 image 입니다.
        NoneType: None

    Example:
        ```python
        error, augmented_image = ImageProcessor.adjust_contrast(image=image, contrast=0.7)
        ```

    Trainer Config:
        ```python
        {
            "_target_": "adjust_contrast",
            "contrast_limit": List[float],  # contrast 최소 최대 범위
        }
        ```
    """
    return adjust_contrast(image=image, contrast=contrast), None
adjust_hue 
adjust_hue(image: ndarray, hue: float = 0.0) -> Tuple[ndarray, None]

image의 색조 (hue)를 변경합니다.

Parameters:

  • image (ndarray) –

    augmentation을 적용할 image 입니다. 

    data type: uint8
channel: H x W / H x W x 1  - Gray
         H x W x 3 - RGB
         H x W x 4 - RGBA

  • hue (float, default: 0.0 ) –

    색조 변화 강도입니다. 유효 범위는 다음과 같습니다. [-1.00, 1.00]. Defaults to 0.0.

Returns:

  • ndarray –

    np.ndarray: augmentation이 적용된 image 입니다.

  • NoneType ( None ) –

    None

Example 
error, augmented_image = ImageProcessor.adjust_hue(image=image, hue=0.7)
Trainer Config 
{
    "_target_": "adjust_hue",
    "hue_limit": List[float],  # hue 최소 최대 범위
}
Source code in SaigeToolkit/data/transform/augmentation/api.py 
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
def adjust_hue(image: np.ndarray, hue: float = 0.0) -> Tuple[np.ndarray, None]:
    """image의 색조 (hue)를 변경합니다.

    Args:
        image (np.ndarray): augmentation을 적용할 image 입니다.
            ```
            data type: uint8
            channel: H x W / H x W x 1  - Gray
                     H x W x 3 - RGB
                     H x W x 4 - RGBA
            ```
        hue (float, optional): 색조 변화 강도입니다. 유효 범위는 다음과 같습니다. [-1.00, 1.00]. Defaults to 0.0.

    Returns:
        np.ndarray: augmentation이 적용된 image 입니다.
        NoneType: None

    Example:
        ```python
        error, augmented_image = ImageProcessor.adjust_hue(image=image, hue=0.7)
        ```

    Trainer Config:
        ```python
        {
            "_target_": "adjust_hue",
            "hue_limit": List[float],  # hue 최소 최대 범위
        }
        ```
    """
    return adjust_hue(image=image, hue=hue), None
adjust_saturation 
adjust_saturation(image: ndarray, saturation: float = 0.0) -> Tuple[ndarray, None]

image의 채도 (saturation)를 변경합니다.

Parameters:

  • image (ndarray) –

    augmentation을 적용할 image 입니다. 

    data type: uint8
channel: H x W / H x W x 1  - Gray
         H x W x 3 - RGB
         H x W x 4 - RGBA

  • saturation (float, default: 0.0 ) –

    채도 변화 강도입니다. 유효 범위는 다음과 같습니다. [-1.00, 1.00]. Defaults to 0.0.

Returns:

  • ndarray –

    np.ndarray: augmentation이 적용된 image 입니다.

  • NoneType ( None ) –

    None

Example 
error, augmented_image = ImageProcessor.adjust_saturation(image=image, saturation=0.7)
Trainer Config 
{
    "_target_": "adjust_saturation",
    "saturation_limit": List[float],  # saturation 최소 최대 범위
}
Source code in SaigeToolkit/data/transform/augmentation/api.py 
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
def adjust_saturation(image: np.ndarray, saturation: float = 0.0) -> Tuple[np.ndarray, None]:
    """image의 채도 (saturation)를 변경합니다.

    Args:
        image (np.ndarray): augmentation을 적용할 image 입니다.
            ```
            data type: uint8
            channel: H x W / H x W x 1  - Gray
                     H x W x 3 - RGB
                     H x W x 4 - RGBA
            ```
        saturation (float, optional): 채도 변화 강도입니다. 유효 범위는 다음과 같습니다. [-1.00, 1.00]. Defaults to 0.0.

    Returns:
        np.ndarray: augmentation이 적용된 image 입니다.
        NoneType: None

    Example:
        ```python
        error, augmented_image = ImageProcessor.adjust_saturation(image=image, saturation=0.7)
        ```

    Trainer Config:
        ```python
        {
            "_target_": "adjust_saturation",
            "saturation_limit": List[float],  # saturation 최소 최대 범위
        }
        ```
    """
    return adjust_saturation(image=image, saturation=saturation), None
adjust_gamma 
adjust_gamma(image: ndarray, gamma: float = 0.0) -> Tuple[ndarray, None]

image의 gamma를 조절하여 밝기를 변화시킵니다.

Parameters:

  • image (ndarray) –

    augmentation을 적용할 image 입니다. 

    data type: uint8
channel: H x W / H x W x 1  - Gray
         H x W x 3 - RGB
         H x W x 4 - RGBA

  • gamma (float, default: 0.0 ) –

    조절할 gamma value 입니다. 유효 범위는 다음과 같습니다. [-1.0, 1.0]. Defaults to 0.0.

Returns:

  • ndarray –

    np.ndarray: augmentation이 적용된 image 입니다.

  • NoneType ( None ) –

    None

Example 
error, augmented_image = ImageProcessor.adjust_gamma(image=image, gamma=0.5)
Trainer Config 
{
    "_target_": "adjust_gamma",
    "gamma_limit": List[float],  # gamma 최소 최대 범위
}
Source code in SaigeToolkit/data/transform/augmentation/api.py 
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
def adjust_gamma(image: np.ndarray, gamma: float = 0.0) -> Tuple[np.ndarray, None]:
    """image의 gamma를 조절하여 밝기를 변화시킵니다.

    Args:
        image (np.ndarray): augmentation을 적용할 image 입니다.
            ```
            data type: uint8
            channel: H x W / H x W x 1  - Gray
                     H x W x 3 - RGB
                     H x W x 4 - RGBA
            ```
        gamma (float, optional): 조절할 gamma value 입니다. 유효 범위는 다음과 같습니다. [-1.0, 1.0]. Defaults to 0.0.

    Returns:
        np.ndarray: augmentation이 적용된 image 입니다.
        NoneType: None

    Example:
        ```python
        error, augmented_image = ImageProcessor.adjust_gamma(image=image, gamma=0.5)
        ```

    Trainer Config:
        ```python
        {
            "_target_": "adjust_gamma",
            "gamma_limit": List[float],  # gamma 최소 최대 범위
        }
        ```
    """
    return adjust_gamma(image=image, gamma=gamma), None
adjust_brightness_contrast 
adjust_brightness_contrast(image: ndarray, brightness: float = 0.0, contrast: float = 0.0) -> Tuple[ndarray, None]

image의 밝기 (brightness), 대비 (contrast)를 변경합니다.

Parameters:

  • image (ndarray) –

    augmentation을 적용할 image 입니다. 

    data type: uint8
channel: H x W / H x W x 1  - Gray
         H x W x 3 - RGB
         H x W x 4 - RGBA

  • brightness (float, default: 0.0 ) –

    밝기를 담당하는 요소입니다. 유효 범위는 다음과 같습니다. [-1.00, 1.00]. Defaults to 0.0.

  • contrast (float, default: 0.0 ) –

    대비를 담당하는 요소입니다. 유효 범위는 다음과 같습니다. [-1.00, 1.00]. Defaults to 0.0.

Returns:

  • ndarray –

    np.ndarray: augmentation이 적용된 image 입니다.

  • NoneType ( None ) –

    None

Example 
error, augmented_image = ImageProcessor.adjust_brightness_contrast(image=image, brightness=0.3, contrast=0.7)
Trainer Config 
{
    "_target_": "adjust_brightness_contrast",
    "brightness_limit": List[float],  # brightness 최소 최대 범위
    "contrast_limit": List[float],  # contrast 최소 최대 범위
}
Source code in SaigeToolkit/data/transform/augmentation/api.py 
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
def adjust_brightness_contrast(
    image: np.ndarray, brightness: float = 0.0, contrast: float = 0.0
) -> Tuple[np.ndarray, None]:
    """image의 밝기 (brightness), 대비 (contrast)를 변경합니다.

    Args:
        image (np.ndarray): augmentation을 적용할 image 입니다.
            ```
            data type: uint8
            channel: H x W / H x W x 1  - Gray
                     H x W x 3 - RGB
                     H x W x 4 - RGBA
            ```
        brightness (float, optional): 밝기를 담당하는 요소입니다. 유효 범위는 다음과 같습니다. [-1.00, 1.00]. Defaults to 0.0.
        contrast (float, optional): 대비를 담당하는 요소입니다. 유효 범위는 다음과 같습니다. [-1.00, 1.00]. Defaults to 0.0.

    Returns:
        np.ndarray: augmentation이 적용된 image 입니다.
        NoneType: None

    Example:
        ```python
        error, augmented_image = ImageProcessor.adjust_brightness_contrast(image=image, brightness=0.3, contrast=0.7)
        ```

    Trainer Config:
        ```python
        {
            "_target_": "adjust_brightness_contrast",
            "brightness_limit": List[float],  # brightness 최소 최대 범위
            "contrast_limit": List[float],  # contrast 최소 최대 범위
        }
        ```
    """
    return (
        adjust_brightness_contrast(
            image=image, brightness=brightness, contrast=contrast, brightness_by_max=True
        ),
        None,
    )
iso_noise 
iso_noise(image: ndarray, color_shift: float = 0.0, intensity: float = 0.0) -> Tuple[ndarray, None]

Apply camera sensor noise.

Parameters:

  • image (ndarray) –

    augmentation을 적용할 image 입니다. 

    data type: uint8
channel: H x W / H x W x 1  - Gray
         H x W x 3 - RGB
         H x W x 4 - RGBA

  • color_shift (float, default: 0.0 ) –

    variance range for color hue change. Measured as a fraction of 360 degree Hue angle in HLS colorspace. 유효 범위는 다음과 같습니다. [0.00, 1.00]. Defaults to 0.0.

  • intensity (float, default: 0.0 ) –

    Multiplicative factor that control strength of color and luminace noise. 유효 범위는 다음과 같습니다. [0.00, 2.00]. Defaults to 0.0.

Returns:

  • ndarray –

    np.ndarray: augmentation이 적용된 image 입니다.

  • NoneType ( None ) –

    None

Example 
error, augmented_image = ImageProcessor.iso_noise(image=image, color_shift=0.2, intensity=0.2)
Trainer Config 
{
    "_target_": "iso_noise",
    "color_shift_limit": List[float],  # color_shift 최소 최대 범위
    "intensity_limit": List[float],  # intensity 최소 최대 범위
}
Source code in SaigeToolkit/data/transform/augmentation/api.py 
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
def iso_noise(
    image: np.ndarray, color_shift: float = 0.0, intensity: float = 0.0
) -> Tuple[np.ndarray, None]:
    """Apply camera sensor noise.

    Args:
        image (np.ndarray): augmentation을 적용할 image 입니다.
            ```
            data type: uint8
            channel: H x W / H x W x 1  - Gray
                     H x W x 3 - RGB
                     H x W x 4 - RGBA
            ```
        color_shift (float, optional): variance range for color hue change. Measured as a fraction of 360 degree Hue angle in HLS colorspace.
                                       유효 범위는 다음과 같습니다. [0.00, 1.00]. Defaults to 0.0.
        intensity (float, optional): Multiplicative factor that control strength of color and luminace noise.
                                     유효 범위는 다음과 같습니다. [0.00, 2.00]. Defaults to 0.0.

    Returns:
        np.ndarray: augmentation이 적용된 image 입니다.
        NoneType: None

    Example:
        ```python
        error, augmented_image = ImageProcessor.iso_noise(image=image, color_shift=0.2, intensity=0.2)
        ```

    Trainer Config:
        ```python
        {
            "_target_": "iso_noise",
            "color_shift_limit": List[float],  # color_shift 최소 최대 범위
            "intensity_limit": List[float],  # intensity 최소 최대 범위
        }
        ```
    """
    return (
        iso_noise(image=image, color_shift=color_shift, intensity=intensity, random_state=0),
        None,
    )
ratio_jitter 
ratio_jitter(image: ndarray, proportion: Optional[int] = 0, fixed_aug_params: Optional[Dict] = None) -> Tuple[ndarray, Dict]

image에 random하게 padding과 crop을 한 뒤, 원래 size로 resize 하는 과정을 통해 image의 가로 세로 비율을 변경합니다. NOTE: 프리뷰에서는 네변에 각각 [0, proportion] 범위에서 crop 혹은 padding한 예시를 보여줍니다.

Parameters:

  • image (ndarray) –

    augmentation을 적용할 image 입니다. 

    data type: uint8
channel: H x W / H x W x 1  - Gray
         H x W x 3 - RGB
         H x W x 4 - RGBA

  • proportion (int, default: 0 ) –

    padding 혹은 crop을 할 비율(단위: 백분율)입니다. 유효 범위는 다음과 같습니다. [0, 50]. Defaults to 0. (fixed_aug_params=None일 때만 작동합니다.)

  • fixed_aug_params (Dict, default: None ) –

    각 변에 대해서 고정된 crop 혹은 padding을 직접 정해주고자 할 때 사용합니다.

Returns:

  • ndarray –

    np.ndarray: augmentation이 적용된 image 입니다.

  • Dict ( Dict ) –

    각 변에 적용된 proportioin 비율(-50 ~ 50 사이의 실수, 양수이면 crop, 음수이면 pad)들입니다. 

    {
    "proportion_left": float,
    "proportion_right": float,
    "proportion_top": float,
    "proportion_bottom": float,
}

각 변에 대해 crop 혹은 padding할 비율을 직접 설정 
error, augmented_image = ImageProcessor.ratio_jitter(
    image=image,
    proportion=None,
    fixed_aug_params={
        "proportion_left": 2.0,
        "proportion_right": 2.0,
        "proportion_top": 2.0,
        "proportion_bottom": 2.0,
    }
)
각 변에 [-proportion, proportion] 범위에서 랜덤하게 crop 혹은 padding 적용 
error, augmented_image = ImageProcessor.ratio_jitter(image=image, proportion=2)
Trainer Config 
{
    "_target_": "ratio_jitter",
    "proportion_limit": int,  # proportion 최대 범위
}
Source code in SaigeToolkit/data/transform/augmentation/api.py 
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
def ratio_jitter(
    image: np.ndarray,
    proportion: Optional[int] = 0,
    fixed_aug_params: Optional[Dict] = None,
) -> Tuple[np.ndarray, Dict]:
    """image에 random하게 padding과 crop을 한 뒤, 원래 size로 resize 하는 과정을 통해 image의 가로 세로 비율을 변경합니다.
    NOTE: 프리뷰에서는 네변에 각각 [0, proportion] 범위에서 crop 혹은 padding한 예시를 보여줍니다.

    Args:
        image (np.ndarray): augmentation을 적용할 image 입니다.
            ```
            data type: uint8
            channel: H x W / H x W x 1  - Gray
                     H x W x 3 - RGB
                     H x W x 4 - RGBA
            ```
        proportion (int, optional): padding 혹은 crop을 할 비율(단위: 백분율)입니다. 유효 범위는 다음과 같습니다. [0, 50]. Defaults to 0. (`fixed_aug_params=None`일 때만 작동합니다.)
        fixed_aug_params (Dict, optional): 각 변에 대해서 고정된 crop 혹은 padding을 직접 정해주고자 할 때 사용합니다.

    Returns:
        np.ndarray: augmentation이 적용된 image 입니다.
        Dict: 각 변에 적용된 proportioin 비율(-50 ~ 50 사이의 실수, 양수이면 crop, 음수이면 pad)들입니다.
            ```python
            {
                "proportion_left": float,
                "proportion_right": float,
                "proportion_top": float,
                "proportion_bottom": float,
            }
            ```

    Example1: 각 변에 대해 crop 혹은 padding할 비율을 직접 설정
        ```python
        error, augmented_image = ImageProcessor.ratio_jitter(
            image=image,
            proportion=None,
            fixed_aug_params={
                "proportion_left": 2.0,
                "proportion_right": 2.0,
                "proportion_top": 2.0,
                "proportion_bottom": 2.0,
            }
        )
        ```

    Example2: 각 변에 [-proportion, proportion] 범위에서 랜덤하게 crop 혹은 padding 적용
        ```python
        error, augmented_image = ImageProcessor.ratio_jitter(image=image, proportion=2)
        ```

    Trainer Config:
        ```python
        {
            "_target_": "ratio_jitter",
            "proportion_limit": int,  # proportion 최대 범위
        }
        ```
    """
    aug_param_keys = ["proportion_left", "proportion_right", "proportion_top", "proportion_bottom"]
    if fixed_aug_params is None:
        check_value(proportion, 0, 50)
        proportion_limit = [-proportion, proportion]
        fixed_aug_params = {k: random.uniform(*proportion_limit) for k in aug_param_keys}
    else:
        for k in aug_param_keys:
            check_value(fixed_aug_params[k], -50, 50)

    return (
        ratio_jitter(
            image=image,
            proportion_left=fixed_aug_params["proportion_left"] / 100,
            proportion_right=fixed_aug_params["proportion_right"] / 100,
            proportion_top=fixed_aug_params["proportion_top"] / 100,
            proportion_bottom=fixed_aug_params["proportion_bottom"] / 100,
            resampling="bilinear",
            border_mode=cv2.BORDER_CONSTANT,
            value=0,
        ),
        fixed_aug_params,
    )
zoom 
zoom(image: ndarray, ratio: float = 1.0) -> Tuple[ndarray, None]

image에 zoom in/out 효과를 줍니다.

Parameters:

  • image (ndarray) –

    augmentation을 적용할 image 입니다. 

    data type: uint8
channel: H x W / H x W x 1  - Gray
         H x W x 3 - RGB
         H x W x 4 - RGBA

  • ratio (float, default: 1.0 ) –

    zoom in/out 할 비율입니다. 1보다 크면 zoom in을 1보다 작으면 zoom out을 합니다. 유효 범위는 다음과 같습니다. [0.01, 100.00]. Defaults to 1.0.

Returns:

  • ndarray –

    np.ndarray: augmentation이 적용된 image 입니다.

  • NoneType ( None ) –

    None

Example 
error, augmented_image = ImageProcessor.zoom(image=image, ratio=2.0)
Trainer Config 
{
    "_target_": "zoom",
    "ratio_limit": List[float],  # ratio 최소 최대 범위
}
Source code in SaigeToolkit/data/transform/augmentation/api.py 
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
def zoom(image: np.ndarray, ratio: float = 1.0) -> Tuple[np.ndarray, None]:
    """image에 zoom in/out 효과를 줍니다.

    Args:
        image (np.ndarray): augmentation을 적용할 image 입니다.
            ```
            data type: uint8
            channel: H x W / H x W x 1  - Gray
                     H x W x 3 - RGB
                     H x W x 4 - RGBA
            ```
        ratio (float, optional): zoom in/out 할 비율입니다. 1보다 크면 zoom in을 1보다 작으면 zoom out을 합니다.
                                 유효 범위는 다음과 같습니다. [0.01, 100.00]. Defaults to 1.0.

    Returns:
        np.ndarray: augmentation이 적용된 image 입니다.
        NoneType: None

    Example:
        ```python
        error, augmented_image = ImageProcessor.zoom(image=image, ratio=2.0)
        ```

    Trainer Config:
        ```python
        {
            "_target_": "zoom",
            "ratio_limit": List[float],  # ratio 최소 최대 범위
        }
        ```
    """
    return (
        zoom(
            image=image,
            ratio=ratio,
            h_start=0,
            w_start=0,
            resampling="bilinear",
            border_mode=cv2.BORDER_CONSTANT,
            value=0,
        ),
        None,
    )
random_resized_crop_and_pad 
random_resized_crop_and_pad(image: ndarray, scale: float = 1.0, aspect_ratio: float = 1.0, height: Optional[int] = None, width: Optional[int] = None) -> Tuple[ndarray, None]

원본 image의 scale 비율의 면적을 가지면서 가로 세로 비가 aspect_ratio인 image를 random하게 crop 한 뒤, (crop image가 원본 image 보다 커지는 경우 padding을 통해 해결합니다.) 크기가 (height, width)가 되도록 resize를 합니다.

Parameters:

  • image (ndarray) –

    augmentation을 적용할 image 입니다. 

    data type: uint8
channel: H x W / H x W x 1  - Gray
         H x W x 3 - RGB
         H x W x 4 - RGBA

  • scale (float, default: 1.0 ) –

    crop할 image의 면적을 나타내는 값입니다. 실제 면적은 [원본 이미지의 면적 * scale] 입니다. 유효 범위는 다음과 같습니다. [0.01, 1.00]. Defaults to 1.0.

  • aspect_ratio (float, default: 1.0 ) –

    crop할 image의 가로 세로 비를 나타내는 값입니다. 유효 범위는 다음과 같습니다. [0.10, 10.00]. Defaults to 1.0.

  • height (int, default: None ) –

    crop image를 resize할 height 입니다. None인 경우 입력 이미지의 원본 height를 사용합니다.

  • width (int, default: None ) –

    crop image를 resize할 width 입니다. None인 경우 입력 이미지의 원본 width를 사용합니다.

Returns:

  • ndarray –

    np.ndarray: augmentation이 적용된 image 입니다.

  • NoneType ( None ) –

    None

Example 
error, augmented_image = ImageProcessor.random_resized_crop_and_pad(image=image,
                                                                    scale=0.5,
                                                                    aspect_ratio=2,
                                                                    height=256,
                                                                    width=256)
Trainer Config 
{
    "_target_": "random_resized_crop_and_pad",
    "scale_limit": List[float],  # scale 최소 최대 범위
    "aspect_ratio_limit": List[float],  # aspect_ratio 최소 최대 범위
    "height": Optional[int],  # crop 후 resize할 이미지 height (None인 경우 원본 height 사용)
    "width": Optional[int],  # crop 후 resize할 이미지 width (None인 경우 원본 width 사용)
}
Source code in SaigeToolkit/data/transform/augmentation/api.py 
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
def random_resized_crop_and_pad(
    image: np.ndarray,
    scale: float = 1.0,
    aspect_ratio: float = 1.0,
    height: Optional[int] = None,
    width: Optional[int] = None,
) -> Tuple[np.ndarray, None]:
    """원본 image의 scale 비율의 면적을 가지면서 가로 세로 비가 aspect_ratio인 image를 random하게 crop 한 뒤,
       (crop image가 원본 image 보다 커지는 경우 padding을 통해 해결합니다.)
       크기가 (height, width)가 되도록 resize를 합니다.

    Args:
        image (np.ndarray): augmentation을 적용할 image 입니다.
            ```
            data type: uint8
            channel: H x W / H x W x 1  - Gray
                     H x W x 3 - RGB
                     H x W x 4 - RGBA
            ```
        scale (float, optional): crop할 image의 면적을 나타내는 값입니다. 실제 면적은 [원본 이미지의 면적 * scale] 입니다.
                                 유효 범위는 다음과 같습니다. [0.01, 1.00]. Defaults to 1.0.
        aspect_ratio (float, optional): crop할 image의 가로 세로 비를 나타내는 값입니다.
                                        유효 범위는 다음과 같습니다. [0.10, 10.00]. Defaults to 1.0.
        height (int): crop image를 resize할 height 입니다. None인 경우 입력 이미지의 원본 height를 사용합니다.
        width (int): crop image를 resize할 width 입니다. None인 경우 입력 이미지의 원본 width를 사용합니다.

    Returns:
        np.ndarray: augmentation이 적용된 image 입니다.
        NoneType: None

    Example:
        ```python
        error, augmented_image = ImageProcessor.random_resized_crop_and_pad(image=image,
                                                                            scale=0.5,
                                                                            aspect_ratio=2,
                                                                            height=256,
                                                                            width=256)
        ```

    Trainer Config:
        ```python
        {
            "_target_": "random_resized_crop_and_pad",
            "scale_limit": List[float],  # scale 최소 최대 범위
            "aspect_ratio_limit": List[float],  # aspect_ratio 최소 최대 범위
            "height": Optional[int],  # crop 후 resize할 이미지 height (None인 경우 원본 height 사용)
            "width": Optional[int],  # crop 후 resize할 이미지 width (None인 경우 원본 width 사용)
        }
        ```
    """
    return (
        random_resized_crop_and_pad(
            image=image,
            scale=scale,
            aspect_ratio=aspect_ratio,
            h_start=0,
            w_start=0,
            height=height,
            width=width,
            resampling="bilinear",
            border_mode=cv2.BORDER_CONSTANT,
            value=0,
        ),
        None,
    )
light_reflect 
light_reflect(image: ndarray, radius: float = 0.0) -> Tuple[ndarray, None]

image에 원형 빛을 비춘 것 같은 효과를 줍니다.

Parameters:

  • image (ndarray) –

    augmentation을 적용할 image 입니다. 

    data type: uint8
channel: H x W / H x W x 1  - Gray
         H x W x 3 - RGB
         H x W x 4 - RGBA

  • radius (float, default: 0.0 ) –

    원형 빛의 반지름 크기 입니다. 유효 범위는 다음과 같습니다. [0.00, 1.00]. Defaults to 0.0.

Returns:

  • ndarray –

    np.ndarray: augmentation이 적용된 image 입니다.

  • NoneType ( None ) –

    None

Example 
error, augmented_image = ImageProcessor.light_reflect(image=image, radius=0.30)
Trainer Config 
{
    "_target_": "light_reflect",
    "radius_limit": List[float],  # radius 최소 최대 범위
}
Source code in SaigeToolkit/data/transform/augmentation/api.py 
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
def light_reflect(image: np.ndarray, radius: float = 0.0) -> Tuple[np.ndarray, None]:
    """image에 원형 빛을 비춘 것 같은 효과를 줍니다.

    Args:
        image (np.ndarray): augmentation을 적용할 image 입니다.
            ```
            data type: uint8
            channel: H x W / H x W x 1  - Gray
                     H x W x 3 - RGB
                     H x W x 4 - RGBA
            ```
        radius (float, optional): 원형 빛의 반지름 크기 입니다. 유효 범위는 다음과 같습니다. [0.00, 1.00]. Defaults to 0.0.

    Returns:
        np.ndarray: augmentation이 적용된 image 입니다.
        NoneType: None

    Example:
        ```python
        error, augmented_image = ImageProcessor.light_reflect(image=image, radius=0.30)
        ```

    Trainer Config:
        ```python
        {
            "_target_": "light_reflect",
            "radius_limit": List[float],  # radius 최소 최대 범위
        }
        ```
    """
    return (
        light_reflect(image=image, xc=0.5, yc=0.5, x_radius=radius, y_radius=radius, angle=0),
        None,
    )
perspective_transform 
perspective_transform(image: ndarray, intensity: Optional[int] = 0, fixed_aug_params: Optional[Dict] = None) -> Tuple[ndarray, Dict]

이미지를 투영 변환(Perspective Transform)합니다. image를 다른 각도(시점)에서 바라본 형태로 변환합니다. Perspective Transform 을 위한 네개의 도착점의 좌표 (offset)은 다음과 같이 샘플링됩니다. offset_top_left: 원본 이미지의 좌측 상단 모서리를 얼마만큼 중심부로 이동시킬 지에 대한 실수 값이 들어있습니다. 즉, Perspective Transform 의 좌측 상단점의 도착점은 아래와 같이 계산할 수 있습니다.

```
x` = 0 + width * offset_top_left[0]
y` = 0 + height * offset_top_left[1]
point_dst = (x`, y`)
```
offset_bottom_right

offset_top_left 와 동일하되 우측 하단 점을 나타냅니다. Perspective Transform 의 우측 하단점의 도착점은 아래와 같이 계산할 수 있습니다.

x` = width - width * offset_bottom_right[0]
y` = height - height * offset_bottom_right[1]
point_dst = (x`, y`)

offset_top_right: offset_top_left 와 동일하되 우측 상단 점을 나타냅니다. offset_bottom_left: offset_top_left 와 동일하되 좌측 하단 점을 나타냅니다.

Parameters:

  • image (ndarray) –

    augmentation을 적용할 image 입니다. 

    data type: uint8
channel: H x W / H x W x 1  - Gray
         H x W x 3 - RGB
         H x W x 4 - RGBA

  • intensity (int, default: 0 ) –

    perspective transform 을 적용 강도(단위: 백분율)입니다. intensity 범위 내에서 랜덤한 값으로 샘플된 네개의 offset 을 이용해 perspective transform 을 수행합니다. 유효 범위는 [0, 49] 로, 각 도착지 점들은 이미지의 중간 선을 지나칠 수 없습니다. 이를 통해 이미지가 반전되는 정도의 왜곡을 방지합니다. Defaults to 0. (fixed_aug_params=None일 때만 작동합니다.)

  • fixed_aug_params (Dict, default: None ) –

    각 꼭짓점에 대해서 offset 비율을 직접 정해주고자 할 때 사용합니다.

Returns:

  • ndarray –

    np.ndarray: augmentation이 적용된 image 입니다.

  • Dict ( Dict ) –

    각 꼭짓점에 적용된 offset들입니다. 

    {
    "offset_top_left": Tuple[float, float],
    "offset_top_right": Tuple[float, float],
    "offset_bottom_right": Tuple[float, float],
    "offset_bottom_left": Tuple[float, float],
}

각 꼭짓점의 offset 비율을 직접 설정 
error, augmented_image = ImageProcessor.perspective_transform(
    image=image,
    fixed_aug_params={
        "offset_top_left": (25.0, 30.0),
        "offset_top_right": (10.0, 40.0),
        "offset_bottom_right": (0.0, 20.0),
        "offset_bottom_left": (20.0, 35.0),
    }
)
각 꼭짓점에 [0, intensity] 범위에서 랜덤하게 offset 비율을 적용 
error, augmented_image = ImageProcessor.perspective_transform(
    image=image,
    intensity=10,
)
Trainer Config 
{
    "_target_": "perspective_transform",
    "intensity_limit": List[float],  # intensity 최소 최대 범위
}
Source code in SaigeToolkit/data/transform/augmentation/api.py 
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
def perspective_transform(
    image: np.ndarray,
    intensity: Optional[int] = 0,
    fixed_aug_params: Optional[Dict] = None,
) -> Tuple[np.ndarray, Dict]:
    """이미지를 투영 변환(Perspective Transform)합니다. image를 다른 각도(시점)에서 바라본 형태로 변환합니다.
    Perspective Transform 을 위한 네개의 도착점의 좌표 (offset)은 다음과 같이 샘플링됩니다.
    offset_top_left:
        원본 이미지의 좌측 상단 모서리를 얼마만큼 중심부로 이동시킬 지에 대한 실수 값이 들어있습니다.
        즉, Perspective Transform 의 좌측 상단점의 도착점은 아래와 같이 계산할 수 있습니다.

        ```
        x` = 0 + width * offset_top_left[0]
        y` = 0 + height * offset_top_left[1]
        point_dst = (x`, y`)
        ```

    offset_bottom_right:
        `offset_top_left` 와 동일하되 우측 하단 점을 나타냅니다.
        Perspective Transform 의 우측 하단점의 도착점은 아래와 같이 계산할 수 있습니다.

        ```
        x` = width - width * offset_bottom_right[0]
        y` = height - height * offset_bottom_right[1]
        point_dst = (x`, y`)
        ```

    offset_top_right: `offset_top_left` 와 동일하되 우측 상단 점을 나타냅니다.
    offset_bottom_left: `offset_top_left` 와 동일하되 좌측 하단 점을 나타냅니다.


    Args:
        image (np.ndarray): augmentation을 적용할 image 입니다.
            ```
            data type: uint8
            channel: H x W / H x W x 1  - Gray
                     H x W x 3 - RGB
                     H x W x 4 - RGBA
            ```
        intensity (int, optional): perspective transform 을 적용 강도(단위: 백분율)입니다.
            intensity 범위 내에서 랜덤한 값으로 샘플된 네개의 offset 을 이용해 perspective transform 을 수행합니다.
            유효 범위는 [0, 49] 로, 각 도착지 점들은 이미지의 중간 선을 지나칠 수 없습니다.
            이를 통해 이미지가 반전되는 정도의 왜곡을 방지합니다. Defaults to 0. (`fixed_aug_params=None`일 때만 작동합니다.)
        fixed_aug_params (Dict, optional): 각 꼭짓점에 대해서 offset 비율을 직접 정해주고자 할 때 사용합니다.

    Returns:
        np.ndarray: augmentation이 적용된 image 입니다.
        Dict: 각 꼭짓점에 적용된 offset들입니다.
            ```python
            {
                "offset_top_left": Tuple[float, float],
                "offset_top_right": Tuple[float, float],
                "offset_bottom_right": Tuple[float, float],
                "offset_bottom_left": Tuple[float, float],
            }
            ```

    Example1:  각 꼭짓점의 offset 비율을 직접 설정
        ```python
        error, augmented_image = ImageProcessor.perspective_transform(
            image=image,
            fixed_aug_params={
                "offset_top_left": (25.0, 30.0),
                "offset_top_right": (10.0, 40.0),
                "offset_bottom_right": (0.0, 20.0),
                "offset_bottom_left": (20.0, 35.0),
            }
        )
        ```

    Example2:  각 꼭짓점에 [0, intensity] 범위에서 랜덤하게 offset 비율을 적용
        ```python
        error, augmented_image = ImageProcessor.perspective_transform(
            image=image,
            intensity=10,
        )
        ```

    Trainer Config:
        ```python
        {
            "_target_": "perspective_transform",
            "intensity_limit": List[float],  # intensity 최소 최대 범위
        }
        ```
    """
    aug_param_keys = [
        "offset_top_left",
        "offset_top_right",
        "offset_bottom_right",
        "offset_bottom_left",
    ]
    if fixed_aug_params is None:
        check_value(intensity, 0, 49)
        fixed_aug_params = {
            k: tuple((random.uniform(0, intensity), random.uniform(0, intensity)))
            for k in aug_param_keys
        }
    else:
        for k in aug_param_keys:
            offset_x, offset_y = fixed_aug_params[k]
            check_value(offset_x, 0, 49)
            check_value(offset_y, 0, 49)

    return (
        perspective_transform(
            image=image,
            **fixed_aug_params,
        ),
        fixed_aug_params,
    )
augment_function
adjust_saturation 
adjust_saturation(image: ImageType, saturation: float = 0.0) -> ImageType

adjust_saturation

Parameters:

  • image (ImageType) –

    입력 이미지

  • saturation (float, default: 0.0 ) –

    변형 강도, [-1.0, 1.0] 범위, Defaults to 0.0.

Returns:

  • ImageType ( ImageType ) –

    결과 이미지

Note

Ablumentation의 AF.shift_hsv()와 다른 알고리즘을 사용합니다. AF.shift_hsv()의 경우 색이 없는 픽셀을 붉은 색으로 변형합니다. 이 함수의 경우 색이 없는 픽셀은 변형하지 않습니다.

Source code in SaigeToolkit/data/transform/augmentation/augment_function.py 
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
@support_gray
@support_rgba
def adjust_saturation(image: ImageType, saturation: float = 0.0) -> ImageType:
    """adjust_saturation

    Args:
        image (ImageType): 입력 이미지
        saturation (float, optional): 변형 강도, [-1.0, 1.0] 범위, Defaults to 0.0.

    Returns:
        ImageType: 결과 이미지

    Note:
        Ablumentation의 AF.shift_hsv()와 다른 알고리즘을 사용합니다.
        AF.shift_hsv()의 경우 색이 없는 픽셀을 붉은 색으로 변형합니다.
        이 함수의 경우 색이 없는 픽셀은 변형하지 않습니다.

    """
    check_value(saturation, -1.00, 1.00)
    saturation = (saturation + 1) ** 2
    gray = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)
    result = image * saturation + gray[..., None] * (1 - saturation)

    dtype = image.dtype
    if dtype == np.uint8:
        result = np.clip(result, 0, 255)
    elif dtype == np.uint16:
        result = np.clip(result, 0, 65535)
    result = result.astype(dtype)

    return result
random_resized_crop 
random_resized_crop(image: ImageType, h_scale: float = 1.0, w_scale: float = 1.0, h_start: float = 0.0, w_start: float = 0.0, resampling: str = 'bilinear') -> Union[ImageType, MaskType]

Crop the given image to given scale and then resize it to original size.

Parameters:

  • image (ImageType) –

    original image

  • h_scale (float, default: 1.0 ) –

    crop height scale. Defaults to 1.0.

  • w_scale (float, default: 1.0 ) –

    crop width scale. Defaults to 1.0.

  • h_start (float, default: 0.0 ) –

    crop height start ratio. Defaults to 0.0.

  • w_start (float, default: 0.0 ) –

    crop width start ratio. Defaults to 0.0.

  • resampling (str, default: 'bilinear' ) –

    interpolation method. Defaults to "bilinear".

Returns:

Source code in SaigeToolkit/data/transform/augmentation/augment_function.py 
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
def random_resized_crop(
    image: ImageType,
    h_scale: float = 1.0,
    w_scale: float = 1.0,
    h_start: float = 0.0,
    w_start: float = 0.0,
    resampling: str = "bilinear",
) -> Union[ImageType, MaskType]:
    """Crop the given image to given scale and then resize it to original size.

    Args:
        image (ImageType): original image
        h_scale (float, optional): crop height scale. Defaults to 1.0.
        w_scale (float, optional): crop width scale. Defaults to 1.0.
        h_start (float, optional): crop height start ratio. Defaults to 0.0.
        w_start (float, optional): crop width start ratio. Defaults to 0.0.
        resampling (str, optional): interpolation method. Defaults to "bilinear".

    Returns:
        Union[ImageType, MaskType]: augmented data
    """
    check_value(h_scale, 0.01, 1.00)
    check_value(w_scale, 0.01, 1.00)
    check_value(h_start, 0.00, 1.00)
    check_value(w_start, 0.00, 1.00)

    h_original, w_original = image.shape[:2]
    target_height = int(round(h_original * h_scale))
    target_width = int(round(w_original * w_scale))

    # crop
    image = AFCrops.random_crop(
        img=image,
        crop_height=target_height,
        crop_width=target_width,
        h_start=min(h_start, 1.0 - 1e-5),
        w_start=min(w_start, 1.0 - 1e-5),
    )

    # resize
    target_size = (w_original, h_original)
    image = resize_image(image, target_size, resampling)

    return image
calculate_transform_matrix 
calculate_transform_matrix(width: int, height: int, offset_top_left: OffsetType, offset_top_right: OffsetType, offset_bottom_right: OffsetType, offset_bottom_left: OffsetType) -> ndarray

Calculates the perspective transformation matrix using the given width, height, and corner points of a rectangle.

Parameters:

  • width (int) –

    Width of the original image.

  • height (int) –

    Height of the original image.

  • offset_top_left (OffsetType) –

    The offset ratio of the top-left corner point. Represented by the coordinates (x, y) and has a range of [0, 49]. Calculate xand y according to the procedure below.

    x` = width * offset_top_left[0]
y` = height * offset_top_left[1]
  • offset_bottom_right (OffsetType) –

    The offset ratio of the bottom-right corner point. Calculate xand y according to the procedure below.

    x` = width - width * offset_bottom_right[0]
y` = hegiht - height * offset_bottom_right[1]
  • offset_top_right (OffsetType) –

    The offset ratio of the top-left corner point.

  • offset_bottom_left (OffsetType) –

    The offset ratio of the bottom-left corner point.

Returns:

  • ndarray –

    np.ndarray: The 4x3 perspective transform matrix.

Source code in SaigeToolkit/data/transform/augmentation/augment_function.py 
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
def calculate_transform_matrix(
    width: int,
    height: int,
    offset_top_left: OffsetType,
    offset_top_right: OffsetType,
    offset_bottom_right: OffsetType,
    offset_bottom_left: OffsetType,
) -> np.ndarray:
    """
    Calculates the perspective transformation matrix
    using the given width, height, and corner points of a rectangle.

    Parameters:
        width (int): Width of the original image.
        height (int): Height of the original image.
        offset_top_left (OffsetType):
            The offset ratio of the top-left corner point.
            Represented by the coordinates (x, y) and has a range of [0, 49].
            Calculate x` and y` according to the procedure below.

            ```
            x` = width * offset_top_left[0]
            y` = height * offset_top_left[1]
            ```

        offset_bottom_right (OffsetType):
            The offset ratio of the bottom-right corner point.
            Calculate x` and y` according to the procedure below.

            ```
            x` = width - width * offset_bottom_right[0]
            y` = hegiht - height * offset_bottom_right[1]
            ```

        offset_top_right (OffsetType): The offset ratio of the top-left corner point.
        offset_bottom_left (OffsetType): The offset ratio of the bottom-left corner point.

    Returns:
        np.ndarray: The 4x3 perspective transform matrix.

    """
    points_from = np.float32([[0, 0], [width, 0], [width, height], [0, height]])

    point_top_left: OffsetType = (
        (offset_top_left[0] / 100) * width,
        (offset_top_left[1] / 100) * height,
    )
    point_top_right: OffsetType = (
        width - (offset_top_right[0] / 100) * width,
        (offset_top_right[1] / 100) * height,
    )
    point_bottom_right: OffsetType = (
        width - (offset_bottom_right[0] / 100) * width,
        height - (offset_bottom_right[1] / 100) * height,
    )
    point_bottom_left: OffsetType = (
        (offset_bottom_left[0] / 100) * width,
        height - (offset_bottom_left[1] / 100) * height,
    )
    points_to = np.float32(
        [
            point_top_left,
            point_top_right,
            point_bottom_right,
            point_bottom_left,
        ]
    )
    transform_matrix = cv2.getPerspectiveTransform(points_from, points_to)

    return transform_matrix
augment_transform

Implement ImageTransform classes for image augmentation.

ImageTransform 
ImageTransform(prob: float = 0.5)

Bases: BaseTransform

Image와 라벨에 적용되는 Transform 입니다.

Source code in SaigeToolkit/data/transform/augmentation/base_transform.py 
14
15
16
def __init__(self, prob: float = 0.5) -> None:
    self.prob = prob
    self._additional_targets = {}
RatioJitter 
RatioJitter(proportion_limit: Optional[Union[List[float], int, float]] = None, resampling: str = 'bilinear', border_mode: int = cv2.BORDER_CONSTANT, value: Union[int, float, List[int], List[float]] = 0, mask_value: Union[int, float] = 0, **kwargs)

Bases: ImageSizeParams, ImageTransform

crop, pad, and resize to original image size

Source code in SaigeToolkit/data/transform/augmentation/augment_transform.py 
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
def __init__(
    self,
    proportion_limit: Optional[Union[List[float], int, float]] = None,
    resampling: str = "bilinear",
    border_mode: int = cv2.BORDER_CONSTANT,
    value: Union[int, float, List[int], List[float]] = 0,
    mask_value: Union[int, float] = 0,
    **kwargs,
) -> None:
    super().__init__(**kwargs)

    if proportion_limit is None:
        proportion_limit = [-0.10, 0.10]

    if isinstance(proportion_limit, int):
        # NOTE: int 입력은 백분율로 적용.
        proportion_limit = [-abs(proportion_limit) / 100, abs(proportion_limit) / 100]

    if isinstance(proportion_limit, float):
        # NOTE: float 입력은 0~1 스케일로 적용.
        proportion_limit = [-abs(proportion_limit), abs(proportion_limit)]

    check_range(proportion_limit, -0.50, 0.50)

    self.proportion_limit = proportion_limit
    self.resampling = resampling
    self.border_mode = border_mode
    self.value = value
    self.mask_value = mask_value
RandomResizedCrop 
RandomResizedCrop(scale_limit: Optional[List[float]] = None, aspect_ratio_limit: Optional[List[float]] = None, resampling: str = 'bilinear', **kwargs)

Bases: ImageSizeParams, ImageTransform

Crop a random part of the input and rescale it to original size

Parameters:

  • scale_limit (Optional[List[float]], default: None ) –

    range of size of the origin size cropped. Defaults to [0.45, 1.00].

  • aspect_ratio_limit (Optional[List[float]], default: None ) –

    range of aspect ratio of the origin aspect ratio cropped. Defaults to [0.50, 2.00].

  • resampling (str, default: 'bilinear' ) –

    interpolation method. Defaults to "bilinear".

Source code in SaigeToolkit/data/transform/augmentation/augment_transform.py 
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
def __init__(
    self,
    scale_limit: Optional[List[float]] = None,
    aspect_ratio_limit: Optional[List[float]] = None,
    resampling: str = "bilinear",
    **kwargs,
) -> None:
    """
    Args:
        scale_limit (Optional[List[float]], optional): range of size of the origin size cropped. Defaults to [0.45, 1.00].
        aspect_ratio_limit (Optional[List[float]], optional): range of aspect ratio of the origin aspect ratio cropped. Defaults to [0.50, 2.00].
        resampling (str, optional): interpolation method. Defaults to "bilinear".
    """
    super().__init__(**kwargs)

    if scale_limit is None:
        scale_limit = [0.45, 1.00]
    if aspect_ratio_limit is None:
        aspect_ratio_limit = [0.50, 2.00]

    check_range(scale_limit, 0.01, 1.00)
    check_range(aspect_ratio_limit, 0.10, 10.00)

    self.scale_limit = scale_limit
    self.aspect_ratio_limit = aspect_ratio_limit
    self.resampling = resampling
RandomResizedCropAndPad 
RandomResizedCropAndPad(scale_limit: Optional[List[float]] = None, aspect_ratio_limit: Optional[List[float]] = None, height: Optional[int] = None, width: Optional[int] = None, resampling: str = 'bilinear', border_mode: int = cv2.BORDER_CONSTANT, value: Union[int, float, List[int], List[float]] = 0, mask_value: Union[int, float] = 0, **kwargs)

Bases: ImageSizeParams, ImageTransform

pad, crop and resize to original image size

Source code in SaigeToolkit/data/transform/augmentation/augment_transform.py 
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
def __init__(
    self,
    scale_limit: Optional[List[float]] = None,
    aspect_ratio_limit: Optional[List[float]] = None,
    height: Optional[int] = None,
    width: Optional[int] = None,
    resampling: str = "bilinear",
    border_mode: int = cv2.BORDER_CONSTANT,
    value: Union[int, float, List[int], List[float]] = 0,
    mask_value: Union[int, float] = 0,
    **kwargs,
) -> None:
    super().__init__(**kwargs)

    if scale_limit is None:
        scale_limit = [0.45, 1.00]
    if aspect_ratio_limit is None:
        aspect_ratio_limit = [0.50, 2.00]

    check_range(scale_limit, 0.01, 1.00)
    check_range(aspect_ratio_limit, 0.10, 10.00)

    self.height = height
    self.width = width
    self.scale_limit = scale_limit
    self.aspect_ratio_limit = aspect_ratio_limit
    self.resampling = resampling
    self.border_mode = border_mode
    self.value = value
    self.mask_value = mask_value
RandomErasing 
RandomErasing(scale: Tuple[float, float] = (0.02, 0.33), ratio: Tuple[float, float] = (0.2, 3.3), value: Union[int, Tuple[int, int, int], str] = 0, randomly_select_values: bool = False, **kwargs)

Bases: ImageTransform

Source code in SaigeToolkit/data/transform/augmentation/augment_transform.py 
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
def __init__(
    self,
    scale: Tuple[float, float] = (0.02, 0.33),
    ratio: Tuple[float, float] = (0.2, 3.3),
    value: Union[int, Tuple[int, int, int], str] = 0,
    randomly_select_values: bool = False,
    **kwargs,
) -> None:
    super().__init__(**kwargs)
    self.scale = scale
    self.ratio = ratio
    self._value = value
    self.randomly_select_values = randomly_select_values
_get_random_erase_params 
_get_random_erase_params()

This is modified version of get_params of random erasing see: https://pytorch.org/vision/main/_modules/torchvision/transforms/transforms.html#RandomErasing.forward

Source code in SaigeToolkit/data/transform/augmentation/augment_transform.py 
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
def _get_random_erase_params(self):
    """
    This is modified version of get_params of random erasing
    see: https://pytorch.org/vision/main/_modules/torchvision/transforms/transforms.html#RandomErasing.forward
    """
    area = 1

    log_ratio = torch.log(torch.tensor(self.ratio))
    for _ in range(10):
        erase_area = area * torch.empty(1).uniform_(self.scale[0], self.scale[1]).item()
        aspect_ratio = torch.exp(torch.empty(1).uniform_(log_ratio[0], log_ratio[1])).item()

        h_in_ratio = math.sqrt(erase_area * aspect_ratio)
        w_in_ratio = math.sqrt(erase_area / aspect_ratio)

        if not (h_in_ratio < 1 and w_in_ratio < 1):
            continue

        x_in_ratio = random.uniform(0, 1 - w_in_ratio)
        y_in_ratio = random.uniform(0, 1 - h_in_ratio)

        return x_in_ratio, y_in_ratio, h_in_ratio, w_in_ratio

    # Return Original Image
    return 0, 0, 1.0, 1.0
base_transform

Data augmentation을 위한 기본 Transform의 interface를 정의합니다.

BaseTransform 
BaseTransform(prob: float = 0.5)

Transform의 기본 interface를 정의합니다.

Source code in SaigeToolkit/data/transform/augmentation/base_transform.py 
14
15
16
def __init__(self, prob: float = 0.5) -> None:
    self.prob = prob
    self._additional_targets = {}
get_params_from_data 
get_params_from_data(data_for_params: ParamsType) -> ParamsType

이 함수는 input으로부터 parameter들을 뽑을 때 사용됩니다.

Parameters:

  • data_for_params (ParamsType) –

    params을 추출할 데이터를 입력으로 갖습니다.

Returns:

  • ParamsType ( ParamsType ) –

    params로 쓰일 데이터를 반환합니다.

Source code in SaigeToolkit/data/transform/augmentation/base_transform.py 
42
43
44
45
46
47
48
49
50
51
def get_params_from_data(self, data_for_params: ParamsType) -> ParamsType:
    """이 함수는 input으로부터 parameter들을 뽑을 때 사용됩니다.

    Args:
        data_for_params (ParamsType): params을 추출할 데이터를 입력으로 갖습니다.

    Returns:
        ParamsType: params로 쓰일 데이터를 반환합니다.
    """
    return {}
builder
compose
base_compose

Data augmentation을 위한 기본 Compose의 interface를 정의합니다.

BaseCompose 
BaseCompose(transforms: Sequence[Union[BaseTransform, BaseCompose]], prob: float = 1.0)

여러 Data Transform들을 하나로 묶어서 관리해주는 Compose의 기본 interface를 정의합니다.

Source code in SaigeToolkit/data/transform/augmentation/compose/base_compose.py 
14
15
16
17
18
def __init__(
    self, transforms: Sequence[Union[BaseTransform, BaseCompose]], prob: float = 1.0
) -> None:
    self.transforms = transforms
    self.prob = prob
builder
compose

box_function

preserve_coordinates 
preserve_coordinates(func)

Box augmentation이 (left, top, right, bottom) coordinate system을 기반으로 구현 되어있기 때문에, input bboxes의 coordinate system을 확인하고 augmentation에 맞는 coordinate system으로 변환하고, augmentation이 끝나면 다시 기존 coordinate system으로 변경하여 출력합니다.

  • np.ndarray의 경우 coordinate system을 체크할 수 없기 때문에 (left, top, right, bottom) coordinate system이라고 가정합니다.

  • NumpyBBoxes의 경우 NumpyBBoxes 내부 변수 coordinate과 내부 함수 convert_coordinate를 활용하여 구현됩니다.

Source code in SaigeToolkit/data/transform/box_function.py 
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
def preserve_coordinates(func):
    """
    Box augmentation이 (left, top, right, bottom) coordinate system을 기반으로 구현 되어있기 때문에,
    input bboxes의 coordinate system을 확인하고 augmentation에 맞는 coordinate system으로 변환하고,
    augmentation이 끝나면 다시 기존 coordinate system으로 변경하여 출력합니다.

    - np.ndarray의 경우
    coordinate system을 체크할 수 없기 때문에 (left, top, right, bottom) coordinate system이라고 가정합니다.

    - NumpyBBoxes의 경우
    NumpyBBoxes 내부 변수 coordinate과 내부 함수 convert_coordinate를 활용하여 구현됩니다.

    """
    _FUNCTIONAL_BBOX_COORDINATE = "xyxy"

    def wrapped_function_for_ndarray(bboxes: np.ndarray, *args, **kwargs) -> np.ndarray:
        new_bboxes = func(bboxes, *args, **kwargs)

        return new_bboxes

    def wrapped_function_for_numpyboxes(bboxes: NumpyBoxes, *args, **kwargs) -> NumpyBoxes:
        # original coordinate -> (left, top, right, bottom)
        original_coordinate = bboxes.coordinate  # record original coordinate
        bboxes = bboxes.convert_coordinate(_FUNCTIONAL_BBOX_COORDINATE)  # -> (left, top, right, bottom)

        # box augmentation
        new_bboxes = func(bboxes, *args, **kwargs)
        if not isinstance(new_bboxes, bboxes.__class__):
            new_bboxes = bboxes.__class__(new_bboxes, _FUNCTIONAL_BBOX_COORDINATE)

        # (left, top, right, bottom) -> original coordinate
        new_bboxes = new_bboxes.convert_coordinate(original_coordinate)

        return new_bboxes

    @wraps(func)
    def wrapped_function(bboxes: BBoxesType, *args, **kwargs) -> BBoxesType:
        # isinstance가 상속된 type도 true를 return 하기 때문에 여기서는 type(instance) == class로 체크
        if isinstance(bboxes, NumpyBoxes):
            new_bboxes = wrapped_function_for_numpyboxes(bboxes, *args, **kwargs)
        elif isinstance(bboxes, np.ndarray):
            new_bboxes = wrapped_function_for_ndarray(bboxes, *args, **kwargs)
        else:
            raise NotImplementedError

        return new_bboxes

    return wrapped_function
resize_box 
resize_box(bboxes: BBoxesType, image_size: Tuple[int], tw: int, th: int) -> BBoxesType

bbox resize from (w, h) to (tw, th)

Source code in SaigeToolkit/data/transform/box_function.py 
62
63
64
65
66
67
68
69
70
71
72
@preserve_coordinates
def resize_box(bboxes: BBoxesType, image_size: Tuple[int], tw: int, th: int) -> BBoxesType:
    """bbox resize from (w, h) to (tw, th)"""
    dtype = bboxes.dtype
    w, h = image_size
    if w == tw and h == th:
        return bboxes
    else:
        w_scale = tw / w
        h_scale = th / h
        return (bboxes * (w_scale, h_scale, w_scale, h_scale)).astype(dtype)
crop_box 
crop_box(bboxes: BBoxesType, cropping_box: Union[ndarray, List[int]]) -> BBoxesType

bbox crop. An image is cropped at (new_left, new_top, new_right, new_bottom)

Source code in SaigeToolkit/data/transform/box_function.py 
75
76
77
78
79
80
81
82
83
84
@preserve_coordinates
def crop_box(bboxes: BBoxesType, cropping_box: Union[np.ndarray, List[int]]) -> BBoxesType:
    """bbox crop. An image is cropped at (new_left, new_top, new_right, new_bottom)"""
    dtype = bboxes.dtype
    new_left, new_top, new_right, new_bottom = map(round, cropping_box)
    new_h = new_bottom - new_top
    new_bboxes = bboxes - (new_left, new_top, new_left, new_top)
    new_bboxes[:, [0, 1]] = np.maximum(new_bboxes[:, [0, 1]], 0)
    new_bboxes[:, [2, 3]] = np.minimum(new_bboxes[:, [2, 3]], (new_right - new_left, new_h))
    return new_bboxes.astype(dtype)

function_util

Define utility functions for data augmentation.

image_function

read_image_size 
read_image_size(image: Union[Image, Tensor, ndarray]) -> ImageSizeType

image size: (W, H)

Source code in SaigeToolkit/data/transform/image_function.py 
11
12
13
14
15
16
17
18
19
20
def read_image_size(image: Union[Image.Image, torch.Tensor, np.ndarray]) -> ImageSizeType:
    """image size: (W, H)"""
    if isinstance(image, Image.Image):
        return image.size
    elif isinstance(image, torch.Tensor) and image.ndim in (2, 3, 4):  # HW, CHW, BCHW
        return (image.shape[-1], image.shape[-2])
    elif isinstance(image, np.ndarray) and image.ndim in (2, 3):  # HW, HWC
        return (image.shape[1], image.shape[0])
    else:
        raise NotImplementedError
resize_image 
resize_image(image: Union[Image, Tensor, ndarray], target_size: ImageSizeType, resampling: str = 'bilinear', use_cv2_for_numpy: bool = True) -> Union[Image, Tensor, ndarray]

이미지를 resize합니다.

Parameters:

  • image (Union[Image, Tensor, ndarray]) –

    image data

  • target_size (ImageSizeType) –

    [W, H]

  • resampling (str, default: 'bilinear' ) –

    resampling method. Defaults to "bilinear".

  • use_cv2_for_numpy (bool, default: True ) –

    use cv2 instead of PIL for faster numpy array image resizing. Defaults to True.

Returns:

  • Union[Image, Tensor, ndarray] –

    Union[Image.Image, torch.Tensor, np.ndarray]: resized image

Source code in SaigeToolkit/data/transform/image_function.py 
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
def resize_image(
    image: Union[Image.Image, torch.Tensor, np.ndarray],
    target_size: ImageSizeType,
    resampling: str = "bilinear",
    use_cv2_for_numpy: bool = True,
) -> Union[Image.Image, torch.Tensor, np.ndarray]:
    """이미지를 resize합니다.

    Args:
        image (Union[Image.Image, torch.Tensor, np.ndarray]): image data
        target_size (ImageSizeType): [W, H]
        resampling (str): resampling method. Defaults to "bilinear".
        use_cv2_for_numpy (bool): use cv2 instead of PIL for faster numpy array image resizing. Defaults to True.

    Returns:
        Union[Image.Image, torch.Tensor, np.ndarray]: resized image
    """
    if isinstance(image, Image.Image):
        image = image.resize(target_size, RESAMPLE_PIL[resampling])
    elif isinstance(image, torch.Tensor):  # CHW, BCHW
        is_three_dim = image.ndim == 3
        if is_three_dim:  # CHW -> 1CHW
            image = image.unsqueeze(0)
        image = torch.nn.functional.interpolate(
            image,
            size=(target_size[1], target_size[0]),
            mode=RESAMPLE_TORCH[resampling],
            antialias=None if resampling == "nearest" else True,
        )
        if is_three_dim:
            image = image.squeeze(0)
    elif isinstance(image, np.ndarray):  # HWC, HW
        if use_cv2_for_numpy:
            # NOTE: cv2 resize is different from PIL/torch (inaccurate but fast)
            image = cv2.resize(image, dsize=target_size, interpolation=RESAMPLE_CV2[resampling])
        else:
            is_single_channel = image.ndim == 3 and image.shape[-1] == 1
            if is_single_channel:
                image = image[:, :, 0]
            image = Image.fromarray(image)
            image = image.resize(target_size, RESAMPLE_PIL[resampling])
            image = np.array(image)
            if is_single_channel:
                image = image[:, :, np.newaxis]
    return image
resize_mask 
resize_mask(mask: Union[Image, Tensor, ndarray], target_size: ImageSizeType) -> Union[Image, Tensor, ndarray]

mask 이미지를 resize합니다. (NEAREST resampling)

Parameters:

  • mask (Union[Image, Tensor, ndarray]) –

    mask image

  • target_size (ImageSizeType) –

    [W, H]

Returns:

  • Union[Image, Tensor, ndarray] –

    Union[Image.Image, torch.Tensor, np.ndarray]: resized mask image

Source code in SaigeToolkit/data/transform/image_function.py 
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
def resize_mask(
    mask: Union[Image.Image, torch.Tensor, np.ndarray],
    target_size: ImageSizeType,
) -> Union[Image.Image, torch.Tensor, np.ndarray]:
    """mask 이미지를 resize합니다. (NEAREST resampling)

    Args:
        mask (Union[Image.Image, torch.Tensor, np.ndarray]): mask image
        target_size (ImageSizeType): [W, H]

    Returns:
        Union[Image.Image, torch.Tensor, np.ndarray]: resized mask image
    """
    if isinstance(mask, Image.Image):
        mask = mask.resize(target_size, Image.Resampling.NEAREST)
    elif isinstance(mask, torch.Tensor):  # HW, BHW
        is_two_dim = mask.ndim == 2
        if is_two_dim:  # HW -> 1HW
            mask = mask.unsqueeze(0)
        mask = torch.nn.functional.interpolate(
            mask.unsqueeze(1),  # BHW -> B1HW
            size=(target_size[1], target_size[0]),
            mode="nearest-exact",
        ).squeeze(1)
        if is_two_dim:
            mask = mask.squeeze(0)
    elif isinstance(mask, np.ndarray):  # HW
        mask = Image.fromarray(mask)
        mask = mask.resize(target_size, Image.Resampling.NEAREST)
        mask = np.array(mask)
    return mask
resize_array 
resize_array(array: Union[Tensor, ndarray], target_size: ImageSizeType, resampling: str = 'nearest') -> Union[Tensor, ndarray]

[HW, BHW]의 array를 resize합니다.

Parameters:

  • array (Union[Tensor, ndarray]) –

    array data [HW, BHW]

  • target_size (ImageSizeType) –

    [W, H]

  • resampling (str, default: 'nearest' ) –

    resampling method. Defaults to "nearest".

Returns:

  • Union[Tensor, ndarray] –

    Union[torch.Tensor, np.ndarray]: resized array

Source code in SaigeToolkit/data/transform/image_function.py 
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
def resize_array(
    array: Union[torch.Tensor, np.ndarray],
    target_size: ImageSizeType,
    resampling: str = "nearest",
) -> Union[torch.Tensor, np.ndarray]:
    """[HW, BHW]의 array를 resize합니다.

    Args:
        array (Union[torch.Tensor, np.ndarray]): array data [HW, BHW]
        target_size (ImageSizeType): [W, H]
        resampling (str, optional): resampling method. Defaults to "nearest".

    Returns:
        Union[torch.Tensor, np.ndarray]: resized array
    """
    is_np_array = isinstance(array, np.ndarray)
    if is_np_array:  # np.ndarray -> torch.Tensor
        array = torch.tensor(array)

    is_two_dim = array.ndim == 2
    if is_two_dim:  # HW -> 1HW
        array = array.unsqueeze(0)

    array = torch.nn.functional.interpolate(
        array.unsqueeze(1),  # BHW -> B1HW
        size=(target_size[1], target_size[0]),
        mode=RESAMPLE_TORCH[resampling],
        antialias=None if resampling == "nearest" else True,
    ).squeeze(1)

    if is_two_dim:  # 1HW -> HW
        array = array.squeeze(0)

    if is_np_array:  # torch.Tensor -> np.ndarray
        array = array.numpy()

    return array
crop 
crop(image: Union[Image, Tensor, ndarray], coordinates: Union[List[int], Tuple[int, int, int, int]]) -> Union[Image, Tensor, ndarray]

이미지의 coordinates 좌표영역을 크롭합니다. coordinates가 이미지를 벗어나는 경우 zero padding 합니다.

Parameters:

  • image (Union[Image, Tensor, ndarray]) –

    image

  • coordinates (Union[List[int], Tuple[int, int, int, int]]) –

    [left, top, right, bottom]

Returns:

  • Union[Image, Tensor, ndarray] –

    Union[Image.Image, torch.Tensor, np.ndarray]: cropped image

Source code in SaigeToolkit/data/transform/image_function.py 
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
def crop(
    image: Union[Image.Image, torch.Tensor, np.ndarray],
    coordinates: Union[List[int], Tuple[int, int, int, int]],
) -> Union[Image.Image, torch.Tensor, np.ndarray]:
    """이미지의 coordinates 좌표영역을 크롭합니다. coordinates가 이미지를 벗어나는 경우 zero padding 합니다.

    Args:
        image (Union[Image.Image, torch.Tensor, np.ndarray]): image
        coordinates (Union[List[int], Tuple[int, int, int, int]]): [left, top, right, bottom]

    Returns:
        Union[Image.Image, torch.Tensor, np.ndarray]: cropped image
    """
    if isinstance(image, Image.Image):
        return image.crop(coordinates)
    else:
        left, top, right, bottom = coordinates
        image_w, image_h = read_image_size(image)
        cropped_shape = (bottom - top, right - left)
        if image.ndim == 3:
            cropped_shape = (*cropped_shape, image.shape[2])
        if isinstance(image, torch.Tensor):
            cropped_image = torch.zeros(*cropped_shape, dtype=image.dtype, device=image.device)
        else:
            cropped_image = np.zeros(cropped_shape, dtype=image.dtype)
        cropped_image[
            np.clip(0, top, bottom) - top : np.clip(image_h, top, bottom) - top,
            np.clip(0, left, right) - left : np.clip(image_w, left, right) - left,
        ] = image[
            np.clip(top, 0, image_h) : np.clip(bottom, 0, image_h),
            np.clip(left, 0, image_w) : np.clip(right, 0, image_w),
        ]
        return cropped_image
add_constant_margin 
add_constant_margin(image: Union[Image, Tensor, ndarray], left: int, top: int, right: int, bottom: int, value: Union[float, int]) -> Union[Image, Tensor, ndarray]

image에 left, top, right, bottom 만큼 constant value로 padding 합니다.

Parameters:

  • image (Union[Image, Tensor, ndarray]) –

    image

  • left (int) –

    image의 왼쪽 padding size 입니다.

  • top (int) –

    image의 위쪽 padding size 입니다.

  • right (int) –

    image의 오른쪽 padding size 입니다.

  • bottom (int) –

    image의 아래쪽 padding size 입니다.

  • value (Union[float, int]) –

    padding된 영역에 들어갈 value 입니다.

Source code in SaigeToolkit/data/transform/image_function.py 
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
def add_constant_margin(
    image: Union[Image.Image, torch.Tensor, np.ndarray],
    left: int,
    top: int,
    right: int,
    bottom: int,
    value: Union[float, int],
) -> Union[Image.Image, torch.Tensor, np.ndarray]:
    """image에 left, top, right, bottom 만큼 constant value로 padding 합니다.

    Args:
        image (Union[Image.Image, torch.Tensor, np.ndarray]): image
        left (int): image의 왼쪽 padding size 입니다.
        top (int): image의 위쪽 padding size 입니다.
        right (int): image의 오른쪽 padding size 입니다.
        bottom (int): image의 아래쪽 padding size 입니다.
        value (Union[float, int]): padding된 영역에 들어갈 value 입니다.
    """
    width, height = read_image_size(image)
    new_width = width + left + right
    new_height = height + top + bottom
    if isinstance(image, Image.Image):
        if image.mode == "RGB":
            value = [int(value)] * 3
        elif image.mode == "L":
            value = [int(value)]
        else:
            raise NotImplementedError
        result = Image.new(image.mode, (new_width, new_height), tuple(value))
        result.paste(image, (left, top))
    elif isinstance(image, np.ndarray):
        result = np.ones((new_height, new_width) + image.shape[2:]) * value
        result = result.astype(image.dtype)
        result[top : top + height, left : left + width] = image
    elif isinstance(image, torch.Tensor):
        result = torch.ones(image.shape[:-2] + (new_height, new_width), device=image.device) * value
        result = result.to(image.dtype)
        result[..., top : top + height, left : left + width] = image
    else:
        raise NotImplementedError

    return result
add_constant_margin_array 
add_constant_margin_array(array: Union[Tensor, ndarray], left: int, top: int, right: int, bottom: int, value: Union[float, int]) -> Union[Tensor, ndarray]

array에 left, top, right, bottom 만큼 constant value로 padding 합니다.

Parameters:

  • array (Union[Tensor, ndarray]) –

    array data [HW, BHW]

  • left (int) –

    array의 왼쪽 padding size 입니다.

  • top (int) –

    array의 위쪽 padding size 입니다.

  • right (int) –

    array의 오른쪽 padding size 입니다.

  • bottom (int) –

    array의 아래쪽 padding size 입니다.

  • value (Union[float, int]) –

    padding된 영역에 들어갈 value 입니다.

Source code in SaigeToolkit/data/transform/image_function.py 
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
def add_constant_margin_array(
    array: Union[torch.Tensor, np.ndarray],
    left: int,
    top: int,
    right: int,
    bottom: int,
    value: Union[float, int],
) -> Union[torch.Tensor, np.ndarray]:
    """array에 left, top, right, bottom 만큼 constant value로 padding 합니다.

    Args:
        array (Union[torch.Tensor, np.ndarray]): array data [HW, BHW]
        left (int): array의 왼쪽 padding size 입니다.
        top (int): array의 위쪽 padding size 입니다.
        right (int): array의 오른쪽 padding size 입니다.
        bottom (int): array의 아래쪽 padding size 입니다.
        value (Union[float, int]): padding된 영역에 들어갈 value 입니다.
    """
    if isinstance(array, np.ndarray):
        is_ndarray = True
        array = torch.from_numpy(array)
    else:
        is_ndarray = False

    result = torch.nn.functional.pad(array, (left, right, top, bottom), value=value)

    if is_ndarray:
        result = result.numpy(force=True)

    return result
fill_pixels_with_mask 
fill_pixels_with_mask(image: Union[Image, ndarray], bool_mask: ndarray, value: Union[int, float] = 0) -> Union[Image, ndarray]

image 중 bool_mask=True인 픽셀들을 value로 채웁니다.

Parameters:

  • image (Union[Image, ndarray]) –

    image (HW or HWC)

  • bool_mask (ndarray) –

    boolean mask (HW)

  • value (Union[int, float], default: 0 ) –

    . Defaults to 0.

Returns:

  • Union[Image, ndarray] –

    Union[Image.Image, np.ndarray]: 결과 이미지

Note
  • image가 Image.Image인 경우, 결과 이미지도 Image.Image로 반환합니다.
  • image의 값이 inplace로 변경됩니다. 값이 변경되지 않길 원한다면, copy를 해주세요.
Source code in SaigeToolkit/data/transform/image_function.py 
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
def fill_pixels_with_mask(
    image: Union[Image.Image, np.ndarray],
    bool_mask: np.ndarray,
    value: Union[int, float] = 0,
) -> Union[Image.Image, np.ndarray]:
    """image 중 bool_mask=True인 픽셀들을 value로 채웁니다.

    Args:
        image (Union[Image.Image, np.ndarray]): image (HW or HWC)
        bool_mask (np.ndarray): boolean mask (HW)
        value (Union[int, float], optional): . Defaults to 0.

    Returns:
        Union[Image.Image, np.ndarray]: 결과 이미지

    Note:
        - image가 Image.Image인 경우, 결과 이미지도 Image.Image로 반환합니다.
        - image의 값이 inplace로 변경됩니다. 값이 변경되지 않길 원한다면, copy를 해주세요.
    """
    is_pil = isinstance(image, Image.Image)
    image = np.asarray(image)

    assert image.ndim in [2, 3] and image.shape[:2] == bool_mask.shape  # HW or HWC

    if image.ndim > bool_mask.ndim:  # max ndim difference is 1
        bool_mask = np.expand_dims(bool_mask, axis=-1)

    image = np.where(bool_mask, value, image)

    if is_pil:
        image = Image.fromarray(image)

    return image
convert_image_mode 
convert_image_mode(image: Union[Image, ndarray], mode: str, copy: bool = False) -> Union[Image, ndarray]

convert image mode

Parameters:

  • image (Union[Image, ndarray]) –

    image

  • mode (str) –

    "RGB" or "L"

  • copy (bool, default: False ) –

    to copy data. Defaults to False.

Returns:

  • Union[Image, ndarray] –

    Union[Image.Image, np.ndarray]: converted image

Note

RGB -> L 변환 시 Image.Image와 np.ndarray 연산 결과가 다를 수 있음. (PIL과 cv2 에서 변환식은 L = R * 299/1000 + G * 587/1000 + B * 114/1000 로 동일하나 소숫점 처리 방식이 다름)

Source code in SaigeToolkit/data/transform/image_function.py 
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
def convert_image_mode(
    image: Union[Image.Image, np.ndarray], mode: str, copy: bool = False
) -> Union[Image.Image, np.ndarray]:
    """convert image mode

    Args:
        image (Union[Image.Image, np.ndarray]): image
        mode (str): "RGB" or "L"
        copy (bool, optional): to copy data. Defaults to False.

    Returns:
        Union[Image.Image, np.ndarray]: converted image

    Note:
        RGB -> L 변환 시 Image.Image와 np.ndarray 연산 결과가 다를 수 있음.
        (PIL과 cv2 에서 변환식은 L = R * 299/1000 + G * 587/1000 + B * 114/1000 로 동일하나 소숫점 처리 방식이 다름)
    """
    if mode not in IMAGE_MODES:
        raise NotImplementedError

    if isinstance(image, Image.Image):
        if image.mode != mode:
            image = image.convert(mode)
        elif copy:
            image = image.copy()
    elif isinstance(image, np.ndarray):
        # check original mode & regularize (RGB: HW3 / L: HW)
        if image.ndim == 3:
            if image.shape[2] == 3:
                original_mode = "RGB"
            elif image.shape[2] == 1:
                original_mode = "L"
                image = image[:, :, 0]  # HW1 -> HW (no copy)
            elif image.shape[2] == 4:
                original_mode = "RGB"
                image = image[:, :, :3]  # RGBA -> RGB (no copy)
            else:
                raise NotImplementedError
        elif image.ndim == 2:
            original_mode = "L"
        else:
            raise NotImplementedError
        # convert
        if original_mode == "RGB" and mode == "L":
            image = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)
        elif original_mode == "L" and mode == "RGB":
            image = cv2.cvtColor(image, cv2.COLOR_GRAY2RGB)
        elif copy:
            image = image.copy()
    else:
        raise NotImplementedError
    return image

image_load

ImageLoader 
ImageLoader(image_mode: Union[str, List[str]] = 'RGB', to_numpy: bool = True)

여러 형태의 데이터를 입력으로 받아, 전처리 과정을 거쳐 PIL Image 혹은 np.ndarray 로 return 합니다. 현재 지원하는 데이터는 다음과 같습니다. - 데이터 타입: [image path, np.ndarray, PIL Image] - color: ["RGB", "Gray"] - bit: [8, 16]

입력으로 받은 데이터에 따른 출력값은 다음과 같습니다. | image path | np.ndarray | PIL | RGB; 8 | PIL(RGB;8) | PIL(RGB;8) | PIL(RGB;8) | RGB;16 | PIL(RGB;8) | PIL(RGB;8) | - | Gray; 8 | PIL(L;8) | PIL(L;8) | PIL(L;8) | Gray;16 | PIL(L;8) | PIL(L;8) | PIL(L;8) |

Note1

PIL은 RGB;16을 지원하지 않습니다. 따라서 PIL(RGB;16)은 입력으로 들어올 수 없습니다.

Note2

PIL은 "I;16" 모드로 Gray;16을 지원하지만, PIL 내부의 convert 함수를 써서 Gray;8로 변환시 값이 overflow 나는 issue가 있습니다.

Note3

이미지는 기본적으로 np.ndarray로 로드되며, PIL.Image.Image로 로드하려면 to_numpy=False를 세팅하세요.

Note4

image_mode는 "RGB", "L" 중 하나를 지원하며, multipage (이미지 리스트) 인 경우 각 페이지의 모드를 리스트로 입력하세요. 예시: 1, 3번째 페이지는 RGB 이고 2번째 페이지는 L 인 경우 image_mode = ["RGB", "L", "RGB"]

Source code in SaigeToolkit/data/transform/image_load.py 
41
42
43
44
45
46
47
def __init__(self, image_mode: Union[str, List[str]] = "RGB", to_numpy: bool = True) -> None:
    if isinstance(image_mode, str):
        image_mode = [image_mode]
    elif not set(image_mode).issubset(set(IMAGE_MODES)):
        raise ValueError
    self.image_mode = image_mode
    self.to_numpy = to_numpy
call_method 
call_method(image: Union[Image, ndarray, str, List], **data) -> Dict

make [PIL Image or np.ndarray] from PIL.Image, np.ndarray, or path string

Parameters:

  • image (Union[Image, ndarray, str, List]) –

    PIL.Image, array, path string or list of it.

Returns:

  • Dict ( Dict ) –

    { "image": Union[PIL.Image.Image, np.ndarray, List[PIL.Image.Image], List[np.ndarray]], **input_dict, }

Source code in SaigeToolkit/data/transform/image_load.py 
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
def call_method(self, image: Union[Image.Image, np.ndarray, str, List], **data) -> Dict:
    """make [PIL Image or np.ndarray] from PIL.Image, np.ndarray, or path string

    Args:
        image (Union[Image.Image, np.ndarray, str, List]): PIL.Image, array, path string or list of it.

    Returns:
        Dict:
            {
                "image": Union[PIL.Image.Image, np.ndarray, List[PIL.Image.Image], List[np.ndarray]],
                **input_dict,
            }
    """
    multipage = isinstance(image, List)
    if not multipage:
        image = [image]

    if len(image) != len(self.image_mode):
        raise ValueError("number of images should match len(image_mode)")

    # 이미지들을 각자의 모드로 로드 (RGB 또는 L).
    loaded_images = []
    for image_i, image_mode_i in zip(image, self.image_mode):
        image_i = self.load(image_i)
        image_i = convert_image_mode(image=image_i, mode=image_mode_i, copy=False)
        loaded_images.append(image_i)

    if multipage:
        # multipage 이미지들의 사이즈가 모두 동일해야함.
        image_sizes = set(read_image_size(item) for item in loaded_images)
        if len(image_sizes) > 1:
            raise ValueError("sizef of images in multipage should be identical")
    else:
        loaded_images = loaded_images[0]

    data["image"] = loaded_images
    return data
load_from_path 
load_from_path(path: str) -> Union[Image, ndarray]

Loading function using PIL.Image library. This function is introduced because {exif_transpose} must be processed. {exif_transpose} fix rotated image binary data using {EXIF} information. Without {exif_transpose}, network would accidently learn randomly rotated image data.

Parameters:

  • path (str) –

    path to image file

Returns:

  • Union[Image, ndarray] –

    Union[Image.Image, np.ndarray]: image

Source code in SaigeToolkit/data/transform/image_load.py 
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
def load_from_path(self, path: str) -> Union[Image.Image, np.ndarray]:
    """Loading function using PIL.Image library.
    This function is introduced because {exif_transpose} must be processed.
    {exif_transpose} fix rotated image binary data using {EXIF} information.
    Without {exif_transpose}, network would accidently learn randomly rotated image data.

    Args:
        path (str): path to image file

    Returns:
        Union[Image.Image, np.ndarray]: image
    """
    with open(path, "rb") as f:
        image = Image.open(f)
        image = ImageOps.exif_transpose(image)

    if self.to_numpy:
        image = np.array(image, dtype=np.uint16 if self._is_16bit(image) else np.uint8)

    if self._is_16bit(image):
        image = self._convert_16_to_8(image)

    return image
_convert_16_to_8 staticmethod 
_convert_16_to_8(image: Union[Image, ndarray]) -> Union[Image, ndarray]

이미지 픽셀당 비트수를 16에서 8로 변화합니다. 입출력 데이터 타입이 같습니다. (pil로 받으면 pil을 ndarray로 받을 시 ndarray를 출력합니다.)

Source code in SaigeToolkit/data/transform/image_load.py 
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
@staticmethod
def _convert_16_to_8(image: Union[Image.Image, np.ndarray]) -> Union[Image.Image, np.ndarray]:
    """
    이미지 픽셀당 비트수를 16에서 8로 변화합니다.
    입출력 데이터 타입이 같습니다. (pil로 받으면 pil을 ndarray로 받을 시 ndarray를 출력합니다.)
    """

    def _convert_16_to_8_np(array: np.ndarray) -> np.ndarray:
        return (array >> 8).astype(np.uint8)

    if isinstance(image, Image.Image):
        assert "I" in image.mode
        array_16bit = np.array(image, dtype=np.uint16)
        array_8bit = _convert_16_to_8_np(array_16bit)
        ret_image = Image.fromarray(array_8bit)
    elif isinstance(image, np.ndarray):
        assert image.dtype == np.uint16
        ret_image = _convert_16_to_8_np(image)
    else:
        raise NotImplementedError

    return ret_image

polygon_function

resize_polygon 
resize_polygon(polygons: PolygonType, image_size: Tuple[int], tw: int, th: int) -> PolygonType

Resize polygon from (w, h) to (tw, th)

Source code in SaigeToolkit/data/transform/polygon_function.py 
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
def resize_polygon(polygons: PolygonType, image_size: Tuple[int], tw: int, th: int) -> PolygonType:
    """Resize polygon from (w, h) to (tw, th)"""
    if len(polygons) == 0:
        return polygons

    w, h = image_size
    if w == tw and h == th:
        return polygons

    dtype = polygons[0].dtype

    w_scale = tw / w
    h_scale = th / h

    new_polygons = deepcopy(polygons)

    for polygon in new_polygons:
        polygon[:, 0] = polygon[:, 0] * w_scale
        polygon[:, 1] = polygon[:, 1] * h_scale
        polygon = polygon.astype(dtype)

    return new_polygons
translate_polygon 
translate_polygon(polygons: PolygonType, offset: Tuple[int]) -> PolygonType

Translate polyfon from [(x1, y1), ... ] to [(x1 - x_offset), (y1 - y_offset), ...]

Source code in SaigeToolkit/data/transform/polygon_function.py 
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
def translate_polygon(
    polygons: PolygonType,
    offset: Tuple[int],
) -> PolygonType:
    """Translate polyfon from [(x1, y1), ... ] to [(x1 - x_offset), (y1 - y_offset), ...]"""
    if len(polygons) == 0:
        return polygons

    if offset[0] == 0 and offset[1] == 0:
        return polygons

    dtype = polygons[0].dtype

    x_offset, y_offset = np.array(offset).astype(dtype)

    new_polygons = deepcopy(polygons)

    for polygon in new_polygons:
        polygon[:, 0] -= x_offset
        polygon[:, 1] -= y_offset

    return new_polygons

resize

Resizer 
Resizer(size: Optional[Union[ImageSizeType, int]] = None, scale: Optional[Union[int, float]] = None, area_sqrt: Optional[Union[int, float]] = None, max_size: Optional[Union[ImageSizeType, int]] = None, round: Optional[int] = None, round_type: str = 'round', resampling: str = 'bilinear', image_only: bool = False, use_cv2_for_numpy: bool = True)

Module for resizing PIL, torch, numpy images Resizer의 작동 방식은 다음과 같습니다. 1. target size 계산 2. target size 미세 조정 3. 데이터에 resize 적용

Args에 따라 1, 2번의 작동 방식이 달라집니다. 1. target size 계산 - resize 될 target size를 계산합니다. - target size는 4개의 args에 영향을 받을 수 있습니다. (size, scale, area_sqrt, max_size) 하나의 Resizer는 4개중 하나의 args만 사용할 수 있으며, 2개 이상의 args가 None이 아닐시 error를 raise 합니다. 1-1. size: image를 size로 resize 합니다. aspect ratio가 변경될 수 있습니다. 1-2. scale: image를 scale배로 늘리거나 줄입니다. aspect ratio는 유지됩니다. 1-3. area_sqrt: image의 면적이 area_sqrt^2이 되도록 이미지를 늘리거나 줄입니다. aspect ratio는 유지됩니다. 1-4. max_size: image의 size가 max_size보다 클 경우 max_size로 resize 합니다. aspect ratio를 유지하기 위해, width/height중 더 많이 줄어야 하는 비율에 맞추어 전체를 resize합니다.

  1. target size 미세 조정
  2. resize된 이미지가 특정 값의 배수가 되도록 target size를 미세 조정 합니다. model의 입력으로 넣을 때, image size가 특정 값의 배수가 되어야 하기 때문에 필요합니다.
    1. target size 계산의 size parameter와 함께 사용할 수 없습니다. (고정 size이기 때문)
  4. target size 미세 조정은 총 2개의 args에 영향을 받을 수 있습니다. (round, round_type) round가 None이면 미세 조정을 하지 않습니다, round_type에 따라 다른 방식의 미세 조정을 적용합니다. 2-1. round: 미세 조정시 반올림합니다. (target size보다 작거나 크거나 같습니다.) 2-2. floor: 미세 조정시 내림합니다. (target size보다 작거나 같습니다.) 2-3. ceil: 미세 조정시 올림합니다. (target size보다 크거나 같습니다.)
  5. 미세 조정된 target_size의 width혹은 height가 0이 될 경우, round 값으로 변경해줍니다. (ex. size=(3, 3), round=8이면 round_type에 상관없이 size=(8, 8)이 됨.)

Parameters:

  • size (Optional[Union[ImageSizeType, int]], default: None ) –

    target image size. Defaults to None.

  • scale (Optional[Union[int, float]], default: None ) –

    target image scale. Defaults to None.

  • area_sqrt (Optional[Union[int, float]], default: None ) –

    target image sqrt area. Defaults to None.

  • max_size (Optional[Union[ImageSizeType, int]], default: None ) –

    target maximum image size. Defaults to None.

  • round (Optional[int], default: None ) –

    round image size. Defaults to None.

  • round_type (Optional[str], default: 'round' ) –

    type of round image. One of ["round", "floor", "ceil"]. Defaults to "round".

  • resampling (str, default: 'bilinear' ) –

    One of ["nearest", "bilinear", "bicubic"]. Defaults to "bilinear".

  • image_only (bool, default: False ) –

    to resize image only. Defaults to False.

  • use_cv2_for_numpy (bool, default: True ) –

    use cv2 instead of PIL for faster numpy array image resizing. Defaults to True.

Raises:

  • ResizerParameterValueError –

    target size parameter가 2개 이상 들어오면 error를 raise 합니다. round_type이 ["round", "floor", "ceil"]안에 없으면 error를 raise 합니다. size와 미세조정 parameter가 함께 들어오면 error를 raise 합니다.

Source code in SaigeToolkit/data/transform/resize.py 
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
def __init__(
    self,
    size: Optional[Union[ImageSizeType, int]] = None,
    scale: Optional[Union[int, float]] = None,
    area_sqrt: Optional[Union[int, float]] = None,
    max_size: Optional[Union[ImageSizeType, int]] = None,
    round: Optional[int] = None,
    round_type: str = "round",
    resampling: str = "bilinear",
    image_only: bool = False,
    use_cv2_for_numpy: bool = True,
) -> None:
    """Module for resizing PIL, torch, numpy images
    Resizer의 작동 방식은 다음과 같습니다.
    1. target size 계산
    2. target size 미세 조정
    3. 데이터에 resize 적용

    Args에 따라 1, 2번의 작동 방식이 달라집니다.
    1. target size 계산
    - resize 될 target size를 계산합니다.
    - target size는 4개의 args에 영향을 받을 수 있습니다. (size, scale, area_sqrt, max_size)
      하나의 Resizer는 4개중 하나의 args만 사용할 수 있으며, 2개 이상의 args가 None이 아닐시 error를 raise 합니다.
        1-1. size: image를 size로 resize 합니다. aspect ratio가 변경될 수 있습니다.
        1-2. scale: image를 scale배로 늘리거나 줄입니다. aspect ratio는 유지됩니다.
        1-3. area_sqrt: image의 면적이 area_sqrt^2이 되도록 이미지를 늘리거나 줄입니다. aspect ratio는 유지됩니다.
        1-4. max_size: image의 size가 max_size보다 클 경우 max_size로 resize 합니다.
                        aspect ratio를 유지하기 위해, width/height중 더 많이 줄어야 하는 비율에 맞추어 전체를 resize합니다.

    2. target size 미세 조정
    - resize된 이미지가 특정 값의 배수가 되도록 target size를 미세 조정 합니다.
      model의 입력으로 넣을 때, image size가 특정 값의 배수가 되어야 하기 때문에 필요합니다.
    - 1. target size 계산의 size parameter와 함께 사용할 수 없습니다. (고정 size이기 때문)
    - target size 미세 조정은 총 2개의 args에 영향을 받을 수 있습니다. (round, round_type)
      round가 None이면 미세 조정을 하지 않습니다, round_type에 따라 다른 방식의 미세 조정을 적용합니다.
        2-1. round: 미세 조정시 반올림합니다. (target size보다 작거나 크거나 같습니다.)
        2-2. floor: 미세 조정시 내림합니다. (target size보다 작거나 같습니다.)
        2-3. ceil: 미세 조정시 올림합니다. (target size보다 크거나 같습니다.)
    - 미세 조정된 target_size의 width혹은 height가 0이 될 경우, round 값으로 변경해줍니다.
      (ex. size=(3, 3), round=8이면 round_type에 상관없이 size=(8, 8)이 됨.)

    Args:
        size (Optional[Union[ImageSizeType, int]], optional): target image size. Defaults to None.
        scale (Optional[Union[int, float]], optional): target image scale. Defaults to None.
        area_sqrt (Optional[Union[int, float]], optional): target image sqrt area. Defaults to None.
        max_size (Optional[Union[ImageSizeType, int]], optional): target maximum image size. Defaults to None.
        round (Optional[int], optional): round image size. Defaults to None.
        round_type (Optional[str], optional): type of round image. One of ["round", "floor", "ceil"]. Defaults to "round".
        resampling (str): One of ["nearest", "bilinear", "bicubic"]. Defaults to "bilinear".
        image_only (bool): to resize image only. Defaults to False.
        use_cv2_for_numpy (bool): use cv2 instead of PIL for faster numpy array image resizing. Defaults to True.

    Raises:
        ResizerParameterValueError: target size parameter가 2개 이상 들어오면 error를 raise 합니다.
                                    round_type이 ["round", "floor", "ceil"]안에 없으면 error를 raise 합니다.
                                    size와 미세조정 parameter가 함께 들어오면 error를 raise 합니다.
    """

    self._check_parameter(
        size=size,
        scale=scale,
        area_sqrt=area_sqrt,
        max_size=max_size,
        round=round,
        round_type=round_type,
    )

    size = self._preprocess_size(size)
    max_size = self._preprocess_size(max_size)

    if scale == 1:
        scale = None

    self.size = size
    self.scale = scale
    self.area_sqrt = area_sqrt
    self.max_size = max_size
    self.round = round
    self.round_type = round_type

    if resampling not in RESAMPLE_PIL or resampling not in RESAMPLE_TORCH:
        raise ValueError
    self.resampling = resampling

    self.image_only = image_only
    self.use_cv2_for_numpy = use_cv2_for_numpy
compute_input_size staticmethod 
compute_input_size(data: Dict) -> ImageSizeType

target size를 계산하기 위해 input size를 가져옵니다. Note: data에 image 혹은 mask data는 있다고 가정하며, 둘의 size는 같다고 가정합니다.

Source code in SaigeToolkit/data/transform/resize.py 
135
136
137
138
139
140
141
142
143
144
145
146
147
148
@staticmethod
def compute_input_size(data: Dict) -> ImageSizeType:
    """target size를 계산하기 위해 input size를 가져옵니다.
    Note: data에 image 혹은 mask data는 있다고 가정하며, 둘의 size는 같다고 가정합니다.
    """
    if "image" in data:
        multipage = isinstance(data["image"], List)
        input_size = read_image_size(data["image"][0] if multipage else data["image"])
    elif "mask" in data:
        input_size = read_image_size(data["mask"])
    else:
        raise NotImplementedError

    return input_size
compute_target_size 
compute_target_size(input_size: Union[ImageSizeType, ndarray]) -> ImageSizeType

compute resize target size from input image size

Parameters:

Returns:

Source code in SaigeToolkit/data/transform/resize.py 
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
def compute_target_size(self, input_size: Union[ImageSizeType, np.ndarray]) -> ImageSizeType:
    """compute resize target size from input image size

    Args:
        input_size (Union[ImageSizeType, np.ndarray]): (width, height)

    Returns:
        ImageSizeType: (width, height)
    """
    return self._compute_target_size(
        input_size=input_size,
        size=self.size,
        scale=self.scale,
        area_sqrt=self.area_sqrt,
        max_size=self.max_size,
        round=self.round,
        round_type=self.round_type,
    )
InspectionSizeResizer 
InspectionSizeResizer(inspection_size_wh: Optional[ImageSizeType] = None, resizer: Optional[Resizer] = None)

Bases: Resizer

원본 이미지가 inspection_size를 넘지 않도록 resize 했을 때의 image scale만큼 resize를 해주는 resizer를 생성하는 class 입니다.

inspection_size는 원본 이미지에 대해 적용하지만, 실제 연산은 ROI가 먼저 계산되기 때문에, 원본 이미지가 inspection_size를 넘지 않도록 resize 했을 때의 image scale을 미리 계산해두고, 해당 scale만큼 resize를 하는 resizer를 생성하여 계산합니다.

Source code in SaigeToolkit/data/transform/resize.py 
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
def __init__(
    self,
    inspection_size_wh: Optional[ImageSizeType] = None,
    resizer: Optional[Resizer] = None,
) -> None:
    self._check_inspection_size_type(inspection_size_wh)
    self._check_inspection_size_value(inspection_size_wh)

    initial_params = {}
    if resizer is not None:
        initial_params["resampling"] = resizer.resampling
        initial_params["image_only"] = resizer.image_only

    super().__init__(**initial_params)

    self.inspection_size_wh = self._preprocess_size(inspection_size_wh)
StackedResizerHandler 
StackedResizerHandler(resizer_list: List[Optional[Resizer]])

같은 image에 대해 여러번의 resize가 연속적으로 적용될 때, 여러개의 resize를 하나로 묶어서 최종 target size로 한번에 resize를 해주는 class입니다.

Source code in SaigeToolkit/data/transform/resize.py 
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
def __init__(
    self,
    resizer_list: List[Optional[Resizer]],
) -> None:
    self.resizer_list = resizer_list
    self.is_empty = len(resizer_list) == 0 or all(x is None for x in resizer_list)

    if not self.is_empty:
        resampling = set()
        image_only = set()
        use_cv2_for_numpy = set()
        for resizer in resizer_list:
            if resizer is not None:
                resampling.add(resizer.resampling)
                image_only.add(resizer.image_only)
                use_cv2_for_numpy.add(resizer.use_cv2_for_numpy)

        if len(resampling) > 1 or len(image_only) > 1 or len(use_cv2_for_numpy) > 1:
            raise StackedResizerHandlerParameterValueError

        self.resampling = resampling.pop()
        self.image_only = image_only.pop()
        self.use_cv2_for_numpy = use_cv2_for_numpy.pop()

roi

api
ROIHandlerAPI 
ROIHandlerAPI(**kwargs)

Set-ROI 기능을 위한 API입니다.

Usage 
# 핸들러 빌드. 아래는 simple mode의 예제 config이며, 자세한 설명은 ROIHandlerAPI.set() 함수 참고.
config = {
    "mode": "simple",
    "left": 0.0,
    "top": 0.0,
    "right": 1.0,
    "bottom": 1.0,
    "blind_mask": None,
}
error, roi_hander = ROIHandlerAPI.build(config)

# image에 대한 roi 계산. 리턴 결과 설명은 ROIHandlerAPI.apply() 함수 참고.
image = np.zeros((100, 100, 3), dtype=np.unit8)
error, roi_results = roi_handler.apply(image)

# roi 파라미터 변경
config["left"] = 0.1
error, _  = roi_handler.set(config)

# 변경된 파라미터로 roi 다시 계산
error, roi_results = roi_handler.apply(image)
Source code in SaigeToolkit/data/transform/roi/api.py 
44
45
def __init__(self, **kwargs) -> None:
    self.handler = ROIHandler(**kwargs)
build classmethod 
build(config: Dict) -> ROIHandlerAPI

ROIHandlerAPI를 빌드합니다. Args: config (Dict): ROIHandlerAPI.set의 파라미터와 동일합니다.

Returns:

Source code in SaigeToolkit/data/transform/roi/api.py 
47
48
49
50
51
52
53
54
55
56
57
@classmethod
@error_handler
def build(cls, config: Dict) -> ROIHandlerAPI:
    """ROIHandlerAPI를 빌드합니다.
    Args:
        config (Dict): ROIHandlerAPI.set의 파라미터와 동일합니다.

    Returns:
        ROIHandlerAPI: 빌드된 ROIHandlerAPI
    """
    return cls(**config)
set 
set(config: Dict) -> None

ROI 파라미터를 변경합니다.

Parameters:

  • config (Dict) –

    ROI 파라미터의 dict입니다. mode에 따라 다른 파라미터를 가집니다. 

    # simple mode:
{
    "mode": "simple",  # Simple ROI 모드.
    "left": float,  # ROI의 왼쪽 경계. [0.0, 1.0) 범위의 실수.
    "top": float,  # ROI의 위쪽 경계. [0.0, 1.0) 범위의 실수.
    "right": float,  # ROI의 오른쪽 경계. (right, 1.0] 범위의 실수.
    "bottom": float,  # ROI의 아래쪽 경계. (top, 1.0] 범위의 실수.
    "blind_mask": Optional[np.ndarray],  # blind mask 이미지. None인 경우 blind 적용 안함.
                                         # np.ndarray인 경우 (dtype=uint8, shape=(H_roi, W_roi))이며 픽셀 값은 0 또는 1.
                                         # mask의 값이 1인 영역이 학습/검사 시 마스킹됩니다.
                                         # polygons에는 blind_mask가 적용되지 않습니다.
}
# advanced mode:
{
    "mode": "advanced",  # Advanced ROI 모드.
    "intensity": List[int],  # 필터링할 [최소, 최대] 픽셀값 범위. 각 값은 [0, 255] 범위의 정수.
    "expansion": int,  # 필터링된 픽셀 영역에 대한 확장/축소 정도. [-10, 10] 범위의 정수.
    "inversion": bool,  # True인 경우 필터링된 픽셀 영역을 반전.
    "offset_left": float,  # ROI 박스의 왼쪽 사이즈. [0.0, 2.0] 범위의 실수 이며, 1인 경우 기본 크기.
    "offset_right": float,  # ROI 박스의 오른쪽 사이즈. [0.0, 2.0] 범위의 실수 이며, 1인 경우 기본 크기.
    "offset_top": float,  # ROI 박스의 위쪽 사이즈. [0.0, 2.0] 범위의 실수 이며, 1인 경우 기본 크기.
    "offset_bottom": float,  # ROI 박스의 아래쪽 사이즈. [0.0, 2.0] 범위의 실수 이며, 1인 경우 기본 크기.
    "blind_mask": Optional[np.ndarray],  # blind mask 이미지. None인 경우 blind 적용 안함.
                                         # np.ndarray인 경우 (dtype=uint8, shape=(H_roi, W_roi))이며 픽셀 값은 0 또는 1.
                                         # mask의 값이 1인 영역이 학습/검사 시 마스킹됩니다.
                                         # polygons에는 blind_mask가 적용되지 않습니다.
}

Source code in SaigeToolkit/data/transform/roi/api.py 
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
@error_handler
def set(self, config: Dict) -> None:
    """ROI 파라미터를 변경합니다.

    Args:
        config (Dict): ROI 파라미터의 dict입니다. mode에 따라 다른 파라미터를 가집니다.
            ```python
            # simple mode:
            {
                "mode": "simple",  # Simple ROI 모드.
                "left": float,  # ROI의 왼쪽 경계. [0.0, 1.0) 범위의 실수.
                "top": float,  # ROI의 위쪽 경계. [0.0, 1.0) 범위의 실수.
                "right": float,  # ROI의 오른쪽 경계. (right, 1.0] 범위의 실수.
                "bottom": float,  # ROI의 아래쪽 경계. (top, 1.0] 범위의 실수.
                "blind_mask": Optional[np.ndarray],  # blind mask 이미지. None인 경우 blind 적용 안함.
                                                     # np.ndarray인 경우 (dtype=uint8, shape=(H_roi, W_roi))이며 픽셀 값은 0 또는 1.
                                                     # mask의 값이 1인 영역이 학습/검사 시 마스킹됩니다.
                                                     # polygons에는 blind_mask가 적용되지 않습니다.
            }
            # advanced mode:
            {
                "mode": "advanced",  # Advanced ROI 모드.
                "intensity": List[int],  # 필터링할 [최소, 최대] 픽셀값 범위. 각 값은 [0, 255] 범위의 정수.
                "expansion": int,  # 필터링된 픽셀 영역에 대한 확장/축소 정도. [-10, 10] 범위의 정수.
                "inversion": bool,  # True인 경우 필터링된 픽셀 영역을 반전.
                "offset_left": float,  # ROI 박스의 왼쪽 사이즈. [0.0, 2.0] 범위의 실수 이며, 1인 경우 기본 크기.
                "offset_right": float,  # ROI 박스의 오른쪽 사이즈. [0.0, 2.0] 범위의 실수 이며, 1인 경우 기본 크기.
                "offset_top": float,  # ROI 박스의 위쪽 사이즈. [0.0, 2.0] 범위의 실수 이며, 1인 경우 기본 크기.
                "offset_bottom": float,  # ROI 박스의 아래쪽 사이즈. [0.0, 2.0] 범위의 실수 이며, 1인 경우 기본 크기.
                "blind_mask": Optional[np.ndarray],  # blind mask 이미지. None인 경우 blind 적용 안함.
                                                     # np.ndarray인 경우 (dtype=uint8, shape=(H_roi, W_roi))이며 픽셀 값은 0 또는 1.
                                                     # mask의 값이 1인 영역이 학습/검사 시 마스킹됩니다.
                                                     # polygons에는 blind_mask가 적용되지 않습니다.
            }
            ```
    """
    self.handler.set(**config)
apply 
apply(image: ndarray) -> Dict

image에 대한 ROI 좌표 및 기타 결과를 계산합니다.

Parameters:

  • image (ndarray) –

    ROI를 적용할 입력 이미지.

Returns:

  • Dict ( Dict ) –

    ROI 계산 결과 dict 입니다. mode에 따라 다른 결과 값들을 가집니다. 

    # simple mode:
{
    "roi_coordinates": List[int],  # [left, top, right, bottom].
}
# advanced mode:
{
    "roi_coordinates": List[int],  # [left, top, right, bottom].
    "filtered_image": np.ndarray(uint8, shape=(H, W)),  # intensity, expansion, inversion이 적용된 중간 결과 이미지 입니다. (픽셀값: 0 or 1)
}

Source code in SaigeToolkit/data/transform/roi/api.py 
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
@error_handler
def apply(self, image: np.ndarray) -> Dict:
    """`image`에 대한 ROI 좌표 및 기타 결과를 계산합니다.

    Args:
        image (np.ndarray): ROI를 적용할 입력 이미지.

    Returns:
        Dict: ROI 계산 결과 dict 입니다. mode에 따라 다른 결과 값들을 가집니다.
            ```python
            # simple mode:
            {
                "roi_coordinates": List[int],  # [left, top, right, bottom].
            }
            # advanced mode:
            {
                "roi_coordinates": List[int],  # [left, top, right, bottom].
                "filtered_image": np.ndarray(uint8, shape=(H, W)),  # intensity, expansion, inversion이 적용된 중간 결과 이미지 입니다. (픽셀값: 0 or 1)
            }
            ```
    """
    return self.handler.roi_calculator(image=image, get_intermediate_results=True)
roi_calculator
ROICalculator

Bases: ABC

ROI 좌표 계산을 위한 추상 클래스입니다.

RelativeBoxROI 
RelativeBoxROI(**kwargs)

Bases: ROICalculator

이미지 크기에 비례하는 상대좌표 박스로 ROI를 계산합니다.

Source code in SaigeToolkit/data/transform/roi/roi_calculator.py 
43
44
def __init__(self, **kwargs) -> None:
    self.set(**kwargs)
PixelIntensityROI 
PixelIntensityROI(**kwargs)

Bases: ROICalculator

픽셀값이 intensity 범위에 들어오는 픽셀만 필터링한 뒤, 필터링 된 픽셀들의 컨투어를 찾고, 가장 면적이 큰 컨투어를 감싸는 최소 박스로 ROI를 계산합니다.

Source code in SaigeToolkit/data/transform/roi/roi_calculator.py 
74
75
def __init__(self, **kwargs) -> None:
    self.set(**kwargs)
AutoRelativeBoxROI 
AutoRelativeBoxROI(padding: str = 'medium')

Bases: RelativeBoxROI

RelativeBoxROI class with automatic coordinate calculation.

Monostate pattern applied to prevent ROI coordinate mismatch between train ~ validation dataset.

Attributes:

  • left (float) –

    ROI coordinates.

  • top (float) –

    ROI coordinates.

  • right (float) –

    ROI coordinates.

  • bottom (float) –

    ROI coordinates.

  • is_ready (bol) –

    Whether auto ROI coordinates is set.

  • image_hw (Optional[List[int]]) –

    dataset image size.

  • discard_outer_polygons (bool) –

    Flag for discarding polygons outside of current ROI region.

  • expand_ratio –

    Ratio for expanding ROI region. Larger value means more padding.

Example
in dataset building...

@property def files(self): return self._files

@files.setter def files(self, value): self._files = value

if isinstance(self.transform.roi_handler.roi_calculator, AutoRelativeBoxROI):
    self.transform.roi_handler.roi_calculator.autoupdate_roi_coordinate(self)
                                                                        self is dataset

```

Source code in SaigeToolkit/data/transform/roi/roi_calculator.py 
228
229
230
231
232
233
234
def __init__(self, padding: str = "medium"):
    self.__dict__ = self.__shared_state

    if padding not in self.REGION_EXPAND_RATIO.keys():
        raise AutoROIParameterValueError(f"AutoROI mode {padding} not available.")

    self.expand_ratio = self.REGION_EXPAND_RATIO[padding]
autoupdate_roi_coordinate 
autoupdate_roi_coordinate(dataset: Dataset) -> bool

Automatically update ROI coordinate using dataset information.

Returns success flag.

Source code in SaigeToolkit/data/transform/roi/roi_calculator.py 
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
def autoupdate_roi_coordinate(self, dataset: Dataset) -> bool:
    """Automatically update ROI coordinate using dataset information.

    Returns success flag.
    """

    # XXX: only called once
    if self.is_ready:
        logger.info("ROI coordinate already set. Update only once")
        return True

    marginal_label_region_xyxy = np.array([np.inf, np.inf, 0, 0])

    for data_idx in range(len(dataset)):
        file = dataset.files[data_idx]
        data_dict = dataset.load_image(file)
        data_dict.update(dataset.load_label(file))

        data_image: Union[Image.Image, np.ndarray] = data_dict["image"]
        current_img_hw = list(read_image_size(data_image))[::-1]

        if self.image_hw:
            if self.image_hw != current_img_hw:
                logger.warn("Variation in image size detected.. autoROI update aborted.")
                logger.warn(
                    f"Expected image size: {self.image_hw} / Given image size: {current_img_hw}"
                )
                return False

        self.image_hw = current_img_hw

        for polygon in data_dict["polygons"]:
            x_min, x_max = np.min(polygon[:, 0]), np.max(polygon[:, 0])
            y_min, y_max = np.min(polygon[:, 1]), np.max(polygon[:, 1])

            if x_min < marginal_label_region_xyxy[0]:
                marginal_label_region_xyxy[0] = x_min

            if y_min < marginal_label_region_xyxy[1]:
                marginal_label_region_xyxy[1] = y_min

            if x_max > marginal_label_region_xyxy[2]:
                marginal_label_region_xyxy[2] = x_max

            if y_max > marginal_label_region_xyxy[3]:
                marginal_label_region_xyxy[3] = y_max

    if not _check_region_xyxy_is_valid(marginal_label_region_xyxy):
        logger.warning(
            "AutoROI setting update failed. Needs at least one training data & valid polygon."
        )
        return False

    marginal_label_region_xyxy = _expand_region_xyxy(marginal_label_region_xyxy, self.expand_ratio)
    marginal_label_region_xyxy = _fit_region_into_img_size(
        marginal_label_region_xyxy,
        self.image_hw,
    )

    self.left = marginal_label_region_xyxy[0] / self.image_hw[1]
    self.top = marginal_label_region_xyxy[1] / self.image_hw[0]
    self.right = marginal_label_region_xyxy[2] / self.image_hw[1]
    self.bottom = marginal_label_region_xyxy[3] / self.image_hw[0]

    self.is_ready = True
    marginal_label_region_ratio = [self.left, self.top, self.right, self.bottom]
    logger.info(f"AutoROI setting success. ROI coordinates: {marginal_label_region_ratio}")

    return True
roi_handler
ROIHandler 
ROIHandler(**kwargs)

ROI 기능을 수행합니다. 이미지에 대한 ROI 좌표를 계산해 크롭하고, 크롭된 이미지에 blind_mask를 적용해 마스크 영역의 픽셀 값을 0으로 치환합니다.

Source code in SaigeToolkit/data/transform/roi/roi_handler.py 
27
28
def __init__(self, **kwargs) -> None:
    self.set(**kwargs)

transform

Transform 
Transform(image_mode: Union[str, List[str]] = 'RGB', inspection_size_wh: Optional[ImageSizeType] = None, roi: Optional[Dict] = None, resize: Optional[Dict] = None, augmentation: Optional[Dict] = None, roi_mask_first: bool = True)
Source code in SaigeToolkit/data/transform/transform.py 
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
def __init__(
    self,
    image_mode: Union[str, List[str]] = "RGB",
    inspection_size_wh: Optional[ImageSizeType] = None,
    roi: Optional[Dict] = None,
    resize: Optional[Dict] = None,
    augmentation: Optional[Dict] = None,
    roi_mask_first: bool = True,  # 하위 호환성을 위해 roi_mask_first=True를 기본값으로 설정
):
    self.roi_mask_first = roi_mask_first
    self.image_loader = ImageLoader(image_mode=image_mode)
    self.roi_handler = ROIHandler(**roi) if roi is not None else None
    self.base_resizer = Resizer(**resize) if resize is not None else None

    # inspection_size_wh 설정 시 그 값에 따라 초기화
    self.inspection_size_resizer: Optional[InspectionSizeResizer] = None
    self.stacked_resizer_handler: StackedResizerHandler = None

    # inspection_size_wh setter에서 inspection_size_resizer와 stacked_resizer_handler 생성
    self.inspection_size_wh = inspection_size_wh

    if augmentation is None:
        self.augmentation = None
    elif "_target_" in augmentation:
        self.augmentation = build_augmentation(augmentation)
    else:
        raise NotImplementedError("Old version of augmentation config is not supported.")
__call__ 
__call__(data: Dict, warmup: bool = False) -> Dict

data Dict에 transform을 적용합니다.

Parameters:

  • data (Dict) –

    data Dictionary

  • warmup (bool, default: False ) –

    InferenceHandler warmup시에 사용하는 파라미터 입니다. True일 경우, 현재 transform을 적용했을 때 나올 수 있는 가장 큰 image size로 transform을 적용합니다. Transform operation 중 ROI의 경우 input image에 따라 output size가 매번 바뀔 수 있기 때문에 해당 옵션이 추가되었습니다. Defaults to False.

Returns:

  • Dict ( Dict ) –

    transform이 적용된 데이터 Dict입니다.

Source code in SaigeToolkit/data/transform/transform.py 
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
def __call__(self, data: Dict, warmup: bool = False) -> Dict:
    """data Dict에 transform을 적용합니다.

    Args:
        data (Dict): data Dictionary
        warmup (bool, optional): InferenceHandler warmup시에 사용하는 파라미터 입니다. True일 경우,
                                    현재 transform을 적용했을 때 나올 수 있는 가장 큰 image size로
                                    transform을 적용합니다.
                                 Transform operation 중 ROI의 경우 input image에 따라 output size가
                                    매번 바뀔 수 있기 때문에 해당 옵션이 추가되었습니다.
                                 Defaults to False.

    Returns:
        Dict: transform이 적용된 데이터 Dict입니다.
    """
    # transform_params 설정
    if "transform_params" not in data:
        data["transform_params"] = {
            "operation_stack": [
                self.Operation.ROI,
                self.Operation.InspectionSize,
                self.Operation.Resize,
            ],
            "operation_params": {},
        }

    operation_params: Dict = data["transform_params"]["operation_params"]

    # load PIL Image
    data = self.image_loader(**data)

    # 원본 이미지가 inspection_size를 넘지 않도록 resize 했을 때의 image scale 계산
    if self.inspection_size_resizer is not None:
        self.inspection_size_resizer.set_scale_from_data(**data)

    # ROI crop 적용
    if self.roi_handler is not None:
        data, roi_revert_params = self.roi_handler.apply_crop(
            return_revert_params=True, warmup=warmup, **data
        )
        if self.roi_mask_first:
            data = self.roi_handler.apply_mask(**data)

        if self.Operation.ROI in operation_params:
            raise KeyError
        operation_params[self.Operation.ROI] = roi_revert_params

    # inspection_size와 resize_factor를 하나로 묶어서 resize
    data, revert_param_list = self.stacked_resizer_handler(return_revert_params=True, **data)
    inspection_size_revert_params, resizer_revert_params = revert_param_list

    if inspection_size_revert_params is not None:
        if self.Operation.InspectionSize in operation_params:
            raise KeyError
        operation_params[self.Operation.InspectionSize] = inspection_size_revert_params

    if resizer_revert_params is not None:
        if self.Operation.Resize in operation_params:
            raise KeyError
        operation_params[self.Operation.Resize] = resizer_revert_params

    # # ROI blind mask 적용
    if self.roi_handler is not None and not self.roi_mask_first:
        data = self.roi_handler.apply_mask(**data)

    # data augmentation
    if self.augmentation is not None:
        data = self.augmentation(**data)

    return data
get_resize_scale classmethod 
get_resize_scale(transform_params: Dict) -> List[float]

before_transform_image_size -> after_transform_input_size가 되기 위한 scale을 구합니다. before_transform_image_size * scale = after_transform_input_size

Parameters:

  • transform_params (Dict) –

    transform시에 저장해둔 parameter 입니다.

Returns:

  • List[float] –

    List[float]: before_transform_image_size * scale = after_transform_input_size인 scale scale: [scale_width, scale_height]

Source code in SaigeToolkit/data/transform/transform.py 
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
@classmethod
def get_resize_scale(cls, transform_params: Dict) -> List[float]:
    """before_transform_image_size -> after_transform_input_size가 되기 위한 scale을 구합니다.
    before_transform_image_size * scale = after_transform_input_size

    Args:
        transform_params (Dict): transform시에 저장해둔 parameter 입니다.

    Returns:
        List[float]: before_transform_image_size * scale = after_transform_input_size인 scale
                     scale: [scale_width, scale_height]
    """
    operation_params: Dict = transform_params["operation_params"]
    inspection_size_revert_params = operation_params.get(cls.Operation.InspectionSize, None)
    resizer_revert_params = operation_params.get(cls.Operation.Resize, None)

    if inspection_size_revert_params is None and resizer_revert_params is None:
        resize_scale = [1.0, 1.0]
    else:
        if inspection_size_revert_params is None:
            image_size_before_resize = resizer_revert_params["image_size_before_resize"]
            image_size_after_resize = resizer_revert_params["image_size_after_resize"]
        elif resizer_revert_params is None:
            image_size_before_resize = inspection_size_revert_params["image_size_before_resize"]
            image_size_after_resize = inspection_size_revert_params["image_size_after_resize"]
        else:
            image_size_before_resize = inspection_size_revert_params["image_size_before_resize"]
            image_size_after_resize = resizer_revert_params["image_size_after_resize"]

        resize_scale = (
            np.array(image_size_after_resize) / np.array(image_size_before_resize)
        ).tolist()

    return resize_scale
_get_next_revert_operation staticmethod 
_get_next_revert_operation(transform_params: Dict) -> Optional[Operation]

다음으로 할 revert operation을 가져옵니다. operation_stack에서 operation이 제거되지는 않습니다.

Parameters:

  • transform_params (Dict) –

    transform시에 저장해둔 parameter 입니다.

Returns:

  • Optional[Operation] –

    Optional[Operation]: 남아있는 revert operation이 있으면 operation을 없으면 None을 반환합니다.

Source code in SaigeToolkit/data/transform/transform.py 
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
@staticmethod
def _get_next_revert_operation(transform_params: Dict) -> Optional[Operation]:
    """다음으로 할 revert operation을 가져옵니다. operation_stack에서 operation이 제거되지는 않습니다.

    Args:
        transform_params (Dict): transform시에 저장해둔 parameter 입니다.

    Returns:
        Optional[Operation]: 남아있는 revert operation이 있으면 operation을 없으면 None을 반환합니다.
    """
    operation_stack: List = transform_params["operation_stack"]
    next_operation = None
    if len(operation_stack) > 0:
        next_operation = operation_stack[-1]

    return next_operation
revert classmethod 
revert(data: Dict, transform_params: Dict, operation: Optional[Operation] = None) -> Dict

summary

Parameters:

  • data (Dict) –

    revert operation을 적용할 data dictionary입니다. data는 다음과 같은 구조를 지닙니다. { "key1": { "data" (Union[np.ndarray, torch.Tensor, List[Dict]]): 실제 data입니다. "data_type" (str): 해당 data의 type 입니다. 현재 ["array", "objects"]를 지원합니다. }, "key2": { "data" (Union[np.ndarray, torch.Tensor, List[Dict]]): 실제 data입니다. "data_type" (str): 해당 data의 type 입니다. 현재 ["array", "objects"]를 지원합니다. }, ... }

  • transform_params (Dict) –

    transform시에 저장해둔 parameter 입니다.

  • operation (Optional[Operation], default: None ) –

    transform에서 revert가 가능한 operation 입니다. Enum class인 Transform.Operation에 있는 항목들을 지원합니다. operation이 None이 아닐 시, 해당 operation 까지 revert를 적용하고, None일 시, 그 다음 revert operation을 적용합니다. Defaults to None.

Raises:

  • RevertOperationNotFoundError –

    args로 넣은 operation이 남은 revert operation 중에 없을 때 에러를 발생합니다.

Returns:

  • Dict ( Dict ) –

    revert operation이 적용된 data dictionary 입니다. 구조는 Args의 data와 같습니다.

Source code in SaigeToolkit/data/transform/transform.py 
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
@classmethod
def revert(
    cls,
    data: Dict,
    transform_params: Dict,
    operation: Optional[Operation] = None,
) -> Dict:
    """_summary_

    Args:
        data (Dict): revert operation을 적용할 data dictionary입니다. data는 다음과 같은 구조를 지닙니다.
            {
                "key1": {
                    "data" (Union[np.ndarray, torch.Tensor, List[Dict]]): 실제 data입니다.
                    "data_type" (str): 해당 data의 type 입니다. 현재 ["array", "objects"]를 지원합니다.
                },
                "key2": {
                    "data" (Union[np.ndarray, torch.Tensor, List[Dict]]): 실제 data입니다.
                    "data_type" (str): 해당 data의 type 입니다. 현재 ["array", "objects"]를 지원합니다.
                },
                ...
            }
        transform_params (Dict): transform시에 저장해둔 parameter 입니다.
        operation (Optional[Operation], optional): transform에서 revert가 가능한 operation 입니다.
                                                    Enum class인 Transform.Operation에 있는 항목들을 지원합니다.
                                                    operation이 None이 아닐 시, 해당 operation 까지 revert를 적용하고,
                                                    None일 시, 그 다음 revert operation을 적용합니다.
                                                    Defaults to None.

    Raises:
        RevertOperationNotFoundError: args로 넣은 operation이 남은 revert operation 중에 없을 때 에러를 발생합니다.

    Returns:
        Dict: revert operation이 적용된 data dictionary 입니다. 구조는 Args의 data와 같습니다.
    """
    # operation이 None인 경우 다음 revert operation 하나를 진행
    if operation is None:
        data = cls._revert(data=data, transform_params=transform_params)
    # operation이 None이 아닌 경우
    else:
        # 해당 operation이 operation_stack에 있는지 확인
        operation_stack: List = transform_params["operation_stack"]
        if operation not in operation_stack:
            raise RevertOperationNotFoundError
        # 해당 operation까지 revert를 적용
        while cls._get_next_revert_operation(transform_params=transform_params) != operation:
            data = cls._revert(data=data, transform_params=transform_params)
        data = cls._revert(data=data, transform_params=transform_params)

    return data
is_oversized staticmethod 
is_oversized(transform_params: Optional[Dict]) -> bool

InspectionSize 연산에서 resize 되었는지 여부를 판단

Source code in SaigeToolkit/data/transform/transform.py 
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
@staticmethod
def is_oversized(transform_params: Optional[Dict]) -> bool:
    """InspectionSize 연산에서 resize 되었는지 여부를 판단"""
    if transform_params is None:
        return False

    inspection_size_params = get_tree_node_with_default(
        transform_params, ["operation_params", Transform.Operation.InspectionSize], None
    )
    if inspection_size_params is None:
        return False

    size1 = tuple(inspection_size_params["image_size_before_resize"])
    size2 = tuple(inspection_size_params["image_size_after_resize"])

    if size1 != size2:
        return True
    else:
        return False