Skip to content

crop

Module diagram

classDiagram
  class crop {
  }
  class base_cropper {
  }
  class builder {
  }
  class edge_cropper {
  }
  class ocr_cropper {
  }
  class polygon_cropper {
  }
  crop --> builder
  builder --> base_cropper
  builder --> edge_cropper
  builder --> ocr_cropper
  builder --> polygon_cropper
  edge_cropper --> base_cropper
  polygon_cropper --> base_cropper

data.crop

데이터 전처리를 위한 cropper 클래스를 제공합니다.

모든 cropper 클래스는 BaseCropper 클래스를 상속받아 구현되어 있으며, get_n_patch와 call 메소드를 구현해야 합니다.

  • get_n_patch 메소드는 이미지를 잘라낼 때 몇 개의 패치로 나눌지를 결정합니다.
  • call 메소드는 이미지를 입력받아 패치로 나누어 반환합니다.

사용 방식에 대해서는 dataset/base_dataset.py를 참고해주세요.

get_crop_fn 

get_crop_fn(_target_: Optional[str] = None, **cfg_crop: dict) -> Optional[BaseCropper]

getting crop function

Parameters:

  • cfg_crop (dict, default: {} ) –

    config dict for building cropper

Returns:

  • Optional[BaseCropper] –

    Optional[BaseCropper]: saige cropper object. if cfg_crop is None, return None

Source code in SaigeToolkit/data/crop/builder.py 
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
def get_crop_fn(_target_: Optional[str] = None, **cfg_crop: dict) -> Optional[BaseCropper]:
    """getting crop function

    Args:
        cfg_crop (dict): config dict for building cropper

    Returns:
        Optional[BaseCropper]: saige cropper object.
            if cfg_crop is None, return None
    """
    if _target_ is None:
        return None
    cropper = get_crop_class(_target_)
    logger.info(f"[{'CROPPER'.center(9)}] {_target_} [params] {cfg_crop}")
    return cropper(**cfg_crop)

base_cropper

Cropper 클래스의 추상화를 위한 BaseCropper 클래스를 제공합니다.

모든 Cropper 클래스는 BaseCropper 클래스를 상속받아 구현되어야 하며, get_n_patch와 call 메소드를 구현해야 합니다.

builder

get_crop_fn 

get_crop_fn(_target_: Optional[str] = None, **cfg_crop: dict) -> Optional[BaseCropper]

getting crop function

Parameters:

  • cfg_crop (dict, default: {} ) –

    config dict for building cropper

Returns:

  • Optional[BaseCropper] –

    Optional[BaseCropper]: saige cropper object. if cfg_crop is None, return None

Source code in SaigeToolkit/data/crop/builder.py 
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
def get_crop_fn(_target_: Optional[str] = None, **cfg_crop: dict) -> Optional[BaseCropper]:
    """getting crop function

    Args:
        cfg_crop (dict): config dict for building cropper

    Returns:
        Optional[BaseCropper]: saige cropper object.
            if cfg_crop is None, return None
    """
    if _target_ is None:
        return None
    cropper = get_crop_class(_target_)
    logger.info(f"[{'CROPPER'.center(9)}] {_target_} [params] {cfg_crop}")
    return cropper(**cfg_crop)

get_crop_class 

get_crop_class(cfg_crop_name: str) -> Type[BaseCropper]

getting crop class from name

Parameters:

  • cfg_crop_name (str) –

    cropper name

Returns:

  • Type[BaseCropper] –

    Type[BaseCropper]: cropper class

Source code in SaigeToolkit/data/crop/builder.py 
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
def get_crop_class(cfg_crop_name: str) -> Type[BaseCropper]:
    """getting crop class from name

    Args:
        cfg_crop_name (str): cropper name

    Returns:
        Type[BaseCropper]: cropper class
    """

    try:
        return CROPPER_IMPLEMENTATION[cfg_crop_name]
    except KeyError as e:
        raise RuntimeError(
            f"Cropper {cfg_crop_name} not implemented. Supported cropper list: {CROPPER_IMPLEMENTATION.keys()}"
        ) from e
    except Exception as e:
        logger.error(f"Error: {e}")
        raise e

edge_cropper

Crop image patches along the contour of mask (or polygon)

Imported from DefectGeneration repository. generation/data/crop/edge_cropper.py

WindowState

Bases: Enum

relation state enum between center point and target point

WindowManager 

WindowManager(width: int = 256, height: int = 256, strict_inner_patch: bool = True)

Manager for cropping window size, relation and window coordinates.

Attributes:

  • width (int) –

    window width for image patch.

  • half_width (int) –

    half of window width for image patch.

  • height (int) –

    window height for image patch.

  • half_height (int) –

    half of window height for image patch.

  • strict_inner_patch (bool) –

    whether not allowing outer area of image.

Source code in SaigeToolkit/data/crop/edge_cropper.py 
45
46
47
48
49
50
51
52
53
54
55
def __init__(
    self,
    width: int = 256,
    height: int = 256,
    strict_inner_patch: bool = True,
):
    self.width = width
    self.half_width = width // 2
    self.height = height
    self.half_height = height // 2
    self.strict_inner_patch = strict_inner_patch
check_window 
check_window(point: Iterable[int], center: Iterable[int]) -> WindowState

check whether target point is inside of window, returns WindowState

Parameters:

  • point (Iterable[int]) –

    target point to be checked

  • center (Iterable[int]) –

    reference center point

Returns:

Source code in SaigeToolkit/data/crop/edge_cropper.py 
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
def check_window(
    self, point: Iterable[int], center: Iterable[int]
) -> WindowState:  # XXX: is this ok?
    """check whether target point is inside of window, returns WindowState

    Args:
        point (Iterable[int]): target point to be checked
        center (Iterable[int]): reference center point

    Returns:
        WindowState: relation state enum
    """
    width_distance = point[0] - center[0]
    abs_width_distance = abs(width_distance)
    height_distance = point[1] - center[1]
    abs_height_distance = abs(height_distance)

    if abs_width_distance <= self.half_width and abs_height_distance <= self.half_height:
        return WindowState.IN

    if abs_width_distance > abs_height_distance:
        if width_distance > 0:
            return WindowState.WIDTH_POSITIVE_OUT
        else:
            return WindowState.WIDTH_NEGATIVE_OUT
    else:
        if height_distance > 0:
            return WindowState.HEIGHT_POSITIVE_OUT
        else:
            return WindowState.HEIGHT_NEGATIVE_OUT
is_in_window 
is_in_window(point: Iterable[int], center: Iterable[int]) -> bool

is target point inside of window

Parameters:

  • point (Iterable[int]) –

    target point to be checked

  • center (Iterable[int]) –

    reference center point

Returns:

  • bool ( bool ) –

    boolean result

Source code in SaigeToolkit/data/crop/edge_cropper.py 
88
89
90
91
92
93
94
95
96
97
98
99
def is_in_window(self, point: Iterable[int], center: Iterable[int]) -> bool:
    """is target point inside of window

    Args:
        point (Iterable[int]): target point to be checked
        center (Iterable[int]): reference center point

    Returns:
        bool: boolean result
    """
    window_state = self.check_window(point, center)
    return window_state == WindowState.IN
is_in_any_window 
is_in_any_window(point: Iterable[int], centers: Iterable[Iterable[int]]) -> bool

is target point inside of any of windows

Parameters:

  • point (Iterable[int]) –

    target point to be checked

  • center (Iterable[int]) –

    list of reference center points

Returns:

  • bool ( bool ) –

    boolean result

Source code in SaigeToolkit/data/crop/edge_cropper.py 
101
102
103
104
105
106
107
108
109
110
111
def is_in_any_window(self, point: Iterable[int], centers: Iterable[Iterable[int]]) -> bool:
    """is target point inside of any of windows

    Args:
        point (Iterable[int]): target point to be checked
        center (Iterable[int]): list of reference center points

    Returns:
        bool: boolean result
    """
    return any(self.is_in_window(point, c) for c in centers)
get_coordinates_from_center 
get_coordinates_from_center(h_center: int, w_center: int, h: int, w: int) -> Tuple[int, int, int, int]

get left/right top/bottom coordinates from picked center point.

Parameters:

  • h_center (int) –

    picked center point h-coordinate

  • w_center (int) –

    picked center point w-coordinate

  • h (int) –

    image size height

  • w (int) –

    image size width

Returns:

  • Tuple[int, int, int, int] –

    Tuple[int, int, int, int]: (crop_left, crop_top, crop_right, crop_bottom)

Source code in SaigeToolkit/data/crop/edge_cropper.py 
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
def get_coordinates_from_center(
    self, h_center: int, w_center: int, h: int, w: int
) -> Tuple[int, int, int, int]:
    """get left/right top/bottom coordinates from picked center point.

    Args:
        h_center (int): picked center point h-coordinate
        w_center (int): picked center point w-coordinate
        h (int): image size height
        w (int): image size width

    Returns:
        Tuple[int, int, int, int]: (crop_left, crop_top, crop_right, crop_bottom)
    """
    if self.strict_inner_patch:
        w_center = np.clip(w_center, self.half_width, w - self.width + self.half_width)
        h_center = np.clip(h_center, self.half_height, h - self.height + self.half_height)
    crop_left = w_center - self.half_width
    crop_right = crop_left + self.width
    crop_top = h_center - self.half_height
    crop_bottom = crop_top + self.height
    return (crop_left, crop_top, crop_right, crop_bottom)

CenterWithSatellite dataclass 

CenterWithSatellite(point: Iterable[int], window_manager: WindowManager, satellite: Optional[Iterable[int]] = None)

Center point with 'satellites'. 'Satellites' is a point within center point's window, but not the center point. Also intra distance between 'satellites' should shorter than window size.

Attributes:

  • coordinate (Iterable[int]) –

    center point coordinate

  • window_manager (WindowManager) –

    WindowManager

  • max_width_distance (int) –

    max width-wise positive distance of satellites

  • max_height_distance (int) –

    max height-wise positive distance of satellites

  • min_width_distance (int) –

    max width-wise negative distance of satellites

  • min_height_distance (int) –

    max height-wise negative distance of satellites

  • satellite_points (List[Dict[str, any]]) –

    satellite_points data

Source code in SaigeToolkit/data/crop/edge_cropper.py 
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
def __init__(
    self,
    point: Iterable[int],
    window_manager: WindowManager,
    satellite: Optional[Iterable[int]] = None,
):
    self.coordinate = point
    self.window_manager = window_manager

    self.max_width_distance = 0
    self.max_height_distance = 0
    self.min_width_distance = 0
    self.min_height_distance = 0

    self.satellite_points = {}
    if satellite is not None:
        self.calculate_relation_as_satellite(satellite)
calculate_relation_as_satellite 
calculate_relation_as_satellite(point: Iterable[int]) -> Union[Dict[str, Any], List[Dict[str, Any]]]

calculate relation between target point and center point. case 1: If target point is outside of center point's window, target point should be considered as next center point candidate.

case 2

Even when target point is in the window, check whether distance between satellites is larger than window size. If so, target point should be considered as next center point candidate.

Parameters:

  • point (Iterable[int]) –

    target point to be checked

Returns:

  • Union[Dict[str, Any], List[Dict[str, Any]]] –

    Union[Dict[str, Any], List[Dict[str, Any]]]: Dict[str, Any]: simple relation between center point and target point List[Dict[str, Any]]: reltations between satellite points and target point

Source code in SaigeToolkit/data/crop/edge_cropper.py 
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
def calculate_relation_as_satellite(
    self, point: Iterable[int]
) -> Union[Dict[str, Any], List[Dict[str, Any]]]:
    """calculate relation between target point and center point.
    case 1:
        If target point is outside of center point's window,
        target point should be considered as next center point candidate.

    case 2:
        Even when target point is in the window,
        check whether distance between satellites is larger than window size.
        If so, target point should be considered as next center point candidate.

    Args:
        point (Iterable[int]): target point to be checked

    Returns:
        Union[Dict[str, Any], List[Dict[str, Any]]]:
            Dict[str, Any]: simple relation between center point and target point
            List[Dict[str, Any]]: reltations between satellite points and target point
    """
    width_distance = point[0] - self.coordinate[0]
    height_distance = point[1] - self.coordinate[1]
    relation = self.window_manager.check_window(point, self.coordinate)

    # target point is outside of center point's window.
    # Do not update satellite and return the relationship.
    if relation != WindowState.IN:
        return {"coordinate": self.coordinate, "relation": relation}

    satellites_intra_distance_errors = []
    if width_distance > self.max_width_distance:
        if abs(width_distance) + abs(self.min_width_distance) > self.window_manager.half_width:
            self._compare_point_and_calculate_error(
                "min_width", point, satellites_intra_distance_errors
            )
        else:
            # else, update as satellite point.
            self.max_width_distance = width_distance
            self.satellite_points.update({"max_width": {"coordinate": point, "relation": relation}})

    if height_distance > self.max_height_distance:
        if abs(height_distance) + abs(self.min_height_distance) > self.window_manager.half_height:
            self._compare_point_and_calculate_error(
                "min_height", point, satellites_intra_distance_errors
            )
        else:
            # else, update as satellite point.
            self.max_height_distance = height_distance
            self.satellite_points.update({"max_height": {"coordinate": point, "relation": relation}})

    if width_distance < self.min_width_distance:
        if abs(width_distance) + abs(self.max_width_distance) > self.window_manager.half_width:
            self._compare_point_and_calculate_error(
                "max_width", point, satellites_intra_distance_errors
            )
        else:
            # else, update as satellite point.
            self.min_width_distance = width_distance
            self.satellite_points.update({"min_width": {"coordinate": point, "relation": relation}})

    if height_distance < self.min_height_distance:
        if abs(height_distance) + abs(self.max_height_distance) > self.window_manager.half_height:
            self._compare_point_and_calculate_error(
                "max_height", point, satellites_intra_distance_errors
            )
        else:
            # else, update as satellite point.
            self.min_height_distance = height_distance
            self.satellite_points.update({"min_height": {"coordinate": point, "relation": relation}})

    if satellites_intra_distance_errors:
        # any distance between satellites larger than window
        return satellites_intra_distance_errors

    return {"coordinate": self.coordinate, "relation": relation}
_compare_point_and_calculate_error 
_compare_point_and_calculate_error(key, point, satellites_intra_distance_errors)

Refactored from the original code from Defect Generation (Function extracted).

Source code in SaigeToolkit/data/crop/edge_cropper.py 
248
249
250
251
252
253
254
255
def _compare_point_and_calculate_error(self, key, point, satellites_intra_distance_errors):
    """Refactored from the original code from Defect Generation (Function extracted)."""
    # distance between satellites is larger than window size.
    compare_point = (
        self.satellite_points[key]["coordinate"] if key in self.satellite_points else self.coordinate
    )
    _relation = self.window_manager.check_window(point, compare_point)
    satellites_intra_distance_errors.append({"coordinate": compare_point, "relation": _relation})

EdgeCropper 

EdgeCropper(crop_w: int = 512, crop_h: int = 512, random_sample: bool = False, patch_per_polygon: int = 1, strict_inner_patch: bool = True, ok_patch_prob: float = 0.01)

Bases: BaseCropper

Crop image patches along the contour of mask (or polygon)

Attributes:

  • window_manager (WindowManager) –

    WindowManager

  • random_sample (bool) –

    whether randomly select the first point of each polygon

  • patch_per_polygon (bool) –

    number of repeat for cropping patches around each polygon.

Source code in SaigeToolkit/data/crop/edge_cropper.py 
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
def __init__(
    self,
    crop_w: int = 512,
    crop_h: int = 512,
    random_sample: bool = False,
    patch_per_polygon: int = 1,
    strict_inner_patch: bool = True,
    ok_patch_prob: float = 0.01,
) -> None:
    self.crop_w = crop_w
    self.crop_h = crop_h
    self.window_manager = WindowManager(crop_w, crop_h, strict_inner_patch)
    self.random_sample = random_sample
    self.patch_per_polygon = patch_per_polygon
    self.ok_patch_prob = ok_patch_prob
    self.n_defect_patch_made = 0

    self.init_centers()
get_n_patch 
get_n_patch(polygons: List[ndarray]) -> int

PolygonCropper crops all polygon areas {self.patch_per_polygon} times per polygons, and also crops random position of image {self.random_patch_per_img} times per images. Therefore, resultant number of patches is weighted-sum result as coded below.

Parameters:

  • polygons (List[ndarray]) –

    polygons of single image in dataset.

Returns:

  • int ( int ) –

    number of patches to be cropped in single image

Source code in SaigeToolkit/data/crop/edge_cropper.py 
289
290
291
292
293
294
295
296
297
298
299
300
def get_n_patch(self, polygons: List[np.ndarray]) -> int:
    """PolygonCropper crops all polygon areas {self.patch_per_polygon} times per polygons,
    and also crops random position of image {self.random_patch_per_img} times per images.
    Therefore, resultant number of patches is weighted-sum result as coded below.

    Args:
        polygons (List[np.ndarray]): polygons of single image in dataset.

    Returns:
        int: number of patches to be cropped in single image
    """
    raise NotImplementedError
__call__ 
__call__(image: Union[Image, ndarray], mask: Union[Image, ndarray], polygons: Optional[List[ndarray]] = None, **kwargs) -> List[dict]

crop image into image patches.

Parameters:

  • image (Union[Image, ndarray]) –

    original image

  • polygons (List[ndarray], default: None ) –

    polygon data (# of polygons, (4, 2)) polygon should have 4 points

  • mask (Union[Image, ndarray]) –

    segmentation mask image

Returns:

  • List[dict] –

    List[dict]: list of cropped image data dict

Source code in SaigeToolkit/data/crop/edge_cropper.py 
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
def __call__(
    self,
    image: Union[Image.Image, np.ndarray],
    mask: Union[Image.Image, np.ndarray],
    polygons: Optional[List[np.ndarray]] = None,
    **kwargs,
) -> List[dict]:
    """crop image into image patches.

    Args:
        image (Union[Image.Image, np.ndarray]): original image
        polygons (List[np.ndarray]): polygon data
            (# of polygons, (4, 2)) polygon should have 4 points
        mask (Union[Image.Image, np.ndarray]): segmentation mask image

    Returns:
        List[dict]: list of cropped image data dict
    """

    w, h = read_image_size(image)

    if polygons is None:
        polygons, _ = cv2.findContours(np.asarray(mask), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
        polygons = [polygon[:, 0] for polygon in polygons]  # TODO: check..

    l_data = []
    self.init_centers()

    # crop defect polygon
    _polygons = random.sample(polygons, len(polygons)) if self.random_sample else polygons
    for polygon in _polygons:
        polygon = subdivide_points_into_min_resolution(list(polygon), 32, 32)
        for _ in range(self.patch_per_polygon):
            max_iter_per_polygon = 3 if self.random_sample else 1  # TODO: as class variable?
            min_num_point_not_included = None
            for _ in range(max_iter_per_polygon):
                new_centers = self.pick_centers_from_polygon(polygon)

                num_point_not_included = sum(
                    not self.window_manager.is_in_any_window(p, [*self.centers, *new_centers])
                    for p in polygon
                )
                if num_point_not_included == 0:
                    self.centers.extend(new_centers)
                    break

                if min_num_point_not_included is None:
                    min_num_point_not_included = num_point_not_included
                    new_centers_candidate = new_centers

                if num_point_not_included < min_num_point_not_included:
                    min_num_point_not_included = num_point_not_included
                    new_centers_candidate = new_centers

            else:
                # If fail to get `num_point_not_included==0` sample,
                # use min `num_point_not_included` sample.
                new_centers = new_centers_candidate
                self.centers.extend(new_centers)

            for center in new_centers:
                single_defect_patch_data = self.get_data_dict_from_center_point(
                    image, mask, center, **kwargs
                )
                l_data.append(single_defect_patch_data)

    self.n_defect_patch_made += len(l_data)

    # crop ok region
    # XXX: OK patch portion can be larger w.r.t num of ok images.
    probability_multiplier = max(self.n_defect_patch_made, 1)
    if random.random() < self.ok_patch_prob * probability_multiplier:
        num_to_make_ok_patch = ceil(self.ok_patch_prob * probability_multiplier)
        max_iteration = 5
        for _ in range(num_to_make_ok_patch):
            for _ in range(max_iteration):
                if self.window_manager.strict_inner_patch:
                    window_half_w = self.window_manager.half_width
                    window_half_h = self.window_manager.half_height

                    # Cannot sample random window center.
                    if w - window_half_w < window_half_w or h - window_half_h < window_half_h:
                        logger.warning(f"Crop size is larger than image size: ({w}, {h}).")
                        continue

                    random_center = (
                        random.randint(window_half_w, w - window_half_w),
                        random.randint(window_half_h, h - window_half_h),
                    )
                else:
                    random_center = random.randint(0, w), random.randint(0, h)
                if not self.window_manager.is_in_any_window(random_center, self.centers):
                    self.centers.append(random_center)
                    self.n_defect_patch_made -= 1 / self.ok_patch_prob
                    break
            else:
                # Max trial ended with no ok patch.
                return l_data

            single_ok_patch_data = self.get_data_dict_from_center_point(
                image, mask, random_center, **kwargs
            )
            l_data.append(single_ok_patch_data)

    return l_data
get_proper_point 
get_proper_point(state_as_satellite: Union[Dict[str, Any], List[Dict[str, Any]]], must_contain_coordinate: List[Iterable[int]], point_in: Iterable[int], point_out: Iterable[int]) -> Optional[Iterable[int]]

get proper next center point that contains every must_contain_coordinate

Parameters:

  • state_as_satellite (Union[Dict[str, Any], List[Dict[str, Any]]]) –

    relation between center point(s) and target point.

  • must_contain_coordinate (List[Iterable[int]]) –

    points should be in next window

  • point_in (Iterable[int]) –

    point inside window from previous boundary

  • point_out (Iterable[int]) –

    point outside window from previous boundary

Returns:

  • Optional[Iterable[int]] –

    Optional[Iterable[int]]: proper next center (or None)

Source code in SaigeToolkit/data/crop/edge_cropper.py 
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
def get_proper_point(
    self,
    state_as_satellite: Union[Dict[str, Any], List[Dict[str, Any]]],
    must_contain_coordinate: List[Iterable[int]],
    point_in: Iterable[int],
    point_out: Iterable[int],
) -> Optional[Iterable[int]]:
    """get proper next center point that contains every `must_contain_coordinate`

    Args:
        state_as_satellite (Union[Dict[str, Any], List[Dict[str, Any]]]):
            relation between center point(s) and target point.
        must_contain_coordinate (List[Iterable[int]]): points should be in next window
        point_in (Iterable[int]): point inside window from previous boundary
        point_out (Iterable[int]): point outside window from previous boundary

    Returns:
        Optional[Iterable[int]]: proper next center (or None)
    """

    if isinstance(state_as_satellite, list):
        for u_r in state_as_satellite:
            p_dst = self.get_proper_point(u_r, must_contain_coordinate, point_in, point_out)
            if p_dst is not None:
                break

        else:
            return None

    else:
        relation, inner_point = state_as_satellite["relation"], state_as_satellite["coordinate"]
        p_dst = self.get_destination_point(relation, inner_point, point_in, point_out)

        if p_dst is None:
            return None

        for _p in must_contain_coordinate:
            if not self.window_manager.is_in_window(_p, p_dst):
                return None

    return p_dst
get_destination_point 
get_destination_point(relation: WindowState, point_ref: Iterable[int], point_in: Iterable[int], point_out: Iterable[int]) -> ndarray

calculate point a-window-away from reference point

Parameters:

  • relation (WindowState) –

    relation between reference point and next center

  • point_ref (Iterable[int]) –

    reference point to calculate next center

  • point_in (Iterable[int]) –

    point inside window from previous boundary

  • point_out (Iterable[int]) –

    point outside window from previous boundary

Returns:

  • ndarray –

    np.ndarray: result destination point

Source code in SaigeToolkit/data/crop/edge_cropper.py 
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
def get_destination_point(
    self,
    relation: WindowState,
    point_ref: Iterable[int],
    point_in: Iterable[int],
    point_out: Iterable[int],
) -> np.ndarray:
    """calculate point a-window-away from reference point

    Args:
        relation (WindowState): relation between reference point and next center
        point_ref (Iterable[int]): reference point to calculate next center
        point_in (Iterable[int]): point inside window from previous boundary
        point_out (Iterable[int]): point outside window from previous boundary

    Returns:
        np.ndarray: result destination point
    """
    w_in, h_in = point_in
    w_out, h_out = point_out

    def _get_interior_point(a_dst, a_in, a_out, b_in, b_out):
        if (a_out != a_in) and (a_in <= a_dst <= a_out or a_out <= a_dst <= a_in):
            rate = (a_dst - a_in) / (a_out - a_in)
        else:
            # if a_dst is not interior between two points, update a_dst to the middle point.
            a_dst = int(round((a_out + a_in) / 2))
            rate = 0.5

        b_dst = b_in + int(round(rate * (b_out - b_in)))
        return a_dst, b_dst

    if relation == WindowState.WIDTH_POSITIVE_OUT:
        w_dst = point_ref[0] + self.window_manager.half_width
        w_dst, h_dst = _get_interior_point(w_dst, w_in, w_out, h_in, h_out)
    elif relation == WindowState.WIDTH_NEGATIVE_OUT:
        w_dst = point_ref[0] - self.window_manager.half_width
        w_dst, h_dst = _get_interior_point(w_dst, w_in, w_out, h_in, h_out)
    elif relation == WindowState.HEIGHT_POSITIVE_OUT:
        h_dst = point_ref[1] + self.window_manager.half_height
        h_dst, w_dst = _get_interior_point(h_dst, h_in, h_out, w_in, w_out)
    elif relation == WindowState.HEIGHT_NEGATIVE_OUT:
        h_dst = point_ref[1] - self.window_manager.half_height
        h_dst, w_dst = _get_interior_point(h_dst, h_in, h_out, w_in, w_out)

    else:
        # if relation is `WindowState.In`, find farthest point and set as target point.
        width_distance = w_out - point_ref[0]
        height_distance = h_out - point_ref[1]
        if abs(width_distance) > abs(height_distance):
            if width_distance > 0:
                relation = WindowState.WIDTH_POSITIVE_OUT
            else:
                relation = WindowState.WIDTH_NEGATIVE_OUT
        else:
            if height_distance > 0:
                relation = WindowState.HEIGHT_POSITIVE_OUT
            else:
                relation = WindowState.HEIGHT_NEGATIVE_OUT

        dst = self.get_destination_point(relation, point_ref, point_in, point_out)

        return dst

    return np.array([w_dst, h_dst])
remove_inner_points_and_get_outer_points 
remove_inner_points_and_get_outer_points(center: Iterable[int], points: List[Iterable[int]]) -> Tuple[List[Iterable[int]], Iterable[int], Iterable[int]]

once center decided, check other points and remove insiders.

Parameters:

  • center (Iterable[int]) –

    next center coordinate

  • points (List[Iterable[int]]) –

    current points on polygon

Returns:

  • Tuple[List[Iterable[int]], Iterable[int], Iterable[int]] –

    Tuple[List[Iterable[int]], Iterable[int], Iterable[int]]: - List[Iterable[int]]: clean-up'd polygon points - Iterable[int]: boundary point (positive direction) - Iterable[int]: boundary point (negative direction)

Source code in SaigeToolkit/data/crop/edge_cropper.py 
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
def remove_inner_points_and_get_outer_points(
    self, center: Iterable[int], points: List[Iterable[int]]
) -> Tuple[List[Iterable[int]], Iterable[int], Iterable[int]]:
    """once center decided, check other points and remove insiders.

    Args:
        center (Iterable[int]): next center coordinate
        points (List[Iterable[int]]): current points on polygon

    Returns:
        Tuple[List[Iterable[int]], Iterable[int], Iterable[int]]:
            - List[Iterable[int]]: clean-up'd polygon points
            - Iterable[int]: boundary point (positive direction)
            - Iterable[int]: boundary point (negative direction)
    """
    idx_positive_direction, idx_negative_direction = 0, 0
    p_dst, n_dst = None, None

    for idx, p in enumerate(points):
        relation = self.window_manager.check_window(p, center)
        if relation == WindowState.IN:
            continue

        idx_positive_direction = idx - 1
        p_dst = self.get_destination_point(
            relation,
            center,
            points[idx_positive_direction],
            points[idx_positive_direction + 1],
        )
        break

    for idx, p in enumerate(points[::-1]):
        relation = self.window_manager.check_window(p, center)
        if relation == WindowState.IN:
            continue

        idx_negative_direction = -idx
        n_dst = self.get_destination_point(
            relation,
            center,
            points[idx_negative_direction],
            points[idx_negative_direction - 1],
        )
        break

    if p_dst is None:
        return [], p_dst, n_dst

    points = (
        points[idx_positive_direction + 1 : idx_negative_direction]
        if idx_negative_direction < 0
        else points[idx_positive_direction + 1 :]
    )

    return points, p_dst, n_dst

subdivide_points_into_min_resolution 

subdivide_points_into_min_resolution(points: List[Iterable[int]], width_resolution: int = 32, height_resolution: int = 32) -> List[Iterable[int]]

refine cv2.findContours result. long straight edge in mask results in long distance between two points.

Parameters:

  • points (List[Iterable[int]]) –

    cv2.findContours result

  • width_resolution (int, default: 32 ) –

    maximum distance width-wise. Defaults to 32.

  • height_resolution (int, default: 32 ) –

    maximum distance height-wise. Defaults to 32.

Returns:

  • List[Iterable[int]] –

    List[Iterable[int]]: refined polygon points

Source code in SaigeToolkit/data/crop/edge_cropper.py 
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
def subdivide_points_into_min_resolution(
    points: List[Iterable[int]], width_resolution: int = 32, height_resolution: int = 32
) -> List[Iterable[int]]:
    """refine cv2.findContours result.
    long straight edge in mask results in long distance between two points.

    Args:
        points (List[Iterable[int]]): cv2.findContours result
        width_resolution (int, optional): maximum distance width-wise. Defaults to 32.
        height_resolution (int, optional): maximum distance height-wise. Defaults to 32.

    Returns:
        List[Iterable[int]]: refined polygon points
    """

    idx_inspected = 0

    while idx_inspected < len(points):
        for idx in range(idx_inspected, len(points)):
            point_prev = points[idx]
            point_next = points[idx + 1] if idx + 1 < len(points) else points[0]

            w_length = abs(point_next[0] - point_prev[0])
            if w_length > width_resolution:
                num_insert = ceil(w_length / width_resolution) - 1
                break

            h_length = abs(point_next[1] - point_prev[1])
            if h_length > height_resolution:
                num_insert = ceil(h_length / height_resolution) - 1
                break
        else:
            break

        for n in range(num_insert - 1, -1, -1):
            rate = (n + 1) / (num_insert + 1)
            point_prev = points[idx]
            point_next = points[idx + 1] if idx + 1 < len(points) else points[0]
            points.insert(
                idx + 1, point_prev + np.round(rate * (point_next - point_prev)).astype(np.int32)
            )

        idx_inspected = idx + n

    return points

get_mid_point 

get_mid_point(points: List[Iterable[int]]) -> Iterable[int]

In case failed to get next center point, get mid point from remaining polygon points.

Parameters:

  • points (List[Iterable[int]]) –

    remaining polygon points

Returns:

  • Iterable[int] –

    Iterable[int]: next center point (mid)

Source code in SaigeToolkit/data/crop/edge_cropper.py 
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
def get_mid_point(points: List[Iterable[int]]) -> Iterable[int]:
    """In case failed to get next center point,
    get mid point from remaining polygon points.

    Args:
        points (List[Iterable[int]]): remaining polygon points

    Returns:
        Iterable[int]: next center point (mid)
    """
    if len(points) == 1:
        return points[0]

    np_points = np.array(points)
    length_between_points = np.linalg.norm(np_points[1:] - np_points[:-1], axis=1)
    length_half = np.sum(length_between_points) / 2
    for idx in range(len(length_between_points)):
        if np.sum(length_between_points[:idx]) >= length_half:
            break

    length_desired = length_half - np.sum(length_between_points[: idx - 1])
    length_entire = length_between_points[idx - 1]
    rate = length_desired / length_entire

    return points[idx] + np.round(rate * (points[idx + 1] - points[idx])).astype(np.int32)

ocr_cropper

OcrCropper 

OcrCropper(chr_height: int = 32, crop_jitter: Optional[float] = None, crop_jitter_rate: Optional[List[float]] = None, save_polygons: bool = False)

Cropper class for OCR task.

Attributes:

  • chr_height (int) –

    cropped image patch is resized to be the same height as chr_height.

  • crop_jitter (Optional[float]) –

    if not None, cropping box is jittered horizontally with ratio crop_jitter.

  • crop_jitter_rate (Optional[List[float]]) –

    if not None, cropping box points are jittered to any direction with ratio crop_jitter_rate.

  • save_polygons (bool) –

    whether polygon label is preserved after cropping.

initializing OcrCropper. all initializing input is set to class attribute.

Parameters:

  • chr_height (int, default: 32 ) –

    Defaults to 32.

  • crop_jitter (Optional[float], default: None ) –

    Defaults to None.

  • crop_jitter_rate (Optional[List[float]], default: None ) –

    Defaults to None.

  • save_polygons (bool, default: False ) –

    Defaults to False.

Source code in SaigeToolkit/data/crop/ocr_cropper.py 
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
def __init__(
    self,
    chr_height: int = 32,
    crop_jitter: Optional[float] = None,
    crop_jitter_rate: Optional[List[float]] = None,
    save_polygons: bool = False,
) -> None:
    """initializing OcrCropper.
    all initializing input is set to class attribute.

    Args:
        chr_height (int): Defaults to 32.
        crop_jitter (Optional[float], optional): Defaults to None.
        crop_jitter_rate (Optional[List[float]], optional): Defaults to None.
        save_polygons (bool): Defaults to False.
    """
    self.chr_height = chr_height
    self.crop_jitter = crop_jitter
    self.crop_jitter_rate = crop_jitter_rate
    self.save_polygons = save_polygons
get_n_patch 
get_n_patch(polygons: List[List[ndarray]]) -> int

OcrCropper only crops all polygon areas, unlike SegCropper. Therefore, resultant number of patches is simply equals to number of polygons.

Parameters:

  • polygons (List[List[ndarray]]) –

    polygons of all images in dataset.

Returns:

  • int ( int ) –

    total number of patches to be cropped

Source code in SaigeToolkit/data/crop/ocr_cropper.py 
47
48
49
50
51
52
53
54
55
56
57
def get_n_patch(self, polygons: List[List[np.ndarray]]) -> int:
    """OcrCropper only crops all polygon areas, unlike SegCropper.
    Therefore, resultant number of patches is simply equals to number of polygons.

    Args:
        polygons (List[List[np.ndarray]]): polygons of all images in dataset.

    Returns:
        int: total number of patches to be cropped
    """
    return len(polygons)
__call__ 
__call__(image: Union[Image, ndarray], polygons: List[ndarray], strings: List[str], ignore: List[bool], is_vertical: Optional[List[bool]] = None, **kwargs) -> List[dict]

crop image into image patches. polygon, string, ignore data should be same length.

Parameters:

  • image (Union[Image, ndarray]) –

    original image

  • polygons (List[ndarray]) –

    polygon data (# of polygons, (4, 2)) polygon should have 4 points

  • strings (List[str]) –

    string data

  • ignore (List[bool]) –

    ignore notation data

  • is_vertical (List[bool], default: None ) –

    is_vertical notation data

Returns:

  • List[dict] –

    List[dict]: list of cropped image data dict

Source code in SaigeToolkit/data/crop/ocr_cropper.py 
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
def __call__(
    self,
    image: Union[Image.Image, np.ndarray],
    polygons: List[np.ndarray],
    strings: List[str],
    ignore: List[bool],
    is_vertical: Optional[List[bool]] = None,
    **kwargs,
) -> List[dict]:
    """crop image into image patches.
    polygon, string, ignore data should be same length.

    Args:
        image (Union[Image.Image, np.ndarray]): original image
        polygons (List[np.ndarray]): polygon data
            (# of polygons, (4, 2)) polygon should have 4 points
        strings (List[str]): string data
        ignore (List[bool]): ignore notation data
        is_vertical (List[bool]): is_vertical notation data

    Returns:
        List[dict]: list of cropped image data dict
    """
    image = np.asarray(image)

    if is_vertical is None:
        is_vertical = [False] * len(polygons)

    l_data = []
    for pol, string, ign, is_vert in zip(polygons, strings, ignore, is_vertical):
        if ign:
            continue

        max_trial = 10
        trial = 0
        while trial < max_trial:
            trial += 1

            tops, bots = self.jitter_polygon(pol)
            widths, heights = self.calculate_width_height(tops, bots)

            if all(np.array(widths + heights) >= 1):
                break

            if trial == max_trial:
                logger.info("Max trial exeeded.. no jitter applied.")
                tops = pol[: len(pol) // 2]
                bots = pol[: len(pol) // 2 - 1 : -1]
                widths, heights = self.calculate_width_height(tops, bots)

        if any(np.array(widths + heights) < 1):
            logger.warning(f"Invalid patch shape (jittered) encounter.. string: {string}")
            continue

        text_patch_list = []
        for i, (w, h) in enumerate(zip(widths, heights)):
            width = int(np.round((w / h * self.chr_height)))
            if width == 0:
                continue

            startpoints = np.array([tops[i], tops[i + 1], bots[i + 1], bots[i]], np.float32)
            endpoints = np.array(
                [
                    [0, 0],
                    [width - 1, 0],
                    [width - 1, self.chr_height - 1],
                    [0, self.chr_height - 1],
                ],
                np.float32,
            )

            affine_matrix = cv2.getPerspectiveTransform(startpoints, endpoints)
            text_patch = cv2.warpPerspective(
                image, affine_matrix, (width, self.chr_height), borderValue=0
            )
            if i != 0:
                # remove duplicated leftmost column
                # border area is duplicated in adjacent patches.
                text_patch = text_patch[:, 1:, :]
            text_patch_list.append(text_patch)

        text_patch = np.concatenate(text_patch_list, axis=1)

        if int(text_patch.shape[1] / (self.chr_height / 8)) + 1 >= len(string):
            patch_data = {
                "image": Image.fromarray(text_patch),
                "strings": string,
                "is_vertical": is_vert,
            }

            if self.save_polygons:
                patch_data.update(polygons=pol)

            l_data.append(patch_data)
        else:
            logger.warning(
                f"Invalid patch length! string: {string} patch width: {text_patch.shape[1]}"
            )

    return l_data
calculate_width_height 
calculate_width_height(tops: ndarray, bots: ndarray) -> Tuple[List[ndarray], List[ndarray]]

calculate width and heights for every four polygon points (2 tops and 2 bots). Assuming that polygon is composed of top-points and bot-points, and every top and bot points are paired.

Parameters:

  • tops (ndarray) –

    top points of polygon

  • bots (ndarray) –

    bot points of polygon

Returns:

  • Tuple[List[ndarray], List[ndarray]] –

    Tuple[List[np.ndarray]]: width and height list for every four points.

Source code in SaigeToolkit/data/crop/ocr_cropper.py 
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
def calculate_width_height(
    self, tops: np.ndarray, bots: np.ndarray
) -> Tuple[List[np.ndarray], List[np.ndarray]]:
    """calculate width and heights for every four polygon points (2 tops and 2 bots).
    Assuming that polygon is composed of top-points and bot-points,
    and every top and bot points are paired.

    Args:
        tops (np.ndarray): top points of polygon
        bots (np.ndarray): bot points of polygon

    Returns:
        Tuple[List[np.ndarray]]: width and height list for every four points.
    """

    widths = []
    for i in range(len(tops) - 1):
        widths.append(
            np.mean(
                (
                    np.linalg.norm(tops[i + 1] - tops[i]),
                    np.linalg.norm(bots[i + 1] - bots[i]),
                )
            )
        )

    heights = []
    for t, b in zip(tops, bots):
        heights.append(np.linalg.norm(b - t))
    heights = [np.mean([heights[i + 1], heights[i]]) for i in range(len(heights) - 1)]

    return widths, heights
jitter_polygon 
jitter_polygon(polygon: ndarray) -> Tuple[ndarray, ndarray]

jittering polygon points and split into tops and bots points

Parameters:

  • polygon (ndarray) –

    original polygon data

Returns:

  • Tuple[ndarray, ndarray] –

    Tuple[np.ndarray, np.ndarray]: jittered tops, bots points

Source code in SaigeToolkit/data/crop/ocr_cropper.py 
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
def jitter_polygon(self, polygon: np.ndarray) -> Tuple[np.ndarray, np.ndarray]:
    """jittering polygon points and split into tops and bots points

    Args:
        polygon (np.ndarray): original polygon data

    Returns:
        Tuple[np.ndarray, np.ndarray]: jittered tops, bots points
    """
    tops = deepcopy(polygon[: len(polygon) // 2])
    bots = deepcopy(polygon[: len(polygon) // 2 - 1 : -1])

    if self.crop_jitter is not None:

        heights = []
        for i, (t, b) in enumerate(zip(tops, bots)):
            heights.append(np.linalg.norm(b - t))

            crop_jitter = self.crop_jitter * heights[-1]

            x_jitter = random.uniform(-crop_jitter, crop_jitter)
            tops[i, 0] += x_jitter
            bots[i, 0] += x_jitter

        height = np.mean(heights)
        y_jitter = self.crop_jitter * height

        tops[:, 1] += random.uniform(-y_jitter, y_jitter)
        bots[:, 1] += random.uniform(-y_jitter, y_jitter)

    elif self.crop_jitter_rate is not None:

        for i, (t, b) in enumerate(zip(tops, bots)):
            height = np.linalg.norm(b - t)

            crop_jitter_short = self.crop_jitter_rate[0] * height
            crop_jitter_long = self.crop_jitter_rate[1] * height

            top_y_jitter = random.uniform(-crop_jitter_long, crop_jitter_short)
            bot_y_jitter = random.uniform(-crop_jitter_short, crop_jitter_long)
            if i == 0:
                x_jitter = random.uniform(-crop_jitter_long, crop_jitter_short)
            elif i == len(tops) - 1:
                x_jitter = random.uniform(-crop_jitter_short, crop_jitter_long)
            else:
                x_jitter = random.uniform(-crop_jitter_short, crop_jitter_short)

            tops[i, :] += (x_jitter, top_y_jitter)
            bots[i, :] += (x_jitter, bot_y_jitter)

    # polygon_out = np.concatenate((tops, bots[::-1]), axis=0)

    return tops, bots

polygon_cropper

PolygonCropper 

PolygonCropper(mode: str = 'center', crop_w: int = 512, crop_h: int = 512, patch_per_polygon: int = 1, random_patch_per_img: int = 1, max_patch_per_img: Optional[int] = None, force_random_patch_normal: bool = False, strict_inner_patch: bool = False, fixed_polygon_order: bool = False)

Bases: BaseCropper

Crop image patches using polygons

Attributes:

  • mode (str) –

    cropping mode. ["center", "defectrandom"] available

  • crop_w (int) –

    cropping width for image patch.

  • crop_h (int) –

    cropping height for image patch.

  • half_w (int) –

    half of cropping width.

  • half_h (int) –

    half of cropping height.

  • patch_per_polygon (int) –

    number of repeat for cropping patch around each polygon.

  • random_patch_per_img (int) –

    number of random patch cropping per image.

  • force_random_patch_normal (bool) –

    whether random patch should not contain polygon area.

  • strict_inner_patch (bool) –

    whether not allowing outer area of image.

  • fixed_polygon_order (bool) –

    whether order of polygons is fixed during cropping.

All input is directly set to class attribute.

Parameters:

  • mode (str, default: 'center' ) –

    Defaults to "center".

  • crop_w (int, default: 512 ) –

    Defaults to 512.

  • crop_h (int, default: 512 ) –

    Defaults to 512.

  • patch_per_polygon (int, default: 1 ) –

    Defaults to 1.

  • random_patch_per_img (int, default: 1 ) –

    Defaults to 1.

  • force_random_patch_normal (bool, default: False ) –

    Defaults to False.

  • strict_inner_patch (bool, default: False ) –

    Defaults to False.

  • fixed_polygon_order (bool, default: False ) –

    Defaults to False.

Source code in SaigeToolkit/data/crop/polygon_cropper.py 
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
def __init__(
    self,
    mode: str = "center",
    crop_w: int = 512,
    crop_h: int = 512,
    patch_per_polygon: int = 1,
    random_patch_per_img: int = 1,
    max_patch_per_img: Optional[int] = None,
    force_random_patch_normal: bool = False,
    strict_inner_patch: bool = False,
    fixed_polygon_order: bool = False,
) -> None:
    """All input is directly set to class attribute.

    Args:
        mode (str, optional): Defaults to "center".
        crop_w (int, optional): Defaults to 512.
        crop_h (int, optional): Defaults to 512.
        patch_per_polygon (int, optional): Defaults to 1.
        random_patch_per_img (int, optional): Defaults to 1.
        force_random_patch_normal (bool, optional): Defaults to False.
        strict_inner_patch (bool, optional): Defaults to False.
        fixed_polygon_order (bool, optional): Defaults to False.
    """
    self.mode = mode
    self.crop_w = crop_w
    self.crop_h = crop_h
    self.patch_per_polygon = patch_per_polygon
    self.random_patch_per_img = random_patch_per_img
    self.max_patch_per_img = max_patch_per_img
    self.force_random_patch_normal = force_random_patch_normal
    self.strict_inner_patch = strict_inner_patch
    self.fixed_polygon_order = fixed_polygon_order
get_n_patch 
get_n_patch(polygons: List[ndarray]) -> int

PolygonCropper crops all polygon areas {self.patch_per_polygon} times per polygons, and also crops random position of image {self.random_patch_per_img} times per images. Therefore, resultant number of patches is weighted-sum result as coded below.

Parameters:

  • polygons (List[ndarray]) –

    polygons of single image in dataset.

Returns:

  • int ( int ) –

    number of patches to be cropped in single image

Source code in SaigeToolkit/data/crop/polygon_cropper.py 
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
def get_n_patch(self, polygons: List[np.ndarray]) -> int:
    """PolygonCropper crops all polygon areas {self.patch_per_polygon} times per polygons,
    and also crops random position of image {self.random_patch_per_img} times per images.
    Therefore, resultant number of patches is weighted-sum result as coded below.

    Args:
        polygons (List[np.ndarray]): polygons of single image in dataset.

    Returns:
        int: number of patches to be cropped in single image
    """
    n_patch = len(polygons) * self.patch_per_polygon + self.random_patch_per_img
    if self.max_patch_per_img is not None:
        n_patch = min(n_patch, self.max_patch_per_img)

    return n_patch
pick_point 
pick_point(polygon: ndarray) -> Tuple[int, int]

picking cropping center point.

Parameters:

  • polygon (ndarray) –

    original polygon data

Raises:

  • NotImplementedError –

    only two modes ["center", "defectrandom"] available

Returns:

  • Tuple[int, int] –

    Tuple[int, int]: picked point, (h_center, w_center)

Source code in SaigeToolkit/data/crop/polygon_cropper.py 
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
def pick_point(self, polygon: np.ndarray) -> Tuple[int, int]:
    """picking cropping center point.

    Args:
        polygon (np.ndarray): original polygon data

    Raises:
        NotImplementedError: only two modes ["center", "defectrandom"] available

    Returns:
        Tuple[int, int]: picked point, (h_center, w_center)
    """
    plg = list(zip(*polygon))
    pts_w, pts_h = np.array(plg[0]), np.array(plg[1])
    h_min, h_max = np.min(pts_h), np.max(pts_h)
    w_min, w_max = np.min(pts_w), np.max(pts_w)

    if self.mode == "center":
        h_center = int(np.round((h_min + h_max) / 2.0))
        w_center = int(np.round((w_min + w_max) / 2.0))

    elif self.mode == "defectrandom":
        count = 0
        max_count = 10
        while True:
            h_center = np.random.randint(h_min, h_max + 1)
            w_center = np.random.randint(w_min, w_max + 1)

            if cv2.pointPolygonTest(polygon, (w_center, h_center), False) >= 0:
                break

            if count > max_count:
                h_center = int(np.round((h_min + h_max) / 2.0))
                w_center = int(np.round((w_min + w_max) / 2.0))
                break
            count += 1

    else:
        raise NotImplementedError

    return h_center, w_center
get_coordinates_from_center 
get_coordinates_from_center(h_center: int, w_center: int, h: int, w: int) -> Tuple[int, int, int, int]

get left/right top/bottom coordinates from picked center point.

Parameters:

  • h_center (int) –

    picked center point h-coordinate

  • w_center (int) –

    picked center point w-coordinate

  • h (int) –

    image size height

  • w (int) –

    image size width

Returns:

  • Tuple[int, int, int, int] –

    Tuple[int, int, int, int]: (crop_left, crop_top, crop_right, crop_bottom)

Source code in SaigeToolkit/data/crop/polygon_cropper.py 
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
def get_coordinates_from_center(
    self, h_center: int, w_center: int, h: int, w: int
) -> Tuple[int, int, int, int]:
    """get left/right top/bottom coordinates from picked center point.

    Args:
        h_center (int): picked center point h-coordinate
        w_center (int): picked center point w-coordinate
        h (int): image size height
        w (int): image size width

    Returns:
        Tuple[int, int, int, int]: (crop_left, crop_top, crop_right, crop_bottom)
    """
    if self.strict_inner_patch:
        w_center = np.clip(w_center, self.half_w, w - self.crop_w + self.half_w)
        h_center = np.clip(h_center, self.half_h, h - self.crop_h + self.half_h)
    crop_left = w_center - self.half_w
    crop_right = crop_left + self.crop_w
    crop_top = h_center - self.half_h
    crop_bottom = crop_top + self.crop_h
    return (crop_left, crop_top, crop_right, crop_bottom)
__call__ 
__call__(image: Union[Image, ndarray], mask: Union[Image, ndarray], polygons: Optional[List[ndarray]] = None, **kwargs) -> List[dict]

crop image into image patches.

Parameters:

  • image (Union[Image, ndarray]) –

    original image

  • polygons (List[ndarray], default: None ) –

    polygon data (# of polygons, (4, 2)) polygon should have 4 points

  • mask (Union[Image, ndarray]) –

    segmentation mask image

Returns:

  • List[dict] –

    List[dict]: list of cropped image data dict

Source code in SaigeToolkit/data/crop/polygon_cropper.py 
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
def __call__(
    self,
    image: Union[Image.Image, np.ndarray],
    mask: Union[Image.Image, np.ndarray],
    polygons: Optional[List[np.ndarray]] = None,
    **kwargs,
) -> List[dict]:
    """crop image into image patches.

    Args:
        image (Union[Image.Image, np.ndarray]): original image
        polygons (List[np.ndarray]): polygon data
            (# of polygons, (4, 2)) polygon should have 4 points
        mask (Union[Image.Image, np.ndarray]): segmentation mask image

    Returns:
        List[dict]: list of cropped image data dict
    """

    w, h = read_image_size(image)

    if polygons is None:
        polygons, _ = cv2.findContours(np.array(mask), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
        polygons = [polygon[:, 0] for polygon in polygons]  # TODO: check..

    l_data = []

    # crop defect polygon
    _polygons = polygons if self.fixed_polygon_order else random.sample(polygons, len(polygons))
    for polygon in _polygons:
        for _ in range(self.patch_per_polygon):
            h_center, w_center = self.pick_point(polygon)
            crop_coordinates = self.get_coordinates_from_center(h_center, w_center, h, w)
            l_data.append(
                {
                    "image": crop(image, crop_coordinates),
                    "mask": crop(mask, crop_coordinates),
                    **kwargs,
                }
            )
            if (
                self.max_patch_per_img is not None
                and len(l_data) + self.random_patch_per_img >= self.max_patch_per_img
            ):
                break
        else:
            continue
        break

    # crop undefect patches
    for _ in range(self.random_patch_per_img):
        while True:
            crop_coordinates = self.get_coordinates_from_center(
                np.random.randint(h), np.random.randint(w), h, w
            )

            patch_lbl = crop(mask, crop_coordinates)
            if (np.array(patch_lbl).sum() == 0) or (not self.force_random_patch_normal):
                break

        l_data.append(
            {
                "image": crop(image, crop_coordinates),
                "mask": patch_lbl,
                **kwargs,
            }
        )

    return l_data