Skip to content

builder

data.crop.builder

logger module-attribute 

logger = getLogger('SaigeResearch')

CROPPER_IMPLEMENTATION module-attribute 

CROPPER_IMPLEMENTATION = {'ocr': OcrCropper, 'polygon': PolygonCropper, 'edge': EdgeCropper}

BaseCropper

Bases: ABC

get_n_patch abstractmethod 

get_n_patch(*_) -> int
Source code in SaigeToolkit/data/crop/base_cropper.py 
11
12
13
@abstractmethod
def get_n_patch(self, *_) -> int:
    pass

__call__ abstractmethod 

__call__(*_) -> Any
Source code in SaigeToolkit/data/crop/base_cropper.py 
15
16
17
@abstractmethod
def __call__(self, *_) -> Any:
    pass

PolygonCropper 

PolygonCropper(mode: str = 'center', crop_w: int = 512, crop_h: int = 512, patch_per_polygon: int = 1, random_patch_per_img: int = 1, max_patch_per_img: Optional[int] = None, force_random_patch_normal: bool = False, strict_inner_patch: bool = False, fixed_polygon_order: bool = False)

Bases: BaseCropper

Crop image patches using polygons

Attributes:

  • mode (str) –

    cropping mode. ["center", "defectrandom"] available

  • crop_w (int) –

    cropping width for image patch.

  • crop_h (int) –

    cropping height for image patch.

  • half_w (int) –

    half of cropping width.

  • half_h (int) –

    half of cropping height.

  • patch_per_polygon (int) –

    number of repeat for cropping patch around each polygon.

  • random_patch_per_img (int) –

    number of random patch cropping per image.

  • force_random_patch_normal (bool) –

    whether random patch should not contain polygon area.

  • strict_inner_patch (bool) –

    whether not allowing outer area of image.

  • fixed_polygon_order (bool) –

    whether order of polygons is fixed during cropping.

All input is directly set to class attribute.

Parameters:

  • mode (str, default: 'center' ) –

    Defaults to "center".

  • crop_w (int, default: 512 ) –

    Defaults to 512.

  • crop_h (int, default: 512 ) –

    Defaults to 512.

  • patch_per_polygon (int, default: 1 ) –

    Defaults to 1.

  • random_patch_per_img (int, default: 1 ) –

    Defaults to 1.

  • force_random_patch_normal (bool, default: False ) –

    Defaults to False.

  • strict_inner_patch (bool, default: False ) –

    Defaults to False.

  • fixed_polygon_order (bool, default: False ) –

    Defaults to False.

Source code in SaigeToolkit/data/crop/polygon_cropper.py 
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
def __init__(
    self,
    mode: str = "center",
    crop_w: int = 512,
    crop_h: int = 512,
    patch_per_polygon: int = 1,
    random_patch_per_img: int = 1,
    max_patch_per_img: Optional[int] = None,
    force_random_patch_normal: bool = False,
    strict_inner_patch: bool = False,
    fixed_polygon_order: bool = False,
) -> None:
    """All input is directly set to class attribute.

    Args:
        mode (str, optional): Defaults to "center".
        crop_w (int, optional): Defaults to 512.
        crop_h (int, optional): Defaults to 512.
        patch_per_polygon (int, optional): Defaults to 1.
        random_patch_per_img (int, optional): Defaults to 1.
        force_random_patch_normal (bool, optional): Defaults to False.
        strict_inner_patch (bool, optional): Defaults to False.
        fixed_polygon_order (bool, optional): Defaults to False.
    """
    self.mode = mode
    self.crop_w = crop_w
    self.crop_h = crop_h
    self.patch_per_polygon = patch_per_polygon
    self.random_patch_per_img = random_patch_per_img
    self.max_patch_per_img = max_patch_per_img
    self.force_random_patch_normal = force_random_patch_normal
    self.strict_inner_patch = strict_inner_patch
    self.fixed_polygon_order = fixed_polygon_order

mode instance-attribute 

mode = mode

crop_w property writable 

crop_w

crop_h property writable 

crop_h

patch_per_polygon instance-attribute 

patch_per_polygon = patch_per_polygon

random_patch_per_img instance-attribute 

random_patch_per_img = random_patch_per_img

max_patch_per_img instance-attribute 

max_patch_per_img = max_patch_per_img

force_random_patch_normal instance-attribute 

force_random_patch_normal = force_random_patch_normal

strict_inner_patch instance-attribute 

strict_inner_patch = strict_inner_patch

fixed_polygon_order instance-attribute 

fixed_polygon_order = fixed_polygon_order

get_n_patch 

get_n_patch(polygons: List[ndarray]) -> int

PolygonCropper crops all polygon areas {self.patch_per_polygon} times per polygons, and also crops random position of image {self.random_patch_per_img} times per images. Therefore, resultant number of patches is weighted-sum result as coded below.

Parameters:

  • polygons (List[ndarray]) –

    polygons of single image in dataset.

Returns:

  • int ( int ) –

    number of patches to be cropped in single image

Source code in SaigeToolkit/data/crop/polygon_cropper.py 
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
def get_n_patch(self, polygons: List[np.ndarray]) -> int:
    """PolygonCropper crops all polygon areas {self.patch_per_polygon} times per polygons,
    and also crops random position of image {self.random_patch_per_img} times per images.
    Therefore, resultant number of patches is weighted-sum result as coded below.

    Args:
        polygons (List[np.ndarray]): polygons of single image in dataset.

    Returns:
        int: number of patches to be cropped in single image
    """
    n_patch = len(polygons) * self.patch_per_polygon + self.random_patch_per_img
    if self.max_patch_per_img is not None:
        n_patch = min(n_patch, self.max_patch_per_img)

    return n_patch

pick_point 

pick_point(polygon: ndarray) -> Tuple[int, int]

picking cropping center point.

Parameters:

  • polygon (ndarray) –

    original polygon data

Raises:

  • NotImplementedError –

    only two modes ["center", "defectrandom"] available

Returns:

  • Tuple[int, int] –

    Tuple[int, int]: picked point, (h_center, w_center)

Source code in SaigeToolkit/data/crop/polygon_cropper.py 
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
def pick_point(self, polygon: np.ndarray) -> Tuple[int, int]:
    """picking cropping center point.

    Args:
        polygon (np.ndarray): original polygon data

    Raises:
        NotImplementedError: only two modes ["center", "defectrandom"] available

    Returns:
        Tuple[int, int]: picked point, (h_center, w_center)
    """
    plg = list(zip(*polygon))
    pts_w, pts_h = np.array(plg[0]), np.array(plg[1])
    h_min, h_max = np.min(pts_h), np.max(pts_h)
    w_min, w_max = np.min(pts_w), np.max(pts_w)

    if self.mode == "center":
        h_center = int(np.round((h_min + h_max) / 2.0))
        w_center = int(np.round((w_min + w_max) / 2.0))

    elif self.mode == "defectrandom":
        count = 0
        max_count = 10
        while True:
            h_center = np.random.randint(h_min, h_max + 1)
            w_center = np.random.randint(w_min, w_max + 1)

            if cv2.pointPolygonTest(polygon, (w_center, h_center), False) >= 0:
                break

            if count > max_count:
                h_center = int(np.round((h_min + h_max) / 2.0))
                w_center = int(np.round((w_min + w_max) / 2.0))
                break
            count += 1

    else:
        raise NotImplementedError

    return h_center, w_center

get_coordinates_from_center 

get_coordinates_from_center(h_center: int, w_center: int, h: int, w: int) -> Tuple[int, int, int, int]

get left/right top/bottom coordinates from picked center point.

Parameters:

  • h_center (int) –

    picked center point h-coordinate

  • w_center (int) –

    picked center point w-coordinate

  • h (int) –

    image size height

  • w (int) –

    image size width

Returns:

  • Tuple[int, int, int, int] –

    Tuple[int, int, int, int]: (crop_left, crop_top, crop_right, crop_bottom)

Source code in SaigeToolkit/data/crop/polygon_cropper.py 
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
def get_coordinates_from_center(
    self, h_center: int, w_center: int, h: int, w: int
) -> Tuple[int, int, int, int]:
    """get left/right top/bottom coordinates from picked center point.

    Args:
        h_center (int): picked center point h-coordinate
        w_center (int): picked center point w-coordinate
        h (int): image size height
        w (int): image size width

    Returns:
        Tuple[int, int, int, int]: (crop_left, crop_top, crop_right, crop_bottom)
    """
    if self.strict_inner_patch:
        w_center = np.clip(w_center, self.half_w, w - self.crop_w + self.half_w)
        h_center = np.clip(h_center, self.half_h, h - self.crop_h + self.half_h)
    crop_left = w_center - self.half_w
    crop_right = crop_left + self.crop_w
    crop_top = h_center - self.half_h
    crop_bottom = crop_top + self.crop_h
    return (crop_left, crop_top, crop_right, crop_bottom)

__call__ 

__call__(image: Union[Image, ndarray], mask: Union[Image, ndarray], polygons: Optional[List[ndarray]] = None, **kwargs) -> List[dict]

crop image into image patches.

Parameters:

  • image (Union[Image, ndarray]) –

    original image

  • polygons (List[ndarray], default: None ) –

    polygon data (# of polygons, (4, 2)) polygon should have 4 points

  • mask (Union[Image, ndarray]) –

    segmentation mask image

Returns:

  • List[dict] –

    List[dict]: list of cropped image data dict

Source code in SaigeToolkit/data/crop/polygon_cropper.py 
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
def __call__(
    self,
    image: Union[Image.Image, np.ndarray],
    mask: Union[Image.Image, np.ndarray],
    polygons: Optional[List[np.ndarray]] = None,
    **kwargs,
) -> List[dict]:
    """crop image into image patches.

    Args:
        image (Union[Image.Image, np.ndarray]): original image
        polygons (List[np.ndarray]): polygon data
            (# of polygons, (4, 2)) polygon should have 4 points
        mask (Union[Image.Image, np.ndarray]): segmentation mask image

    Returns:
        List[dict]: list of cropped image data dict
    """

    w, h = read_image_size(image)

    if polygons is None:
        polygons, _ = cv2.findContours(np.array(mask), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
        polygons = [polygon[:, 0] for polygon in polygons]  # TODO: check..

    l_data = []

    # crop defect polygon
    _polygons = polygons if self.fixed_polygon_order else random.sample(polygons, len(polygons))
    for polygon in _polygons:
        for _ in range(self.patch_per_polygon):
            h_center, w_center = self.pick_point(polygon)
            crop_coordinates = self.get_coordinates_from_center(h_center, w_center, h, w)
            l_data.append(
                {
                    "image": crop(image, crop_coordinates),
                    "mask": crop(mask, crop_coordinates),
                    **kwargs,
                }
            )
            if (
                self.max_patch_per_img is not None
                and len(l_data) + self.random_patch_per_img >= self.max_patch_per_img
            ):
                break
        else:
            continue
        break

    # crop undefect patches
    for _ in range(self.random_patch_per_img):
        while True:
            crop_coordinates = self.get_coordinates_from_center(
                np.random.randint(h), np.random.randint(w), h, w
            )

            patch_lbl = crop(mask, crop_coordinates)
            if (np.array(patch_lbl).sum() == 0) or (not self.force_random_patch_normal):
                break

        l_data.append(
            {
                "image": crop(image, crop_coordinates),
                "mask": patch_lbl,
                **kwargs,
            }
        )

    return l_data

OcrCropper 

OcrCropper(chr_height: int = 32, crop_jitter: Optional[float] = None, crop_jitter_rate: Optional[List[float]] = None, save_polygons: bool = False)

Cropper class for OCR task.

Attributes:

  • chr_height (int) –

    cropped image patch is resized to be the same height as chr_height.

  • crop_jitter (Optional[float]) –

    if not None, cropping box is jittered horizontally with ratio crop_jitter.

  • crop_jitter_rate (Optional[List[float]]) –

    if not None, cropping box points are jittered to any direction with ratio crop_jitter_rate.

  • save_polygons (bool) –

    whether polygon label is preserved after cropping.

initializing OcrCropper. all initializing input is set to class attribute.

Parameters:

  • chr_height (int, default: 32 ) –

    Defaults to 32.

  • crop_jitter (Optional[float], default: None ) –

    Defaults to None.

  • crop_jitter_rate (Optional[List[float]], default: None ) –

    Defaults to None.

  • save_polygons (bool, default: False ) –

    Defaults to False.

Source code in SaigeToolkit/data/crop/ocr_cropper.py 
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
def __init__(
    self,
    chr_height: int = 32,
    crop_jitter: Optional[float] = None,
    crop_jitter_rate: Optional[List[float]] = None,
    save_polygons: bool = False,
) -> None:
    """initializing OcrCropper.
    all initializing input is set to class attribute.

    Args:
        chr_height (int): Defaults to 32.
        crop_jitter (Optional[float], optional): Defaults to None.
        crop_jitter_rate (Optional[List[float]], optional): Defaults to None.
        save_polygons (bool): Defaults to False.
    """
    self.chr_height = chr_height
    self.crop_jitter = crop_jitter
    self.crop_jitter_rate = crop_jitter_rate
    self.save_polygons = save_polygons

chr_height instance-attribute 

chr_height = chr_height

crop_jitter instance-attribute 

crop_jitter = crop_jitter

crop_jitter_rate instance-attribute 

crop_jitter_rate = crop_jitter_rate

save_polygons instance-attribute 

save_polygons = save_polygons

get_n_patch 

get_n_patch(polygons: List[List[ndarray]]) -> int

OcrCropper only crops all polygon areas, unlike SegCropper. Therefore, resultant number of patches is simply equals to number of polygons.

Parameters:

  • polygons (List[List[ndarray]]) –

    polygons of all images in dataset.

Returns:

  • int ( int ) –

    total number of patches to be cropped

Source code in SaigeToolkit/data/crop/ocr_cropper.py 
47
48
49
50
51
52
53
54
55
56
57
def get_n_patch(self, polygons: List[List[np.ndarray]]) -> int:
    """OcrCropper only crops all polygon areas, unlike SegCropper.
    Therefore, resultant number of patches is simply equals to number of polygons.

    Args:
        polygons (List[List[np.ndarray]]): polygons of all images in dataset.

    Returns:
        int: total number of patches to be cropped
    """
    return len(polygons)

__call__ 

__call__(image: Union[Image, ndarray], polygons: List[ndarray], strings: List[str], ignore: List[bool], is_vertical: Optional[List[bool]] = None, **kwargs) -> List[dict]

crop image into image patches. polygon, string, ignore data should be same length.

Parameters:

  • image (Union[Image, ndarray]) –

    original image

  • polygons (List[ndarray]) –

    polygon data (# of polygons, (4, 2)) polygon should have 4 points

  • strings (List[str]) –

    string data

  • ignore (List[bool]) –

    ignore notation data

  • is_vertical (List[bool], default: None ) –

    is_vertical notation data

Returns:

  • List[dict] –

    List[dict]: list of cropped image data dict

Source code in SaigeToolkit/data/crop/ocr_cropper.py 
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
def __call__(
    self,
    image: Union[Image.Image, np.ndarray],
    polygons: List[np.ndarray],
    strings: List[str],
    ignore: List[bool],
    is_vertical: Optional[List[bool]] = None,
    **kwargs,
) -> List[dict]:
    """crop image into image patches.
    polygon, string, ignore data should be same length.

    Args:
        image (Union[Image.Image, np.ndarray]): original image
        polygons (List[np.ndarray]): polygon data
            (# of polygons, (4, 2)) polygon should have 4 points
        strings (List[str]): string data
        ignore (List[bool]): ignore notation data
        is_vertical (List[bool]): is_vertical notation data

    Returns:
        List[dict]: list of cropped image data dict
    """
    image = np.asarray(image)

    if is_vertical is None:
        is_vertical = [False] * len(polygons)

    l_data = []
    for pol, string, ign, is_vert in zip(polygons, strings, ignore, is_vertical):
        if ign:
            continue

        max_trial = 10
        trial = 0
        while trial < max_trial:
            trial += 1

            tops, bots = self.jitter_polygon(pol)
            widths, heights = self.calculate_width_height(tops, bots)

            if all(np.array(widths + heights) >= 1):
                break

            if trial == max_trial:
                logger.info("Max trial exeeded.. no jitter applied.")
                tops = pol[: len(pol) // 2]
                bots = pol[: len(pol) // 2 - 1 : -1]
                widths, heights = self.calculate_width_height(tops, bots)

        if any(np.array(widths + heights) < 1):
            logger.warning(f"Invalid patch shape (jittered) encounter.. string: {string}")
            continue

        text_patch_list = []
        for i, (w, h) in enumerate(zip(widths, heights)):
            width = int(np.round((w / h * self.chr_height)))
            if width == 0:
                continue

            startpoints = np.array([tops[i], tops[i + 1], bots[i + 1], bots[i]], np.float32)
            endpoints = np.array(
                [
                    [0, 0],
                    [width - 1, 0],
                    [width - 1, self.chr_height - 1],
                    [0, self.chr_height - 1],
                ],
                np.float32,
            )

            affine_matrix = cv2.getPerspectiveTransform(startpoints, endpoints)
            text_patch = cv2.warpPerspective(
                image, affine_matrix, (width, self.chr_height), borderValue=0
            )
            if i != 0:
                # remove duplicated leftmost column
                # border area is duplicated in adjacent patches.
                text_patch = text_patch[:, 1:]
            text_patch_list.append(text_patch)

        text_patch = np.concatenate(text_patch_list, axis=1)

        if int(text_patch.shape[1] / (self.chr_height / 8)) + 1 >= len(string):
            patch_data = {
                "image": Image.fromarray(text_patch),
                "strings": string,
                "is_vertical": is_vert,
            }

            if self.save_polygons:
                patch_data.update(polygons=pol)

            l_data.append(patch_data)
        else:
            logger.warning(
                f"Invalid patch length! string: {string} patch width: {text_patch.shape[1]}"
            )

    return l_data

calculate_width_height 

calculate_width_height(tops: ndarray, bots: ndarray) -> Tuple[List[ndarray], List[ndarray]]

calculate width and heights for every four polygon points (2 tops and 2 bots). Assuming that polygon is composed of top-points and bot-points, and every top and bot points are paired.

Parameters:

  • tops (ndarray) –

    top points of polygon

  • bots (ndarray) –

    bot points of polygon

Returns:

  • Tuple[List[ndarray], List[ndarray]] –

    Tuple[List[np.ndarray]]: width and height list for every four points.

Source code in SaigeToolkit/data/crop/ocr_cropper.py 
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
def calculate_width_height(
    self, tops: np.ndarray, bots: np.ndarray
) -> Tuple[List[np.ndarray], List[np.ndarray]]:
    """calculate width and heights for every four polygon points (2 tops and 2 bots).
    Assuming that polygon is composed of top-points and bot-points,
    and every top and bot points are paired.

    Args:
        tops (np.ndarray): top points of polygon
        bots (np.ndarray): bot points of polygon

    Returns:
        Tuple[List[np.ndarray]]: width and height list for every four points.
    """

    widths = []
    for i in range(len(tops) - 1):
        widths.append(
            np.mean(
                (
                    np.linalg.norm(tops[i + 1] - tops[i]),
                    np.linalg.norm(bots[i + 1] - bots[i]),
                )
            )
        )

    heights = []
    for t, b in zip(tops, bots):
        heights.append(np.linalg.norm(b - t))
    heights = [np.mean([heights[i + 1], heights[i]]) for i in range(len(heights) - 1)]

    return widths, heights

jitter_polygon 

jitter_polygon(polygon: ndarray) -> Tuple[ndarray, ndarray]

jittering polygon points and split into tops and bots points

Parameters:

  • polygon (ndarray) –

    original polygon data

Returns:

  • Tuple[ndarray, ndarray] –

    Tuple[np.ndarray, np.ndarray]: jittered tops, bots points

Source code in SaigeToolkit/data/crop/ocr_cropper.py 
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
def jitter_polygon(self, polygon: np.ndarray) -> Tuple[np.ndarray, np.ndarray]:
    """jittering polygon points and split into tops and bots points

    Args:
        polygon (np.ndarray): original polygon data

    Returns:
        Tuple[np.ndarray, np.ndarray]: jittered tops, bots points
    """
    tops = deepcopy(polygon[: len(polygon) // 2])
    bots = deepcopy(polygon[: len(polygon) // 2 - 1 : -1])

    if self.crop_jitter is not None:

        heights = []
        for i, (t, b) in enumerate(zip(tops, bots)):
            heights.append(np.linalg.norm(b - t))

            crop_jitter = self.crop_jitter * heights[-1]

            x_jitter = random.uniform(-crop_jitter, crop_jitter)
            tops[i, 0] += x_jitter
            bots[i, 0] += x_jitter

        height = np.mean(heights)
        y_jitter = self.crop_jitter * height

        tops[:, 1] += random.uniform(-y_jitter, y_jitter)
        bots[:, 1] += random.uniform(-y_jitter, y_jitter)

    elif self.crop_jitter_rate is not None:

        for i, (t, b) in enumerate(zip(tops, bots)):
            height = np.linalg.norm(b - t)

            crop_jitter_short = self.crop_jitter_rate[0] * height
            crop_jitter_long = self.crop_jitter_rate[1] * height

            top_y_jitter = random.uniform(-crop_jitter_long, crop_jitter_short)
            bot_y_jitter = random.uniform(-crop_jitter_short, crop_jitter_long)
            if i == 0:
                x_jitter = random.uniform(-crop_jitter_long, crop_jitter_short)
            elif i == len(tops) - 1:
                x_jitter = random.uniform(-crop_jitter_short, crop_jitter_long)
            else:
                x_jitter = random.uniform(-crop_jitter_short, crop_jitter_short)

            tops[i, :] += (x_jitter, top_y_jitter)
            bots[i, :] += (x_jitter, bot_y_jitter)

    # polygon_out = np.concatenate((tops, bots[::-1]), axis=0)

    return tops, bots

EdgeCropper 

EdgeCropper(crop_w: int = 512, crop_h: int = 512, random_sample: bool = False, patch_per_polygon: int = 1, strict_inner_patch: bool = True, ok_patch_prob: float = 0.01)

Bases: BaseCropper

Crop image patches along the contour of mask (or polygon)

Attributes:

Source code in SaigeToolkit/data/crop/edge_cropper.py 
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
def __init__(
    self,
    crop_w: int = 512,
    crop_h: int = 512,
    random_sample: bool = False,
    patch_per_polygon: int = 1,
    strict_inner_patch: bool = True,
    ok_patch_prob: float = 0.01,
) -> None:
    self.crop_w = crop_w
    self.crop_h = crop_h
    self.window_manager = WindowManager(crop_w, crop_h, strict_inner_patch)
    self.random_sample = random_sample
    self.patch_per_polygon = patch_per_polygon
    self.ok_patch_prob = ok_patch_prob
    self.n_defect_patch_made = 0

    self.init_centers()

crop_w instance-attribute 

crop_w = crop_w

crop_h instance-attribute 

crop_h = crop_h

window_manager instance-attribute 

window_manager = WindowManager(crop_w, crop_h, strict_inner_patch)

random_sample instance-attribute 

random_sample = random_sample

patch_per_polygon instance-attribute 

patch_per_polygon = patch_per_polygon

ok_patch_prob instance-attribute 

ok_patch_prob = ok_patch_prob

n_defect_patch_made instance-attribute 

n_defect_patch_made = 0

init_centers 

init_centers()
Source code in SaigeToolkit/data/crop/edge_cropper.py 
289
290
def init_centers(self):
    self.centers = []

get_n_patch 

get_n_patch(polygons: List[ndarray]) -> int

PolygonCropper crops all polygon areas {self.patch_per_polygon} times per polygons, and also crops random position of image {self.random_patch_per_img} times per images. Therefore, resultant number of patches is weighted-sum result as coded below.

Parameters:

  • polygons (List[ndarray]) –

    polygons of single image in dataset.

Returns:

  • int ( int ) –

    number of patches to be cropped in single image

Source code in SaigeToolkit/data/crop/edge_cropper.py 
292
293
294
295
296
297
298
299
300
301
302
303
def get_n_patch(self, polygons: List[np.ndarray]) -> int:
    """PolygonCropper crops all polygon areas {self.patch_per_polygon} times per polygons,
    and also crops random position of image {self.random_patch_per_img} times per images.
    Therefore, resultant number of patches is weighted-sum result as coded below.

    Args:
        polygons (List[np.ndarray]): polygons of single image in dataset.

    Returns:
        int: number of patches to be cropped in single image
    """
    raise NotImplementedError

__call__ 

__call__(image: Union[Image, ndarray], mask: Union[Image, ndarray], polygons: Optional[List[ndarray]] = None, **kwargs) -> List[dict]

crop image into image patches.

Parameters:

  • image (Union[Image, ndarray]) –

    original image

  • polygons (List[ndarray], default: None ) –

    polygon data (# of polygons, (4, 2)) polygon should have 4 points

  • mask (Union[Image, ndarray]) –

    segmentation mask image

Returns:

  • List[dict] –

    List[dict]: list of cropped image data dict

Source code in SaigeToolkit/data/crop/edge_cropper.py 
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
def __call__(
    self,
    image: Union[Image.Image, np.ndarray],
    mask: Union[Image.Image, np.ndarray],
    polygons: Optional[List[np.ndarray]] = None,
    **kwargs,
) -> List[dict]:
    """crop image into image patches.

    Args:
        image (Union[Image.Image, np.ndarray]): original image
        polygons (List[np.ndarray]): polygon data
            (# of polygons, (4, 2)) polygon should have 4 points
        mask (Union[Image.Image, np.ndarray]): segmentation mask image

    Returns:
        List[dict]: list of cropped image data dict
    """

    w, h = read_image_size(image)

    if polygons is None:
        _polygons, _ = cv2.findContours(np.asarray(mask), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
        polygons = [polygon[:, 0] for polygon in _polygons]  # TODO: check..

    l_data = []
    self.init_centers()

    # crop defect polygon
    _polygons = random.sample(polygons, len(polygons)) if self.random_sample else polygons
    for polygon in _polygons:
        polygon = subdivide_points_into_min_resolution(list(polygon), 32, 32)
        for _ in range(self.patch_per_polygon):
            max_iter_per_polygon = 3 if self.random_sample else 1  # TODO: as class variable?
            min_num_point_not_included = None
            for _ in range(max_iter_per_polygon):
                new_centers = self.pick_centers_from_polygon(polygon)

                num_point_not_included = sum(
                    not self.window_manager.is_in_any_window(p, [*self.centers, *new_centers])
                    for p in polygon
                )
                if num_point_not_included == 0:
                    self.centers.extend(new_centers)
                    break

                if min_num_point_not_included is None:
                    min_num_point_not_included = num_point_not_included
                    new_centers_candidate = new_centers

                if num_point_not_included < min_num_point_not_included:
                    min_num_point_not_included = num_point_not_included
                    new_centers_candidate = new_centers

            else:
                # If fail to get `num_point_not_included==0` sample,
                # use min `num_point_not_included` sample.
                new_centers = new_centers_candidate
                self.centers.extend(new_centers)

            for center in new_centers:
                single_defect_patch_data = self.get_data_dict_from_center_point(
                    image, mask, center, **kwargs
                )
                l_data.append(single_defect_patch_data)

    self.n_defect_patch_made += len(l_data)

    # crop ok region
    # XXX: OK patch portion can be larger w.r.t num of ok images.
    probability_multiplier = max(self.n_defect_patch_made, 1)
    if random.random() < self.ok_patch_prob * probability_multiplier:
        num_to_make_ok_patch = ceil(self.ok_patch_prob * probability_multiplier)
        max_iteration = 5
        for _ in range(num_to_make_ok_patch):
            for _ in range(max_iteration):
                if self.window_manager.strict_inner_patch:
                    window_half_w = self.window_manager.half_width
                    window_half_h = self.window_manager.half_height

                    # Cannot sample random window center.
                    if w - window_half_w < window_half_w or h - window_half_h < window_half_h:
                        logger.warning(f"Crop size is larger than image size: ({w}, {h}).")
                        continue

                    random_center = (
                        random.randint(window_half_w, w - window_half_w),
                        random.randint(window_half_h, h - window_half_h),
                    )
                else:
                    random_center = random.randint(0, w), random.randint(0, h)
                if not self.window_manager.is_in_any_window(random_center, self.centers):
                    self.centers.append(random_center)
                    self.n_defect_patch_made -= 1 / self.ok_patch_prob
                    break
            else:
                # Max trial ended with no ok patch.
                return l_data

            single_ok_patch_data = self.get_data_dict_from_center_point(
                image, mask, random_center, **kwargs
            )
            l_data.append(single_ok_patch_data)

    return l_data

get_data_dict_from_center_point 

get_data_dict_from_center_point(image: Union[Image, ndarray], mask: Union[Image, ndarray], center, **kwargs)
Source code in SaigeToolkit/data/crop/edge_cropper.py 
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
def get_data_dict_from_center_point(
    self,
    image: Union[Image.Image, np.ndarray],
    mask: Union[Image.Image, np.ndarray],
    center,
    **kwargs,
):
    w, h = read_image_size(image)
    w_center, h_center = center
    crop_coordinates = self.window_manager.get_coordinates_from_center(
        int(h_center), int(w_center), int(h), int(w)
    )
    return {
        "image": crop(image, crop_coordinates),
        "mask": crop(mask, crop_coordinates),
        "crop_coordinates": crop_coordinates,
        **kwargs,
    }

pick_centers_from_polygon 

pick_centers_from_polygon(polygon_points: List[Iterable[int]]) -> List[Iterable[int]]
Source code in SaigeToolkit/data/crop/edge_cropper.py 
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
def pick_centers_from_polygon(self, polygon_points: List[Iterable[int]]) -> List[Iterable[int]]:
    if self.random_sample:
        max_iter_init = 10 if self.random_sample else 1  # TODO: as class variable?
        for _ in range(max_iter_init):
            idx_start = random.choice(range(len(polygon_points)))

            if not self.window_manager.is_in_any_window(polygon_points[idx_start], self.centers):
                break

            # if fail to get proper init index, restart random choice
        else:
            # if fail to get proper init index in given repeat num, return []
            return []
    else:
        idx_start = 0

    polygon_points = [*polygon_points[idx_start:], *polygon_points[:idx_start]]
    new_centers = [polygon_points[0]]

    (
        polygon_points,
        inner_point_positive,
        inner_point_negative,
    ) = self.remove_inner_points_and_get_outer_points(new_centers[-1], polygon_points)
    if inner_point_positive is None:
        return new_centers

    while polygon_points:
        previous_boundary = CenterWithSatellite(inner_point_positive, self.window_manager)
        idx_inside_checked = 0
        for idx, p in enumerate(polygon_points):
            if self.window_manager.is_in_any_window(p, [*self.centers, *new_centers]):
                continue

            state_as_satellite = previous_boundary.calculate_relation_as_satellite(p)

            # if current point and boundary point of previous center
            # (or current point and satellites of boundary point of previous center)
            # in the condition [cannot be contained in one window],
            # break from searching next center point candidate.
            if (
                isinstance(state_as_satellite, list)
                or state_as_satellite["relation"] != WindowState.IN
            ):
                break
            else:
                # if condition failed, current point is considered as `point inside window`.
                idx_inside_checked = idx

        else:
            polygon_points_for_mid_point = [
                _point
                for _point in polygon_points
                if not self.window_manager.is_in_any_window(_point, [*self.centers, *new_centers])
            ]
            if polygon_points_for_mid_point:
                new_centers.append(get_mid_point(polygon_points_for_mid_point))
            break

        # When finding next center, following points should be contained in the window:
        # boundary point of previous edge and its satelitte points.
        # TODO: 이거 Center 클래스에 넣자
        must_contain_coordinate = [
            previous_boundary.coordinate,
            *[sate["coordinate"] for sate in previous_boundary.satellite_points.values()],
        ]

        for idx_positive_direction in range(idx - 1, idx_inside_checked - 1, -1):
            # `polygon_points[idx_positive_direction]` should in window of previouss edge point,
            # so the next center can be placed right outside of previous window.
            relation_prev_point = self.window_manager.check_window(
                polygon_points[idx_positive_direction], previous_boundary.coordinate
            )
            if relation_prev_point != WindowState.IN:
                continue

            # if needed, state_as_satellite for outside point should be re-calculated.
            if idx_positive_direction != idx - 1:
                state_as_satellite = previous_boundary.calculate_relation_as_satellite(
                    polygon_points[idx_positive_direction + 1]
                )

            p_dst = self.get_proper_point(
                state_as_satellite,
                must_contain_coordinate,
                polygon_points[idx_positive_direction],
                polygon_points[idx_positive_direction + 1],
            )

            # success finding new center point, break
            if p_dst is not None:
                break

        # failed to find more center points, break.
        else:
            break

        new_centers.append(p_dst)
        polygon_points, inner_point_positive, _check = self.remove_inner_points_and_get_outer_points(
            new_centers[-1], polygon_points[idx_positive_direction:]
        )

    return new_centers

get_proper_point 

get_proper_point(state_as_satellite: Union[Dict[str, Any], List[Dict[str, Any]]], must_contain_coordinate: List[Iterable[int]], point_in: Iterable[int], point_out: Iterable[int]) -> Optional[Iterable[int]]

get proper next center point that contains every must_contain_coordinate

Parameters:

  • state_as_satellite (Union[Dict[str, Any], List[Dict[str, Any]]]) –

    relation between center point(s) and target point.

  • must_contain_coordinate (List[Iterable[int]]) –

    points should be in next window

  • point_in (Iterable[int]) –

    point inside window from previous boundary

  • point_out (Iterable[int]) –

    point outside window from previous boundary

Returns:

  • Optional[Iterable[int]] –

    Optional[Iterable[int]]: proper next center (or None)

Source code in SaigeToolkit/data/crop/edge_cropper.py 
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
def get_proper_point(
    self,
    state_as_satellite: Union[Dict[str, Any], List[Dict[str, Any]]],
    must_contain_coordinate: List[Iterable[int]],
    point_in: Iterable[int],
    point_out: Iterable[int],
) -> Optional[Iterable[int]]:
    """get proper next center point that contains every `must_contain_coordinate`

    Args:
        state_as_satellite (Union[Dict[str, Any], List[Dict[str, Any]]]):
            relation between center point(s) and target point.
        must_contain_coordinate (List[Iterable[int]]): points should be in next window
        point_in (Iterable[int]): point inside window from previous boundary
        point_out (Iterable[int]): point outside window from previous boundary

    Returns:
        Optional[Iterable[int]]: proper next center (or None)
    """

    if isinstance(state_as_satellite, list):
        for u_r in state_as_satellite:
            p_dst = self.get_proper_point(u_r, must_contain_coordinate, point_in, point_out)
            if p_dst is not None:
                break

        else:
            return None

    else:
        relation, inner_point = state_as_satellite["relation"], state_as_satellite["coordinate"]
        p_dst = self.get_destination_point(relation, inner_point, point_in, point_out)

        if p_dst is None:
            return None

        for _p in must_contain_coordinate:
            if not self.window_manager.is_in_window(_p, p_dst):
                return None

    return p_dst

get_destination_point 

get_destination_point(relation: WindowState, point_ref: Iterable[int], point_in: Iterable[int], point_out: Iterable[int]) -> ndarray

calculate point a-window-away from reference point

Parameters:

  • relation (WindowState) –

    relation between reference point and next center

  • point_ref (Iterable[int]) –

    reference point to calculate next center

  • point_in (Iterable[int]) –

    point inside window from previous boundary

  • point_out (Iterable[int]) –

    point outside window from previous boundary

Returns:

  • ndarray –

    np.ndarray: result destination point

Source code in SaigeToolkit/data/crop/edge_cropper.py 
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
def get_destination_point(
    self,
    relation: WindowState,
    point_ref: Iterable[int],
    point_in: Iterable[int],
    point_out: Iterable[int],
) -> np.ndarray:
    """calculate point a-window-away from reference point

    Args:
        relation (WindowState): relation between reference point and next center
        point_ref (Iterable[int]): reference point to calculate next center
        point_in (Iterable[int]): point inside window from previous boundary
        point_out (Iterable[int]): point outside window from previous boundary

    Returns:
        np.ndarray: result destination point
    """
    w_in, h_in = point_in
    w_out, h_out = point_out

    def _get_interior_point(a_dst, a_in, a_out, b_in, b_out):
        if (a_out != a_in) and (a_in <= a_dst <= a_out or a_out <= a_dst <= a_in):
            rate = (a_dst - a_in) / (a_out - a_in)
        else:
            # if a_dst is not interior between two points, update a_dst to the middle point.
            a_dst = int(round((a_out + a_in) / 2))
            rate = 0.5

        b_dst = b_in + int(round(rate * (b_out - b_in)))
        return a_dst, b_dst

    if relation == WindowState.WIDTH_POSITIVE_OUT:
        w_dst = point_ref[0] + self.window_manager.half_width
        w_dst, h_dst = _get_interior_point(w_dst, w_in, w_out, h_in, h_out)
    elif relation == WindowState.WIDTH_NEGATIVE_OUT:
        w_dst = point_ref[0] - self.window_manager.half_width
        w_dst, h_dst = _get_interior_point(w_dst, w_in, w_out, h_in, h_out)
    elif relation == WindowState.HEIGHT_POSITIVE_OUT:
        h_dst = point_ref[1] + self.window_manager.half_height
        h_dst, w_dst = _get_interior_point(h_dst, h_in, h_out, w_in, w_out)
    elif relation == WindowState.HEIGHT_NEGATIVE_OUT:
        h_dst = point_ref[1] - self.window_manager.half_height
        h_dst, w_dst = _get_interior_point(h_dst, h_in, h_out, w_in, w_out)

    else:
        # if relation is `WindowState.In`, find farthest point and set as target point.
        width_distance = w_out - point_ref[0]
        height_distance = h_out - point_ref[1]
        if abs(width_distance) > abs(height_distance):
            if width_distance > 0:
                relation = WindowState.WIDTH_POSITIVE_OUT
            else:
                relation = WindowState.WIDTH_NEGATIVE_OUT
        else:
            if height_distance > 0:
                relation = WindowState.HEIGHT_POSITIVE_OUT
            else:
                relation = WindowState.HEIGHT_NEGATIVE_OUT

        dst = self.get_destination_point(relation, point_ref, point_in, point_out)

        return dst

    return np.array([w_dst, h_dst])

remove_inner_points_and_get_outer_points 

remove_inner_points_and_get_outer_points(center: Iterable[int], points: List[Iterable[int]]) -> Tuple[List[Iterable[int]], Iterable[int], Iterable[int]]

once center decided, check other points and remove insiders.

Parameters:

  • center (Iterable[int]) –

    next center coordinate

  • points (List[Iterable[int]]) –

    current points on polygon

Returns:

  • Tuple[List[Iterable[int]], Iterable[int], Iterable[int]] –

    Tuple[List[Iterable[int]], Iterable[int], Iterable[int]]: - List[Iterable[int]]: clean-up'd polygon points - Iterable[int]: boundary point (positive direction) - Iterable[int]: boundary point (negative direction)

Source code in SaigeToolkit/data/crop/edge_cropper.py 
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
def remove_inner_points_and_get_outer_points(
    self, center: Iterable[int], points: List[Iterable[int]]
) -> Tuple[List[Iterable[int]], Iterable[int], Iterable[int]]:
    """once center decided, check other points and remove insiders.

    Args:
        center (Iterable[int]): next center coordinate
        points (List[Iterable[int]]): current points on polygon

    Returns:
        Tuple[List[Iterable[int]], Iterable[int], Iterable[int]]:
            - List[Iterable[int]]: clean-up'd polygon points
            - Iterable[int]: boundary point (positive direction)
            - Iterable[int]: boundary point (negative direction)
    """
    idx_positive_direction, idx_negative_direction = 0, 0
    p_dst, n_dst = None, None

    for idx, p in enumerate(points):
        relation = self.window_manager.check_window(p, center)
        if relation == WindowState.IN:
            continue

        idx_positive_direction = idx - 1
        p_dst = self.get_destination_point(
            relation,
            center,
            points[idx_positive_direction],
            points[idx_positive_direction + 1],
        )
        break

    for idx, p in enumerate(points[::-1]):
        relation = self.window_manager.check_window(p, center)
        if relation == WindowState.IN:
            continue

        idx_negative_direction = -idx
        n_dst = self.get_destination_point(
            relation,
            center,
            points[idx_negative_direction],
            points[idx_negative_direction - 1],
        )
        break

    if p_dst is None:
        return [], p_dst, n_dst

    points = (
        points[idx_positive_direction + 1 : idx_negative_direction]
        if idx_negative_direction < 0
        else points[idx_positive_direction + 1 :]
    )

    return points, p_dst, n_dst

get_crop_fn 

get_crop_fn(_target_: Optional[str] = None, **cfg_crop: dict) -> Optional[BaseCropper]

getting crop function

Parameters:

  • cfg_crop (dict, default: {} ) –

    config dict for building cropper

Returns:

  • Optional[BaseCropper] –

    Optional[BaseCropper]: saige cropper object. if cfg_crop is None, return None

Source code in SaigeToolkit/data/crop/builder.py 
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
def get_crop_fn(_target_: Optional[str] = None, **cfg_crop: dict) -> Optional[BaseCropper]:
    """getting crop function

    Args:
        cfg_crop (dict): config dict for building cropper

    Returns:
        Optional[BaseCropper]: saige cropper object.
            if cfg_crop is None, return None
    """
    if _target_ is None:
        return None
    cropper = get_crop_class(_target_)
    logger.info(f"[{'CROPPER'.center(9)}] {_target_} [params] {cfg_crop}")
    return cropper(**cfg_crop)

get_crop_class 

get_crop_class(cfg_crop_name: str) -> Type[BaseCropper]

getting crop class from name

Parameters:

  • cfg_crop_name (str) –

    cropper name

Returns:

  • Type[BaseCropper] –

    Type[BaseCropper]: cropper class

Source code in SaigeToolkit/data/crop/builder.py 
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
def get_crop_class(cfg_crop_name: str) -> Type[BaseCropper]:
    """getting crop class from name

    Args:
        cfg_crop_name (str): cropper name

    Returns:
        Type[BaseCropper]: cropper class
    """

    try:
        return CROPPER_IMPLEMENTATION[cfg_crop_name]
    except KeyError as e:
        raise RuntimeError(
            f"Cropper {cfg_crop_name} not implemented. Supported cropper list: {CROPPER_IMPLEMENTATION.keys()}"
        ) from e
    except Exception as e:
        logger.error(f"Error: {e}")
        raise e