Skip to content

class_sampler

data.batch_sampler.class_sampler

BatchSampler 

BatchSampler(dataset: Dataset, batch_size: int = 1, repeat: bool = False, shuffle: bool = False, drop_last: bool = False)

Bases: Registerable, ABC

Baseclass for all batch_sampler classes.

Attributes:

  • dataset (Dataset) –

    dataset to sample from.

  • batch_size (int) –

    batch size.

  • repeat (bool) –

    repeat data when data is not enough for batch size.

  • shuffle (bool) –

    shuffle data.

  • drop_last (bool) –

    drop last data when data is not enough for batch size.

Note
  • batch_size, shuffle, and drop_last should be popped from dataloader config, then passed to the batch sampler config.
  • repeat: for SaigeVision2 api, should be set as True.
    • if number of data is enough for batch size, then repeat is neglected.
  • drop_last: for SaigeVision2 api, should be set as True.
    • to prevent the last batch from being smaller than the specified batch size.
    • if False, even when repeat is set as True, the last batch will be smaller than the specified batch size.
  • for Training, shuffle should be set as True.
    • drop_last and repeat are optional.
  • for Validation, repeat, shuffle, and drop_last should be set as False.

Examples:

>>> cfg_sampler.update(
...     {
...         "shuffle": cfg_dataloader.pop("shuffle", False),
...         "drop_last": cfg_dataloader.pop("drop_last", False),
...         "batch_size": cfg_dataloader.pop("batch_size", 1),
...     }
... )
... batch_sampler = build_batch_sampler(dataset=dataset, **cfg_sampler)
... dataloader = build_dataloader(
...     dataset,
...     collate_fn=collate_fn,
...     batch_sampler=batch_sampler,
...     **cfg_dataloader,
... )
Source code in SaigeToolkit/data/batch_sampler/base_sampler.py 
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
def __init__(
    self,
    dataset: data.Dataset,
    batch_size: int = 1,
    repeat: bool = False,
    shuffle: bool = False,
    drop_last: bool = False,
) -> None:
    self.dataset = dataset

    self.batch_size = batch_size
    self.repeat = repeat

    self.shuffle = shuffle
    self.drop_last = drop_last

    self._n_repeat = None
    self._buffer_length = None

ClassBalancedSampler 

ClassBalancedSampler(dataset: Dataset, shuffle: bool = True, **kwargs)

Bases: BatchSampler

Sampler implementation for simple class balancing.

Uses class_index value in dataset file dictionaries. When more than one classes exist in single image, the multiplier of the rarest class within image will be applied to the data.

Raises:

  • IndexError –

    Will be raised when class index exceeds MAX_NUM_CLS.

Notes
  • TODO: Weighted balancing.
  • TODO: Cropper applicability.
Source code in SaigeToolkit/data/batch_sampler/class_sampler.py 
26
27
28
29
30
31
32
33
def __init__(
    self,
    dataset: Dataset,
    shuffle: bool = True,
    **kwargs,
) -> None:
    super().__init__(dataset, shuffle=shuffle, **kwargs)
    self.label_list, self.class_multiplier = self._make_weights_for_balanced_classes()