Skip to content

builder

data.batch_sampler.builder

BatchSampler 

BatchSampler(dataset: Dataset, batch_size: int = 1, repeat: bool = False, shuffle: bool = False, drop_last: bool = False)

Bases: Registerable, ABC

Baseclass for all batch_sampler classes.

Attributes:

  • dataset (Dataset) –

    dataset to sample from.

  • batch_size (int) –

    batch size.

  • repeat (bool) –

    repeat data when data is not enough for batch size.

  • shuffle (bool) –

    shuffle data.

  • drop_last (bool) –

    drop last data when data is not enough for batch size.

Note
  • batch_size, shuffle, and drop_last should be popped from dataloader config, then passed to the batch sampler config.
  • repeat: for SaigeVision2 api, should be set as True.
    • if number of data is enough for batch size, then repeat is neglected.
  • drop_last: for SaigeVision2 api, should be set as True.
    • to prevent the last batch from being smaller than the specified batch size.
    • if False, even when repeat is set as True, the last batch will be smaller than the specified batch size.
  • for Training, shuffle should be set as True.
    • drop_last and repeat are optional.
  • for Validation, repeat, shuffle, and drop_last should be set as False.

Examples:

>>> if dataloader_class.__name__ == "Full2PatchDataLoader":
...     cfg_sampler.update(
...         {
...             "shuffle": cfg_dataloader.get("shuffle", False),
...             "drop_last": cfg_dataloader.get("drop_last", False),
...         }
...     )
... else:
...     cfg_sampler.update(
...         {
...             "shuffle": cfg_dataloader.pop("shuffle", False),
...             "drop_last": cfg_dataloader.pop("drop_last", False),
...             "batch_size": cfg_dataloader.pop("batch_size", 1),
...         }
...     )
... batch_sampler = build_batch_sampler(dataset=dataset, **cfg_sampler)
... dataloader = build_dataloader(
...     dataset,
...     collate_fn=collate_fn,
...     batch_sampler=batch_sampler,
...     **cfg_dataloader,
... )
Source code in SaigeToolkit/data/batch_sampler/base_sampler.py 
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
def __init__(
    self,
    dataset: data.Dataset,
    batch_size: int = 1,
    repeat: bool = False,
    shuffle: bool = False,
    drop_last: bool = False,
) -> None:
    self.dataset = dataset

    self.batch_size = batch_size
    self.repeat = repeat

    self.shuffle = shuffle
    self.drop_last = drop_last

    self._n_repeat = None
    self._buffer_length = None

build_batch_sampler 

build_batch_sampler(_target_: str, **config) -> BatchSampler
Source code in SaigeToolkit/data/batch_sampler/builder.py 
4
5
def build_batch_sampler(_target_: str, **config) -> BatchSampler:
    return BatchSampler.registry[_target_](**config)