Skip to content

norm

model.backbone.norm

FrozenBatchNorm2d 

FrozenBatchNorm2d(n: int)

Bases: Module

BatchNorm2d where the batch statistics and the affine parameters are fixed

Attributes:

  • weight (Tensor) –

    batch_norm weight. not nn.Parameter for freezing values.

  • bias (Tensor) –

    batch_norm bias. not nn.Parameter for freezing values.

  • running_mean (Tensor) –

    batch_norm running_mean. not nn.Parameter for freezing values.

  • running_var (Tensor) –

    batch_norm running_var. not nn.Parameter for freezing values.

Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved.

Initializing FrozenBatchNorm2d

Parameters:

  • n (int) –

    number of channels for torch.Tensor

Source code in SaigeToolkit/model/backbone/norm.py 
21
22
23
24
25
26
27
28
29
30
31
def __init__(self, n: int) -> None:
    """Initializing FrozenBatchNorm2d

    Args:
        n (int): number of channels for torch.Tensor
    """
    super(FrozenBatchNorm2d, self).__init__()
    self.register_buffer("weight", torch.ones(n))
    self.register_buffer("bias", torch.zeros(n))
    self.register_buffer("running_mean", torch.zeros(n))
    self.register_buffer("running_var", torch.ones(n))

forward 

forward(x: Tensor) -> Tensor

forward function for FrozenBatchNorm2d

Parameters:

  • x (Tensor) –

    input tensor

Returns:

  • Tensor –

    torch.Tensor: batch_normalized output tensor

Source code in SaigeToolkit/model/backbone/norm.py 
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
def forward(self, x: torch.Tensor) -> torch.Tensor:
    """forward function for FrozenBatchNorm2d

    Args:
        x (torch.Tensor): input tensor

    Returns:
        torch.Tensor: batch_normalized output tensor
    """
    # Cast all fixed parameters to half() if necessary
    if x.dtype == torch.float16:
        self.weight = self.weight.half()
        self.bias = self.bias.half()
        self.running_mean = self.running_mean.half()
        self.running_var = self.running_var.half()
    scale = self.weight * self.running_var.rsqrt()
    bias = self.bias - self.running_mean * scale
    scale = scale.reshape(1, -1, 1, 1)
    bias = bias.reshape(1, -1, 1, 1)
    return x * scale + bias

get_norm 

get_norm(name: str) -> Type[Module]

getting customized batch norm class

Parameters:

  • name (str) –

    batch norm class name

Returns:

  • Type[Module] –

    Type[nn.Module]: custom batch norm class

Source code in SaigeToolkit/model/backbone/norm.py 
55
56
57
58
59
60
61
62
63
64
def get_norm(name: str) -> Type[nn.Module]:
    """getting customized batch norm class

    Args:
        name (str): batch norm class name

    Returns:
        Type[nn.Module]: custom batch norm class
    """
    return {"fixed_batch_norm": FrozenBatchNorm2d}[name]