zamba.models.slowfast_models¶
SlowFast
¶
Bases: ZambaVideoClassificationLightningModule
Pretrained SlowFast model for fine-tuning with the following architecture:
Input -> SlowFast Base (including trainable Backbone) -> Res Basic Head -> Output
Attributes:
Name | Type | Description |
---|---|---|
backbone |
Module
|
When scheduling the backbone to train with the
|
base |
Module
|
The entire model prior to the head. |
head |
Module
|
The trainable head. |
_backbone_output_dim |
int
|
Dimensionality of the backbone output (and head input). |
Source code in zamba/models/slowfast_models.py
11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 |
|
__init__(backbone_mode='train', post_backbone_dropout=None, output_with_global_average=True, head_dropout_rate=None, head_hidden_layer_sizes=None, finetune_from=None, **kwargs)
¶
Initializes the SlowFast model.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
backbone_mode |
str
|
If "eval", treat the backbone as a feature extractor and set to evaluation mode in all forward passes. |
'train'
|
post_backbone_dropout |
float
|
Dropout that operates on the output of the backbone + pool (before the fully-connected layer in the head). |
None
|
output_with_global_average |
bool
|
If True, apply an adaptive average pooling operation after the fully-connected layer in the head. |
True
|
head_dropout_rate |
float
|
Optional dropout rate applied after backbone and between projection layers in the head. |
None
|
head_hidden_layer_sizes |
tuple of int
|
If not None, the size of hidden layers in the head multilayer perceptron. |
None
|
finetune_from |
pathlike or str
|
If not None, load an existing model from the path and resume training from an existing model. |
None
|
Source code in zamba/models/slowfast_models.py
27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 |
|
initialize_from_torchub()
¶
Loads SlowFast model from torchhub and prepares ZambaVideoClassificationLightningModule by removing the head and setting the backbone and base.
Source code in zamba/models/slowfast_models.py
94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 |
|