zamba.models.config¶
BackboneFinetuneConfig
¶
Bases: ZambaBaseModel
Configuration containing parameters to be used for backbone finetuning.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
unfreeze_backbone_at_epoch |
int
|
Epoch at which the backbone will be unfrozen. Defaults to 5. |
required |
backbone_initial_ratio_lr |
float
|
Used to scale down the backbone learning rate compared to rest of model. Defaults to 0.01. |
required |
multiplier |
int or float
|
Multiply the learning rate by a constant value at the end of each epoch. Defaults to 1. |
required |
pre_train_bn |
bool
|
Train batch normalization layers prior to finetuning. False is recommended for slowfast models and True is recommended for time distributed models. Defaults to False. |
required |
train_bn |
bool
|
Make batch normalization trainable. Defaults to False. |
required |
verbose |
bool
|
Display current learning rate for model and backbone. Defaults to True. |
required |
Source code in zamba/models/config.py
259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 |
|
EarlyStoppingConfig
¶
Bases: ZambaBaseModel
Configuration containing parameters to be used for early stopping.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
monitor |
str
|
Metric to be monitored. Options are "val_macro_f1" or "val_loss". Defaults to "val_macro_f1". |
required |
patience |
int
|
Number of epochs with no improvement after which training will be stopped. Defaults to 5. |
required |
verbose |
bool
|
Verbosity mode. Defaults to True. |
required |
mode |
str
|
Options are "min" or "max". In "min" mode, training will stop when the quantity monitored has stopped decreasing and in "max" mode it will stop when the quantity monitored has stopped increasing. If None, mode will be inferred from monitor. Defaults to None. |
required |
Source code in zamba/models/config.py
285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 |
|
ModelConfig
¶
Bases: ZambaBaseModel
Contains all configs necessary to use a model for training or inference. Must contain a train_config or a predict_config at a minimum.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
video_loader_config |
VideoLoaderConfig
|
An instantiated VideoLoaderConfig. If None, will use default video loader config for model specified in TrainConfig or PredictConfig. |
required |
train_config |
TrainConfig
|
An instantiated TrainConfig. Defaults to None. |
required |
predict_config |
PredictConfig
|
An instantiated PredictConfig. Defaults to None. |
required |
Source code in zamba/models/config.py
868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 |
|
ModelEnum
¶
Bases: str
, Enum
Shorthand names of models supported by zamba.
Source code in zamba/models/config.py
53 54 55 56 57 58 59 |
|
MonitorEnum
¶
Bases: str
, Enum
Validation metric to monitor for early stopping. Training is stopped when no improvement is observed.
Source code in zamba/models/config.py
62 63 64 65 66 67 |
|
PredictConfig
¶
Bases: ZambaBaseModel
Configuration for using a model for inference.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
filepaths |
FilePath
|
Path to a CSV containing videos for inference, with one row per video in the data_dir. There must be a column called 'filepath' (absolute or relative to the data_dir). If None, uses all files in data_dir. Defaults to None. |
required |
data_dir |
DirectoryPath
|
Path to a directory containing videos for inference. Defaults to the working directory. |
required |
model_name |
str
|
Name of the model to use for inference. Options are: time_distributed, slowfast, european, blank_nonblank. Defaults to time_distributed. |
required |
checkpoint |
FilePath
|
Path to a custom checkpoint file (.ckpt) generated by zamba that can be used to generate predictions. If None, defaults to a pretrained model. Defaults to None. |
required |
gpus |
int
|
Number of GPUs to use for inference. Defaults to all of the available GPUs found on the machine. |
required |
num_workers |
int
|
Number of subprocesses to use for data loading. 0 means that the data will be loaded in the main process. The maximum value is the number of CPUs in the system. Defaults to 3. |
required |
batch_size |
int
|
Batch size to use for inference. Defaults to 2. |
required |
save |
bool
|
Whether to save out predictions. If False, predictions are not saved. Defaults to True. |
required |
save_dir |
Path
|
An optional directory in which to save the model predictions and configuration yaml. If no save_dir is specified and save=True, outputs will be written to the current working directory. Defaults to None. |
required |
overwrite |
bool
|
If True, overwrite outputs in save_dir if they exist. Defaults to False. |
required |
dry_run |
bool
|
Perform inference on a single batch for testing. Predictions will not be saved. Defaults to False. |
required |
proba_threshold |
float
|
Probability threshold for classification. If specified, binary predictions are returned with 1 being greater than the threshold and 0 being less than or equal to the threshold. If None, return probability scores for each species. Defaults to None. |
required |
output_class_names |
bool
|
Output the species with the highest probability score as a single prediction for each video. If False, return probabilty scores for each species. Defaults to False. |
required |
weight_download_region |
str
|
s3 region to download pretrained weights from. Options are "us" (United States), "eu" (Europe), or "asia" (Asia Pacific). Defaults to "us". |
required |
skip_load_validation |
bool
|
By default, zamba runs a check to verify that all videos can be loaded and skips files that cannot be loaded. This can be time intensive, depending on how many videos there are. If you are very confident all your videos can be loaded, you can set this to True and skip this check. Defaults to False. |
required |
model_cache_dir |
Path
|
Cache directory where downloaded model weights will be saved. If None and no environment variable is set, will use your default cache directory. Defaults to None. |
required |
Source code in zamba/models/config.py
698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 |
|
SchedulerConfig
¶
Bases: ZambaBaseModel
Configuration containing parameters for a custom pytorch learning rate scheduler. See https://pytorch.org/docs/stable/optim.html for options.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
scheduler |
str
|
Name of learning rate scheduler to use. See https://pytorch.org/docs/stable/optim.html for options. |
required |
scheduler_params |
dict
|
Parameters passed to learning rate scheduler upon initialization (eg. {"milestones": [1], "gamma": 0.5, "verbose": True}). Defaults to None. |
required |
Source code in zamba/models/config.py
318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 |
|
TrainConfig
¶
Bases: ZambaBaseModel
Configuration for training a model.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
labels |
FilePath or pandas DataFrame
|
Path to a CSV or pandas DataFrame containing labels for training, with one row per label. There must be columns called 'filepath' (absolute or relative to the data_dir) and 'label', and optionally columns called 'split' ("train", "val", or "holdout") and 'site'. Labels must be specified to train a model. |
required |
data_dir |
DirectoryPath
|
Path to a directory containing training videos. Defaults to the working directory. |
required |
model_name |
str
|
Name of the model to use for training. Options are: time_distributed, slowfast, european, blank_nonblank. Defaults to time_distributed. |
required |
checkpoint |
FilePath
|
Path to a custom checkpoint file (.ckpt) generated by zamba that can be used to resume training. If None, defaults to a pretrained model. Defaults to None. |
required |
scheduler_config |
SchedulerConfig or str
|
Config for setting up the learning rate scheduler on the model. If "default", uses scheduler that was used for training. If None, will not use a scheduler. Defaults to "default". |
required |
dry_run |
(bool or int, Optional)
|
Run one training and validation batch for one epoch to detect any bugs prior to training the full model. Disables tuners, checkpoint callbacks, loggers, and logger callbacks. Defaults to False. |
required |
batch_size |
int
|
Batch size to use for training. Defaults to 2. |
required |
auto_lr_find |
bool
|
Use a learning rate finder algorithm when calling trainer.tune() to try to find an optimal initial learning rate. Defaults to False. The learning rate finder is not guaranteed to find a good learning rate; depending on the dataset, it can select a learning rate that leads to poor model training. Use with caution. |
required |
backbone_finetune_params |
BackboneFinetuneConfig
|
Set parameters to finetune a backbone model to align with the current learning rate. Defaults to a BackboneFinetuneConfig(unfreeze_backbone_at_epoch=5, backbone_initial_ratio_lr=0.01, multiplier=1, pre_train_bn=False, train_bn=False, verbose=True). |
required |
gpus |
int
|
Number of GPUs to train on applied per node. Defaults to all of the available GPUs found on the machine. |
required |
num_workers |
int
|
Number of subprocesses to use for data loading. 0 means that the data will be loaded in the main process. The maximum value is the number of CPUs in the system. Defaults to 3. |
required |
max_epochs |
int
|
Stop training once this number of epochs is reached. Disabled by default (None), which stops training at 1000 epochs. |
required |
early_stopping_config |
EarlyStoppingConfig
|
Configuration for early stopping, which monitors a metric during training and stops training when the metric stops improving. Defaults to EarlyStoppingConfig(monitor='val_macro_f1', patience=5, verbose=True, mode='max'). |
required |
weight_download_region |
str
|
s3 region to download pretrained weights from. Options are "us" (United States), "eu" (Europe), or "asia" (Asia Pacific). Defaults to "us". |
required |
split_proportions |
dict
|
Proportions used to divide data into training, validation, and holdout sets if a if a "split" column is not included in labels. Defaults to "train": 3, "val": 1, "holdout": 1. |
required |
save_dir |
Path
|
Path to a directory where training files
will be saved. Files include the best model checkpoint ( |
required |
overwrite |
bool
|
If True, will save outputs in |
required |
from_scratch |
bool
|
Instantiate the model with base weights. This means starting with ImageNet weights for image-based models (time_distributed, european, and blank_nonblank) and Kinetics weights for video-based models (slowfast). Defaults to False. |
required |
use_default_model_labels |
bool
|
By default, output the full set of default model labels rather than just the species in the labels file. Only applies if the provided labels are a subset of the default model labels. If set to False, will replace the model head for finetuning and output only the species in the provided labels file. |
required |
model_cache_dir |
Path
|
Cache directory where downloaded model weights will be saved. If None and the MODEL_CACHE_DIR environment variable is not set, uses your default cache directory. Defaults to None. |
required |
Source code in zamba/models/config.py
348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 |
|
preprocess_labels(values)
¶
One hot encode, add splits, and check for binary case.
Replaces values['labels'] with modified DataFrame.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
values |
dictionary containing 'labels' and other config info |
required |
Source code in zamba/models/config.py
583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 |
|
validate_provided_species_and_use_default_model_labels(values)
¶
If the model species are the desired output, the labels file must contain a subset of the model species.
Source code in zamba/models/config.py
551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 |
|
ZambaBaseModel
¶
Bases: BaseModel
Set defaults for all models that inherit from the pydantic base model.
Source code in zamba/models/config.py
250 251 252 253 254 255 256 |
|
check_files_exist_and_load(df, data_dir, skip_load_validation)
¶
Check whether files in file list exist and can be loaded with ffmpeg. Warn and skip files that don't exist or can't be loaded.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
df |
DataFrame
|
DataFrame with a "filepath" column |
required |
data_dir |
Path
|
Data folder to prepend if filepath is not an absolute path. |
required |
skip_load_validation |
bool
|
Skip ffprobe check that verifies all videos can be loaded. |
required |
Returns:
Type | Description |
---|---|
pd.DataFrame: DataFrame with valid and loadable videos. |
Source code in zamba/models/config.py
91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 |
|
get_filepaths(cls, values)
¶
If no file list is passed, get all files in data directory. Warn if there
are unsupported suffixes. Filepaths is set to a dataframe, where column filepath
contains files with valid suffixes.
Source code in zamba/models/config.py
216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 |
|
make_split(labels, values)
¶
Add a split column to labels
.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
labels |
DataFrame with one row per video |
required | |
values |
dictionary with config info |
required |
Source code in zamba/models/config.py
635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 |
|
validate_gpus(gpus)
¶
Ensure the number of GPUs requested is equal to or less than the number of GPUs available on the machine.
Source code in zamba/models/config.py
70 71 72 73 74 75 76 |
|
validate_model_cache_dir(model_cache_dir)
¶
Set up cache directory for downloading model weight. Order of priority is: config argument, environment variable, or user's default cache dir.
Source code in zamba/models/config.py
79 80 81 82 83 84 85 86 87 88 |
|
validate_model_name_and_checkpoint(cls, values)
¶
Ensures a checkpoint file or model name is provided. If a model name is provided, looks up the corresponding public checkpoint file from the official configs. Download the checkpoint if it does not yet exist.
Source code in zamba/models/config.py
175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 |
|