Experiments
Experiment
Experiment class handles all the internal stuff like boiler plate code for training, calling callbacks,metrics,etc. One can override some ``train_step``, ``val_step`` and ``batch_step`` in experiment class perform custom training and validation.
- class torchflare.experiments.Experiment(num_epochs: int, fp16: bool = False, device: str = 'cuda', seed: int = 42)[source]
Simple class for handling boilerplate code for training, validation and Inference.
- Parameters
num_epochs (int) – The number of epochs to save model.
fp16 (bool) – Set this to True if you want to use mixed precision training.
device (str) – The device where you want train your model. One of cuda or cpu.
seed (int) – The seed to ensure reproducibility.
Examples
import torch import torchmetrics import torchflare.callbacks as cbs from torchflare.experiments import Experiment # Defining Training/Validation Dataloaders train_dl = SomeTrainDataloader() valid_dl = SomeValidDataloader() # Defining params optimizer = "Adam" optimizer_params = {"lr": 1e-4} criterion = "cross_entropy" num_epochs = 10 num_classes = 4 # Defining the list of metrics metric_list = [ torchmetrics.Accuracy(num_classes = num_classes) ] # Defining the list of callbacks callbacks = [ cbs.EarlyStopping(monitor="accuracy", mode="max"), cbs.ModelCheckpoint(monitor="accuracy", mode="max"), cbs.ReduceLROnPlateau(mode="max", patience=3), # Defining Scheduler callback. ] # Defining the model config which contains model, optimizer, criterion. config = ModelConfig( nn_module=SomeModelClass, module_params={"num_features": 200, "num_classes": 5}, optimizer=optimizer, optimizer_params=optimizer_params, criterion=criterion, ) # Creating Experiment and setting the params. exp = Experiment( num_epochs=num_epochs, fp16=True, device=device, seed=42, ) # Compiling the experiment exp.compile_experiment( model_config=config, metrics=metric_list, callbacks=callbacks, main_metric="accuracy", ) # Running the experiment exp.fit_loader(train_dl=train_dl, valid_dl=valid_dl)
- compile_experiment(model_config: ModelConfig, callbacks: List = None, metrics: List = None, main_metric: Optional[str] = None) None[source]
Configures the model for training and validation.
- Parameters
model_config – An ModelConfig object which holds information about models, optimizer and criterion.
callbacks (List) – The list of callbacks to be used.
metrics (List) – The list of metrics to be used.
main_metric (str) – The name of main metric to be monitored. Use lower case version. For examples , use ‘accuracy’ instead of ‘Accuracy’.
Note
Supports all the schedulers implemented in pytorch/transformers except SWA. Support for custom scheduling will be added soon.
- fit(x: Union[torch.Tensor, numpy.ndarray], y: Union[torch.Tensor, numpy.ndarray], val_data: Optional[Union[Tuple, List]] = None, batch_size: int = 64, dataloader_kwargs: Optional[Dict] = None)[source]
Train and validate the model on training and validation dataset.
- Parameters
x – A numpy array(or array-like) or torch.tensor for inputs to the model.
y – Target data. Same type as input data coule numpy array(or array-like) or torch.tensors.
val_data – A tuple or list (x_val , y_val) of numpy arrays or torch.tensors.
batch_size (int) – The batch size to be used for training and validation.
dataloader_kwargs (Dict) – Keyword arguments to pass to the PyTorch dataloaders created internally. By default, shuffle=True is passed for the training dataloader but this can be overriden by using this argument.
Note
Model will only be saved when ModelCheckpoint callback is used.
- fit_loader(train_dl: torch.utils.data.DataLoader, valid_dl: Optional[torch.utils.data.DataLoader] = None)[source]
Train and validate the model using dataloaders.
- Parameters
train_dl (DataLoader) – The training dataloader.
valid_dl (DataLoader) – The validation dataloader.
Note
Model will only be saved when ModelCheckpoint callback is used.
- predict(x: Union[torch.Tensor, numpy.ndarray], path_to_model: str, batch_size: int = 64, dataloader_kwargs: Optional[Dict] = None, device: str = 'cuda')
Method to perform inference on test data.
- Parameters
x – A numpy array(or array-like) or torch.tensor for inputs to the model.
batch_size – The batch size to be used for inference.
device – The device on which you want to perform inference.
dataloader_kwargs – Keyword arguments to pass to the PyTorch dataloader which is created internally.
path_to_model – The full path to the model.
- predict_on_loader(path_to_model: str, test_dl: torch.utils.data.DataLoader, device: str = 'cuda') torch.Tensor
Method to perform inference on test dataloader.
- Parameters
test_dl (DataLoader) – The dataloader to be use for testing.
device (str) – The device on which you want to perform inference.
path_to_model (str) – The full path to model
- Yields
Output per batch.
- train_step() Dict[source]
Method to perform train step.
The train step includes forward pass, loss evaluation, backward pass.
Note
Use self.backend attribute for doing zero_grad backward pass etc. It is compulsory for train_step to return a dictionary with loss. If you are using metrics then train_step should return a dictionary with both predictions and loss.
- Returns
A dictionary with train_step results:
{ "predictions" : The train batch predictions, "loss" : The loss for the train_step }
- val_step() Dict[source]
Method to perform validation step.
The train step includes forward pass, loss evaluation.
Note
It is compulsory for val_step to return a dictionary with loss. If you are using metrics then val_step should return a dictionary with both predictions and loss.
- Returns
A dictionary with val_step results:
{ "predictions" : The validation batch predictions, "loss" : The loss for the val_step }
ModelConfig
- class torchflare.experiments.ModelConfig(nn_module: Union[torch.nn.Module, Dict], module_params: Dict, optimizer: Union[str, Dict, torch.optim.Optimizer, Any], optimizer_params: Dict, criterion: Union[Callable, Dict, str], model_dict: bool = False, optimizer_dict: bool = False)[source]
Model Config to initialize model related parameters, optimizers and criterion.
- Parameters
nn_module – Uninstantiated PyTorch model class or dictionary of model classes.
module_params – The params required to init the nn_module.
optimizer – The name or uninstantiated optimizer class or dictionary of optimizer classes to be used.
optimizer_params – The params to init the optimizer class.
critertion – The critertion or dictionary of criterion to be used.
Example
from torchflare.experiments import ModelConfig config = ModelConfig( nn_module=SomeModelClass, module_params={"in_features": 100, "out_features": 50}, optimizer="Adam", optimizer_params={"lr": 3e-4}, criterion="binary_cross_entropy", )
Note
If you are having multiple models and optimizers ensure that both the model keys and optimizer keys are same.