TorchModel

class lightautoml.ml_algo.dl_model.TorchModel(default_params=None, freeze_defaults=True, timer=None, optimization_search_space={})

Bases: TabularMLAlgo

Neural net for tabular datasets.

default_params:

• bs: Batch size.

• num_workers: Number of threads for multiprocessing.

• max_length: Max sequence length.

• opt_params: Dict with optim params.

• scheduler_params: Dict with scheduler params.

• is_snap: Use snapshots.

• snap_params: Dict with SE parameters.

• init_bias: Init last linear bias by mean target values.

• n_epochs: Number of training epochs.

• input_bn: Use 1d batch norm for input data.

• emb_dropout: Dropout probability.

• emb_ratio: Ratio for embedding size = (x + 1) // emb_ratio.

• max_emb_size: Max embedding size.

• device: Torch device or str.

• use_cont: Use numeric data.

• use_cat: Use category data.

• use_text: Use text data.

• lang: Text language.

• bert_name: Name of HuggingFace transformer model.

• pooling: Type of pooling strategy for bert model.

• deterministic: CUDNN backend.

• multigpu: Use Data Parallel.

• model_with_emb: Use model with custom embeddings.

• loss: Torch loss or str or func with (y_pred, y_true) args.

• loss_params: Dict with loss params.

• loss_on_logits: Calculate loss on logits or on predictions of model for classification tasks.

• clip_grad: Clip gradient before loss backprop.

• clip_grad_params: Dict with clip_grad params.

• dataset: Class for data retrieval

• tuned: Tune custom model

• num_init_features: Scale input dimension to another one

• use_noise: Use Noise

• use_bn: Use BatchNorm

• path_to_save: Path to save model checkpoints, None - stay in memory.

• random_state: Random state to take subsample.

• verbose_inside: Number of steps between verbose inside epoch or None.

• verbose: Verbose every N epochs.

freeze_defaults:

• True : params may be rewrited depending on dataset.

• False: params may be changed only manually or with tuning.

timer: Timer instance or None.

static get_mean_target(target, task_name)

Get target mean / inverse sigmoid transformation to init bias in last layer of network.

Parameters:

• target – Target values.

• task_name (str) – One of the available task names

Returns:

Array with bias values.

init_params_on_input(train_valid_iterator)

Get model parameters depending on dataset parameters.

Parameters:

train_valid_iterator – Classic cv-iterator.

Return type:

dict

Returns:

Parameters of model.

get_dataloaders_from_dicts(data_dict)

Construct dataloaders depending on stage.

Parameters:

data_dict (Dict) – Dict with (stage_name, data) (key, value).

Returns:

Dataloaders.

fit_predict(train_valid_iterator)

Fit and then predict accordig the strategy that uses train_valid_iterator.

If item uses more then one time it will predict mean value of predictions. If the element is not used in training then the prediction will be numpy.nan for this item

Parameters:

train_valid_iterator (TrainValidIterator) – Classic cv-iterator.

Return type:

NumpyDataset

Returns:

Dataset with predicted values.

fit_predict_single_fold(train, valid)

Implements training and prediction on single fold.

Parameters:

• train – Train Dataset.

• valid – Validation Dataset.

Returns:

Tuple (model, predicted_values).

predict_single_fold(model, dataset)

Predict target values for dataset.

Parameters:

• model (any) – Neural net object or dict or str.

• dataset (Union[NumpyDataset, CSRSparseDataset, PandasDataset]) – Test dataset.

Return type:

ndarray

Returns:

Predicted target values.