BOREP

class lightautoml.text.dl_transformers.BOREP(embed_size=300, proj_size=300, pooling='mean', max_length=200, init='orthogonal', pos_encoding=False, **kwargs)

Bases: Module

Class to compute Bag of Random Embedding Projections sentence embeddings from words embeddings.

Bag of Random Embedding Projections sentence embeddings.

Parameters:

• embed_size (int) – Size of word embeddings.

• proj_size (int) – Size of output sentence embedding.

• pooling (str) – Pooling type.

• max_length (int) – Maximum length of sentence.

• init (str) – Type of weight initialization.

• pos_encoding (bool) – Add positional embedding.

• **kwargs (Any) – Ignored params.

Note

There are several pooling types:

• ‘max’: Maximum on seq_len dimension for non masked inputs.

• ‘mean’: Mean on seq_len dimension for non masked inputs.

• ‘sum’: Sum on seq_len dimension for non masked inputs.

For init parameter there are several options:

• ‘orthogonal’: Orthogonal init.

• ‘normal’: Normal with std 0.1.

• ‘uniform’: Uniform from -0.1 to 0.1.

• ‘kaiming’: Uniform kaiming init.

• ‘xavier’: Uniform xavier init.

get_out_shape()

Output shape.

Return type:

int

Returns:

Int with module output shape.

get_name()

Module name.

Return type:

str

Returns:

String with module name.

forward(inp)

Forward-pass.

Return type:

Tensor