char_cnn

char_cnn

class hanlp.layers.embeddings.char_cnn.CharCNN(field: str, embed: Union[int, hanlp.layers.embeddings.embedding.Embedding], num_filters: int, ngram_filter_sizes: Tuple[int, ...] = (2, 3, 4, 5), conv_layer_activation: str = 'ReLU', output_dim: Optional[int] = None, vocab_size=None)[source]

A CnnEncoder is a combination of multiple convolution layers and max pooling layers. The input to this module is of shape (batch_size, num_tokens, input_dim), and the output is of shape (batch_size, output_dim).

The CNN has one convolution layer for each ngram filter size. Each convolution operation gives out a vector of size num_filters. The number of times a convolution layer will be used is num_tokens - ngram_size + 1. The corresponding maxpooling layer aggregates all these outputs from the convolution layer and outputs the max.

This operation is repeated for every ngram size passed, and consequently the dimensionality of the output after maxpooling is len(ngram_filter_sizes) * num_filters. This then gets (optionally) projected down to a lower dimensional output, specified by output_dim.

We then use a fully connected layer to project in back to the desired output_dim. For more details, refer to “A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural Networks for Sentence Classification”, Zhang and Wallace 2016, particularly Figure 1.

See allennlp.modules.seq2vec_encoders.cnn_encoder.CnnEncoder, Apache 2.0

Parameters

  • field – The field in samples this encoder will work on.

  • embed – An Embedding object or the feature size to create an Embedding object.

  • num_filters – This is the output dim for each convolutional layer, which is the number of “filters” learned by that layer.

  • ngram_filter_sizes – This specifies both the number of convolutional layers we will create and their sizes. The default of (2, 3, 4, 5) will have four convolutional layers, corresponding to encoding ngrams of size 2 to 5 with some number of filters.

  • conv_layer_activation – Activation, optional (default=`torch.nn.ReLU`) Activation to use after the convolution layers.

  • output_dim – After doing convolutions and pooling, we’ll project the collected features into a vector of this size. If this value is None, we will just return the result of the max pooling, giving an output of shape len(ngram_filter_sizes) * num_filters.

  • vocab_size – The size of character vocab.

Returns

A tensor of shape (batch_size, output_dim).

forward(batch: dict, **kwargs)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hanlp.layers.embeddings.char_cnn.CharCNNEmbedding(field, embed: Union[int, hanlp.layers.embeddings.embedding.Embedding], num_filters: int, ngram_filter_sizes: Tuple[int, ...] = (2, 3, 4, 5), conv_layer_activation: str = 'ReLU', output_dim: Optional[int] = None, min_word_length=None)[source]
Parameters

  • field – The character field in samples this encoder will work on.

  • embed – An Embedding object or the feature size to create an Embedding object.

  • num_filters – This is the output dim for each convolutional layer, which is the number of “filters” learned by that layer.

  • ngram_filter_sizes – This specifies both the number of convolutional layers we will create and their sizes. The default of (2, 3, 4, 5) will have four convolutional layers, corresponding to encoding ngrams of size 2 to 5 with some number of filters.

  • conv_layer_activation – Activation, optional (default=`torch.nn.ReLU`) Activation to use after the convolution layers.

  • output_dim – After doing convolutions and pooling, we’ll project the collected features into a vector of this size. If this value is None, we will just return the result of the max pooling, giving an output of shape len(ngram_filter_sizes) * num_filters.

  • min_word_length – For ngram filter with max size, the input (chars) is required to have at least max size chars.

module(vocabs: hanlp.common.transform.VocabDict, **kwargs) → Optional[torch.nn.modules.module.Module][source]

Build a module for this embedding.

Parameters

**kwargs – Containing vocabs, training etc. Not finalized for now.

Returns

A module.

transform(vocabs: hanlp.common.transform.VocabDict, **kwargs) → Optional[Callable][source]

Build a transform function for this embedding.

Parameters

**kwargs – Containing vocabs, training etc. Not finalized for now.

Returns

A transform function.

previous

fasttext

next

char_rnn