Biaffine Named Entity Recognition.
BiaffineNamedEntityRecognition(trn: str = None, dev: str = None, tst: str = None, sampler_builder: hanlp.common.dataset.SamplerBuilder = None, dependencies: str = None, scalar_mix: hanlp.layers.scalar_mix.ScalarMixWithDropoutBuilder = None, use_raw_hidden_states=False, lr=None, separate_optimizer=False, doc_level_offset=True, is_flat_ner=True, tagset=None, ret_tokens=' ', ffnn_size=150, loss_reduction='mean', **kwargs)¶
An implementation of Named Entity Recognition as Dependency Parsing (Yu et al. 2020). It treats every possible span as a candidate of entity and predicts its entity label. Non-entity spans are assigned NULL label to be excluded. The label prediction is done with a biaffine layer (Dozat & Manning 2017). As it makes no assumption about the spans, it naturally supports flat NER and nested NER.
trn – Path to training set.
dev – Path to dev set.
tst – Path to test set.
sampler_builder – A builder which builds a sampler.
dependencies – Its dependencies on other tasks.
scalar_mix – A builder which builds a ScalarMixWithDropout object.
use_raw_hidden_states – Whether to use raw hidden states from transformer without any pooling.
lr – Learning rate for this task.
separate_optimizer – Use customized separate optimizer for this task.
Trueto indicate the offsets in
jsonlinesare of document level.
Truefor flat NER, otherwise nested NER.
tagset – Optional tagset to prune entities outside of this tagset from datasets.
ret_tokens – A delimiter between tokens in entities so that the surface form of an entity can be rebuilt.
ffnn_size – Feedforward size for MLPs extracting the head/tail representations.
loss_reduction – The loss reduction used in aggregating losses.
**kwargs – Not used.
build_dataloader(data, transform: hanlp.common.transform.TransformList = None, training=False, device=None, logger: logging.Logger = None, gradient_accumulation=1, **kwargs) → torch.utils.data.dataloader.DataLoader¶
Build a dataloader for training or evaluation.
data – Either a path or a list of samples.
transform – The transform from MTL, which is usually [TransformerSequenceTokenizer, FieldLength(‘token’)]
training – Whether this method is called on training set.
device – The device dataloader is intended to work with.
logger – Logger for printing message indicating progress.
cache – Whether the dataloader should be cached.
gradient_accumulation – Gradient accumulation to be passed to sampler builder.
**kwargs – Additional experimental arguments.
Implement this to build metric(s).
**kwargs – The subclass decides the method signature.
build_model(encoder_size, training=True, **kwargs) → torch.nn.modules.module.Module¶
Trueif called during training.