transformertc.classification

Inference (prediction) functions for Token level Classification with BERT

Module Contents

transformertc.classification.logger

transformertc.classification.features_from_tokens(idtokens, tokenizer, label2id, ignore_label_id, max_length)

transformertc.classification.extract_features(texts, tokenizer, label2id, ignore_label_id, max_length, n_jobs=-1)

Creates a dataset at inference time. Args:

transformertc.classification.extract_bio(positions, tokens, preds, scores)

transformertc.classification.extract_simple(positions, tokens, preds, scores)

transformertc.classification.convert_classification_to_result(data, extract_f, id2label, task_format='BIO')

Convert a tuple containing all the data for a single example.

transformertc.classification.convert_classifications_to_result(texts, all_features, all_predictions, all_scores, id2label, task_format='BIO', n_jobs=-1)

Convert model predictions to a list of ResultTC.

transformertc.classification.pytorch_classify(model, device, all_features, batch_size=None, no_tqdm=False)