transformers/docs/source/en/model_doc/nezha.md

This model was released on 2019-08-31 and added to Hugging Face Transformers on 2023-06-20 and contributed by sijunhe.

Warning

This model is in maintenance mode only, we don’t accept any new PRs changing its code.

If you run into any issues running this model, please reinstall the last version that supported this model: v4.40.2. You can do so by running the following command: pip install -U transformers==4.40.2.

Nezha

NEZHA: Neural ContextualiZed representation for CHinese lAnguage understanding presents NEZHA, a pre-trained language model for Chinese NLU tasks. NEZHA is based on BERT with enhancements such as Functional Relative Positional Encoding, Whole Word Masking, Mixed Precision Training, and the LAMB Optimizer. Experiments demonstrate that NEZHA achieves top performance on tasks like named entity recognition, sentence matching, sentiment classification, and natural language inference.

import torch
from transformers import pipeline

pipeline = pipeline(task="fill-mask", model="sijunhe/nezha-cn-base", dtype="auto")
pipeline("植物通过[MASK]合作用产生能量")

import torch
from transformers import AutoModelForMaskedLM, AutoTokenizer

model = AutoModelForMaskedLM.from_pretrained("sijunhe/nezha-cn-base", dtype="auto")
tokenizer = AutoTokenizer.from_pretrained("sijunhe/nezha-cn-base")

inputs = tokenizer("植物通过[MASK]合作用产生能量", return_tensors="pt")
outputs = model(**inputs)
mask_token_id = tokenizer.mask_token_id
mask_position = (inputs.input_ids == tokenizer.mask_token_id).nonzero(as_tuple=True)[1]
predicted_word = tokenizer.decode(outputs.logits[0, mask_position].argmax(dim=-1))
print(f"Predicted word: {predicted_word}")

NezhaConfig

autodoc NezhaConfig

NezhaModel

autodoc NezhaModel - forward

NezhaForPreTraining

autodoc NezhaForPreTraining - forward

NezhaForMaskedLM

autodoc NezhaForMaskedLM - forward

NezhaForNextSentencePrediction

autodoc NezhaForNextSentencePrediction - forward

NezhaForSequenceClassification

autodoc NezhaForSequenceClassification - forward

NezhaForMultipleChoice

autodoc NezhaForMultipleChoice - forward

NezhaForTokenClassification

autodoc NezhaForTokenClassification - forward

NezhaForQuestionAnswering

autodoc NezhaForQuestionAnswering - forward