transformers/docs/source/en/model_doc/mt5.md

This model was released on 2020-10-22 and added to Hugging Face Transformers on 2020-11-17 and contributed by patrickvonplaten.

mT5

mT5 leverages a unified text-to-text format and scale to achieve state-of-the-art results across various multilingual NLP tasks. It was pre-trained on a Common Crawl-based dataset covering 101 languages. The model demonstrates superior performance on multilingual benchmarks and includes a technique to mitigate accidental translation in zero-shot scenarios. mT5 requires fine-tuning for specific tasks and does not benefit from task prefixes during single-task fine-tuning, unlike the original T5. Google provides several variants of mT5, ranging from small to XXL sizes.

import torch
from transformers import pipeline

pipeline = pipeline(task="text2text-generation", model="csebuetnlp/mT5_multilingual_XLSum", dtype="auto")
pipeline("""
Plants are remarkable organisms that produce their own food using a method called photosynthesis.
This process involves converting sunlight, carbon dioxide, and water into glucose, which provides energy for growth.
Plants play a crucial role in sustaining life on Earth by generating oxygen and serving as the foundation of most ecosystems.
"""
)

import torch
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

model = AutoModelForSeq2SeqLM.from_pretrained("csebuetnlp/mT5_multilingual_XLSum", dtype="auto")
tokenizer = AutoTokenizer.from_pretrained("csebuetnlp/mT5_multilingual_XLSum")

text="""
Plants are remarkable organisms that produce their own food using a method called photosynthesis.
This process involves converting sunlight, carbon dioxide, and water into glucose, which provides energy for growth.
Plants play a crucial role in sustaining life on Earth by generating oxygen and serving as the foundation of most ecosystems.
"""
inputs = tokenizer(text, return_tensors="pt")
outputs = model.generate(**inputs, max_length=50)
print(tokenizer.decode(outputs[0]))

Usage tips

• Fine-tune mT5 for downstream tasks. The model was only pretrained on the mc4 dataset, which doesn't include task-specific training.

MT5Config

autodoc MT5Config

MT5Tokenizer

autodoc MT5Tokenizer

See [T5Tokenizer] for all details.

MT5TokenizerFast

autodoc MT5TokenizerFast

See [T5TokenizerFast] for all details.

MT5Model

autodoc MT5Model

MT5ForConditionalGeneration

autodoc MT5ForConditionalGeneration

MT5EncoderModel

autodoc MT5EncoderModel

MT5ForSequenceClassification

autodoc MT5ForSequenceClassification

MT5ForTokenClassification

autodoc MT5ForTokenClassification

MT5ForQuestionAnswering

autodoc MT5ForQuestionAnswering