transformers/docs/source/en/model_doc/t5.md

This model was released on 2019-10-23 and added to Hugging Face Transformers on 2020-11-16 and contributed by thomwolf.

T5

T5 explores transfer learning in NLP by converting all language problems into a text-to-text format. This unified framework systematically compares various pretraining objectives, architectures, and datasets across numerous language understanding tasks. Leveraging scale and a new dataset called the "Colossal Clean Crawled Corpus," T5 achieves top results on benchmarks such as summarization, question answering, and text classification. The model and associated dataset are publicly available for further research.

import torch
from transformers import pipeline

pipeline = pipeline(task="text2text-generation", model="google-t5/t5-base", dtype="auto",)
pipeline("translate English to French: Plants create energy through a process known as photosynthesis.")

import torch
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("google-t5/t5-base")
model = AutoModelForSeq2SeqLM.from_pretrained("google-t5/t5-base", dtype="auto",)

inputs = tokenizer("translate English to French: Plants create energy through a process known as photosynthesis.", return_tensors="pt")
outputs = model.generate(**inputs, max_length=50)
print(tokenizer.decode(outputs[0]))

Usage tips

• Pad encoder inputs on the left or right. T5 uses relative scalar embeddings.
• T5 models need a slightly higher learning rate than the default used in [Trainer]. Use values of 1e-4 and 3e-4 for most tasks.

T5Config

autodoc T5Config

T5Tokenizer

autodoc T5Tokenizer - build_inputs_with_special_tokens - get_special_tokens_mask - create_token_type_ids_from_sequences - save_vocabulary

T5TokenizerFast

autodoc T5TokenizerFast

T5Model

autodoc T5Model - forward

T5ForConditionalGeneration

autodoc T5ForConditionalGeneration - forward

T5EncoderModel

autodoc T5EncoderModel - forward

T5ForSequenceClassification

autodoc T5ForSequenceClassification - forward

T5ForTokenClassification

autodoc T5ForTokenClassification - forward

T5ForQuestionAnswering

autodoc T5ForQuestionAnswering - forward