trl/docs/source/models.md

Models

With the AutoModelForCausalLMWithValueHead class TRL supports all decoder model architectures in transformers such as GPT-2, OPT, and GPT-Neo. In addition, with AutoModelForSeq2SeqLMWithValueHead you can use encoder-decoder architectures such as T5. TRL also requires reference models which are frozen copies of the model that is trained. With create_reference_model you can easily create a frozen copy and also share layers between the two models to save memory.

PreTrainedModelWrapper

autodoc PreTrainedModelWrapper

AutoModelForCausalLMWithValueHead

autodoc AutoModelForCausalLMWithValueHead - init - forward - generate - _init_weights

AutoModelForSeq2SeqLMWithValueHead

autodoc AutoModelForSeq2SeqLMWithValueHead - init - forward - generate - _init_weights

create_reference_model

autodoc create_reference_model