frozenleaves/trl
1
0
Fork 0
mirror of https://github.com/huggingface/trl.git synced 2025-10-20 10:03:51 +08:00
Code Issues Packages Projects Releases Wiki Activity

Deprecate BestOfNSampler (#4291)

Browse Source 
Co-authored-by: behroozazarkhalili <ermiaazarkhalili>
Co-authored-by: Behrooz Azarkhalili <80390531+behroozazarkhalili@users.noreply.github.com>
This commit is contained in:
Quentin Gallouédec
2025-10-15 18:06:34 -06:00
committed by GitHub
parent 31caf64778
commit 19d2f97932
2 changed files with 12 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
docs/source/best_of_n.md
View File

@ -1,5 +1,8 @@
# Best of N sampling: Alternative ways to get better model output without RL based fine-tuning # Best of N sampling: Alternative ways to get better model output without RL based fine-tuning
> [!WARNING]
> Best-of-N sampling is deprecated and will be removed in TRL 0.25.0.
Within the extras module is the `best-of-n` sampler class that serves as an alternative method of generating better model output. Within the extras module is the `best-of-n` sampler class that serves as an alternative method of generating better model output.
As to how it fares against the RL based fine-tuning, please look in the `examples` directory for a comparison example As to how it fares against the RL based fine-tuning, please look in the `examples` directory for a comparison example

9
trl/extras/best_of_n_sampler.py
View File

@ -12,6 +12,7 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
import warnings
from typing import Any, Callable, Optional, Union from typing import Any, Callable, Optional, Union
import torch import torch
@ -42,8 +43,16 @@ class BestOfNSampler:
generation_config ([`~transformers.GenerationConfig`], *optional*): generation_config ([`~transformers.GenerationConfig`], *optional*):
Generation config passed to the underlying model's `generate` method. See Generation config passed to the underlying model's `generate` method. See
[`~transformers.GenerationConfig`] for more details. [`~transformers.GenerationConfig`] for more details.
<Deprecated version="0.24.0">
`BestOfNSampler` is deprecated and will be removed in version 0.25.
</Deprecated>
""" """
warnings.warn("`BestOfNSampler` is deprecated and will be removed in TRL 0.25.", FutureWarning, stacklevel=2)
def __init__( def __init__(
self, self,
model: PreTrainedModelWrapper, model: PreTrainedModelWrapper,