* Implemented integration with Comet in `LogCompletionsCallback`. Implemented related integration test.
* Implemented integration with Comet in `CPOTrainer.evaluation_loop()` during logging of `game_log` table.
* Implemented integration with Comet in `CPOTrainer.evaluation_loop()` during logging of `game_log` table.
* Implemented integration with Comet in `DPOTrainer.evaluation_loop()` during logging of `game_log` table.
* Implemented integration with Comet in `BCOTrainer.evaluation_loop()` during logging of `game_log` table.
* Implemented integration with Comet in `KTOTrainer.evaluation_loop()` during logging of `game_log` table.
* Implemented integration with Comet in `ORPOTrainer.evaluation_loop()` during logging of `game_log` table.
* first commit
* uncomment
* other tests adaptations
* Remove unused variable in test_setup_chat_format
* Remove unused import statement
* style
* Add Bart model
* Update BCOTrainerTester class in test_bco_trainer.py
* Update model IDs and tokenizers in test files
* Add new models and processors
* Update model IDs in test files
* Fix formatting issue in test_dataset_formatting.py
* Refactor dataset formatting in test_dataset_formatting.py
* Fix dataset sequence length in SFTTrainerTester
* Remove tokenizer
* Remove print statement
* Add reward_model_path and sft_model_path to PPO trainer
* Fix tokenizer padding issue
* Add chat template for testing purposes in PaliGemma model
* Update PaliGemma model and chat template
* Increase learning rate to speed up test
* Update model names in run_dpo.sh and run_sft.sh scripts
* Update model and dataset names
* Fix formatting issue in test_dataset_formatting.py
* Fix formatting issue in test_dataset_formatting.py
* Remove unused chat template
* Update model generation script
* additional models
* Update model references in test files
* Remove unused imports in test_online_dpo_trainer.py
* Add is_llm_blender_available import and update reward_tokenizer
* Refactor test_online_dpo_trainer.py: Move skipped test case decorator
* remove models without chat templates
* Update model names in scripts and tests
* Update model_id in test_modeling_value_head.py
* Update model versions in test files
* Fix formatting issue in test_dataset_formatting.py
* Update embedding model ID in BCOTrainerTester
* Update test_online_dpo_trainer.py with reward model changes
* Update expected formatted text in test_dataset_formatting.py
* Add reward_tokenizer to TestOnlineDPOTrainer
* fix tests
* Add SIMPLE_CHAT_TEMPLATE to T5 tokenizer
* Fix dummy_text format in test_rloo_trainer.py
* Skip outdated test for chatML data collator
* Add new vision language models
* Commented out unused model IDs in test_vdpo_trainer
* Update model and vision configurations in generate_tiny_models.py and test_dpo_trainer.py
* Update model and tokenizer references
* Don't push if it already exists
* Add comment explaining test skip
* Fix model_exists function call and add new models
* Update LlavaForConditionalGeneration model and processor
* `qgallouedec` -> `trl-internal-testing`
* Create mergekit_utils.py
* adding mergekit as an optional dependancy
* adding MergeModel to callbacks
* adding mergekit_utils dependencies to callbacks
* setting lower bound for mergekit
* setting mergekit lower band to 0.0.5.1
* adding support for MergeModelCallBack __init__.py
* adding support for mergemodelcallback
* mergemodelcallback tests
* Update callbacks.py
* Update __init__.py
* Update __init__.py
* Update test_callbacks.py
* Update trl/trainer/callbacks.py
removing ## from docs
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
* Update trl/trainer/callbacks.py
removing ## from docs
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
* Update trl/trainer/callbacks.py
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
* using different dataset for tests
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
* Update trl/mergekit_utils.py
adding types
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
* Update trl/mergekit_utils.py
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
* Apply suggestions from code review
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
* replacing get_last_checkpoint
* renaming Merge to merge_models
* setting mergers default value to linear
* removing unnecessary docs and comments
* adding docstring to Mergeconfig
* adding mergekits link to docstring
* precommit
* removing duplicated import
* typos in mergekit_utils docstring
* fixing tests
* making mergemodelcallback tests optional
* Make import optional
* minor
* use tmp dir in test
* sort
* Add import error checks for mergekit extra
* use a common _merge_and_maybe_push method and compat with windows path
* debug windows
* Update dependencies for mergekit and add test dependencies
* Add assertion to check if merged folder exists in the last checkpoint
* Fix temporary directory cleanup in test_callbacks.py
* Add sys import and skip test for Python versions below 3.10 due to cleanup errors with temp dir
* revert change for debug
---------
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
Co-authored-by: Quentin Gallouédec <quentin.gallouedec@huggingface.co>
Co-authored-by: Kashif Rasul <kashif.rasul@gmail.com>
* Add conditional check for LLMBlender availability in test_judges.py
* Fix import issues and update test requirements
* Remove unused imports
* Add require_peft decorator to test cases
* Fix import_utils module to use correct package name for llm_blender
