* Implemented integration with Comet in `LogCompletionsCallback`. Implemented related integration test.
* Implemented integration with Comet in `CPOTrainer.evaluation_loop()` during logging of `game_log` table.
* Implemented integration with Comet in `CPOTrainer.evaluation_loop()` during logging of `game_log` table.
* Implemented integration with Comet in `DPOTrainer.evaluation_loop()` during logging of `game_log` table.
* Implemented integration with Comet in `BCOTrainer.evaluation_loop()` during logging of `game_log` table.
* Implemented integration with Comet in `KTOTrainer.evaluation_loop()` during logging of `game_log` table.
* Implemented integration with Comet in `ORPOTrainer.evaluation_loop()` during logging of `game_log` table.
* Create mergekit_utils.py
* adding mergekit as an optional dependancy
* adding MergeModel to callbacks
* adding mergekit_utils dependencies to callbacks
* setting lower bound for mergekit
* setting mergekit lower band to 0.0.5.1
* adding support for MergeModelCallBack __init__.py
* adding support for mergemodelcallback
* mergemodelcallback tests
* Update callbacks.py
* Update __init__.py
* Update __init__.py
* Update test_callbacks.py
* Update trl/trainer/callbacks.py
removing ## from docs
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
* Update trl/trainer/callbacks.py
removing ## from docs
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
* Update trl/trainer/callbacks.py
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
* using different dataset for tests
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
* Update trl/mergekit_utils.py
adding types
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
* Update trl/mergekit_utils.py
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
* Apply suggestions from code review
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
* replacing get_last_checkpoint
* renaming Merge to merge_models
* setting mergers default value to linear
* removing unnecessary docs and comments
* adding docstring to Mergeconfig
* adding mergekits link to docstring
* precommit
* removing duplicated import
* typos in mergekit_utils docstring
* fixing tests
* making mergemodelcallback tests optional
* Make import optional
* minor
* use tmp dir in test
* sort
* Add import error checks for mergekit extra
* use a common _merge_and_maybe_push method and compat with windows path
* debug windows
* Update dependencies for mergekit and add test dependencies
* Add assertion to check if merged folder exists in the last checkpoint
* Fix temporary directory cleanup in test_callbacks.py
* Add sys import and skip test for Python versions below 3.10 due to cleanup errors with temp dir
* revert change for debug
---------
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
Co-authored-by: lewtun <lewis.c.tunstall@gmail.com>
Co-authored-by: Quentin Gallouédec <quentin.gallouedec@huggingface.co>
Co-authored-by: Kashif Rasul <kashif.rasul@gmail.com>
* Add conditional check for LLMBlender availability in test_judges.py
* Fix import issues and update test requirements
* Remove unused imports
* Add require_peft decorator to test cases
* Fix import_utils module to use correct package name for llm_blender
* adds option to skip dataset preparation in SFTTrainer
* before changing the template
* adds support for new schema
* a few fixes to data collator to support new schema
* updates args
* precommit
* adds sys prompt to chat template and other fixes
* updates template, fixes collator for multiple images
* precommit
* rename vsft to vstf_llava
* adding integration tests
* adds integration test for vsft
* precommit
* adds back chat template
* docs
* typo
* adds eval, precommit
* adds peft launch args
* formatting
* fixes no deps tests by checking if PIL lib exists
* Update __init__.py
---------
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
* init
* run
* Update custom eval loop to aid DPO debugging (#770)
* sample_during_eval -> generate_during_eval
* Remove unused return_tokens
* Add import utils for W&B, prevent test fails
* Optimize dataloader random batch selection
* Separate prompt and response in logs
Makes it much easier to quickly read the starts of the generations
* Simplify logging
* reset eval steps
* manual merge fixes
* revert merge
* remove self.max_length
* style
* fix max_length
---------
Co-authored-by: Tom Aarsen <37621491+tomaarsen@users.noreply.github.com>
* add fixes in to support PP
* add same logic for enc-dec
* add more checks
* fix 20b issues
* clean up
* update scripts
* dp safety checker
* added multi gpu tests
* fix order
* change
* fix script
* adds a hacky peft example
* fixes bug due to missing "prepare_model_for_training"
* Formatting
* adds peft to requirements
* Update trl/trainer/ppo_trainer.py
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
* gpt neo runs
* changes requested on the PR
* style
* updates to prepare_model_for_int8_training PEFT PR https://github.com/huggingface/peft/pull/105
* updates to prepare_model_for_int8_training PEFT PR https://github.com/huggingface/peft/pull/105
* adds missing 8-bit attribute to modeling base
* adds lr to example script
* adds missing train to trainer
* disables caching temporarily while I debug something
* debugging issues with unstable training
* Fix peft + int8 (#170)
* add fix
* another fix
* Auto stash before merge of "peft-example" and "origin/peft-example"
* adds peft model types to modeling base
* reduces memory usage using adapters and no ref model.
* adds support for EleutherAI/gpt-neox-20b
* example for peft finetune of cm model
* removes hacky research code
* fixing the rebase and some typos
* style
* style2
* adds gradient checkpointing to base model
* cleans up comments
* moves config and other pretrained_model properties to __init__
* make style
* added tests
* change dependency
* Update .github/workflows/tests.yml
* fix test
* fix style and failing tests
* make quality
* revert change
* rm unneeded change
* revert changes
* rm changes
* rm changes
* rm uneeded change
* Update trl/models/modeling_base.py
* revert uneeded changes
* make style
* adapt suggestions
* fix tests
* attempt to fix
* fix
* fix
* add no peft test
* revert
* remove unneded check
* more tests
* fix logic
* add `save_pretrained` support
* fix quality
* clean up
* clean up
* stronger test
* refactor comments
* make style
* attempt to add non-peft tests
* remove test runner
* format
* fix test
* move `train` on top
* fix peft import
* make quality
* fixes typo
* adds peft example to docs
---------
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
Co-authored-by: younesbelakda <younesbelkada@gmail.com>
