18 Commits

Author SHA1 Message Date
24f8d0276c [examples] upgrade code for seed setting (#3387)
* replace set_seed

* update import
2025-02-11 16:31:41 +01:00
52fae0960c Add end_training/destroy_pg to everything and unpin numpy (#3030)
* Add end_training/destroy_pg to everything

* Carry over to AcceleratorState

* If forked, ignore

* More numpy fun

* Skip only init
2024-08-20 10:40:12 -04:00
13e79ccfab Enable more Ruff lints & fix issues (#2419)
* Remove antiquated flake8 and isort configuration

* Bump to Ruff 0.2.1

* Explain ruff options

* Autofix Ruff B010 (static `setattr`)

* Autofix Ruff B009 (static `getattr`)

* Enable Ruff UP (not UP007); auto-fix

* Fix remaining Ruff UP complaints

* Fix a couple more format calls
2024-02-14 08:59:42 -05:00
1bfde6b963 Fp8 integration (#1086)
* Draft of FP8 support

* Missing import

* Fix names

* Conversion is inplace

* Enable fp8 in examples

* Customization point for Recipe

* Auto-enable FP8 depending on compute capability

* Fix typo

* Put back mixed precision arg

* Add debug script

* Add more tests in debug

* Add more stuff to debug

* Don't forget train

* Put the train in the right place

* Add options for selective conversion

* Fix typo

* Properly recurse

* Add more debug utils

* Typo and init

* Last choice

* More fixes

* More options in example

* Remove debug scripts

* Clean up debug and new names

* Add torch.no_grad for conversion

* Optimizer is deconnected from model?

* Re-attach model parameters to optimizer

* Fix extract

* Style

* Cleanup post-rebase

* Deal with apdding

* fix examples

* Update src/accelerate/accelerator.py

Co-authored-by: Sourab Mangrulkar <13534540+pacman100@users.noreply.github.com>

* Address comments

---------

Co-authored-by: Sourab Mangrulkar <13534540+pacman100@users.noreply.github.com>
2023-03-07 09:10:10 -05:00
5002e56704 Update quality tools to 2023 (#1046)
* Setup 2023 tooling for quality

* Result of styling

* Simplify inits and remove isort and flake8 from doc

* Puts back isort skip flag
2023-02-07 13:34:05 -05:00
f0f348921d Don't force mixed precision as no in examples (#1018) 2023-01-27 10:12:27 -05:00
8944975a3c Reenable Gather for Metrics (#590)
* Clean and finish

Co-authored-by: Sylvain Gugger <Sylvain.gugger@gmail.com>
2022-08-02 13:45:17 -04:00
503057132d Skip and raise NotImplementedError for gather_for_metrics for now (#580)
* Skip and raise NotImplementedError for now
2022-07-28 11:56:00 -04:00
e0212893ea Fix gather_for_metrics (#578)
* Fix gather_for_metrics
2022-07-27 14:20:52 -04:00
164943c7d7 Add a gather_for_metrics capability (#540)
* Add test and full implementation
2022-07-21 07:40:37 -04:00
f62ae86cfb Mix precision -> Mixed precision (#388) 2022-05-23 15:02:29 -04:00
209db19dc8 Create a testing framework for example scripts and fix current ones (#313)
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-04-13 13:24:36 -04:00
8595834292 Refactor Examples by Feature (#312)
Splits up examples into by feature scripts
2022-04-11 15:59:13 -04:00
38e6d941fa Update example scripts (#307)
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-04-04 17:19:25 -04:00
2c554b056c Pass lr_scheduler to Accelerator.prepare (#301)
* Work in progress

* Pass scheduler to Accelerator.prapre

* Fix tests

* Apply suggestions from code review

Co-authored-by: Zachary Mueller <muellerzr@gmail.com>

* Style post comments

Co-authored-by: Zachary Mueller <muellerzr@gmail.com>
2022-03-31 09:55:41 -04:00
4fc586f5af [WIP] Add bfloat16 support #243 (#247)
* Add bfloat16 support (#243)

* Compatibility with previous config files

* Fix non-default argument 'num_processes' follows default argument

* Compatibility with previous config files

* Fix user input config

* Add use_fp16 compatibility

* Show dtype

* Verbosity

* Remove dtype verbosity

* Update src/accelerate/accelerator.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/accelerate/commands/launch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Add version checks and fix "no"/None issues

* Update src/accelerate/accelerator.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/accelerate/commands/launch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/accelerate/commands/launch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/accelerate/commands/launch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/accelerate/commands/launch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/accelerate/notebook_launcher.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/accelerate/commands/launch.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Raise error if required PyTorch version is not available

* Style

* Make style

* Update src/accelerate/accelerator.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Delete unused import

* Raise error if requited Pytorch is not available (Fix previous commit)

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-02-10 11:41:29 -05:00
2575bc829e Add KwargsHandlers (#15)
* Add KwargsHandlers

* Typo

* Add more tests
2021-03-23 10:06:59 -04:00
b206aad14b Add CV example (#10)
* Add CV example

* Update README

* Make get_label picklable
2021-03-15 09:33:45 -04:00