This change adds `LockedLayerRepository` as an alternative to
`LayerRepository`. `LockedLayerRepository` allows for locking all kernel
layers that are used at the project level. Example usage:
```
with use_kernel_mapping(
{
"SomeLayer": {
"cuda": LockedLayerRepository(
repo_id="some-org/some-layer",
layer_name="SomeLayer",
)
},
}
):
layer = kernelize(layer, device="cuda", mode=Mode.INFERENCE)
```
This requires that the project has a `pyproject.toml` with kernel
version specifications and `kernel.lock` with the locked kernels.
* first commit
* style
* update
* fix
* different approach
* Polish kernelize
- Process comment from the PR.
- Replacement should be on instances, not the class.
- Remove torch compile checks (not relevant during kernelize). We
might add it back in a different way in another commit: add an
option to `kernelize`.
* Fixup tests
* Fix `torch.compile` support
* Remove some unused code
* Sync the docs
* CI: update Torch versions
---------
Co-authored-by: Daniël de Kok <me@danieldk.eu>
* Disable kernel mappings with `DISABLE_KERNEL_MAPPING=1`
* Rename HF_KERNELS_CACHE to KERNELS_CACHE
But still recognize the old variant for compatibility.
* Add documentation for environment variables
* Add `use_kernel_forward_from_hub` decorator
This decorator replaces a layer's `forward` with the `forward` of
a layer on the hub.
* Add support for registering a mapping for the duration of a context
This change makes `_KERNEL_MAPPING` a context variable and adds a
`use_kernel_mapping` context manager. This allows users to register
a mapping for the duration of a context.
* Update layer docs
* ruff fix
* Remove an old bit from the docs
* Extend layer mapping example
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* Support stringly-typed device type
* Forward-reference `register_kernel_mapping` in monkeypatching section
* Use stringly-typed device name in layer mapping example
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
---------
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
* feat: add docker that includes activation kernel usage example
* feat: improve docker demo and add example script
* feat: improve docker location and update readme
