mirror of
https://github.com/huggingface/kernels.git
synced 2025-11-11 16:50:41 +08:00
14 lines
537 B
Markdown
14 lines
537 B
Markdown
# FAQ
|
|
|
|
## Why is the kernelization step needed?
|
|
|
|
In earlier versions of `kernels`, a layer's `forward` method was replaced
|
|
by `use_kernel_forward_from_hub` and `replace_kernel_forward_from_hub`.
|
|
The new `forward` would dispatch to a kernel based on the device type,
|
|
whether a model was training, etc. However, this approach was
|
|
fundamentally incompatible with `torch.compile` since it relied
|
|
on data-dependent branching.
|
|
|
|
To avoid branching, we have to make dispatch decisions ahead of time,
|
|
which is what the `kernelize` function does.
|