Modeling
Last updated
Last updated
Optimum Furiosa is a utility package for building and running inference with Furiosa NPUs. Optimum can be used to load optimized models from the and create pipelines to run accelerated inference without rewriting your APIs.
The optimum.furiosa.FuriosaAIModelForXXX
model classes are API compatible with Hugging Face models. This means you can just replace your AutoModelForXXX
class with the corresponding FuriosaAIModelForXXX
class in optimum.furiosa
.
You do not need to adapt your code to get it to work with FuriosaAIModelForXXX
classes:
Because the model you want to work with might not be already converted to ONNX, FuriosaAIModel
includes a method to convert vanilla Hugging Face models to ONNX ones. Simply pass export=True
to the from_pretrained
method, and your model will be loaded and converted to ONNX on-the-fly:
Copied
Copied
It is also possible, just as with regular s, to push your FurisoaAIModelForXXX
to the :