# BOINC AI Optimum Habana

## 🤗 Optimum Habana

🤗 Optimum Habana is the interface between the 🤗 Transformers and 🤗 Diffusers libraries and [Habana’s Gaudi processor (HPU)](https://docs.habana.ai/en/latest/index.html). It provides a set of tools that enable easy model loading, training and inference on single- and multi-HPU settings for various downstream tasks as shown in the table below.

HPUs offer fast model training and inference as well as a great price-performance ratio. Check out [this blog post about BERT pre-training](https://huggingface.co/blog/pretraining-bert) and [this article benchmarking Habana Gaudi2 versus Nvidia A100 GPUs](https://huggingface.co/blog/habana-gaudi-2-benchmark) for concrete examples. If you are not familiar with HPUs, we recommend you take a look at [our conceptual guide](https://huggingface.co/docs/optimum/habana/concept_guides/hpu).

The following model architectures, tasks and device distributions have been validated for 🤗 Optimum Habana:

In the tables below, ✅ means single-card, multi-card and DeepSpeed have all been validated.

* Transformers

| Architecture        |                      Training                     |                         Inference                        | Tasks                                                                                                                                                                                                                                                                                                                                                                                                                |
| ------------------- | :-----------------------------------------------: | :------------------------------------------------------: | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| BERT                |                         ✅                         |                             ✅                            | <ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/text-classification">text classification</a></li></ul><ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/question-answering">question answering</a></li></ul><ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/language-modeling">language modeling</a></li></ul>   |
| RoBERTa             |                         ✅                         |                             ✅                            | <ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/question-answering">question answering</a></li></ul><ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/language-modeling">language modeling</a></li></ul>                                                                                                                                           |
| ALBERT              |                         ✅                         |                             ✅                            | <ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/question-answering">question answering</a></li></ul><ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/language-modeling">language modeling</a></li></ul>                                                                                                                                           |
| DistilBERT          |                         ✅                         |                             ✅                            | <ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/question-answering">question answering</a></li></ul><ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/language-modeling">language modeling</a></li></ul>                                                                                                                                           |
| GPT2                |                         ✅                         |                             ✅                            | <ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/language-modeling">language modeling</a></li></ul><ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/text-generation">text generation</a></li></ul>                                                                                                                                                 |
| BLOOM(Z)            |                         ❌                         |                <ul><li>DeepSpeed</li></ul>               | <ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/text-generation">text generation</a></li></ul>                                                                                                                                                                                                                                                                                     |
| StarCoder           |                         ❌                         |               <ul><li>Single card</li></ul>              | <ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/text-generation">text generation</a></li></ul>                                                                                                                                                                                                                                                                                     |
| GPT-J               |            <ul><li>DeepSpeed</li></ul>            | <ul><li>Single card</li></ul><ul><li>DeepSpeed</li></ul> | <ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/language-modeling">language modeling</a></li></ul><ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/text-generation">text generation</a></li></ul>                                                                                                                                                 |
| GPT-NeoX            |            <ul><li>DeepSpeed</li></ul>            |                <ul><li>DeepSpeed</li></ul>               | <ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/language-modeling">language modeling</a></li></ul><ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/text-generation">text generation</a></li></ul>                                                                                                                                                 |
| OPT                 |                         ❌                         |                <ul><li>DeepSpeed</li></ul>               | <ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/text-generation">text generation</a></li></ul>                                                                                                                                                                                                                                                                                     |
| Llama 2 / CodeLlama | <ul><li>DeepSpeed</li></ul><ul><li>LoRA</li></ul> |     <ul><li>DeepSpeed</li></ul><ul><li>LoRA</li></ul>    | <ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/language-modeling">language modeling</a></li></ul><ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/text-generation">text generation</a></li></ul>                                                                                                                                                 |
| StableLM            |                         ❌                         |               <ul><li>Single card</li></ul>              | <ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/text-generation">text generation</a></li></ul>                                                                                                                                                                                                                                                                                     |
| Falcon              |                         ❌                         |               <ul><li>Single card</li></ul>              | <ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/text-generation">text generation</a></li></ul>                                                                                                                                                                                                                                                                                     |
| CodeGen             |                         ❌                         |               <ul><li>Single card</li></ul>              | <ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/text-generation">text generation</a></li></ul>                                                                                                                                                                                                                                                                                     |
| MPT                 |                         ❌                         |               <ul><li>Single card</li></ul>              | <ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/text-generation">text generation</a></li></ul>                                                                                                                                                                                                                                                                                     |
| T5                  |                         ✅                         |                             ✅                            | <ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/summarization">summarization</a></li></ul><ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/translation">translation</a></li></ul><ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/question-answering#fine-tuning-t5-on-squad20">question answering</a></li></ul> |
| BART                |                         ❌                         |               <ul><li>Single card</li></ul>              | <ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/summarization">summarization</a></li></ul><ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/translation">translation</a></li></ul><ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/question-answering#fine-tuning-t5-on-squad20">question answering</a></li></ul> |
| ViT                 |                         ✅                         |                             ✅                            | <ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/image-classification">image classification</a></li></ul>                                                                                                                                                                                                                                                                           |
| Swin                |                         ✅                         |                             ✅                            | <ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/image-classification">image classification</a></li></ul>                                                                                                                                                                                                                                                                           |
| Wav2Vec2            |                         ✅                         |                             ✅                            | <ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/audio-classification">audio classification</a></li></ul><ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/speech-recognition">speech recognition</a></li></ul>                                                                                                                                     |
| CLIP                |                         ✅                         |                             ✅                            | <ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/contrastive-image-text">contrastive image-text training</a></li></ul>                                                                                                                                                                                                                                                              |
| BridgeTower         |                         ✅                         |                             ✅                            | <ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/contrastive-image-text">contrastive image-text training</a></li></ul>                                                                                                                                                                                                                                                              |
| ESMFold             |                         ❌                         |               <ul><li>Single card</li></ul>              | <ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/protein-folding">protein folding</a></li></ul>                                                                                                                                                                                                                                                                                     |

* Diffusers

| Architecture     | Training |           Inference           | \<center>Tasks\</center>                                                                                                                   |
| ---------------- | :------: | :---------------------------: | ------------------------------------------------------------------------------------------------------------------------------------------ |
| Stable Diffusion |     ❌    | <ul><li>Single card</li></ul> | <ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/stable-diffusion">text-to-image generation</a></li></ul> |
| LDM3D            |     ❌    | <ul><li>Single card</li></ul> | <ul><li><a href="https://github.com/huggingface/optimum-habana/tree/main/examples/stable-diffusion">text-to-image generation</a></li></ul> |

Other models and tasks supported by the 🌍 Transformers and 🌍 Diffusers library may also work. You can refer to this [section](https://github.com/huggingface/optimum-habana#how-to-use-it) for using them with 🌍 Optimum Habana. Besides, [this page](https://github.com/huggingface/optimum-habana/tree/main/examples) explains how to modify any [example](https://github.com/huggingface/transformers/tree/main/examples/pytorch) from the 🌍 Transformers library to make it work with 🌍 Optimum Habana.
