# PEFT low level API

## PEFT as a utility library

Let’s cover in this section how you can leverage PEFT’s low level API to inject trainable adapters into any `torch` module. The development of this API has been motivated by the need for super users to not rely on modeling classes that are exposed in PEFT library and still be able to use adapter methods such as LoRA, IA3 and AdaLoRA.

### Supported tuner types

Currently the supported adapter types are the ‘injectable’ adapters, meaning adapters where an inplace modification of the model is sufficient to correctly perform the fine tuning. As such, only [LoRA](https://huggingface.co/docs/peft/developer_guides/conceptual_guides/lora), AdaLoRA and [IA3](https://huggingface.co/docs/peft/developer_guides/conceptual_guides/ia3) are currently supported in this API.

### inject\_adapter\_in\_model method

To perform the adapter injection, simply use `inject_adapter_in_model` method that takes 3 arguments, the PEFT config and the model itself and an optional adapter name. You can also attach multiple adapters in the model if you call multiple times `inject_adapter_in_model` with different adapter names.

Below is a basic example usage of how to inject LoRA adapters into the submodule `linear` of the module `DummyModel`.

Copied

```
import torch
from peft import inject_adapter_in_model, LoraConfig


class DummyModel(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.embedding = torch.nn.Embedding(10, 10)
        self.linear = torch.nn.Linear(10, 10)
        self.lm_head = torch.nn.Linear(10, 10)

    def forward(self, input_ids):
        x = self.embedding(input_ids)
        x = self.linear(x)
        x = self.lm_head(x)
        return x


lora_config = LoraConfig(
    lora_alpha=16,
    lora_dropout=0.1,
    r=64,
    bias="none",
    target_modules=["linear"],
)

model = DummyModel()
model = inject_adapter_in_model(lora_config, model)

dummy_inputs = torch.LongTensor([[0, 1, 2, 3, 4, 5, 6, 7]])
dummy_outputs = model(dummy_inputs)
```

If you print the model, you will notice that the adapters have been correctly injected into the model

Copied

```
DummyModel(
  (embedding): Embedding(10, 10)
  (linear): Linear(
    in_features=10, out_features=10, bias=True
    (lora_dropout): ModuleDict(
      (default): Dropout(p=0.1, inplace=False)
    )
    (lora_A): ModuleDict(
      (default): Linear(in_features=10, out_features=64, bias=False)
    )
    (lora_B): ModuleDict(
      (default): Linear(in_features=64, out_features=10, bias=False)
    )
    (lora_embedding_A): ParameterDict()
    (lora_embedding_B): ParameterDict()
  )
  (lm_head): Linear(in_features=10, out_features=10, bias=True)
)
```

Note that it should be up to users to properly take care of saving the adapters (in case they want to save adapters only), as `model.state_dict()` will return the full state dict of the model. In case you want to extract the adapters state dict you can use the `get_peft_model_state_dict` method:

Copied

```
from peft import get_peft_model_state_dict

peft_state_dict = get_peft_model_state_dict(model)
print(peft_state_dict)
```

### Pros and cons

When to use this API and when to not use it? Let’s discuss in this section the pros and cons

Pros:

* The model gets modified in-place, meaning the model will preserve all its original attributes and methods
* Works for any torch module, and any modality (vision, text, multi-modal)

Cons:

* You need to manually writing BOINC AI `from_pretrained` and `save_pretrained` utility methods if you want to easily save / load adapters from the BOINC AI Hub.
* You cannot use any of the utility method provided by `PeftModel` such as disabling adapters, merging adapters, etc.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://boinc-ai.gitbook.io/peft/developer-guides/peft-low-level-api.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.