> For the complete documentation index, see [llms.txt](https://boinc-ai.gitbook.io/peft/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://boinc-ai.gitbook.io/peft/reference/peft-model.md).

# PEFT model

## Models

[PeftModel](https://huggingface.co/docs/peft/main/en/package_reference/peft_model#peft.PeftModel) is the base model class for specifying the base Transformer model and configuration to apply a PEFT method to. The base `PeftModel` contains methods for loading and saving models from the Hub, and supports the [PromptEncoder](https://huggingface.co/docs/peft/main/en/package_reference/tuners#peft.PromptEncoder) for prompt learning.

### PeftModel

#### class peft.PeftModel

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L81)

( model: PreTrainedModelpeft\_config: PeftConfigadapter\_name: str = 'default' )

Parameters

* **model** ([PreTrainedModel](https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel)) — The base transformer model used for Peft.
* **peft\_config** ([PeftConfig](https://huggingface.co/docs/peft/main/en/package_reference/config#peft.PeftConfig)) — The configuration of the Peft model.
* **adapter\_name** (`str`) — The name of the adapter, defaults to `"default"`.

Base model encompassing various Peft methods.

**Attributes**:

* **base\_model** ([PreTrainedModel](https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel)) — The base transformer model used for Peft.
* **peft\_config** ([PeftConfig](https://huggingface.co/docs/peft/main/en/package_reference/config#peft.PeftConfig)) — The configuration of the Peft model.
* **modules\_to\_save** (`list` of `str`) — The list of sub-module names to save when saving the model.
* **prompt\_encoder** ([PromptEncoder](https://huggingface.co/docs/peft/main/en/package_reference/tuners#peft.PromptEncoder)) — The prompt encoder used for Peft if using [PromptLearningConfig](https://huggingface.co/docs/peft/main/en/package_reference/config#peft.PromptLearningConfig).
* **prompt\_tokens** (`torch.Tensor`) — The virtual prompt tokens used for Peft if using [PromptLearningConfig](https://huggingface.co/docs/peft/main/en/package_reference/config#peft.PromptLearningConfig).
* **transformer\_backbone\_name** (`str`) — The name of the transformer backbone in the base model if using [PromptLearningConfig](https://huggingface.co/docs/peft/main/en/package_reference/config#peft.PromptLearningConfig).
* **word\_embeddings** (`torch.nn.Embedding`) — The word embeddings of the transformer backbone in the base model if using [PromptLearningConfig](https://huggingface.co/docs/peft/main/en/package_reference/config#peft.PromptLearningConfig).

**create\_or\_update\_model\_card**

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L671)

( output\_dir: str )

Updates or create model card to include information about peft:

1. Adds `peft` library tag
2. Adds peft version
3. Adds base model info
4. Adds quantization information if it was used

**disable\_adapter**

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L500)

( )

Disables the adapter module.

**forward**

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L486)

( \*args: Any\*\*kwargs: Any )

Forward pass of the model.

**from\_pretrained**

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L237)

( model: PreTrainedModelmodel\_id: Union\[str, os.PathLike]adapter\_name: str = 'default'is\_trainable: bool = Falseconfig: Optional\[PeftConfig] = None\*\*kwargs: Any )

Parameters

* **model** ([PreTrainedModel](https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel)) — The model to be adapted. The model should be initialized with the [from\_pretrained](https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) method from the 🌍 Transformers library.
* **model\_id** (`str` or `os.PathLike`) — The name of the PEFT configuration to use. Can be either:
  * A string, the `model id` of a PEFT configuration hosted inside a model repo on the BOINC AI Hub.
  * A path to a directory containing a PEFT configuration file saved using the `save_pretrained` method (`./my_peft_config_directory/`).
* **adapter\_name** (`str`, *optional*, defaults to `"default"`) — The name of the adapter to be loaded. This is useful for loading multiple adapters.
* **is\_trainable** (`bool`, *optional*, defaults to `False`) — Whether the adapter should be trainable or not. If `False`, the adapter will be frozen and use for inference
* **config** ([PeftConfig](https://huggingface.co/docs/peft/main/en/package_reference/config#peft.PeftConfig), *optional*) — The configuration object to use instead of an automatically loaded configuation. This configuration object is mutually exclusive with `model_id` and `kwargs`. This is useful when configuration is already loaded before calling `from_pretrained`. kwargs — (`optional`): Additional keyword arguments passed along to the specific PEFT configuration class.

Instantiate a PEFT model from a pretrained model and loaded PEFT weights.

Note that the passed `model` may be modified inplace.

**get\_base\_model**

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L523)

( )

Returns the base model.

**get\_nb\_trainable\_parameters**

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L445)

( )

Returns the number of trainable parameters and number of all parameters in the model.

**get\_prompt**

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L399)

( batch\_size: inttask\_ids: Optional\[torch.Tensor] = None )

Returns the virtual prompts to use for Peft. Only applicable when `peft_config.peft_type != PeftType.LORA`.

**get\_prompt\_embedding\_to\_save**

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L380)

( adapter\_name: str )

Returns the prompt embedding to save when saving the model. Only applicable when `peft_config.peft_type != PeftType.LORA`.

**print\_trainable\_parameters**

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L469)

( )

Prints the number of trainable parameters in the model.

**save\_pretrained**

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L155)

( save\_directory: strsafe\_serialization: bool = Falseselected\_adapters: Optional\[List\[str]] = None\*\*kwargs: Any )

Parameters

* **save\_directory** (`str`) — Directory where the adapter model and configuration files will be saved (will be created if it does not exist).
* **kwargs** (additional keyword arguments, *optional*) — Additional keyword arguments passed along to the `push_to_hub` method.

This function saves the adapter model and the adapter configuration files to a directory, so that it can be reloaded using the [PeftModel.from\_pretrained()](https://huggingface.co/docs/peft/main/en/package_reference/peft_model#peft.PeftModel.from_pretrained) class method, and also used by the `PeftModel.push_to_hub()` method.

**set\_adapter**

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L652)

( adapter\_name: str )

Sets the active adapter.

### PeftModelForSequenceClassification

A `PeftModel` for sequence classification tasks.

#### class peft.PeftModelForSequenceClassification

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L720)

( modelpeft\_config: PeftConfigadapter\_name = 'default' )

Parameters

* **model** ([PreTrainedModel](https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel)) — Base transformer model.
* **peft\_config** ([PeftConfig](https://huggingface.co/docs/peft/main/en/package_reference/config#peft.PeftConfig)) — Peft config.

Peft model for sequence classification tasks.

**Attributes**:

* **config** ([PretrainedConfig](https://huggingface.co/docs/transformers/main/en/main_classes/configuration#transformers.PretrainedConfig)) — The configuration object of the base model.
* **cls\_layer\_name** (`str`) — The name of the classification layer.

Example:

Copied

```
>>> from transformers import AutoModelForSequenceClassification
>>> from peft import PeftModelForSequenceClassification, get_peft_config

>>> config = {
...     "peft_type": "PREFIX_TUNING",
...     "task_type": "SEQ_CLS",
...     "inference_mode": False,
...     "num_virtual_tokens": 20,
...     "token_dim": 768,
...     "num_transformer_submodules": 1,
...     "num_attention_heads": 12,
...     "num_layers": 12,
...     "encoder_hidden_size": 768,
...     "prefix_projection": False,
...     "postprocess_past_key_value_function": None,
... }

>>> peft_config = get_peft_config(config)
>>> model = AutoModelForSequenceClassification.from_pretrained("bert-base-cased")
>>> peft_model = PeftModelForSequenceClassification(model, peft_config)
>>> peft_model.print_trainable_parameters()
trainable params: 370178 || all params: 108680450 || trainable%: 0.3406113979101117
```

### PeftModelForTokenClassification

A `PeftModel` for token classification tasks.

#### class peft.PeftModelForTokenClassification

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L1323)

( modelpeft\_config: PeftConfig = Noneadapter\_name = 'default' )

Parameters

* **model** ([PreTrainedModel](https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel)) — Base transformer model.
* **peft\_config** ([PeftConfig](https://huggingface.co/docs/peft/main/en/package_reference/config#peft.PeftConfig)) — Peft config.

Peft model for token classification tasks.

**Attributes**:

* **config** ([PretrainedConfig](https://huggingface.co/docs/transformers/main/en/main_classes/configuration#transformers.PretrainedConfig)) — The configuration object of the base model.
* **cls\_layer\_name** (`str`) — The name of the classification layer.

Example:

Copied

```
>>> from transformers import AutoModelForSequenceClassification
>>> from peft import PeftModelForTokenClassification, get_peft_config

>>> config = {
...     "peft_type": "PREFIX_TUNING",
...     "task_type": "TOKEN_CLS",
...     "inference_mode": False,
...     "num_virtual_tokens": 20,
...     "token_dim": 768,
...     "num_transformer_submodules": 1,
...     "num_attention_heads": 12,
...     "num_layers": 12,
...     "encoder_hidden_size": 768,
...     "prefix_projection": False,
...     "postprocess_past_key_value_function": None,
... }

>>> peft_config = get_peft_config(config)
>>> model = AutoModelForTokenClassification.from_pretrained("bert-base-cased")
>>> peft_model = PeftModelForTokenClassification(model, peft_config)
>>> peft_model.print_trainable_parameters()
trainable params: 370178 || all params: 108680450 || trainable%: 0.3406113979101117
```

### PeftModelForCausalLM

A `PeftModel` for causal language modeling.

#### class peft.PeftModelForCausalLM

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L909)

( modelpeft\_config: PeftConfigadapter\_name = 'default' )

Parameters

* **model** ([PreTrainedModel](https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel)) — Base transformer model.
* **peft\_config** ([PeftConfig](https://huggingface.co/docs/peft/main/en/package_reference/config#peft.PeftConfig)) — Peft config.

Peft model for causal language modeling.

Example:

Copied

```
>>> from transformers import AutoModelForCausalLM
>>> from peft import PeftModelForCausalLM, get_peft_config

>>> config = {
...     "peft_type": "PREFIX_TUNING",
...     "task_type": "CAUSAL_LM",
...     "inference_mode": False,
...     "num_virtual_tokens": 20,
...     "token_dim": 1280,
...     "num_transformer_submodules": 1,
...     "num_attention_heads": 20,
...     "num_layers": 36,
...     "encoder_hidden_size": 1280,
...     "prefix_projection": False,
...     "postprocess_past_key_value_function": None,
... }

>>> peft_config = get_peft_config(config)
>>> model = AutoModelForCausalLM.from_pretrained("gpt2-large")
>>> peft_model = PeftModelForCausalLM(model, peft_config)
>>> peft_model.print_trainable_parameters()
trainable params: 1843200 || all params: 775873280 || trainable%: 0.23756456724479544
```

### PeftModelForSeq2SeqLM

A `PeftModel` for sequence-to-sequence language modeling.

#### class peft.PeftModelForSeq2SeqLM

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L1078)

( modelpeft\_config: PeftConfigadapter\_name = 'default' )

Parameters

* **model** ([PreTrainedModel](https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel)) — Base transformer model.
* **peft\_config** ([PeftConfig](https://huggingface.co/docs/peft/main/en/package_reference/config#peft.PeftConfig)) — Peft config.

Peft model for sequence-to-sequence language modeling.

Example:

Copied

```
>>> from transformers import AutoModelForSeq2SeqLM
>>> from peft import PeftModelForSeq2SeqLM, get_peft_config

>>> config = {
...     "peft_type": "LORA",
...     "task_type": "SEQ_2_SEQ_LM",
...     "inference_mode": False,
...     "r": 8,
...     "target_modules": ["q", "v"],
...     "lora_alpha": 32,
...     "lora_dropout": 0.1,
...     "fan_in_fan_out": False,
...     "enable_lora": None,
...     "bias": "none",
... }

>>> peft_config = get_peft_config(config)
>>> model = AutoModelForSeq2SeqLM.from_pretrained("t5-base")
>>> peft_model = PeftModelForSeq2SeqLM(model, peft_config)
>>> peft_model.print_trainable_parameters()
trainable params: 884736 || all params: 223843584 || trainable%: 0.3952474242013566
```

### PeftModelForQuestionAnswering

A `PeftModel` for question answering.

#### class peft.PeftModelForQuestionAnswering

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L1496)

( modelpeft\_config: PeftConfig = Noneadapter\_name = 'default' )

Parameters

* **model** ([PreTrainedModel](https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel)) — Base transformer model.
* **peft\_config** ([PeftConfig](https://huggingface.co/docs/peft/main/en/package_reference/config#peft.PeftConfig)) — Peft config.

Peft model for extractive question answering.

**Attributes**:

* **config** ([PretrainedConfig](https://huggingface.co/docs/transformers/main/en/main_classes/configuration#transformers.PretrainedConfig)) — The configuration object of the base model.
* **cls\_layer\_name** (`str`) — The name of the classification layer.

Example:

Copied

```
>>> from transformers import AutoModelForQuestionAnswering
>>> from peft import PeftModelForQuestionAnswering, get_peft_config

>>> config = {
...     "peft_type": "LORA",
...     "task_type": "QUESTION_ANS",
...     "inference_mode": False,
...     "r": 16,
...     "target_modules": ["query", "value"],
...     "lora_alpha": 32,
...     "lora_dropout": 0.05,
...     "fan_in_fan_out": False,
...     "bias": "none",
... }

>>> peft_config = get_peft_config(config)
>>> model = AutoModelForQuestionAnswering.from_pretrained("bert-base-cased")
>>> peft_model = PeftModelForQuestionAnswering(model, peft_config)
>>> peft_model.print_trainable_parameters()
trainable params: 592900 || all params: 108312580 || trainable%: 0.5473971721475013
```

### PeftModelForFeatureExtraction

A `PeftModel` for getting extracting features/embeddings from transformer models.

#### class peft.PeftModelForFeatureExtraction

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L1688)

( modelpeft\_config: PeftConfig = Noneadapter\_name = 'default' )

Parameters

* **model** ([PreTrainedModel](https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel)) — Base transformer model.
* **peft\_config** ([PeftConfig](https://huggingface.co/docs/peft/main/en/package_reference/config#peft.PeftConfig)) — Peft config.

Peft model for extracting features/embeddings from transformer models

**Attributes**:

* **config** ([PretrainedConfig](https://huggingface.co/docs/transformers/main/en/main_classes/configuration#transformers.PretrainedConfig)) — The configuration object of the base model.

Example:

Copied

```
>>> from transformers import AutoModel
>>> from peft import PeftModelForFeatureExtraction, get_peft_config

>>> config = {
...     "peft_type": "LORA",
...     "task_type": "FEATURE_EXTRACTION",
...     "inference_mode": False,
...     "r": 16,
...     "target_modules": ["query", "value"],
...     "lora_alpha": 32,
...     "lora_dropout": 0.05,
...     "fan_in_fan_out": False,
...     "bias": "none",
... }
>>> peft_config = get_peft_config(config)
>>> model = AutoModel.from_pretrained("bert-base-cased")
>>> peft_model = PeftModelForFeatureExtraction(model, peft_config)
>>> peft_model.print_trainable_parameters()
```