# PEFT model

## Models

[PeftModel](https://huggingface.co/docs/peft/main/en/package_reference/peft_model#peft.PeftModel) is the base model class for specifying the base Transformer model and configuration to apply a PEFT method to. The base `PeftModel` contains methods for loading and saving models from the Hub, and supports the [PromptEncoder](https://huggingface.co/docs/peft/main/en/package_reference/tuners#peft.PromptEncoder) for prompt learning.

### PeftModel

#### class peft.PeftModel

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L81)

( model: PreTrainedModelpeft\_config: PeftConfigadapter\_name: str = 'default' )

Parameters

* **model** ([PreTrainedModel](https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel)) — The base transformer model used for Peft.
* **peft\_config** ([PeftConfig](https://huggingface.co/docs/peft/main/en/package_reference/config#peft.PeftConfig)) — The configuration of the Peft model.
* **adapter\_name** (`str`) — The name of the adapter, defaults to `"default"`.

Base model encompassing various Peft methods.

**Attributes**:

* **base\_model** ([PreTrainedModel](https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel)) — The base transformer model used for Peft.
* **peft\_config** ([PeftConfig](https://huggingface.co/docs/peft/main/en/package_reference/config#peft.PeftConfig)) — The configuration of the Peft model.
* **modules\_to\_save** (`list` of `str`) — The list of sub-module names to save when saving the model.
* **prompt\_encoder** ([PromptEncoder](https://huggingface.co/docs/peft/main/en/package_reference/tuners#peft.PromptEncoder)) — The prompt encoder used for Peft if using [PromptLearningConfig](https://huggingface.co/docs/peft/main/en/package_reference/config#peft.PromptLearningConfig).
* **prompt\_tokens** (`torch.Tensor`) — The virtual prompt tokens used for Peft if using [PromptLearningConfig](https://huggingface.co/docs/peft/main/en/package_reference/config#peft.PromptLearningConfig).
* **transformer\_backbone\_name** (`str`) — The name of the transformer backbone in the base model if using [PromptLearningConfig](https://huggingface.co/docs/peft/main/en/package_reference/config#peft.PromptLearningConfig).
* **word\_embeddings** (`torch.nn.Embedding`) — The word embeddings of the transformer backbone in the base model if using [PromptLearningConfig](https://huggingface.co/docs/peft/main/en/package_reference/config#peft.PromptLearningConfig).

**create\_or\_update\_model\_card**

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L671)

( output\_dir: str )

Updates or create model card to include information about peft:

1. Adds `peft` library tag
2. Adds peft version
3. Adds base model info
4. Adds quantization information if it was used

**disable\_adapter**

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L500)

( )

Disables the adapter module.

**forward**

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L486)

( \*args: Any\*\*kwargs: Any )

Forward pass of the model.

**from\_pretrained**

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L237)

( model: PreTrainedModelmodel\_id: Union\[str, os.PathLike]adapter\_name: str = 'default'is\_trainable: bool = Falseconfig: Optional\[PeftConfig] = None\*\*kwargs: Any )

Parameters

* **model** ([PreTrainedModel](https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel)) — The model to be adapted. The model should be initialized with the [from\_pretrained](https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel.from_pretrained) method from the 🌍 Transformers library.
* **model\_id** (`str` or `os.PathLike`) — The name of the PEFT configuration to use. Can be either:
  * A string, the `model id` of a PEFT configuration hosted inside a model repo on the BOINC AI Hub.
  * A path to a directory containing a PEFT configuration file saved using the `save_pretrained` method (`./my_peft_config_directory/`).
* **adapter\_name** (`str`, *optional*, defaults to `"default"`) — The name of the adapter to be loaded. This is useful for loading multiple adapters.
* **is\_trainable** (`bool`, *optional*, defaults to `False`) — Whether the adapter should be trainable or not. If `False`, the adapter will be frozen and use for inference
* **config** ([PeftConfig](https://huggingface.co/docs/peft/main/en/package_reference/config#peft.PeftConfig), *optional*) — The configuration object to use instead of an automatically loaded configuation. This configuration object is mutually exclusive with `model_id` and `kwargs`. This is useful when configuration is already loaded before calling `from_pretrained`. kwargs — (`optional`): Additional keyword arguments passed along to the specific PEFT configuration class.

Instantiate a PEFT model from a pretrained model and loaded PEFT weights.

Note that the passed `model` may be modified inplace.

**get\_base\_model**

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L523)

( )

Returns the base model.

**get\_nb\_trainable\_parameters**

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L445)

( )

Returns the number of trainable parameters and number of all parameters in the model.

**get\_prompt**

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L399)

( batch\_size: inttask\_ids: Optional\[torch.Tensor] = None )

Returns the virtual prompts to use for Peft. Only applicable when `peft_config.peft_type != PeftType.LORA`.

**get\_prompt\_embedding\_to\_save**

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L380)

( adapter\_name: str )

Returns the prompt embedding to save when saving the model. Only applicable when `peft_config.peft_type != PeftType.LORA`.

**print\_trainable\_parameters**

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L469)

( )

Prints the number of trainable parameters in the model.

**save\_pretrained**

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L155)

( save\_directory: strsafe\_serialization: bool = Falseselected\_adapters: Optional\[List\[str]] = None\*\*kwargs: Any )

Parameters

* **save\_directory** (`str`) — Directory where the adapter model and configuration files will be saved (will be created if it does not exist).
* **kwargs** (additional keyword arguments, *optional*) — Additional keyword arguments passed along to the `push_to_hub` method.

This function saves the adapter model and the adapter configuration files to a directory, so that it can be reloaded using the [PeftModel.from\_pretrained()](https://huggingface.co/docs/peft/main/en/package_reference/peft_model#peft.PeftModel.from_pretrained) class method, and also used by the `PeftModel.push_to_hub()` method.

**set\_adapter**

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L652)

( adapter\_name: str )

Sets the active adapter.

### PeftModelForSequenceClassification

A `PeftModel` for sequence classification tasks.

#### class peft.PeftModelForSequenceClassification

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L720)

( modelpeft\_config: PeftConfigadapter\_name = 'default' )

Parameters

* **model** ([PreTrainedModel](https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel)) — Base transformer model.
* **peft\_config** ([PeftConfig](https://huggingface.co/docs/peft/main/en/package_reference/config#peft.PeftConfig)) — Peft config.

Peft model for sequence classification tasks.

**Attributes**:

* **config** ([PretrainedConfig](https://huggingface.co/docs/transformers/main/en/main_classes/configuration#transformers.PretrainedConfig)) — The configuration object of the base model.
* **cls\_layer\_name** (`str`) — The name of the classification layer.

Example:

Copied

```
>>> from transformers import AutoModelForSequenceClassification
>>> from peft import PeftModelForSequenceClassification, get_peft_config

>>> config = {
...     "peft_type": "PREFIX_TUNING",
...     "task_type": "SEQ_CLS",
...     "inference_mode": False,
...     "num_virtual_tokens": 20,
...     "token_dim": 768,
...     "num_transformer_submodules": 1,
...     "num_attention_heads": 12,
...     "num_layers": 12,
...     "encoder_hidden_size": 768,
...     "prefix_projection": False,
...     "postprocess_past_key_value_function": None,
... }

>>> peft_config = get_peft_config(config)
>>> model = AutoModelForSequenceClassification.from_pretrained("bert-base-cased")
>>> peft_model = PeftModelForSequenceClassification(model, peft_config)
>>> peft_model.print_trainable_parameters()
trainable params: 370178 || all params: 108680450 || trainable%: 0.3406113979101117
```

### PeftModelForTokenClassification

A `PeftModel` for token classification tasks.

#### class peft.PeftModelForTokenClassification

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L1323)

( modelpeft\_config: PeftConfig = Noneadapter\_name = 'default' )

Parameters

* **model** ([PreTrainedModel](https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel)) — Base transformer model.
* **peft\_config** ([PeftConfig](https://huggingface.co/docs/peft/main/en/package_reference/config#peft.PeftConfig)) — Peft config.

Peft model for token classification tasks.

**Attributes**:

* **config** ([PretrainedConfig](https://huggingface.co/docs/transformers/main/en/main_classes/configuration#transformers.PretrainedConfig)) — The configuration object of the base model.
* **cls\_layer\_name** (`str`) — The name of the classification layer.

Example:

Copied

```
>>> from transformers import AutoModelForSequenceClassification
>>> from peft import PeftModelForTokenClassification, get_peft_config

>>> config = {
...     "peft_type": "PREFIX_TUNING",
...     "task_type": "TOKEN_CLS",
...     "inference_mode": False,
...     "num_virtual_tokens": 20,
...     "token_dim": 768,
...     "num_transformer_submodules": 1,
...     "num_attention_heads": 12,
...     "num_layers": 12,
...     "encoder_hidden_size": 768,
...     "prefix_projection": False,
...     "postprocess_past_key_value_function": None,
... }

>>> peft_config = get_peft_config(config)
>>> model = AutoModelForTokenClassification.from_pretrained("bert-base-cased")
>>> peft_model = PeftModelForTokenClassification(model, peft_config)
>>> peft_model.print_trainable_parameters()
trainable params: 370178 || all params: 108680450 || trainable%: 0.3406113979101117
```

### PeftModelForCausalLM

A `PeftModel` for causal language modeling.

#### class peft.PeftModelForCausalLM

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L909)

( modelpeft\_config: PeftConfigadapter\_name = 'default' )

Parameters

* **model** ([PreTrainedModel](https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel)) — Base transformer model.
* **peft\_config** ([PeftConfig](https://huggingface.co/docs/peft/main/en/package_reference/config#peft.PeftConfig)) — Peft config.

Peft model for causal language modeling.

Example:

Copied

```
>>> from transformers import AutoModelForCausalLM
>>> from peft import PeftModelForCausalLM, get_peft_config

>>> config = {
...     "peft_type": "PREFIX_TUNING",
...     "task_type": "CAUSAL_LM",
...     "inference_mode": False,
...     "num_virtual_tokens": 20,
...     "token_dim": 1280,
...     "num_transformer_submodules": 1,
...     "num_attention_heads": 20,
...     "num_layers": 36,
...     "encoder_hidden_size": 1280,
...     "prefix_projection": False,
...     "postprocess_past_key_value_function": None,
... }

>>> peft_config = get_peft_config(config)
>>> model = AutoModelForCausalLM.from_pretrained("gpt2-large")
>>> peft_model = PeftModelForCausalLM(model, peft_config)
>>> peft_model.print_trainable_parameters()
trainable params: 1843200 || all params: 775873280 || trainable%: 0.23756456724479544
```

### PeftModelForSeq2SeqLM

A `PeftModel` for sequence-to-sequence language modeling.

#### class peft.PeftModelForSeq2SeqLM

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L1078)

( modelpeft\_config: PeftConfigadapter\_name = 'default' )

Parameters

* **model** ([PreTrainedModel](https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel)) — Base transformer model.
* **peft\_config** ([PeftConfig](https://huggingface.co/docs/peft/main/en/package_reference/config#peft.PeftConfig)) — Peft config.

Peft model for sequence-to-sequence language modeling.

Example:

Copied

```
>>> from transformers import AutoModelForSeq2SeqLM
>>> from peft import PeftModelForSeq2SeqLM, get_peft_config

>>> config = {
...     "peft_type": "LORA",
...     "task_type": "SEQ_2_SEQ_LM",
...     "inference_mode": False,
...     "r": 8,
...     "target_modules": ["q", "v"],
...     "lora_alpha": 32,
...     "lora_dropout": 0.1,
...     "fan_in_fan_out": False,
...     "enable_lora": None,
...     "bias": "none",
... }

>>> peft_config = get_peft_config(config)
>>> model = AutoModelForSeq2SeqLM.from_pretrained("t5-base")
>>> peft_model = PeftModelForSeq2SeqLM(model, peft_config)
>>> peft_model.print_trainable_parameters()
trainable params: 884736 || all params: 223843584 || trainable%: 0.3952474242013566
```

### PeftModelForQuestionAnswering

A `PeftModel` for question answering.

#### class peft.PeftModelForQuestionAnswering

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L1496)

( modelpeft\_config: PeftConfig = Noneadapter\_name = 'default' )

Parameters

* **model** ([PreTrainedModel](https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel)) — Base transformer model.
* **peft\_config** ([PeftConfig](https://huggingface.co/docs/peft/main/en/package_reference/config#peft.PeftConfig)) — Peft config.

Peft model for extractive question answering.

**Attributes**:

* **config** ([PretrainedConfig](https://huggingface.co/docs/transformers/main/en/main_classes/configuration#transformers.PretrainedConfig)) — The configuration object of the base model.
* **cls\_layer\_name** (`str`) — The name of the classification layer.

Example:

Copied

```
>>> from transformers import AutoModelForQuestionAnswering
>>> from peft import PeftModelForQuestionAnswering, get_peft_config

>>> config = {
...     "peft_type": "LORA",
...     "task_type": "QUESTION_ANS",
...     "inference_mode": False,
...     "r": 16,
...     "target_modules": ["query", "value"],
...     "lora_alpha": 32,
...     "lora_dropout": 0.05,
...     "fan_in_fan_out": False,
...     "bias": "none",
... }

>>> peft_config = get_peft_config(config)
>>> model = AutoModelForQuestionAnswering.from_pretrained("bert-base-cased")
>>> peft_model = PeftModelForQuestionAnswering(model, peft_config)
>>> peft_model.print_trainable_parameters()
trainable params: 592900 || all params: 108312580 || trainable%: 0.5473971721475013
```

### PeftModelForFeatureExtraction

A `PeftModel` for getting extracting features/embeddings from transformer models.

#### class peft.PeftModelForFeatureExtraction

[\<source>](https://github.com/huggingface/peft/blob/main/src/peft/peft_model.py#L1688)

( modelpeft\_config: PeftConfig = Noneadapter\_name = 'default' )

Parameters

* **model** ([PreTrainedModel](https://huggingface.co/docs/transformers/main/en/main_classes/model#transformers.PreTrainedModel)) — Base transformer model.
* **peft\_config** ([PeftConfig](https://huggingface.co/docs/peft/main/en/package_reference/config#peft.PeftConfig)) — Peft config.

Peft model for extracting features/embeddings from transformer models

**Attributes**:

* **config** ([PretrainedConfig](https://huggingface.co/docs/transformers/main/en/main_classes/configuration#transformers.PretrainedConfig)) — The configuration object of the base model.

Example:

Copied

```
>>> from transformers import AutoModel
>>> from peft import PeftModelForFeatureExtraction, get_peft_config

>>> config = {
...     "peft_type": "LORA",
...     "task_type": "FEATURE_EXTRACTION",
...     "inference_mode": False,
...     "r": 16,
...     "target_modules": ["query", "value"],
...     "lora_alpha": 32,
...     "lora_dropout": 0.05,
...     "fan_in_fan_out": False,
...     "bias": "none",
... }
>>> peft_config = get_peft_config(config)
>>> model = AutoModel.from_pretrained("bert-base-cased")
>>> peft_model = PeftModelForFeatureExtraction(model, peft_config)
>>> peft_model.print_trainable_parameters()
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://boinc-ai.gitbook.io/peft/reference/peft-model.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.