AutoConfig

AutoConfig

class transformers.AutoConfig

<source>

( )

This is a generic configuration class that will be instantiated as one of the configuration classes of the library when created with the from_pretrained() class method.

This class cannot be instantiated directly using __init__() (throws an error).

from_pretrained

<source>

( pretrained_model_name_or_path**kwargs )

Parameters

• pretrained_model_name_or_path (str or os.PathLike) — Can be either:

  • A string, the model id of a pretrained model configuration hosted inside a model repo on huggingface.co. Valid model ids can be located at the root-level, like bert-base-uncased, or namespaced under a user or organization name, like dbmdz/bert-base-german-cased.

  • A path to a directory containing a configuration file saved using the save_pretrained() method, or the save_pretrained() method, e.g., ./my_model_directory/.

  • A path or url to a saved configuration JSON file, e.g., ./my_model_directory/configuration.json.

• cache_dir (str or os.PathLike, optional) — Path to a directory in which a downloaded pretrained model configuration should be cached if the standard cache should not be used.

• force_download (bool, optional, defaults to False) — Whether or not to force the (re-)download the model weights and configuration files and override the cached versions if they exist.

• resume_download (bool, optional, defaults to False) — Whether or not to delete incompletely received files. Will attempt to resume the download if such a file exists.

• proxies (Dict[str, str], optional) — A dictionary of proxy servers to use by protocol or endpoint, e.g., {'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}. The proxies are used on each request.

• revision (str, optional, defaults to "main") — The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so revision can be any identifier allowed by git.

• return_unused_kwargs (bool, optional, defaults to False) — If False, then this function returns just the final configuration object.

  If True, then this functions returns a Tuple(config, unused_kwargs) where unused_kwargs is a dictionary consisting of the key/value pairs whose keys are not configuration attributes: i.e., the part of kwargs which has not been used to update config and is otherwise ignored.

• trust_remote_code (bool, optional, defaults to False) — Whether or not to allow for custom models defined on the Hub in their own modeling files. This option should only be set to True for repositories you trust and in which you have read the code, as it will execute code present on the Hub on your local machine.

• kwargs(additional keyword arguments, optional) — The values in kwargs of any keys which are configuration attributes will be used to override the loaded values. Behavior concerning key/value pairs whose keys are not configuration attributes is controlled by the return_unused_kwargs keyword parameter.

Instantiate one of the configuration classes of the library from a pretrained model configuration.

The configuration class to instantiate is selected based on the model_type property of the config object that is loaded, or when it’s missing, by falling back to using pattern matching on pretrained_model_name_or_path:

• albert — AlbertConfig (ALBERT model)
• align — AlignConfig (ALIGN model)
• altclip — AltCLIPConfig (AltCLIP model)
• audio-spectrogram-transformer — ASTConfig (Audio Spectrogram Transformer model)
• autoformer — AutoformerConfig (Autoformer model)
• bark — BarkConfig (Bark model)
• bart — BartConfig (BART model)
• beit — BeitConfig (BEiT model)
• bert — BertConfig (BERT model)
• bert-generation — BertGenerationConfig (Bert Generation model)
• big_bird — BigBirdConfig (BigBird model)
• bigbird_pegasus — BigBirdPegasusConfig (BigBird-Pegasus model)
• biogpt — BioGptConfig (BioGpt model)
• bit — BitConfig (BiT model)
• blenderbot — BlenderbotConfig (Blenderbot model)
• blenderbot-small — BlenderbotSmallConfig (BlenderbotSmall model)
• blip — BlipConfig (BLIP model)
• blip-2 — Blip2Config (BLIP-2 model)
• bloom — BloomConfig (BLOOM model)
• bridgetower — BridgeTowerConfig (BridgeTower model)
• bros — BrosConfig (BROS model)
• camembert — CamembertConfig (CamemBERT model)
• canine — CanineConfig (CANINE model)
• chinese_clip — ChineseCLIPConfig (Chinese-CLIP model)
• clap — ClapConfig (CLAP model)
• clip — CLIPConfig (CLIP model)
• clipseg — CLIPSegConfig (CLIPSeg model)
• code_llama — LlamaConfig (CodeLlama model)
• codegen — CodeGenConfig (CodeGen model)
• conditional_detr — ConditionalDetrConfig (Conditional DETR model)
• convbert — ConvBertConfig (ConvBERT model)
• convnext — ConvNextConfig (ConvNeXT model)
• convnextv2 — ConvNextV2Config (ConvNeXTV2 model)
• cpmant — CpmAntConfig (CPM-Ant model)
• ctrl — CTRLConfig (CTRL model)
• cvt — CvtConfig (CvT model)
• data2vec-audio — Data2VecAudioConfig (Data2VecAudio model)
• data2vec-text — Data2VecTextConfig (Data2VecText model)
• data2vec-vision — Data2VecVisionConfig (Data2VecVision model)
• deberta — DebertaConfig (DeBERTa model)
• deberta-v2 — DebertaV2Config (DeBERTa-v2 model)
• decision_transformer — DecisionTransformerConfig (Decision Transformer model)
• deformable_detr — DeformableDetrConfig (Deformable DETR model)
• deit — DeiTConfig (DeiT model)
• deta — DetaConfig (DETA model)
• detr — DetrConfig (DETR model)
• dinat — DinatConfig (DiNAT model)
• dinov2 — Dinov2Config (DINOv2 model)
• distilbert — DistilBertConfig (DistilBERT model)
• donut-swin — DonutSwinConfig (DonutSwin model)
• dpr — DPRConfig (DPR model)
• dpt — DPTConfig (DPT model)
• efficientformer — EfficientFormerConfig (EfficientFormer model)
• efficientnet — EfficientNetConfig (EfficientNet model)
• electra — ElectraConfig (ELECTRA model)
• encodec — EncodecConfig (EnCodec model)
• encoder-decoder — EncoderDecoderConfig (Encoder decoder model)
• ernie — ErnieConfig (ERNIE model)
• ernie_m — ErnieMConfig (ErnieM model)
• esm — EsmConfig (ESM model)
• falcon — FalconConfig (Falcon model)
• flaubert — FlaubertConfig (FlauBERT model)
• flava — FlavaConfig (FLAVA model)
• fnet — FNetConfig (FNet model)
• focalnet — FocalNetConfig (FocalNet model)
• fsmt — FSMTConfig (FairSeq Machine-Translation model)
• funnel — FunnelConfig (Funnel Transformer model)
• git — GitConfig (GIT model)
• glpn — GLPNConfig (GLPN model)
• gpt-sw3 — GPT2Config (GPT-Sw3 model)
• gpt2 — GPT2Config (OpenAI GPT-2 model)
• gpt_bigcode — GPTBigCodeConfig (GPTBigCode model)
• gpt_neo — GPTNeoConfig (GPT Neo model)
• gpt_neox — GPTNeoXConfig (GPT NeoX model)
• gpt_neox_japanese — GPTNeoXJapaneseConfig (GPT NeoX Japanese model)
• gptj — GPTJConfig (GPT-J model)
• gptsan-japanese — GPTSanJapaneseConfig (GPTSAN-japanese model)
• graphormer — GraphormerConfig (Graphormer model)
• groupvit — GroupViTConfig (GroupViT model)
• hubert — HubertConfig (Hubert model)
• ibert — IBertConfig (I-BERT model)
• idefics — IdeficsConfig (IDEFICS model)
• imagegpt — ImageGPTConfig (ImageGPT model)
• informer — InformerConfig (Informer model)
• instructblip — InstructBlipConfig (InstructBLIP model)
• jukebox — JukeboxConfig (Jukebox model)
• layoutlm — LayoutLMConfig (LayoutLM model)
• layoutlmv2 — LayoutLMv2Config (LayoutLMv2 model)
• layoutlmv3 — LayoutLMv3Config (LayoutLMv3 model)
• led — LEDConfig (LED model)
• levit — LevitConfig (LeViT model)
• lilt — LiltConfig (LiLT model)
• llama — LlamaConfig (LLaMA model)
• longformer — LongformerConfig (Longformer model)
• longt5 — LongT5Config (LongT5 model)
• luke — LukeConfig (LUKE model)
• lxmert — LxmertConfig (LXMERT model)
• m2m_100 — M2M100Config (M2M100 model)
• marian — MarianConfig (Marian model)
• markuplm — MarkupLMConfig (MarkupLM model)
• mask2former — Mask2FormerConfig (Mask2Former model)
• maskformer — MaskFormerConfig (MaskFormer model)
• maskformer-swin — MaskFormerSwinConfig (MaskFormerSwin model)

• mbart — MBartConfig (mBART model)
• mctct — MCTCTConfig (M-CTC-T model)
• mega — MegaConfig (MEGA model)
• megatron-bert — MegatronBertConfig (Megatron-BERT model)
• mgp-str — MgpstrConfig (MGP-STR model)
• mistral — MistralConfig (Mistral model)
• mobilebert — MobileBertConfig (MobileBERT model)
• mobilenet_v1 — MobileNetV1Config (MobileNetV1 model)
• mobilenet_v2 — MobileNetV2Config (MobileNetV2 model)
• mobilevit — MobileViTConfig (MobileViT model)
• mobilevitv2 — MobileViTV2Config (MobileViTV2 model)
• mpnet — MPNetConfig (MPNet model)
• mpt — MptConfig (MPT model)
• mra — MraConfig (MRA model)
• mt5 — MT5Config (MT5 model)
• musicgen — MusicgenConfig (MusicGen model)
• mvp — MvpConfig (MVP model)
• nat — NatConfig (NAT model)
• nezha — NezhaConfig (Nezha model)
• nllb-moe — NllbMoeConfig (NLLB-MOE model)
• nougat — VisionEncoderDecoderConfig (Nougat model)
• nystromformer — NystromformerConfig (Nyströmformer model)
• oneformer — OneFormerConfig (OneFormer model)
• open-llama — OpenLlamaConfig (OpenLlama model)
• openai-gpt — OpenAIGPTConfig (OpenAI GPT model)
• opt — OPTConfig (OPT model)
• owlvit — OwlViTConfig (OWL-ViT model)
• pegasus — PegasusConfig (Pegasus model)
• pegasus_x — PegasusXConfig (PEGASUS-X model)
• perceiver — PerceiverConfig (Perceiver model)
• persimmon — PersimmonConfig (Persimmon model)
• pix2struct — Pix2StructConfig (Pix2Struct model)
• plbart — PLBartConfig (PLBart model)
• poolformer — PoolFormerConfig (PoolFormer model)
• pop2piano — Pop2PianoConfig (Pop2Piano model)
• prophetnet — ProphetNetConfig (ProphetNet model)
• pvt — PvtConfig (PVT model)
• qdqbert — QDQBertConfig (QDQBert model)
• rag — RagConfig (RAG model)
• realm — RealmConfig (REALM model)
• reformer — ReformerConfig (Reformer model)
• regnet — RegNetConfig (RegNet model)
• rembert — RemBertConfig (RemBERT model)
• resnet — ResNetConfig (ResNet model)
• retribert — RetriBertConfig (RetriBERT model)
• roberta — RobertaConfig (RoBERTa model)
• roberta-prelayernorm — RobertaPreLayerNormConfig (RoBERTa-PreLayerNorm model)
• roc_bert — RoCBertConfig (RoCBert model)
• roformer — RoFormerConfig (RoFormer model)
• rwkv — RwkvConfig (RWKV model)
• sam — SamConfig (SAM model)
• segformer — SegformerConfig (SegFormer model)
• sew — SEWConfig (SEW model)
• sew-d — SEWDConfig (SEW-D model)
• speech-encoder-decoder — SpeechEncoderDecoderConfig (Speech Encoder decoder model)
• speech_to_text — Speech2TextConfig (Speech2Text model)
• speech_to_text_2 — Speech2Text2Config (Speech2Text2 model)
• speecht5 — SpeechT5Config (SpeechT5 model)
• splinter — SplinterConfig (Splinter model)
• squeezebert — SqueezeBertConfig (SqueezeBERT model)
• swiftformer — SwiftFormerConfig (SwiftFormer model)
• swin — SwinConfig (Swin Transformer model)
• swin2sr — Swin2SRConfig (Swin2SR model)
• swinv2 — Swinv2Config (Swin Transformer V2 model)
• switch_transformers — SwitchTransformersConfig (SwitchTransformers model)
• t5 — T5Config (T5 model)
• table-transformer — TableTransformerConfig (Table Transformer model)
• tapas — TapasConfig (TAPAS model)
• time_series_transformer — TimeSeriesTransformerConfig (Time Series Transformer model)
• timesformer — TimesformerConfig (TimeSformer model)
• timm_backbone — TimmBackboneConfig (TimmBackbone model)

• trajectory_transformer — TrajectoryTransformerConfig (Trajectory Transformer model)
• transfo-xl — TransfoXLConfig (Transformer-XL model)
• trocr — TrOCRConfig (TrOCR model)
• tvlt — TvltConfig (TVLT model)
• umt5 — UMT5Config (UMT5 model)
• unispeech — UniSpeechConfig (UniSpeech model)
• unispeech-sat — UniSpeechSatConfig (UniSpeechSat model)
• upernet — UperNetConfig (UPerNet model)
• van — VanConfig (VAN model)
• videomae — VideoMAEConfig (VideoMAE model)
• vilt — ViltConfig (ViLT model)
• vision-encoder-decoder — VisionEncoderDecoderConfig (Vision Encoder decoder model)
• vision-text-dual-encoder — VisionTextDualEncoderConfig (VisionTextDualEncoder model)
• visual_bert — VisualBertConfig (VisualBERT model)
• vit — ViTConfig (ViT model)
• vit_hybrid — ViTHybridConfig (ViT Hybrid model)
• vit_mae — ViTMAEConfig (ViTMAE model)
• vit_msn — ViTMSNConfig (ViTMSN model)
• vitdet — VitDetConfig (VitDet model)
• vitmatte — VitMatteConfig (ViTMatte model)
• vits — VitsConfig (VITS model)
• vivit — VivitConfig (ViViT model)
• wav2vec2 — Wav2Vec2Config (Wav2Vec2 model)
• wav2vec2-conformer — Wav2Vec2ConformerConfig (Wav2Vec2-Conformer model)
• wavlm — WavLMConfig (WavLM model)
• whisper — WhisperConfig (Whisper model)
• xclip — XCLIPConfig (X-CLIP model)
• xglm — XGLMConfig (XGLM model)
• xlm — XLMConfig (XLM model)
• xlm-prophetnet — XLMProphetNetConfig (XLM-ProphetNet model)
• xlm-roberta — XLMRobertaConfig (XLM-RoBERTa model)
• xlm-roberta-xl — XLMRobertaXLConfig (XLM-RoBERTa-XL model)
• xlnet — XLNetConfig (XLNet model)
• xmod — XmodConfig (X-MOD model)
• yolos — YolosConfig (YOLOS model)
• yoso — YosoConfig (YOSO model)

Examples:

Copied

>>> from transformers import AutoConfig

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-uncased")

>>> # Download configuration from huggingface.co (user-uploaded) and cache.
>>> config = AutoConfig.from_pretrained("dbmdz/bert-base-german-cased")

>>> # If configuration file is in a directory (e.g., was saved using *save_pretrained('./test/saved_model/')*).
>>> config = AutoConfig.from_pretrained("./test/bert_saved_model/")

>>> # Load a specific configuration file.
>>> config = AutoConfig.from_pretrained("./test/bert_saved_model/my_configuration.json")

>>> # Change some config attributes when loading a pretrained config.
>>> config = AutoConfig.from_pretrained("bert-base-uncased", output_attentions=True, foo=False)
>>> config.output_attentions
True

>>> config, unused_kwargs = AutoConfig.from_pretrained(
...     "bert-base-uncased", output_attentions=True, foo=False, return_unused_kwargs=True
... )
>>> config.output_attentions
True

>>> unused_kwargs
{'foo': False}

register

<source>

( model_typeconfigexist_ok = False )

Parameters

• model_type (str) — The model type like “bert” or “gpt”.

• config (PretrainedConfig) — The config to register.

Register a new configuration for this class.