TFAutoModel

class transformers.TFAutoModel

<source>

( *args**kwargs )

This is a generic model class that will be instantiated as one of the base model classes of the library when created with the from_pretrained() class method or the from_config() class method.

This class cannot be instantiated directly using __init__() (throws an error).

from_config

<source>

( **kwargs )

Parameters

• config (PretrainedConfig) — The model class to instantiate is selected based on the configuration class:

  • AlbertConfig configuration class: TFAlbertModel (ALBERT model)

  • BartConfig configuration class: TFBartModel (BART model)

  • BertConfig configuration class: TFBertModel (BERT model)

  • BlenderbotConfig configuration class: TFBlenderbotModel (Blenderbot model)

  • BlenderbotSmallConfig configuration class: TFBlenderbotSmallModel (BlenderbotSmall model)

  • BlipConfig configuration class: TFBlipModel (BLIP model)

  • CLIPConfig configuration class: TFCLIPModel (CLIP model)

  • CTRLConfig configuration class: TFCTRLModel (CTRL model)

  • CamembertConfig configuration class: TFCamembertModel (CamemBERT model)

  • ConvBertConfig configuration class: TFConvBertModel (ConvBERT model)

  • ConvNextConfig configuration class: TFConvNextModel (ConvNeXT model)

  • CvtConfig configuration class: TFCvtModel (CvT model)

  • DPRConfig configuration class: TFDPRQuestionEncoder (DPR model)

  • Data2VecVisionConfig configuration class: TFData2VecVisionModel (Data2VecVision model)

  • DebertaConfig configuration class: TFDebertaModel (DeBERTa model)

  • DebertaV2Config configuration class: TFDebertaV2Model (DeBERTa-v2 model)

  • DeiTConfig configuration class: TFDeiTModel (DeiT model)

  • DistilBertConfig configuration class: TFDistilBertModel (DistilBERT model)

  • EfficientFormerConfig configuration class: TFEfficientFormerModel (EfficientFormer model)

  • ElectraConfig configuration class: TFElectraModel (ELECTRA model)

  • EsmConfig configuration class: TFEsmModel (ESM model)

  • FlaubertConfig configuration class: TFFlaubertModel (FlauBERT model)

  • FunnelConfig configuration class: TFFunnelModel or TFFunnelBaseModel (Funnel Transformer model)

  • GPT2Config configuration class: TFGPT2Model (OpenAI GPT-2 model)

  • GPTJConfig configuration class: TFGPTJModel (GPT-J model)

  • GroupViTConfig configuration class: TFGroupViTModel (GroupViT model)

  • HubertConfig configuration class: TFHubertModel (Hubert model)

  • LEDConfig configuration class: TFLEDModel (LED model)

  • LayoutLMConfig configuration class: TFLayoutLMModel (LayoutLM model)

  • LayoutLMv3Config configuration class: TFLayoutLMv3Model (LayoutLMv3 model)

  • LongformerConfig configuration class: TFLongformerModel (Longformer model)

  • LxmertConfig configuration class: TFLxmertModel (LXMERT model)

  • MBartConfig configuration class: TFMBartModel (mBART model)

  • MPNetConfig configuration class: TFMPNetModel (MPNet model)

  • MT5Config configuration class: TFMT5Model (MT5 model)

  • MarianConfig configuration class: TFMarianModel (Marian model)

  • MobileBertConfig configuration class: TFMobileBertModel (MobileBERT model)

  • MobileViTConfig configuration class: TFMobileViTModel (MobileViT model)

  • OPTConfig configuration class: TFOPTModel (OPT model)

  • OpenAIGPTConfig configuration class: TFOpenAIGPTModel (OpenAI GPT model)

  • PegasusConfig configuration class: TFPegasusModel (Pegasus model)

  • RegNetConfig configuration class: TFRegNetModel (RegNet model)

  • RemBertConfig configuration class: TFRemBertModel (RemBERT model)

  • ResNetConfig configuration class: TFResNetModel (ResNet model)

  • RoFormerConfig configuration class: TFRoFormerModel (RoFormer model)

  • RobertaConfig configuration class: TFRobertaModel (RoBERTa model)

  • RobertaPreLayerNormConfig configuration class: TFRobertaPreLayerNormModel (RoBERTa-PreLayerNorm model)

  • SamConfig configuration class: TFSamModel (SAM model)

  • SegformerConfig configuration class: TFSegformerModel (SegFormer model)

  • Speech2TextConfig configuration class: TFSpeech2TextModel (Speech2Text model)

  • SwinConfig configuration class: TFSwinModel (Swin Transformer model)

  • T5Config configuration class: TFT5Model (T5 model)

  • TapasConfig configuration class: TFTapasModel (TAPAS model)

  • TransfoXLConfig configuration class: TFTransfoXLModel (Transformer-XL model)

  • ViTConfig configuration class: TFViTModel (ViT model)

  • ViTMAEConfig configuration class: TFViTMAEModel (ViTMAE model)

  • VisionTextDualEncoderConfig configuration class: TFVisionTextDualEncoderModel (VisionTextDualEncoder model)

  • Wav2Vec2Config configuration class: TFWav2Vec2Model (Wav2Vec2 model)

  • WhisperConfig configuration class: TFWhisperModel (Whisper model)

  • XGLMConfig configuration class: TFXGLMModel (XGLM model)

  • XLMConfig configuration class: TFXLMModel (XLM model)

  • XLMRobertaConfig configuration class: TFXLMRobertaModel (XLM-RoBERTa model)

  • XLNetConfig configuration class: TFXLNetModel (XLNet model)

Instantiates one of the base model classes of the library from a configuration.

Note: Loading a model from its configuration file does not load the model weights. It only affects the model’s configuration. Use from_pretrained() to load the model weights.

Examples:

Copied

>>> from transformers import AutoConfig, TFAutoModel

>>> # Download configuration from huggingface.co and cache.
>>> config = AutoConfig.from_pretrained("bert-base-cased")
>>> model = TFAutoModel.from_config(config)

from_pretrained

<source>

( *model_args**kwargs )

Parameters

• pretrained_model_name_or_path (str or os.PathLike) — Can be either:

  • A string, the model id of a pretrained model hosted inside a model repo on huggingface.co. Valid model ids can be located at the root-level, like bert-base-uncased, or namespaced under a user or organization name, like dbmdz/bert-base-german-cased.

  • A path to a directory containing model weights saved using save_pretrained(), e.g., ./my_model_directory/.

  • A path or url to a PyTorch state_dict save file (e.g, ./pt_model/pytorch_model.bin). In this case, from_pt should be set to True and a configuration object should be provided as config argument. This loading path is slower than converting the PyTorch model in a TensorFlow model using the provided conversion scripts and loading the TensorFlow model afterwards.

• model_args (additional positional arguments, optional) — Will be passed along to the underlying model __init__() method.

• config (PretrainedConfig, optional) — Configuration for the model to use instead of an automatically loaded configuration. Configuration can be automatically loaded when:

  • The model is a model provided by the library (loaded with the model id string of a pretrained model).

  • The model was saved using save_pretrained() and is reloaded by supplying the save directory.

  • The model is loaded by supplying a local directory as pretrained_model_name_or_path and a configuration JSON file named config.json is found in the directory.

• cache_dir (str or os.PathLike, optional) — Path to a directory in which a downloaded pretrained model configuration should be cached if the standard cache should not be used.

• from_pt (bool, optional, defaults to False) — Load the model weights from a PyTorch checkpoint save file (see docstring of pretrained_model_name_or_path argument).

• force_download (bool, optional, defaults to False) — Whether or not to force the (re-)download of the model weights and configuration files, overriding the cached versions if they exist.

• resume_download (bool, optional, defaults to False) — Whether or not to delete incompletely received files. Will attempt to resume the download if such a file exists.

• proxies (Dict[str, str], optional) — A dictionary of proxy servers to use by protocol or endpoint, e.g., {'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}. The proxies are used on each request.

• output_loading_info(bool, optional, defaults to False) — Whether ot not to also return a dictionary containing missing keys, unexpected keys and error messages.

• local_files_only(bool, optional, defaults to False) — Whether or not to only look at local files (e.g., not try downloading the model).

• revision (str, optional, defaults to "main") — The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so revision can be any identifier allowed by git.

• trust_remote_code (bool, optional, defaults to False) — Whether or not to allow for custom models defined on the Hub in their own modeling files. This option should only be set to True for repositories you trust and in which you have read the code, as it will execute code present on the Hub on your local machine.

• code_revision (str, optional, defaults to "main") — The specific revision to use for the code on the Hub, if the code leaves in a different repository than the rest of the model. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so revision can be any identifier allowed by git.

• kwargs (additional keyword arguments, optional) — Can be used to update the configuration object (after it being loaded) and initiate the model (e.g., output_attentions=True). Behaves differently depending on whether a config is provided or automatically loaded:

  • If a configuration is provided with config, **kwargs will be directly passed to the underlying model’s __init__ method (we assume all relevant updates to the configuration have already been done)

  • If a configuration is not provided, kwargs will be first passed to the configuration class initialization function (from_pretrained()). Each key of kwargs that corresponds to a configuration attribute will be used to override said attribute with the supplied kwargs value. Remaining keys that do not correspond to any configuration attribute will be passed to the underlying model’s __init__ function.

Instantiate one of the base model classes of the library from a pretrained model.

The model class to instantiate is selected based on the model_type property of the config object (either passed as an argument or loaded from pretrained_model_name_or_path if possible), or when it’s missing, by falling back to using pattern matching on pretrained_model_name_or_path:

• albert — TFAlbertModel (ALBERT model)

• bart — TFBartModel (BART model)

• bert — TFBertModel (BERT model)

• blenderbot — TFBlenderbotModel (Blenderbot model)

• blenderbot-small — TFBlenderbotSmallModel (BlenderbotSmall model)

• blip — TFBlipModel (BLIP model)

• camembert — TFCamembertModel (CamemBERT model)

• clip — TFCLIPModel (CLIP model)

• convbert — TFConvBertModel (ConvBERT model)

• convnext — TFConvNextModel (ConvNeXT model)

• ctrl — TFCTRLModel (CTRL model)

• cvt — TFCvtModel (CvT model)

• data2vec-vision — TFData2VecVisionModel (Data2VecVision model)

• deberta — TFDebertaModel (DeBERTa model)

• deberta-v2 — TFDebertaV2Model (DeBERTa-v2 model)

• deit — TFDeiTModel (DeiT model)

• distilbert — TFDistilBertModel (DistilBERT model)

• dpr — TFDPRQuestionEncoder (DPR model)

• efficientformer — TFEfficientFormerModel (EfficientFormer model)

• electra — TFElectraModel (ELECTRA model)

• esm — TFEsmModel (ESM model)

• flaubert — TFFlaubertModel (FlauBERT model)

• funnel — TFFunnelModel or TFFunnelBaseModel (Funnel Transformer model)

• gpt-sw3 — TFGPT2Model (GPT-Sw3 model)

• gpt2 — TFGPT2Model (OpenAI GPT-2 model)

• gptj — TFGPTJModel (GPT-J model)

• groupvit — TFGroupViTModel (GroupViT model)

• hubert — TFHubertModel (Hubert model)

• layoutlm — TFLayoutLMModel (LayoutLM model)

• layoutlmv3 — TFLayoutLMv3Model (LayoutLMv3 model)

• led — TFLEDModel (LED model)

• longformer — TFLongformerModel (Longformer model)

• lxmert — TFLxmertModel (LXMERT model)

• marian — TFMarianModel (Marian model)

• mbart — TFMBartModel (mBART model)

• mobilebert — TFMobileBertModel (MobileBERT model)

• mobilevit — TFMobileViTModel (MobileViT model)

• mpnet — TFMPNetModel (MPNet model)

• mt5 — TFMT5Model (MT5 model)

• openai-gpt — TFOpenAIGPTModel (OpenAI GPT model)

• opt — TFOPTModel (OPT model)

• pegasus — TFPegasusModel (Pegasus model)

• regnet — TFRegNetModel (RegNet model)

• rembert — TFRemBertModel (RemBERT model)

• resnet — TFResNetModel (ResNet model)

• roberta — TFRobertaModel (RoBERTa model)

• roberta-prelayernorm — TFRobertaPreLayerNormModel (RoBERTa-PreLayerNorm model)

• roformer — TFRoFormerModel (RoFormer model)

• sam — TFSamModel (SAM model)

• segformer — TFSegformerModel (SegFormer model)

• speech_to_text — TFSpeech2TextModel (Speech2Text model)

• swin — TFSwinModel (Swin Transformer model)

• t5 — TFT5Model (T5 model)

• tapas — TFTapasModel (TAPAS model)

• transfo-xl — TFTransfoXLModel (Transformer-XL model)

• vision-text-dual-encoder — TFVisionTextDualEncoderModel (VisionTextDualEncoder model)

• vit — TFViTModel (ViT model)

• vit_mae — TFViTMAEModel (ViTMAE model)

• wav2vec2 — TFWav2Vec2Model (Wav2Vec2 model)

• whisper — TFWhisperModel (Whisper model)

• xglm — TFXGLMModel (XGLM model)

• xlm — TFXLMModel (XLM model)

• xlm-roberta — TFXLMRobertaModel (XLM-RoBERTa model)

• xlnet — TFXLNetModel (XLNet model)

Examples:

Copied

>>> from transformers import AutoConfig, TFAutoModel

>>> # Download model and configuration from huggingface.co and cache.
>>> model = TFAutoModel.from_pretrained("bert-base-cased")

>>> # Update configuration during loading
>>> model = TFAutoModel.from_pretrained("bert-base-cased", output_attentions=True)
>>> model.config.output_attentions
True

>>> # Loading from a PyTorch checkpoint file instead of a TensorFlow model (slower)
>>> config = AutoConfig.from_pretrained("./pt_model/bert_pt_model_config.json")
>>> model = TFAutoModel.from_pretrained(
...     "./pt_model/bert_pytorch_model.bin", from_pt=True, config=config
... )