Transformers
  • 🌍GET STARTED
    • Transformers
    • Quick tour
    • Installation
  • 🌍TUTORIALS
    • Run inference with pipelines
    • Write portable code with AutoClass
    • Preprocess data
    • Fine-tune a pretrained model
    • Train with a script
    • Set up distributed training with BOINC AI Accelerate
    • Load and train adapters with BOINC AI PEFT
    • Share your model
    • Agents
    • Generation with LLMs
  • 🌍TASK GUIDES
    • 🌍NATURAL LANGUAGE PROCESSING
      • Text classification
      • Token classification
      • Question answering
      • Causal language modeling
      • Masked language modeling
      • Translation
      • Summarization
      • Multiple choice
    • 🌍AUDIO
      • Audio classification
      • Automatic speech recognition
    • 🌍COMPUTER VISION
      • Image classification
      • Semantic segmentation
      • Video classification
      • Object detection
      • Zero-shot object detection
      • Zero-shot image classification
      • Depth estimation
    • 🌍MULTIMODAL
      • Image captioning
      • Document Question Answering
      • Visual Question Answering
      • Text to speech
    • 🌍GENERATION
      • Customize the generation strategy
    • 🌍PROMPTING
      • Image tasks with IDEFICS
  • 🌍DEVELOPER GUIDES
    • Use fast tokenizers from BOINC AI Tokenizers
    • Run inference with multilingual models
    • Use model-specific APIs
    • Share a custom model
    • Templates for chat models
    • Run training on Amazon SageMaker
    • Export to ONNX
    • Export to TFLite
    • Export to TorchScript
    • Benchmarks
    • Notebooks with examples
    • Community resources
    • Custom Tools and Prompts
    • Troubleshoot
  • 🌍PERFORMANCE AND SCALABILITY
    • Overview
    • 🌍EFFICIENT TRAINING TECHNIQUES
      • Methods and tools for efficient training on a single GPU
      • Multiple GPUs and parallelism
      • Efficient training on CPU
      • Distributed CPU training
      • Training on TPUs
      • Training on TPU with TensorFlow
      • Training on Specialized Hardware
      • Custom hardware for training
      • Hyperparameter Search using Trainer API
    • 🌍OPTIMIZING INFERENCE
      • Inference on CPU
      • Inference on one GPU
      • Inference on many GPUs
      • Inference on Specialized Hardware
    • Instantiating a big model
    • Troubleshooting
    • XLA Integration for TensorFlow Models
    • Optimize inference using `torch.compile()`
  • 🌍CONTRIBUTE
    • How to contribute to transformers?
    • How to add a model to BOINC AI Transformers?
    • How to convert a BOINC AI Transformers model to TensorFlow?
    • How to add a pipeline to BOINC AI Transformers?
    • Testing
    • Checks on a Pull Request
  • 🌍CONCEPTUAL GUIDES
    • Philosophy
    • Glossary
    • What BOINC AI Transformers can do
    • How BOINC AI Transformers solve tasks
    • The Transformer model family
    • Summary of the tokenizers
    • Attention mechanisms
    • Padding and truncation
    • BERTology
    • Perplexity of fixed-length models
    • Pipelines for webserver inference
    • Model training anatomy
  • 🌍API
    • 🌍MAIN CLASSES
      • Agents and Tools
      • 🌍Auto Classes
        • Extending the Auto Classes
        • AutoConfig
        • AutoTokenizer
        • AutoFeatureExtractor
        • AutoImageProcessor
        • AutoProcessor
        • Generic model classes
          • AutoModel
          • TFAutoModel
          • FlaxAutoModel
        • Generic pretraining classes
          • AutoModelForPreTraining
          • TFAutoModelForPreTraining
          • FlaxAutoModelForPreTraining
        • Natural Language Processing
          • AutoModelForCausalLM
          • TFAutoModelForCausalLM
          • FlaxAutoModelForCausalLM
          • AutoModelForMaskedLM
          • TFAutoModelForMaskedLM
          • FlaxAutoModelForMaskedLM
          • AutoModelForMaskGenerationge
          • TFAutoModelForMaskGeneration
          • AutoModelForSeq2SeqLM
          • TFAutoModelForSeq2SeqLM
          • FlaxAutoModelForSeq2SeqLM
          • AutoModelForSequenceClassification
          • TFAutoModelForSequenceClassification
          • FlaxAutoModelForSequenceClassification
          • AutoModelForMultipleChoice
          • TFAutoModelForMultipleChoice
          • FlaxAutoModelForMultipleChoice
          • AutoModelForNextSentencePrediction
          • TFAutoModelForNextSentencePrediction
          • FlaxAutoModelForNextSentencePrediction
          • AutoModelForTokenClassification
          • TFAutoModelForTokenClassification
          • FlaxAutoModelForTokenClassification
          • AutoModelForQuestionAnswering
          • TFAutoModelForQuestionAnswering
          • FlaxAutoModelForQuestionAnswering
          • AutoModelForTextEncoding
          • TFAutoModelForTextEncoding
        • Computer vision
          • AutoModelForDepthEstimation
          • AutoModelForImageClassification
          • TFAutoModelForImageClassification
          • FlaxAutoModelForImageClassification
          • AutoModelForVideoClassification
          • AutoModelForMaskedImageModeling
          • TFAutoModelForMaskedImageModeling
          • AutoModelForObjectDetection
          • AutoModelForImageSegmentation
          • AutoModelForImageToImage
          • AutoModelForSemanticSegmentation
          • TFAutoModelForSemanticSegmentation
          • AutoModelForInstanceSegmentation
          • AutoModelForUniversalSegmentation
          • AutoModelForZeroShotImageClassification
          • TFAutoModelForZeroShotImageClassification
          • AutoModelForZeroShotObjectDetection
        • Audio
          • AutoModelForAudioClassification
          • AutoModelForAudioFrameClassification
          • TFAutoModelForAudioFrameClassification
          • AutoModelForCTC
          • AutoModelForSpeechSeq2Seq
          • TFAutoModelForSpeechSeq2Seq
          • FlaxAutoModelForSpeechSeq2Seq
          • AutoModelForAudioXVector
          • AutoModelForTextToSpectrogram
          • AutoModelForTextToWaveform
        • Multimodal
          • AutoModelForTableQuestionAnswering
          • TFAutoModelForTableQuestionAnswering
          • AutoModelForDocumentQuestionAnswering
          • TFAutoModelForDocumentQuestionAnswering
          • AutoModelForVisualQuestionAnswering
          • AutoModelForVision2Seq
          • TFAutoModelForVision2Seq
          • FlaxAutoModelForVision2Seq
      • Callbacks
      • Configuration
      • Data Collator
      • Keras callbacks
      • Logging
      • Models
      • Text Generation
      • ONNX
      • Optimization
      • Model outputs
      • Pipelines
      • Processors
      • Quantization
      • Tokenizer
      • Trainer
      • DeepSpeed Integration
      • Feature Extractor
      • Image Processor
    • 🌍MODELS
      • 🌍TEXT MODELS
        • ALBERT
        • BART
        • BARThez
        • BARTpho
        • BERT
        • BertGeneration
        • BertJapanese
        • Bertweet
        • BigBird
        • BigBirdPegasus
        • BioGpt
        • Blenderbot
        • Blenderbot Small
        • BLOOM
        • BORT
        • ByT5
        • CamemBERT
        • CANINE
        • CodeGen
        • CodeLlama
        • ConvBERT
        • CPM
        • CPMANT
        • CTRL
        • DeBERTa
        • DeBERTa-v2
        • DialoGPT
        • DistilBERT
        • DPR
        • ELECTRA
        • Encoder Decoder Models
        • ERNIE
        • ErnieM
        • ESM
        • Falcon
        • FLAN-T5
        • FLAN-UL2
        • FlauBERT
        • FNet
        • FSMT
        • Funnel Transformer
        • GPT
        • GPT Neo
        • GPT NeoX
        • GPT NeoX Japanese
        • GPT-J
        • GPT2
        • GPTBigCode
        • GPTSAN Japanese
        • GPTSw3
        • HerBERT
        • I-BERT
        • Jukebox
        • LED
        • LLaMA
        • LLama2
        • Longformer
        • LongT5
        • LUKE
        • M2M100
        • MarianMT
        • MarkupLM
        • MBart and MBart-50
        • MEGA
        • MegatronBERT
        • MegatronGPT2
        • Mistral
        • mLUKE
        • MobileBERT
        • MPNet
        • MPT
        • MRA
        • MT5
        • MVP
        • NEZHA
        • NLLB
        • NLLB-MoE
        • Nyströmformer
        • Open-Llama
        • OPT
        • Pegasus
        • PEGASUS-X
        • Persimmon
        • PhoBERT
        • PLBart
        • ProphetNet
        • QDQBert
        • RAG
        • REALM
        • Reformer
        • RemBERT
        • RetriBERT
        • RoBERTa
        • RoBERTa-PreLayerNorm
        • RoCBert
        • RoFormer
        • RWKV
        • Splinter
        • SqueezeBERT
        • SwitchTransformers
        • T5
        • T5v1.1
        • TAPEX
        • Transformer XL
        • UL2
        • UMT5
        • X-MOD
        • XGLM
        • XLM
        • XLM-ProphetNet
        • XLM-RoBERTa
        • XLM-RoBERTa-XL
        • XLM-V
        • XLNet
        • YOSO
      • 🌍VISION MODELS
        • BEiT
        • BiT
        • Conditional DETR
        • ConvNeXT
        • ConvNeXTV2
        • CvT
        • Deformable DETR
        • DeiT
        • DETA
        • DETR
        • DiNAT
        • DINO V2
        • DiT
        • DPT
        • EfficientFormer
        • EfficientNet
        • FocalNet
        • GLPN
        • ImageGPT
        • LeViT
        • Mask2Former
        • MaskFormer
        • MobileNetV1
        • MobileNetV2
        • MobileViT
        • MobileViTV2
        • NAT
        • PoolFormer
        • Pyramid Vision Transformer (PVT)
        • RegNet
        • ResNet
        • SegFormer
        • SwiftFormer
        • Swin Transformer
        • Swin Transformer V2
        • Swin2SR
        • Table Transformer
        • TimeSformer
        • UperNet
        • VAN
        • VideoMAE
        • Vision Transformer (ViT)
        • ViT Hybrid
        • ViTDet
        • ViTMAE
        • ViTMatte
        • ViTMSN
        • ViViT
        • YOLOS
      • 🌍AUDIO MODELS
        • Audio Spectrogram Transformer
        • Bark
        • CLAP
        • EnCodec
        • Hubert
        • MCTCT
        • MMS
        • MusicGen
        • Pop2Piano
        • SEW
        • SEW-D
        • Speech2Text
        • Speech2Text2
        • SpeechT5
        • UniSpeech
        • UniSpeech-SAT
        • VITS
        • Wav2Vec2
        • Wav2Vec2-Conformer
        • Wav2Vec2Phoneme
        • WavLM
        • Whisper
        • XLS-R
        • XLSR-Wav2Vec2
      • 🌍MULTIMODAL MODELS
        • ALIGN
        • AltCLIP
        • BLIP
        • BLIP-2
        • BridgeTower
        • BROS
        • Chinese-CLIP
        • CLIP
        • CLIPSeg
        • Data2Vec
        • DePlot
        • Donut
        • FLAVA
        • GIT
        • GroupViT
        • IDEFICS
        • InstructBLIP
        • LayoutLM
        • LayoutLMV2
        • LayoutLMV3
        • LayoutXLM
        • LiLT
        • LXMERT
        • MatCha
        • MGP-STR
        • Nougat
        • OneFormer
        • OWL-ViT
        • Perceiver
        • Pix2Struct
        • Segment Anything
        • Speech Encoder Decoder Models
        • TAPAS
        • TrOCR
        • TVLT
        • ViLT
        • Vision Encoder Decoder Models
        • Vision Text Dual Encoder
        • VisualBERT
        • X-CLIP
      • 🌍REINFORCEMENT LEARNING MODELS
        • Decision Transformer
        • Trajectory Transformer
      • 🌍TIME SERIES MODELS
        • Autoformer
        • Informer
        • Time Series Transformer
      • 🌍GRAPH MODELS
        • Graphormer
  • 🌍INTERNAL HELPERS
    • Custom Layers and Utilities
    • Utilities for pipelines
    • Utilities for Tokenizers
    • Utilities for Trainer
    • Utilities for Generation
    • Utilities for Image Processors
    • Utilities for Audio processing
    • General Utilities
    • Utilities for Time Series
Powered by GitBook
On this page
  • Community
  • Community resources:
  • Community notebooks:
  1. DEVELOPER GUIDES

Community resources

Community

This page regroups resources around 🌍 Transformers developed by the community.

Community resources:

Resource
Description
Author

Community notebooks:

Notebook
Description
Author

How to generate lyrics in the style of your favorite artist by fine-tuning a GPT-2 model

How to train T5 for any task using Tensorflow 2. This notebook demonstrates a Question & Answer task implemented in Tensorflow 2 using SQUAD

How to train T5 on SQUAD with Transformers and Nlp

How to fine-tune T5 for classification and multiple choice tasks using a text-to-text format with PyTorch Lightning

How to fine-tune the DialoGPT model on a new dataset for open-dialog conversational chatbots

How to train on sequences as long as 500,000 tokens with Reformer

How to fine-tune BART for summarization with fastai using blurr

How to generate tweets in the style of your favorite Twitter account by fine-tuning a GPT-2 model

A complete tutorial showcasing W&B integration with BOINC AI

How to build a “long” version of existing pretrained models

How to fine-tune longformer model for QA task

How to evaluate longformer on TriviaQA with nlp

How to fine-tune T5 for sentiment span extraction using a text-to-text format with PyTorch Lightning

How to fine-tune DistilBert for multiclass classification with PyTorch

How to fine-tune BERT for multi-label classification using PyTorch

How to fine-tune T5 for summarization in PyTorch and track experiments with WandB

How to speed up fine-tuning by a factor of 2 using dynamic padding / bucketing

How to train a Reformer model with bi-directional self-attention layers

How to increase vocabulary of a pretrained SciBERT model from AllenAI on the CORD dataset and pipeline it.

How to fine-tune BlenderBotSmall for summarization on a custom dataset, using the Trainer API.

How to fine-tune Electra for sentiment analysis and interpret predictions with Captum Integrated Gradients

How to fine-tune a non-English GPT-2 Model with Trainer class

How to fine-tune a DistilBERT Model for Multi Label Classification task

How to fine-tune an ALBERT model or another BERT-based model for the sentence-pair classification task

How to fine-tune a Roberta model for sentiment analysis

How accurate are the answers to questions generated by your seq2seq transformer model?

How to fine-tune DistilBERT for text classification in TensorFlow

How to warm-start a EncoderDecoderModel with a bert-base-uncased checkpoint for summarization on CNN/Dailymail

How to warm-start a shared EncoderDecoderModel with a roberta-base checkpoint for summarization on BBC/XSum

How to fine-tune TapasForQuestionAnswering with a tapas-base checkpoint on the Sequential Question Answering (SQA) dataset

How to evaluate a fine-tuned TapasForSequenceClassification with a tapas-base-finetuned-tabfact checkpoint using a combination of the 🌍 datasets and 🌍 transformers libraries

How to fine-tune mBART using Seq2SeqTrainer for Hindi to English translation

How to fine-tune LayoutLMForTokenClassification on the FUNSD dataset for information extraction from scanned documents

How to fine-tune DistilGPT2 and generate text

How to fine-tune LED on pubmed for long-range summarization

How to effectively evaluate LED on long-range summarization

How to fine-tune LayoutLMForSequenceClassification on the RVL-CDIP dataset for scanned document classification

How to decode CTC sequence with language model adjustment

How to fine-tune BART for summarization in two languages with Trainer class

How to evaluate BigBird on long document question answering on Trivia QA

How to create YouTube captions from any video by transcribing the audio with Wav2Vec

How to fine-tune the Vision Transformer (ViT) on CIFAR-10 using BOINC AI Transformers, Datasets and PyTorch Lightning

How to fine-tune the Vision Transformer (ViT) on CIFAR-10 using BOINC AI Transformers, Datasets and the 🌍 Trainer

How to evaluate LukeForEntityClassification on the Open Entity dataset

How to evaluate LukeForEntityPairClassification on the TACRED dataset

How to evaluate LukeForEntitySpanClassification on the CoNLL-2003 dataset

How to evaluate BigBirdPegasusForConditionalGeneration on PubMed dataset

How to leverage a pretrained Wav2Vec2 model for Emotion Classification on the MEGA dataset

How to use a trained DetrForObjectDetection model to detect objects in an image and visualize attention

How to fine-tune DetrForObjectDetection on a custom object detection dataset

How to fine-tune T5 on a Named Entity Recognition Task

PreviousNotebooks with examplesNextCustom Tools and Prompts

Last updated 1 year ago

A set of flashcards based on the that has been put into a form which can be easily learned/revised using an open source, cross platform app specifically designed for long term knowledge retention. See this .

🌍

🌍

🌍

🌍
BOINC AI Transformers Glossary Flashcards
Transformers Docs Glossary
Anki
Introductory video on how to use the flashcards
Darigov Research
Fine-tune a pre-trained Transformer to generate lyrics
Aleksey Korshuk
Train T5 in Tensorflow 2
Muhammad Harris
Train T5 on TPU
Suraj Patil
Fine-tune T5 for Classification and Multiple Choice
Suraj Patil
Fine-tune DialoGPT on New Datasets and Languages
Nathan Cooper
Long Sequence Modeling with Reformer
Patrick von Platen
Fine-tune BART for Summarization
Wayde Gilliam
Fine-tune a pre-trained Transformer on anyone’s tweets
Boris Dayma
Optimize
BOINC AI models with Weights & Biases
Boris Dayma
Pretrain Longformer
Iz Beltagy
Fine-tune Longformer for QA
Suraj Patil
Evaluate Model with
nlp
Patrick von Platen
Fine-tune T5 for Sentiment Span Extraction
Lorenzo Ampil
Fine-tune DistilBert for Multiclass Classification
Abhishek Kumar Mishra
Fine-tune BERT for Multi-label Classification
Abhishek Kumar Mishra
Fine-tune T5 for Summarization
Abhishek Kumar Mishra
Speed up Fine-Tuning in Transformers with Dynamic Padding / Bucketing
Michael Benesty
Pretrain Reformer for Masked Language Modeling
Patrick von Platen
Expand and Fine Tune Sci-BERT
Tanmay Thakur
Fine Tune BlenderBotSmall for Summarization using the Trainer API
Tanmay Thakur
Fine-tune Electra and interpret with Integrated Gradients
Eliza Szczechla
fine-tune a non-English GPT-2 Model with Trainer class
Philipp Schmid
Fine-tune a DistilBERT Model for Multi Label Classification task
Dhaval Taunk
Fine-tune ALBERT for sentence-pair classification
Nadir El Manouzi
Fine-tune Roberta for sentiment analysis
Dhaval Taunk
Evaluating Question Generation Models
Pascal Zoleko
Classify text with DistilBERT and Tensorflow
Peter Bayerle
Leverage BERT for Encoder-Decoder Summarization on CNN/Dailymail
Patrick von Platen
Leverage RoBERTa for Encoder-Decoder Summarization on BBC XSum
Patrick von Platen
Fine-tune TAPAS on Sequential Question Answering (SQA)
Niels Rogge
Evaluate TAPAS on Table Fact Checking (TabFact)
Niels Rogge
Fine-tuning mBART for translation
Vasudev Gupta
Fine-tune LayoutLM on FUNSD (a form understanding dataset)
Niels Rogge
Fine-Tune DistilGPT2 and Generate Text
Aakash Tripathi
Fine-Tune LED on up to 8K tokens
Patrick von Platen
Evaluate LED on Arxiv
Patrick von Platen
Fine-tune LayoutLM on RVL-CDIP (a document image classification dataset)
Niels Rogge
Wav2Vec2 CTC decoding with GPT2 adjustment
Eric Lam
Fine-tune BART for summarization in two languages with Trainer class
Eliza Szczechla
Evaluate Big Bird on Trivia QA
Patrick von Platen
Create video captions using Wav2Vec2
Niklas Muennighoff
Fine-tune the Vision Transformer on CIFAR-10 using PyTorch Lightning
Niels Rogge
Fine-tune the Vision Transformer on CIFAR-10 using the
Trainer
Niels Rogge
Evaluate LUKE on Open Entity, an entity typing dataset
Ikuya Yamada
Evaluate LUKE on TACRED, a relation extraction dataset
Ikuya Yamada
Evaluate LUKE on CoNLL-2003, an important NER benchmark
Ikuya Yamada
Evaluate BigBird-Pegasus on PubMed dataset
Vasudev Gupta
Speech Emotion Classification with Wav2Vec2
Mehrdad Farahani
Detect objects in an image with DETR
Niels Rogge
Fine-tune DETR on a custom object detection dataset
Niels Rogge
Finetune T5 for Named Entity Recognition
Ogundepo Odunayo