Pipelines

The pipelines are a great and easy way to use models for inference. These pipelines are objects that abstract most of the complex code from the library, offering a simple API dedicated to several tasks, including Named Entity Recognition, Masked Language Modeling, Sentiment Analysis, Feature Extraction and Question Answering. See the task summary for examples of use.

There are two categories of pipeline abstractions to be aware about:

The pipeline() which is the most powerful object encapsulating all other pipelines.
Task-specific pipelines are available for audio, computer vision, natural language processing, and multimodal tasks.

The pipeline abstraction

The pipeline abstraction is a wrapper around all the other available pipelines. It is instantiated as any other pipeline but can provide additional quality of life.

Simple call on one item:

Copied

>>> pipe = pipeline("text-classification")
>>> pipe("This restaurant is awesome")
[{'label': 'POSITIVE', 'score': 0.9998743534088135}]

If you want to use a specific model from the hub you can ignore the task if the model on the hub already defines it:

Copied

>>> pipe = pipeline(model="roberta-large-mnli")
>>> pipe("This restaurant is awesome")
[{'label': 'NEUTRAL', 'score': 0.7313136458396912}]

To call a pipeline on many items, you can call it with a list.

Copied

>>> pipe = pipeline("text-classification")
>>> pipe(["This restaurant is awesome", "This restaurant is awful"])
[{'label': 'POSITIVE', 'score': 0.9998743534088135},
 {'label': 'NEGATIVE', 'score': 0.9996669292449951}]

To iterate over full datasets it is recommended to use a dataset directly. This means you don’t need to allocate the whole dataset at once, nor do you need to do batching yourself. This should work just as fast as custom loops on GPU. If it doesn’t don’t hesitate to create an issue.

Copied

import datasets
from transformers import pipeline
from transformers.pipelines.pt_utils import KeyDataset
from tqdm.auto import tqdm

pipe = pipeline("automatic-speech-recognition", model="facebook/wav2vec2-base-960h", device=0)
dataset = datasets.load_dataset("superb", name="asr", split="test")

# KeyDataset (only *pt*) will simply return the item in the dict returned by the dataset item
# as we're not interested in the *target* part of the dataset. For sentence pair use KeyPairDataset
for out in tqdm(pipe(KeyDataset(dataset, "file"))):
    print(out)
    # {"text": "NUMBER TEN FRESH NELLY IS WAITING ON YOU GOOD NIGHT HUSBAND"}
    # {"text": ....}
    # ....

For ease of use, a generator is also possible:

Copied

from transformers import pipeline

pipe = pipeline("text-classification")


def data():
    while True:
        # This could come from a dataset, a database, a queue or HTTP request
        # in a server
        # Caveat: because this is iterative, you cannot use `num_workers > 1` variable
        # to use multiple threads to preprocess data. You can still have 1 thread that
        # does the preprocessing while the main runs the big inference
        yield "This is a test"


for out in pipe(data()):
    print(out)
    # {"text": "NUMBER TEN FRESH NELLY IS WAITING ON YOU GOOD NIGHT HUSBAND"}
    # {"text": ....}
    # ....

transformers.pipeline

Pipelines

The pipeline abstraction

Pipeline batching

Pipeline chunk batching

Pipeline custom code

Implementing a pipeline

Audio

AudioClassificationPipeline

class transformers.AudioClassificationPipeline

AutomaticSpeechRecognitionPipeline

class transformers.AutomaticSpeechRecognitionPipeline

TextToAudioPipeline

class transformers.TextToAudioPipeline

ZeroShotAudioClassificationPipeline

class transformers.ZeroShotAudioClassificationPipeline

Computer vision

DepthEstimationPipeline

class transformers.DepthEstimationPipeline

ImageClassificationPipeline

class transformers.ImageClassificationPipeline

ImageSegmentationPipeline

class transformers.ImageSegmentationPipeline

ImageToImagePipeline

class transformers.ImageToImagePipeline

ObjectDetectionPipeline

class transformers.ObjectDetectionPipeline

VideoClassificationPipeline

class transformers.VideoClassificationPipeline

ZeroShotImageClassificationPipeline

class transformers.ZeroShotImageClassificationPipeline

ZeroShotObjectDetectionPipeline

class transformers.ZeroShotObjectDetectionPipeline

Natural Language Processing

ConversationalPipeline

class transformers.Conversation

class transformers.ConversationalPipeline

FillMaskPipeline

class transformers.FillMaskPipeline

NerPipeline

class transformers.TokenClassificationPipeline

QuestionAnsweringPipeline

class transformers.QuestionAnsweringPipeline

SummarizationPipeline

class transformers.SummarizationPipeline

TableQuestionAnsweringPipeline

class transformers.TableQuestionAnsweringPipeline

TextClassificationPipeline

class transformers.TextClassificationPipeline

TextGenerationPipeline

class transformers.TextGenerationPipeline

Text2TextGenerationPipeline

class transformers.Text2TextGenerationPipeline

TokenClassificationPipeline

class transformers.TokenClassificationPipeline

TranslationPipeline

class transformers.TranslationPipeline

ZeroShotClassificationPipeline

class transformers.ZeroShotClassificationPipeline

Multimodal

DocumentQuestionAnsweringPipeline

class transformers.DocumentQuestionAnsweringPipeline

FeatureExtractionPipeline

class transformers.FeatureExtractionPipeline

ImageToTextPipeline

class transformers.ImageToTextPipeline

VisualQuestionAnsweringPipeline

class transformers.VisualQuestionAnsweringPipeline

Parent class: Pipeline

class transformers.Pipeline