# Inference Client

## Inference

Inference is the process of using a trained model to make predictions on new data. As this process can be compute-intensive, running on a dedicated server can be an interesting option. The `huggingface_hub` library provides an easy way to call a service that runs inference for hosted models. There are several services you can connect to:

* [Inference API](https://huggingface.co/docs/api-inference/index): a service that allows you to run accelerated inference on Hugging Face’s infrastructure for free. This service is a fast way to get started, test different models, and prototype AI products.
* [Inference Endpoints](https://huggingface.co/inference-endpoints): a product to easily deploy models to production. Inference is run by Hugging Face in a dedicated, fully managed infrastructure on a cloud provider of your choice.

These services can be called with the [InferenceClient](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceClient) object. Please refer to [this guide](https://huggingface.co/docs/huggingface_hub/guides/inference) for more information on how to use it.

### Inference Client

#### class huggingface\_hub.InferenceClient

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L105)

( model: typing.Optional\[str] = Nonetoken: typing.Union\[str, bool, NoneType] = Nonetimeout: typing.Optional\[float] = Noneheaders: typing.Union\[typing.Dict\[str, str], NoneType] = Nonecookies: typing.Union\[typing.Dict\[str, str], NoneType] = None )

Parameters

* **model** (`str`, `optional`) — The model to run inference with. Can be a model id hosted on the Hugging Face Hub, e.g. `bigcode/starcoder` or a URL to a deployed Inference Endpoint. Defaults to None, in which case a recommended model is automatically selected for the task.
* **token** (`str`, *optional*) — Hugging Face token. Will default to the locally saved token. Pass `token=False` if you don’t want to send your token to the server.
* **timeout** (`float`, `optional`) — The maximum number of seconds to wait for a response from the server. Loading a new model in Inference API can take up to several minutes. Defaults to None, meaning it will loop until the server is available.
* **headers** (`Dict[str, str]`, `optional`) — Additional headers to send to the server. By default only the authorization and user-agent headers are sent. Values in this dictionary will override the default values.
* **cookies** (`Dict[str, str]`, `optional`) — Additional cookies to send to the server.

Initialize a new Inference Client.

[InferenceClient](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceClient) aims to provide a unified experience to perform inference. The client can be used seamlessly with either the (free) Inference API or self-hosted Inference Endpoints.

**audio\_classification**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L258)

( audio: typing.Union\[bytes, typing.BinaryIO, str, pathlib.Path]model: typing.Optional\[str] = None ) → `List[Dict]`

Parameters

* **audio** (Union\[str, Path, bytes, BinaryIO]) — The audio content to classify. It can be raw audio bytes, a local audio file, or a URL pointing to an audio file.
* **model** (`str`, *optional*) — The model to use for audio classification. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended model for audio classification will be used.

Returns

`List[Dict]`

The classification output containing the predicted label and its confidence.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `HTTPError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `HTTPError` — If the request fails with an HTTP error status code other than HTTP 503.

Perform audio classification on the provided audio content.

Example:

Copied

```
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()
>>> client.audio_classification("audio.flac")
[{'score': 0.4976358711719513, 'label': 'hap'}, {'score': 0.3677836060523987, 'label': 'neu'},...]
```

**automatic\_speech\_recognition**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L296)

( audio: typing.Union\[bytes, typing.BinaryIO, str, pathlib.Path]model: typing.Optional\[str] = None ) → str

Parameters

* **audio** (Union\[str, Path, bytes, BinaryIO]) — The content to transcribe. It can be raw audio bytes, local audio file, or a URL to an audio file.
* **model** (`str`, *optional*) — The model to use for ASR. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended model for ASR will be used.

Returns

str

The transcribed text.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `HTTPError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `HTTPError` — If the request fails with an HTTP error status code other than HTTP 503.

Perform automatic speech recognition (ASR or audio-to-text) on the given audio content.

Example:

Copied

```
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()
>>> client.automatic_speech_recognition("hello_world.flac")
"hello world"
```

**conversational**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L332)

( text: strgenerated\_responses: typing.Optional\[typing.List\[str]] = Nonepast\_user\_inputs: typing.Optional\[typing.List\[str]] = Noneparameters: typing.Union\[typing.Dict\[str, typing.Any], NoneType] = Nonemodel: typing.Optional\[str] = None ) → `Dict`

Parameters

* **text** (`str`) — The last input from the user in the conversation.
* **generated\_responses** (`List[str]`, *optional*) — A list of strings corresponding to the earlier replies from the model. Defaults to None.
* **past\_user\_inputs** (`List[str]`, *optional*) — A list of strings corresponding to the earlier replies from the user. Should be the same length as `generated_responses`. Defaults to None.
* **parameters** (`Dict[str, Any]`, *optional*) — Additional parameters for the conversational task. Defaults to None. For more details about the available parameters, please refer to [this page](https://huggingface.co/docs/api-inference/detailed_parameters#conversational-task)
* **model** (`str`, *optional*) — The model to use for the conversational task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended conversational model will be used. Defaults to None.

Returns

`Dict`

The generated conversational output.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `HTTPError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `HTTPError` — If the request fails with an HTTP error status code other than HTTP 503.

Generate conversational responses based on the given input text (i.e. chat with the API).

Example:

Copied

```
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()
>>> output = client.conversational("Hi, who are you?")
>>> output
{'generated_text': 'I am the one who knocks.', 'conversation': {'generated_responses': ['I am the one who knocks.'], 'past_user_inputs': ['Hi, who are you?']}, 'warnings': ['Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.']}
>>> client.conversational(
...     "Wow, that's scary!",
...     generated_responses=output["conversation"]["generated_responses"],
...     past_user_inputs=output["conversation"]["past_user_inputs"],
... )
```

**document\_question\_answering**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L437)

( image: typing.Union\[bytes, typing.BinaryIO, str, pathlib.Path]question: strmodel: typing.Optional\[str] = None ) → `List[Dict]`

Parameters

* **image** (`Union[str, Path, bytes, BinaryIO]`) — The input image for the context. It can be raw bytes, an image file, or a URL to an online image.
* **question** (`str`) — Question to be answered.
* **model** (`str`, *optional*) — The model to use for the document question answering task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended document question answering model will be used. Defaults to None.

Returns

`List[Dict]`

a list of dictionaries containing the predicted label, associated probability, word ids, and page number.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `HTTPError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `HTTPError` — If the request fails with an HTTP error status code other than HTTP 503.

Answer questions on document images.

Example:

Copied

```
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()
>>> client.document_question_answering(image="https://huggingface.co/spaces/impira/docquery/resolve/2359223c1837a7587402bda0f2643382a6eefeab/invoice.png", question="What is the invoice number?")
[{'score': 0.42515629529953003, 'answer': 'us-001', 'start': 16, 'end': 16}]
```

**feature\_extraction**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L478)

( text: strmodel: typing.Optional\[str] = None ) → `np.ndarray`

Parameters

* **text** (`str`) — The text to embed.
* **model** (`str`, *optional*) — The model to use for the conversational task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended conversational model will be used. Defaults to None.

Returns

`np.ndarray`

The embedding representing the input text as a float32 numpy array.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `HTTPError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `HTTPError` — If the request fails with an HTTP error status code other than HTTP 503.

Generate embeddings for a given text.

Example:

Copied

```
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()
>>> client.feature_extraction("Hi, who are you?")
array([[ 2.424802  ,  2.93384   ,  1.1750331 , ...,  1.240499, -0.13776633, -0.7889173 ],
[-0.42943227, -0.6364878 , -1.693462  , ...,  0.41978157, -2.4336355 ,  0.6162071 ],
...,
[ 0.28552425, -0.928395  , -1.2077185 , ...,  0.76810825, -2.1069427 ,  0.6236161 ]], dtype=float32)
```

**fill\_mask**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L514)

( text: strmodel: typing.Optional\[str] = None ) → `List[Dict]`

Parameters

* **text** (`str`) — a string to be filled from, must contain the \[MASK] token (check model card for exact name of the mask).
* **model** (`str`, *optional*) — The model to use for the fill mask task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended fill mask model will be used. Defaults to None.

Returns

`List[Dict]`

a list of fill mask output dictionaries containing the predicted label, associated probability, token reference, and completed text.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `HTTPError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `HTTPError` — If the request fails with an HTTP error status code other than HTTP 503.

Fill in a hole with a missing word (token to be precise).

Example:

Copied

```
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()
>>> client.fill_mask("The goal of life is <mask>.")
[{'score': 0.06897063553333282,
'token': 11098,
'token_str': ' happiness',
'sequence': 'The goal of life is happiness.'},
{'score': 0.06554922461509705,
'token': 45075,
'token_str': ' immortality',
'sequence': 'The goal of life is immortality.'}]
```

**get\_model\_status**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L1878)

( model: typing.Optional\[str] = None ) → `ModelStatus`

Parameters

* **model** (`str`, *optional*) — Identifier of the model for witch the status gonna be checked. If model is not provided, the model associated with this instance of [InferenceClient](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceClient) will be used. Only InferenceAPI service can be checked so the identifier cannot be a URL.

Returns

`ModelStatus`

An instance of ModelStatus dataclass, containing information, about the state of the model: load, state, compute type and framework.

Get the status of a model hosted on the Inference API.

This endpoint is mostly useful when you already know which model you want to use and want to check its availability. If you want to discover already deployed models, you should rather use [list\_deployed\_models()](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceClient.list_deployed_models).

Example:

Copied

```
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()
>>> client.get_model_status("bigcode/starcoder")
ModelStatus(loaded=True, state='Loaded', compute_type='gpu', framework='text-generation-inference')
```

**image\_classification**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L554)

( image: typing.Union\[bytes, typing.BinaryIO, str, pathlib.Path]model: typing.Optional\[str] = None ) → `List[Dict]`

Parameters

* **image** (`Union[str, Path, bytes, BinaryIO]`) — The image to classify. It can be raw bytes, an image file, or a URL to an online image.
* **model** (`str`, *optional*) — The model to use for image classification. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended model for image classification will be used.

Returns

`List[Dict]`

a list of dictionaries containing the predicted label and associated probability.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `HTTPError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `HTTPError` — If the request fails with an HTTP error status code other than HTTP 503.

Perform image classification on the given image using the specified model.

Example:

Copied

```
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()
>>> client.image_classification("https://upload.wikimedia.org/wikipedia/commons/thumb/4/43/Cute_dog.jpg/320px-Cute_dog.jpg")
[{'score': 0.9779096841812134, 'label': 'Blenheim spaniel'}, ...]
```

**image\_segmentation**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L590)

( image: typing.Union\[bytes, typing.BinaryIO, str, pathlib.Path]model: typing.Optional\[str] = None ) → `List[Dict]`

Parameters

* **image** (`Union[str, Path, bytes, BinaryIO]`) — The image to segment. It can be raw bytes, an image file, or a URL to an online image.
* **model** (`str`, *optional*) — The model to use for image segmentation. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended model for image segmentation will be used.

Returns

`List[Dict]`

A list of dictionaries containing the segmented masks and associated attributes.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `HTTPError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `HTTPError` — If the request fails with an HTTP error status code other than HTTP 503.

Perform image segmentation on the given image using the specified model.

You must have `PIL` installed if you want to work with images (`pip install Pillow`).

Example:

Copied

```
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()
>>> client.image_segmentation("cat.jpg"):
[{'score': 0.989008, 'label': 'LABEL_184', 'mask': <PIL.PngImagePlugin.PngImageFile image mode=L size=400x300 at 0x7FDD2B129CC0>}, ...]
```

**image\_to\_image**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L641)

( image: typing.Union\[bytes, typing.BinaryIO, str, pathlib.Path]prompt: typing.Optional\[str] = Nonenegative\_prompt: typing.Optional\[str] = Noneheight: typing.Optional\[int] = Nonewidth: typing.Optional\[int] = Nonenum\_inference\_steps: typing.Optional\[int] = Noneguidance\_scale: typing.Optional\[float] = Nonemodel: typing.Optional\[str] = None\*\*kwargs ) → `Image`

Parameters

* **image** (`Union[str, Path, bytes, BinaryIO]`) — The input image for translation. It can be raw bytes, an image file, or a URL to an online image.
* **prompt** (`str`, *optional*) — The text prompt to guide the image generation.
* **negative\_prompt** (`str`, *optional*) — A negative prompt to guide the translation process.
* **height** (`int`, *optional*) — The height in pixels of the generated image.
* **width** (`int`, *optional*) — The width in pixels of the generated image.
* **num\_inference\_steps** (`int`, *optional*) — The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference.
* **guidance\_scale** (`float`, *optional*) — Higher guidance scale encourages to generate images that are closely linked to the text `prompt`, usually at the expense of lower image quality.
* **model** (`str`, *optional*) — The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None.

Returns

`Image`

The translated image.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `HTTPError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `HTTPError` — If the request fails with an HTTP error status code other than HTTP 503.

Perform image-to-image translation using a specified model.

You must have `PIL` installed if you want to work with images (`pip install Pillow`).

Example:

Copied

```
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()
>>> image = client.image_to_image("cat.jpg", prompt="turn the cat into a tiger")
>>> image.save("tiger.jpg")
```

**image\_to\_text**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L725)

( image: typing.Union\[bytes, typing.BinaryIO, str, pathlib.Path]model: typing.Optional\[str] = None ) → `str`

Parameters

* **image** (`Union[str, Path, bytes, BinaryIO]`) — The input image to caption. It can be raw bytes, an image file, or a URL to an online image..
* **model** (`str`, *optional*) — The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None.

Returns

`str`

The generated text.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `HTTPError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `HTTPError` — If the request fails with an HTTP error status code other than HTTP 503.

Takes an input image and return text.

Models can have very different outputs depending on your use case (image captioning, optical character recognition (OCR), Pix2Struct, etc). Please have a look to the model card to learn more about a model’s specificities.

Example:

Copied

```
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()
>>> client.image_to_text("cat.jpg")
'a cat standing in a grassy field '
>>> client.image_to_text("https://upload.wikimedia.org/wikipedia/commons/thumb/4/43/Cute_dog.jpg/320px-Cute_dog.jpg")
'a dog laying on the grass next to a flower pot '
```

**list\_deployed\_models**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L761)

( frameworks: typing.Union\[NoneType, str, typing.Literal\['all'], typing.List\[str]] = None ) → `Dict[str, List[str]]`

Parameters

* **frameworks** (`Literal["all"]` or `List[str]` or `str`, *optional*) — The frameworks to filter on. By default only a subset of the available frameworks are tested. If set to “all”, all available frameworks will be tested. It is also possible to provide a single framework or a custom set of frameworks to check.

Returns

`Dict[str, List[str]]`

A dictionary mapping task names to a sorted list of model IDs.

List models currently deployed on the Inference API service.

This helper checks deployed models framework by framework. By default, it will check the 4 main frameworks that are supported and account for 95% of the hosted models. However, if you want a complete list of models you can specify `frameworks="all"` as input. Alternatively, if you know before-hand which framework you are interested in, you can also restrict to search to this one (e.g. `frameworks="text-generation-inference"`). The more frameworks are checked, the more time it will take.

This endpoint is mostly useful for discoverability. If you already know which model you want to use and want to check its availability, you can directly use [get\_model\_status()](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceClient.get_model_status).

Example:

Copied

```
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()

# Discover zero-shot-classification models currently deployed
>>> models = client.list_deployed_models()
>>> models["zero-shot-classification"]
['Narsil/deberta-large-mnli-zero-cls', 'facebook/bart-large-mnli', ...]

# List from only 1 framework
>>> client.list_deployed_models("text-generation-inference")
{'text-generation': ['bigcode/starcoder', 'meta-llama/Llama-2-70b-chat-hf', ...], ...}
```

**object\_detection**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L836)

( image: typing.Union\[bytes, typing.BinaryIO, str, pathlib.Path]model: typing.Optional\[str] = None ) → `List[ObjectDetectionOutput]`

Parameters

* **image** (`Union[str, Path, bytes, BinaryIO]`) — The image to detect objects on. It can be raw bytes, an image file, or a URL to an online image.
* **model** (`str`, *optional*) — The model to use for object detection. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended model for object detection (DETR) will be used.

Returns

`List[ObjectDetectionOutput]`

A list of dictionaries containing the bounding boxes and associated attributes.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `HTTPError` or `ValueError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `HTTPError` — If the request fails with an HTTP error status code other than HTTP 503.
* `ValueError` — If the request output is not a List.

Perform object detection on the given image using the specified model.

You must have `PIL` installed if you want to work with images (`pip install Pillow`).

Example:

Copied

```
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()
>>> client.object_detection("people.jpg"):
[{"score":0.9486683011054993,"label":"person","box":{"xmin":59,"ymin":39,"xmax":420,"ymax":510}}, ... ]
```

**post**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L172)

( json: typing.Union\[str, typing.Dict, typing.List, NoneType] = Nonedata: typing.Union\[bytes, typing.BinaryIO, str, pathlib.Path, NoneType] = Nonemodel: typing.Optional\[str] = Nonetask: typing.Optional\[str] = Nonestream: bool = False ) → bytes

Parameters

* **json** (`Union[str, Dict, List]`, *optional*) — The JSON data to send in the request body. Defaults to None.
* **data** (`Union[str, Path, bytes, BinaryIO]`, *optional*) — The content to send in the request body. It can be raw bytes, a pointer to an opened file, a local file path, or a URL to an online resource (image, audio file,…). If both `json` and `data` are passed, `data` will take precedence. At least `json` or `data` must be provided. Defaults to None.
* **model** (`str`, *optional*) — The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. Will override the model defined at the instance level. Defaults to None.
* **task** (`str`, *optional*) — The task to perform on the inference. Used only to default to a recommended model if `model` is not provided. At least `model` or `task` must be provided. Defaults to None.
* **stream** (`bool`, *optional*) — Whether to iterate over streaming APIs.

Returns

bytes

The raw bytes returned by the server.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `HTTPError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `HTTPError` — If the request fails with an HTTP error status code other than HTTP 503.

Make a POST request to the inference server.

**question\_answering**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L884)

( question: strcontext: strmodel: typing.Optional\[str] = None ) → `Dict`

Parameters

* **question** (`str`) — Question to be answered.
* **context** (`str`) — The context of the question.
* **model** (`str`) — The model to use for the question answering task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint.

Returns

`Dict`

a dictionary of question answering output containing the score, start index, end index, and answer.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `HTTPError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `HTTPError` — If the request fails with an HTTP error status code other than HTTP 503.

Retrieve the answer to a question from a given text.

Example:

Copied

```
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()
>>> client.question_answering(question="What's my name?", context="My name is Clara and I live in Berkeley.")
{'score': 0.9326562285423279, 'start': 11, 'end': 16, 'answer': 'Clara'}
```

**sentence\_similarity**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L925)

( sentence: strother\_sentences: typing.List\[str]model: typing.Optional\[str] = None ) → `List[float]`

Parameters

* **sentence** (`str`) — The main sentence to compare to others.
* **other\_sentences** (`List[str]`) — The list of sentences to compare to.
* **model** (`str`, *optional*) — The model to use for the conversational task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended conversational model will be used. Defaults to None.

Returns

`List[float]`

The embedding representing the input text.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `HTTPError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `HTTPError` — If the request fails with an HTTP error status code other than HTTP 503.

Compute the semantic similarity between a sentence and a list of other sentences by comparing their embeddings.

Example:

Copied

```
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()
>>> client.sentence_similarity(
...     "Machine learning is so easy.",
...     other_sentences=[
...         "Deep learning is so straightforward.",
...         "This is so difficult, like rocket science.",
...         "I can't believe how much I struggled with this.",
...     ],
... )
[0.7785726189613342, 0.45876261591911316, 0.2906220555305481]
```

**summarization**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L972)

( text: strparameters: typing.Union\[typing.Dict\[str, typing.Any], NoneType] = Nonemodel: typing.Optional\[str] = None ) → `str`

Parameters

* **text** (`str`) — The input text to summarize.
* **parameters** (`Dict[str, Any]`, *optional*) — Additional parameters for summarization. Check out this [page](https://huggingface.co/docs/api-inference/detailed_parameters#summarization-task) for more details.
* **model** (`str`, *optional*) — The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None.

Returns

`str`

The generated summary text.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `HTTPError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `HTTPError` — If the request fails with an HTTP error status code other than HTTP 503.

Generate a summary of a given text using a specified model.

Example:

Copied

```
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()
>>> client.summarization("The Eiffel tower...")
'The Eiffel tower is one of the most famous landmarks in the world....'
```

**table\_question\_answering**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L1015)

( table: typing.Dict\[str, typing.Any]query: strmodel: typing.Optional\[str] = None ) → `Dict`

Parameters

* **table** (`str`) — A table of data represented as a dict of lists where entries are headers and the lists are all the values, all lists must have the same size.
* **query** (`str`) — The query in plain text that you want to ask the table.
* **model** (`str`) — The model to use for the table-question-answering task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint.

Returns

`Dict`

a dictionary of table question answering output containing the answer, coordinates, cells and the aggregator used.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `HTTPError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `HTTPError` — If the request fails with an HTTP error status code other than HTTP 503.

Retrieve the answer to a question from information given in a table.

Example:

Copied

```
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()
>>> query = "How many stars does the transformers repository have?"
>>> table = {"Repository": ["Transformers", "Datasets", "Tokenizers"], "Stars": ["36542", "4512", "3934"]}
>>> client.table_question_answering(table, query, model="google/tapas-base-finetuned-wtq")
{'answer': 'AVERAGE > 36542', 'coordinates': [[0, 1]], 'cells': ['36542'], 'aggregator': 'AVERAGE'}
```

**tabular\_classification**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L1060)

( table: typing.Dict\[str, typing.Any]model: str ) → `List`

Parameters

* **table** (`Dict[str, Any]`) — Set of attributes to classify.
* **model** (`str`) — The model to use for the tabular-classification task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint.

Returns

`List`

a list of labels, one per row in the initial table.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `HTTPError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `HTTPError` — If the request fails with an HTTP error status code other than HTTP 503.

Classifying a target category (a group) based on a set of attributes.

Example:

Copied

```
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()
>>> table = {
...     "fixed_acidity": ["7.4", "7.8", "10.3"],
...     "volatile_acidity": ["0.7", "0.88", "0.32"],
...     "citric_acid": ["0", "0", "0.45"],
...     "residual_sugar": ["1.9", "2.6", "6.4"],
...     "chlorides": ["0.076", "0.098", "0.073"],
...     "free_sulfur_dioxide": ["11", "25", "5"],
...     "total_sulfur_dioxide": ["34", "67", "13"],
...     "density": ["0.9978", "0.9968", "0.9976"],
...     "pH": ["3.51", "3.2", "3.23"],
...     "sulphates": ["0.56", "0.68", "0.82"],
...     "alcohol": ["9.4", "9.8", "12.6"],
... }
>>> client.tabular_classification(table=table, model="julien-c/wine-quality")
["5", "5", "5"]
```

**tabular\_regression**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L1104)

( table: typing.Dict\[str, typing.Any]model: str ) → `List`

Parameters

* **table** (`Dict[str, Any]`) — Set of attributes stored in a table. The attributes used to predict the target can be both numerical and categorical.
* **model** (`str`) — The model to use for the tabular-regression task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint.

Returns

`List`

a list of predicted numerical target values.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `HTTPError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `HTTPError` — If the request fails with an HTTP error status code other than HTTP 503.

Predicting a numerical target value given a set of attributes/features in a table.

Example:

Copied

```
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()
>>> table = {
...     "Height": ["11.52", "12.48", "12.3778"],
...     "Length1": ["23.2", "24", "23.9"],
...     "Length2": ["25.4", "26.3", "26.5"],
...     "Length3": ["30", "31.2", "31.1"],
...     "Species": ["Bream", "Bream", "Bream"],
...     "Width": ["4.02", "4.3056", "4.6961"],
... }
>>> client.tabular_regression(table, model="scikit-learn/Fish-Weight")
[110, 120, 130]
```

**text\_classification**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L1143)

( text: strmodel: typing.Optional\[str] = None ) → `List[Dict]`

Parameters

* **text** (`str`) — A string to be classified.
* **model** (`str`, *optional*) — The model to use for the text classification task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended text classification model will be used. Defaults to None.

Returns

`List[Dict]`

a list of dictionaries containing the predicted label and associated probability.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `HTTPError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `HTTPError` — If the request fails with an HTTP error status code other than HTTP 503.

Perform text classification (e.g. sentiment-analysis) on the given text.

Example:

Copied

```
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()
>>> client.text_classification("I like you")
[{'label': 'POSITIVE', 'score': 0.9998695850372314}, {'label': 'NEGATIVE', 'score': 0.0001304351753788069}]
```

**text\_generation**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L1271)

( prompt: strdetails: bool = Falsestream: bool = Falsemodel: typing.Optional\[str] = Nonedo\_sample: bool = Falsemax\_new\_tokens: int = 20best\_of: typing.Optional\[int] = Nonerepetition\_penalty: typing.Optional\[float] = Nonereturn\_full\_text: bool = Falseseed: typing.Optional\[int] = Nonestop\_sequences: typing.Optional\[typing.List\[str]] = Nonetemperature: typing.Optional\[float] = Nonetop\_k: typing.Optional\[int] = Nonetop\_p: typing.Optional\[float] = Nonetruncate: typing.Optional\[int] = Nonetypical\_p: typing.Optional\[float] = Nonewatermark: bool = Falsedecoder\_input\_details: bool = False ) → `Union[str, TextGenerationResponse, Iterable[str], Iterable[TextGenerationStreamResponse]]`

Parameters

* **prompt** (`str`) — Input text.
* **details** (`bool`, *optional*) — By default, text\_generation returns a string. Pass `details=True` if you want a detailed output (tokens, probabilities, seed, finish reason, etc.). Only available for models running on with the `text-generation-inference` backend.
* **stream** (`bool`, *optional*) — By default, text\_generation returns the full generated text. Pass `stream=True` if you want a stream of tokens to be returned. Only available for models running on with the `text-generation-inference` backend.
* **model** (`str`, *optional*) — The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None.
* **do\_sample** (`bool`) — Activate logits sampling
* **max\_new\_tokens** (`int`) — Maximum number of generated tokens
* **best\_of** (`int`) — Generate best\_of sequences and return the one if the highest token logprobs
* **repetition\_penalty** (`float`) — The parameter for repetition penalty. 1.0 means no penalty. See [this paper](https://arxiv.org/pdf/1909.05858.pdf) for more details.
* **return\_full\_text** (`bool`) — Whether to prepend the prompt to the generated text
* **seed** (`int`) — Random sampling seed
* **stop\_sequences** (`List[str]`) — Stop generating tokens if a member of `stop_sequences` is generated
* **temperature** (`float`) — The value used to module the logits distribution.
* **top\_k** (`int`) — The number of highest probability vocabulary tokens to keep for top-k-filtering.
* **top\_p** (`float`) — If set to < 1, only the smallest set of most probable tokens with probabilities that add up to `top_p` or higher are kept for generation.
* **truncate** (`int`) — Truncate inputs tokens to the given size
* **typical\_p** (`float`) — Typical Decoding mass See [Typical Decoding for Natural Language Generation](https://arxiv.org/abs/2202.00666) for more information
* **watermark** (`bool`) — Watermarking with [A Watermark for Large Language Models](https://arxiv.org/abs/2301.10226)
* **decoder\_input\_details** (`bool`) — Return the decoder input token logprobs and ids. You must set `details=True` as well for it to be taken into account. Defaults to `False`.

Returns

`Union[str, TextGenerationResponse, Iterable[str], Iterable[TextGenerationStreamResponse]]`

Generated text returned from the server:

* if `stream=False` and `details=False`, the generated text is returned as a `str` (default)
* if `stream=True` and `details=False`, the generated text is returned token by token as a `Iterable[str]`
* if `stream=False` and `details=True`, the generated text is returned with more details as a [TextGenerationResponse](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.inference._text_generation.TextGenerationResponse)
* if `details=True` and `stream=True`, the generated text is returned token by token as a iterable of [TextGenerationStreamResponse](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.inference._text_generation.TextGenerationStreamResponse)

Raises

`ValidationError` or [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `HTTPError`

* `ValidationError` — If input values are not valid. No HTTP call is made to the server.
* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `HTTPError` — If the request fails with an HTTP error status code other than HTTP 503.

Given a prompt, generate the following text.

It is recommended to have Pydantic installed in order to get inputs validated. This is preferable as it allow early failures.

API endpoint is supposed to run with the `text-generation-inference` backend (TGI). This backend is the go-to solution to run large language models at scale. However, for some smaller models (e.g. “gpt2”) the default `transformers` + `api-inference` solution is still in use. Both approaches have very similar APIs, but not exactly the same. This method is compatible with both approaches but some parameters are only available for `text-generation-inference`. If some parameters are ignored, a warning message is triggered but the process continues correctly.

To learn more about the TGI project, please refer to <https://github.com/huggingface/text-generation-inference>.

Example:

Copied

```
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()

# Case 1: generate text
>>> client.text_generation("The huggingface_hub library is ", max_new_tokens=12)
'100% open source and built to be easy to use.'

# Case 2: iterate over the generated tokens. Useful for large generation.
>>> for token in client.text_generation("The huggingface_hub library is ", max_new_tokens=12, stream=True):
...     print(token)
100
%
open
source
and
built
to
be
easy
to
use
.

# Case 3: get more details about the generation process.
>>> client.text_generation("The huggingface_hub library is ", max_new_tokens=12, details=True)
TextGenerationResponse(
    generated_text='100% open source and built to be easy to use.',
    details=Details(
        finish_reason=<FinishReason.Length: 'length'>,
        generated_tokens=12,
        seed=None,
        prefill=[
            InputToken(id=487, text='The', logprob=None),
            InputToken(id=53789, text=' hugging', logprob=-13.171875),
            (...)
            InputToken(id=204, text=' ', logprob=-7.0390625)
        ],
        tokens=[
            Token(id=1425, text='100', logprob=-1.0175781, special=False),
            Token(id=16, text='%', logprob=-0.0463562, special=False),
            (...)
            Token(id=25, text='.', logprob=-0.5703125, special=False)
        ],
        best_of_sequences=None
    )
)

# Case 4: iterate over the generated tokens with more details.
# Last object is more complete, containing the full generated text and the finish reason.
>>> for details in client.text_generation("The huggingface_hub library is ", max_new_tokens=12, details=True, stream=True):
...     print(details)
...
TextGenerationStreamResponse(token=Token(id=1425, text='100', logprob=-1.0175781, special=False), generated_text=None, details=None)
TextGenerationStreamResponse(token=Token(id=16, text='%', logprob=-0.0463562, special=False), generated_text=None, details=None)
TextGenerationStreamResponse(token=Token(id=1314, text=' open', logprob=-1.3359375, special=False), generated_text=None, details=None)
TextGenerationStreamResponse(token=Token(id=3178, text=' source', logprob=-0.28100586, special=False), generated_text=None, details=None)
TextGenerationStreamResponse(token=Token(id=273, text=' and', logprob=-0.5961914, special=False), generated_text=None, details=None)
TextGenerationStreamResponse(token=Token(id=3426, text=' built', logprob=-1.9423828, special=False), generated_text=None, details=None)
TextGenerationStreamResponse(token=Token(id=271, text=' to', logprob=-1.4121094, special=False), generated_text=None, details=None)
TextGenerationStreamResponse(token=Token(id=314, text=' be', logprob=-1.5224609, special=False), generated_text=None, details=None)
TextGenerationStreamResponse(token=Token(id=1833, text=' easy', logprob=-2.1132812, special=False), generated_text=None, details=None)
TextGenerationStreamResponse(token=Token(id=271, text=' to', logprob=-0.08520508, special=False), generated_text=None, details=None)
TextGenerationStreamResponse(token=Token(id=745, text=' use', logprob=-0.39453125, special=False), generated_text=None, details=None)
TextGenerationStreamResponse(token=Token(
    id=25,
    text='.',
    logprob=-0.5703125,
    special=False),
    generated_text='100% open source and built to be easy to use.',
    details=StreamDetails(finish_reason=<FinishReason.Length: 'length'>, generated_tokens=12, seed=None)
)
```

**text\_to\_image**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L1538)

( prompt: strnegative\_prompt: typing.Optional\[str] = Noneheight: typing.Optional\[float] = Nonewidth: typing.Optional\[float] = Nonenum\_inference\_steps: typing.Optional\[float] = Noneguidance\_scale: typing.Optional\[float] = Nonemodel: typing.Optional\[str] = None\*\*kwargs ) → `Image`

Parameters

* **prompt** (`str`) — The prompt to generate an image from.
* **negative\_prompt** (`str`, *optional*) — An optional negative prompt for the image generation.
* **height** (`float`, *optional*) — The height in pixels of the image to generate.
* **width** (`float`, *optional*) — The width in pixels of the image to generate.
* **num\_inference\_steps** (`int`, *optional*) — The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference.
* **guidance\_scale** (`float`, *optional*) — Higher guidance scale encourages to generate images that are closely linked to the text `prompt`, usually at the expense of lower image quality.
* **model** (`str`, *optional*) — The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None.

Returns

`Image`

The generated image.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `HTTPError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `HTTPError` — If the request fails with an HTTP error status code other than HTTP 503.

Generate an image based on a given text using a specified model.

You must have `PIL` installed if you want to work with images (`pip install Pillow`).

Example:

Copied

```
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()

>>> image = client.text_to_image("An astronaut riding a horse on the moon.")
>>> image.save("astronaut.png")

>>> image = client.text_to_image(
...     "An astronaut riding a horse on the moon.",
...     negative_prompt="low resolution, blurry",
...     model="stabilityai/stable-diffusion-2-1",
... )
>>> image.save("better_astronaut.png")
```

**text\_to\_speech**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L1618)

( text: strmodel: typing.Optional\[str] = None ) → `bytes`

Parameters

* **text** (`str`) — The text to synthesize.
* **model** (`str`, *optional*) — The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None.

Returns

`bytes`

The generated audio.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `HTTPError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `HTTPError` — If the request fails with an HTTP error status code other than HTTP 503.

Synthesize an audio of a voice pronouncing a given text.

Example:

Copied

```
>>> from pathlib import Path
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()

>>> audio = client.text_to_speech("Hello world")
>>> Path("hello_world.flac").write_bytes(audio)
```

**token\_classification**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L1650)

( text: strmodel: typing.Optional\[str] = None ) → `List[Dict]`

Parameters

* **text** (`str`) — A string to be classified.
* **model** (`str`, *optional*) — The model to use for the token classification task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended token classification model will be used. Defaults to None.

Returns

`List[Dict]`

List of token classification outputs containing the entity group, confidence score, word, start and end index.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `HTTPError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `HTTPError` — If the request fails with an HTTP error status code other than HTTP 503.

Perform token classification on the given text. Usually used for sentence parsing, either grammatical, or Named Entity Recognition (NER) to understand keywords contained within text.

Example:

Copied

```
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()
>>> client.token_classification("My name is Sarah Jessica Parker but you can call me Jessica")
[{'entity_group': 'PER',
'score': 0.9971321225166321,
'word': 'Sarah Jessica Parker',
'start': 11,
'end': 31},
{'entity_group': 'PER',
'score': 0.9773476123809814,
'word': 'Jessica',
'start': 52,
'end': 59}]
```

**translation**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L1697)

( text: strmodel: typing.Optional\[str] = None ) → `str`

Parameters

* **text** (`str`) — A string to be translated.
* **model** (`str`, *optional*) — The model to use for the translation task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended translation model will be used. Defaults to None.

Returns

`str`

The generated translated text.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `HTTPError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `HTTPError` — If the request fails with an HTTP error status code other than HTTP 503.

Convert text from one language to another.

Check out <https://huggingface.co/tasks/translation> for more information on how to choose the best model for your specific use case. Source and target languages usually depends on the model.

Example:

Copied

```
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()
>>> client.translation("My name is Wolfgang and I live in Berlin")
'Mein Name ist Wolfgang und ich lebe in Berlin.'
>>> client.translation("My name is Wolfgang and I live in Berlin", model="Helsinki-NLP/opus-mt-en-fr")
"Je m'appelle Wolfgang et je vis à Berlin."
```

**visual\_question\_answering**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L393)

( image: typing.Union\[bytes, typing.BinaryIO, str, pathlib.Path]question: strmodel: typing.Optional\[str] = None ) → `List[Dict]`

Parameters

* **image** (`Union[str, Path, bytes, BinaryIO]`) — The input image for the context. It can be raw bytes, an image file, or a URL to an online image.
* **question** (`str`) — Question to be answered.
* **model** (`str`, *optional*) — The model to use for the visual question answering task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended visual question answering model will be used. Defaults to None.

Returns

`List[Dict]`

a list of dictionaries containing the predicted label and associated probability.

Raises

`InferenceTimeoutError` or `HTTPError`

* `InferenceTimeoutError` — If the model is unavailable or the request times out.
* `HTTPError` — If the request fails with an HTTP error status code other than HTTP 503.

Answering open-ended questions based on an image.

Example:

Copied

```
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()
>>> client.visual_question_answering(
...     image="https://huggingface.co/datasets/mishig/sample_images/resolve/main/tiger.jpg",
...     question="What is the animal doing?"
... )
[{'score': 0.778609573841095, 'answer': 'laying down'},{'score': 0.6957435607910156, 'answer': 'sitting'}, ...]
```

**zero\_shot\_classification**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L1734)

( text: strlabels: typing.List\[str]multi\_label: bool = Falsemodel: typing.Optional\[str] = None ) → `List[Dict]`

Parameters

* **text** (`str`) — The input text to classify.
* **labels** (`List[str]`) — List of string possible labels. There must be at least 2 labels.
* **multi\_label** (`bool`) — Boolean that is set to True if classes can overlap.
* **model** (`str`, *optional*) — The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None.

Returns

`List[Dict]`

List of classification outputs containing the predicted labels and their confidence.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `HTTPError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `HTTPError` — If the request fails with an HTTP error status code other than HTTP 503.

Provide as input a text and a set of candidate labels to classify the input text.

Example:

Copied

```
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()
>>> text = (
...     "A new model offers an explanation for how the Galilean satellites formed around the solar system's"
...     "largest world. Konstantin Batygin did not set out to solve one of the solar system's most puzzling"
...     " mysteries when he went for a run up a hill in Nice, France."
... )
>>> labels = ["space & cosmos", "scientific discovery", "microbiology", "robots", "archeology"]
>>> client.zero_shot_classification(text, labels)
[
    {"label": "scientific discovery", "score": 0.7961668968200684},
    {"label": "space & cosmos", "score": 0.18570658564567566},
    {"label": "microbiology", "score": 0.00730885099619627},
    {"label": "archeology", "score": 0.006258360575884581},
    {"label": "robots", "score": 0.004559356719255447},
]
>>> client.zero_shot_classification(text, labels, multi_label=True)
[
    {"label": "scientific discovery", "score": 0.9829297661781311},
    {"label": "space & cosmos", "score": 0.755190908908844},
    {"label": "microbiology", "score": 0.0005462635890580714},
    {"label": "archeology", "score": 0.00047131875180639327},
    {"label": "robots", "score": 0.00030448526376858354},
]
```

**zero\_shot\_image\_classification**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_client.py#L1806)

( image: typing.Union\[bytes, typing.BinaryIO, str, pathlib.Path]labels: typing.List\[str]model: typing.Optional\[str] = None ) → `List[Dict]`

Parameters

* **image** (`Union[str, Path, bytes, BinaryIO]`) — The input image to caption. It can be raw bytes, an image file, or a URL to an online image.
* **labels** (`List[str]`) — List of string possible labels. There must be at least 2 labels.
* **model** (`str`, *optional*) — The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None.

Returns

`List[Dict]`

List of classification outputs containing the predicted labels and their confidence.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `HTTPError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `HTTPError` — If the request fails with an HTTP error status code other than HTTP 503.

Provide input image and text labels to predict text labels for the image.

Example:

Copied

```
>>> from huggingface_hub import InferenceClient
>>> client = InferenceClient()

>>> client.zero_shot_image_classification(
...     "https://upload.wikimedia.org/wikipedia/commons/thumb/4/43/Cute_dog.jpg/320px-Cute_dog.jpg",
...     labels=["dog", "cat", "horse"],
... )
[{"label": "dog", "score": 0.956}, ...]
```

### Async Inference Client

An async version of the client is also provided, based on `asyncio` and `aiohttp`. To use it, you can either install `aiohttp` directly or use the `[inference]` extra:

Copied

```
pip install --upgrade huggingface_hub[inference]
# or
# pip install aiohttp
```

#### class huggingface\_hub.AsyncInferenceClient

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L89)

( model: typing.Optional\[str] = Nonetoken: typing.Union\[str, bool, NoneType] = Nonetimeout: typing.Optional\[float] = Noneheaders: typing.Union\[typing.Dict\[str, str], NoneType] = Nonecookies: typing.Union\[typing.Dict\[str, str], NoneType] = None )

Parameters

* **model** (`str`, `optional`) — The model to run inference with. Can be a model id hosted on the Hugging Face Hub, e.g. `bigcode/starcoder` or a URL to a deployed Inference Endpoint. Defaults to None, in which case a recommended model is automatically selected for the task.
* **token** (`str`, *optional*) — Hugging Face token. Will default to the locally saved token. Pass `token=False` if you don’t want to send your token to the server.
* **timeout** (`float`, `optional`) — The maximum number of seconds to wait for a response from the server. Loading a new model in Inference API can take up to several minutes. Defaults to None, meaning it will loop until the server is available.
* **headers** (`Dict[str, str]`, `optional`) — Additional headers to send to the server. By default only the authorization and user-agent headers are sent. Values in this dictionary will override the default values.
* **cookies** (`Dict[str, str]`, `optional`) — Additional cookies to send to the server.

Initialize a new Inference Client.

[InferenceClient](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceClient) aims to provide a unified experience to perform inference. The client can be used seamlessly with either the (free) Inference API or self-hosted Inference Endpoints.

**audio\_classification**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L255)

( audio: typing.Union\[bytes, typing.BinaryIO, str, pathlib.Path]model: typing.Optional\[str] = None ) → `List[Dict]`

Parameters

* **audio** (Union\[str, Path, bytes, BinaryIO]) — The audio content to classify. It can be raw audio bytes, a local audio file, or a URL pointing to an audio file.
* **model** (`str`, *optional*) — The model to use for audio classification. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended model for audio classification will be used.

Returns

`List[Dict]`

The classification output containing the predicted label and its confidence.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `aiohttp.ClientResponseError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `aiohttp.ClientResponseError` — If the request fails with an HTTP error status code other than HTTP 503.

Perform audio classification on the provided audio content.

Example:

Copied

```
# Must be run in an async context
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()
>>> await client.audio_classification("audio.flac")
[{'score': 0.4976358711719513, 'label': 'hap'}, {'score': 0.3677836060523987, 'label': 'neu'},...]
```

**automatic\_speech\_recognition**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L294)

( audio: typing.Union\[bytes, typing.BinaryIO, str, pathlib.Path]model: typing.Optional\[str] = None ) → str

Parameters

* **audio** (Union\[str, Path, bytes, BinaryIO]) — The content to transcribe. It can be raw audio bytes, local audio file, or a URL to an audio file.
* **model** (`str`, *optional*) — The model to use for ASR. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended model for ASR will be used.

Returns

str

The transcribed text.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `aiohttp.ClientResponseError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `aiohttp.ClientResponseError` — If the request fails with an HTTP error status code other than HTTP 503.

Perform automatic speech recognition (ASR or audio-to-text) on the given audio content.

Example:

Copied

```
# Must be run in an async context
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()
>>> await client.automatic_speech_recognition("hello_world.flac")
"hello world"
```

**conversational**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L331)

( text: strgenerated\_responses: typing.Optional\[typing.List\[str]] = Nonepast\_user\_inputs: typing.Optional\[typing.List\[str]] = Noneparameters: typing.Union\[typing.Dict\[str, typing.Any], NoneType] = Nonemodel: typing.Optional\[str] = None ) → `Dict`

Parameters

* **text** (`str`) — The last input from the user in the conversation.
* **generated\_responses** (`List[str]`, *optional*) — A list of strings corresponding to the earlier replies from the model. Defaults to None.
* **past\_user\_inputs** (`List[str]`, *optional*) — A list of strings corresponding to the earlier replies from the user. Should be the same length as `generated_responses`. Defaults to None.
* **parameters** (`Dict[str, Any]`, *optional*) — Additional parameters for the conversational task. Defaults to None. For more details about the available parameters, please refer to [this page](https://huggingface.co/docs/api-inference/detailed_parameters#conversational-task)
* **model** (`str`, *optional*) — The model to use for the conversational task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended conversational model will be used. Defaults to None.

Returns

`Dict`

The generated conversational output.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `aiohttp.ClientResponseError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `aiohttp.ClientResponseError` — If the request fails with an HTTP error status code other than HTTP 503.

Generate conversational responses based on the given input text (i.e. chat with the API).

Example:

Copied

```
# Must be run in an async context
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()
>>> output = await client.conversational("Hi, who are you?")
>>> output
{'generated_text': 'I am the one who knocks.', 'conversation': {'generated_responses': ['I am the one who knocks.'], 'past_user_inputs': ['Hi, who are you?']}, 'warnings': ['Setting `pad_token_id` to `eos_token_id`:50256 async for open-end generation.']}
>>> await client.conversational(
...     "Wow, that's scary!",
...     generated_responses=output["conversation"]["generated_responses"],
...     past_user_inputs=output["conversation"]["past_user_inputs"],
... )
```

**document\_question\_answering**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L438)

( image: typing.Union\[bytes, typing.BinaryIO, str, pathlib.Path]question: strmodel: typing.Optional\[str] = None ) → `List[Dict]`

Parameters

* **image** (`Union[str, Path, bytes, BinaryIO]`) — The input image for the context. It can be raw bytes, an image file, or a URL to an online image.
* **question** (`str`) — Question to be answered.
* **model** (`str`, *optional*) — The model to use for the document question answering task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended document question answering model will be used. Defaults to None.

Returns

`List[Dict]`

a list of dictionaries containing the predicted label, associated probability, word ids, and page number.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `aiohttp.ClientResponseError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `aiohttp.ClientResponseError` — If the request fails with an HTTP error status code other than HTTP 503.

Answer questions on document images.

Example:

Copied

```
# Must be run in an async context
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()
>>> await client.document_question_answering(image="https://huggingface.co/spaces/impira/docquery/resolve/2359223c1837a7587402bda0f2643382a6eefeab/invoice.png", question="What is the invoice number?")
[{'score': 0.42515629529953003, 'answer': 'us-001', 'start': 16, 'end': 16}]
```

**feature\_extraction**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L480)

( text: strmodel: typing.Optional\[str] = None ) → `np.ndarray`

Parameters

* **text** (`str`) — The text to embed.
* **model** (`str`, *optional*) — The model to use for the conversational task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended conversational model will be used. Defaults to None.

Returns

`np.ndarray`

The embedding representing the input text as a float32 numpy array.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `aiohttp.ClientResponseError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `aiohttp.ClientResponseError` — If the request fails with an HTTP error status code other than HTTP 503.

Generate embeddings for a given text.

Example:

Copied

```
# Must be run in an async context
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()
>>> await client.feature_extraction("Hi, who are you?")
array([[ 2.424802  ,  2.93384   ,  1.1750331 , ...,  1.240499, -0.13776633, -0.7889173 ],
[-0.42943227, -0.6364878 , -1.693462  , ...,  0.41978157, -2.4336355 ,  0.6162071 ],
...,
[ 0.28552425, -0.928395  , -1.2077185 , ...,  0.76810825, -2.1069427 ,  0.6236161 ]], dtype=float32)
```

**fill\_mask**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L517)

( text: strmodel: typing.Optional\[str] = None ) → `List[Dict]`

Parameters

* **text** (`str`) — a string to be filled from, must contain the \[MASK] token (check model card for exact name of the mask).
* **model** (`str`, *optional*) — The model to use for the fill mask task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended fill mask model will be used. Defaults to None.

Returns

`List[Dict]`

a list of fill mask output dictionaries containing the predicted label, associated probability, token reference, and completed text.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `aiohttp.ClientResponseError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `aiohttp.ClientResponseError` — If the request fails with an HTTP error status code other than HTTP 503.

Fill in a hole with a missing word (token to be precise).

Example:

Copied

```
# Must be run in an async context
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()
>>> await client.fill_mask("The goal of life is <mask>.")
[{'score': 0.06897063553333282,
'token': 11098,
'token_str': ' happiness',
'sequence': 'The goal of life is happiness.'},
{'score': 0.06554922461509705,
'token': 45075,
'token_str': ' immortality',
'sequence': 'The goal of life is immortality.'}]
```

**get\_model\_status**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L1908)

( model: typing.Optional\[str] = None ) → `ModelStatus`

Parameters

* **model** (`str`, *optional*) — Identifier of the model for witch the status gonna be checked. If model is not provided, the model associated with this instance of [InferenceClient](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceClient) will be used. Only InferenceAPI service can be checked so the identifier cannot be a URL.

Returns

`ModelStatus`

An instance of ModelStatus dataclass, containing information, about the state of the model: load, state, compute type and framework.

Get the status of a model hosted on the Inference API.

This endpoint is mostly useful when you already know which model you want to use and want to check its availability. If you want to discover already deployed models, you should rather use [list\_deployed\_models()](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceClient.list_deployed_models).

Example:

Copied

```
# Must be run in an async context
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()
>>> await client.get_model_status("bigcode/starcoder")
ModelStatus(loaded=True, state='Loaded', compute_type='gpu', framework='text-generation-inference')
```

**image\_classification**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L558)

( image: typing.Union\[bytes, typing.BinaryIO, str, pathlib.Path]model: typing.Optional\[str] = None ) → `List[Dict]`

Parameters

* **image** (`Union[str, Path, bytes, BinaryIO]`) — The image to classify. It can be raw bytes, an image file, or a URL to an online image.
* **model** (`str`, *optional*) — The model to use for image classification. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended model for image classification will be used.

Returns

`List[Dict]`

a list of dictionaries containing the predicted label and associated probability.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `aiohttp.ClientResponseError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `aiohttp.ClientResponseError` — If the request fails with an HTTP error status code other than HTTP 503.

Perform image classification on the given image using the specified model.

Example:

Copied

```
# Must be run in an async context
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()
>>> await client.image_classification("https://upload.wikimedia.org/wikipedia/commons/thumb/4/43/Cute_dog.jpg/320px-Cute_dog.jpg")
[{'score': 0.9779096841812134, 'label': 'Blenheim spaniel'}, ...]
```

**image\_segmentation**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L595)

( image: typing.Union\[bytes, typing.BinaryIO, str, pathlib.Path]model: typing.Optional\[str] = None ) → `List[Dict]`

Parameters

* **image** (`Union[str, Path, bytes, BinaryIO]`) — The image to segment. It can be raw bytes, an image file, or a URL to an online image.
* **model** (`str`, *optional*) — The model to use for image segmentation. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended model for image segmentation will be used.

Returns

`List[Dict]`

A list of dictionaries containing the segmented masks and associated attributes.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `aiohttp.ClientResponseError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `aiohttp.ClientResponseError` — If the request fails with an HTTP error status code other than HTTP 503.

Perform image segmentation on the given image using the specified model.

You must have `PIL` installed if you want to work with images (`pip install Pillow`).

Example:

Copied

```
# Must be run in an async context
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()
>>> await client.image_segmentation("cat.jpg"):
[{'score': 0.989008, 'label': 'LABEL_184', 'mask': <PIL.PngImagePlugin.PngImageFile image mode=L size=400x300 at 0x7FDD2B129CC0>}, ...]
```

**image\_to\_image**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L647)

( image: typing.Union\[bytes, typing.BinaryIO, str, pathlib.Path]prompt: typing.Optional\[str] = Nonenegative\_prompt: typing.Optional\[str] = Noneheight: typing.Optional\[int] = Nonewidth: typing.Optional\[int] = Nonenum\_inference\_steps: typing.Optional\[int] = Noneguidance\_scale: typing.Optional\[float] = Nonemodel: typing.Optional\[str] = None\*\*kwargs ) → `Image`

Parameters

* **image** (`Union[str, Path, bytes, BinaryIO]`) — The input image for translation. It can be raw bytes, an image file, or a URL to an online image.
* **prompt** (`str`, *optional*) — The text prompt to guide the image generation.
* **negative\_prompt** (`str`, *optional*) — A negative prompt to guide the translation process.
* **height** (`int`, *optional*) — The height in pixels of the generated image.
* **width** (`int`, *optional*) — The width in pixels of the generated image.
* **num\_inference\_steps** (`int`, *optional*) — The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference.
* **guidance\_scale** (`float`, *optional*) — Higher guidance scale encourages to generate images that are closely linked to the text `prompt`, usually at the expense of lower image quality.
* **model** (`str`, *optional*) — The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None.

Returns

`Image`

The translated image.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `aiohttp.ClientResponseError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `aiohttp.ClientResponseError` — If the request fails with an HTTP error status code other than HTTP 503.

Perform image-to-image translation using a specified model.

You must have `PIL` installed if you want to work with images (`pip install Pillow`).

Example:

Copied

```
# Must be run in an async context
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()
>>> image = await client.image_to_image("cat.jpg", prompt="turn the cat into a tiger")
>>> image.save("tiger.jpg")
```

**image\_to\_text**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L732)

( image: typing.Union\[bytes, typing.BinaryIO, str, pathlib.Path]model: typing.Optional\[str] = None ) → `str`

Parameters

* **image** (`Union[str, Path, bytes, BinaryIO]`) — The input image to caption. It can be raw bytes, an image file, or a URL to an online image..
* **model** (`str`, *optional*) — The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None.

Returns

`str`

The generated text.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `aiohttp.ClientResponseError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `aiohttp.ClientResponseError` — If the request fails with an HTTP error status code other than HTTP 503.

Takes an input image and return text.

Models can have very different outputs depending on your use case (image captioning, optical character recognition (OCR), Pix2Struct, etc). Please have a look to the model card to learn more about a model’s specificities.

Example:

Copied

```
# Must be run in an async context
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()
>>> await client.image_to_text("cat.jpg")
'a cat standing in a grassy field '
>>> await client.image_to_text("https://upload.wikimedia.org/wikipedia/commons/thumb/4/43/Cute_dog.jpg/320px-Cute_dog.jpg")
'a dog laying on the grass next to a flower pot '
```

**list\_deployed\_models**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L769)

( frameworks: typing.Union\[NoneType, str, typing.Literal\['all'], typing.List\[str]] = None ) → `Dict[str, List[str]]`

Parameters

* **frameworks** (`Literal["all"]` or `List[str]` or `str`, *optional*) — The frameworks to filter on. By default only a subset of the available frameworks are tested. If set to “all”, all available frameworks will be tested. It is also possible to provide a single framework or a custom set of frameworks to check.

Returns

`Dict[str, List[str]]`

A dictionary mapping task names to a sorted list of model IDs.

List models currently deployed on the Inference API service.

This helper checks deployed models framework by framework. By default, it will check the 4 main frameworks that are supported and account for 95% of the hosted models. However, if you want a complete list of models you can specify `frameworks="all"` as input. Alternatively, if you know before-hand which framework you are interested in, you can also restrict to search to this one (e.g. `frameworks="text-generation-inference"`). The more frameworks are checked, the more time it will take.

This endpoint is mostly useful for discoverability. If you already know which model you want to use and want to check its availability, you can directly use [get\_model\_status()](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceClient.get_model_status).

Example:

Copied

```
# Must be run in an async contextthon
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()

# Discover zero-shot-classification models currently deployed
>>> models = await client.list_deployed_models()
>>> models["zero-shot-classification"]
['Narsil/deberta-large-mnli-zero-cls', 'facebook/bart-large-mnli', ...]

# List from only 1 framework
>>> await client.list_deployed_models("text-generation-inference")
{'text-generation': ['bigcode/starcoder', 'meta-llama/Llama-2-70b-chat-hf', ...], ...}
```

**object\_detection**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L850)

( image: typing.Union\[bytes, typing.BinaryIO, str, pathlib.Path]model: typing.Optional\[str] = None ) → `List[ObjectDetectionOutput]`

Parameters

* **image** (`Union[str, Path, bytes, BinaryIO]`) — The image to detect objects on. It can be raw bytes, an image file, or a URL to an online image.
* **model** (`str`, *optional*) — The model to use for object detection. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended model for object detection (DETR) will be used.

Returns

`List[ObjectDetectionOutput]`

A list of dictionaries containing the bounding boxes and associated attributes.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `aiohttp.ClientResponseError` or `ValueError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `aiohttp.ClientResponseError` — If the request fails with an HTTP error status code other than HTTP 503.
* `ValueError` — If the request output is not a List.

Perform object detection on the given image using the specified model.

You must have `PIL` installed if you want to work with images (`pip install Pillow`).

Example:

Copied

```
# Must be run in an async context
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()
>>> await client.object_detection("people.jpg"):
[{"score":0.9486683011054993,"label":"person","box":{"xmin":59,"ymin":39,"xmax":420,"ymax":510}}, ... ]
```

**post**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L156)

( json: typing.Union\[str, typing.Dict, typing.List, NoneType] = Nonedata: typing.Union\[bytes, typing.BinaryIO, str, pathlib.Path, NoneType] = Nonemodel: typing.Optional\[str] = Nonetask: typing.Optional\[str] = Nonestream: bool = False ) → bytes

Parameters

* **json** (`Union[str, Dict, List]`, *optional*) — The JSON data to send in the request body. Defaults to None.
* **data** (`Union[str, Path, bytes, BinaryIO]`, *optional*) — The content to send in the request body. It can be raw bytes, a pointer to an opened file, a local file path, or a URL to an online resource (image, audio file,…). If both `json` and `data` are passed, `data` will take precedence. At least `json` or `data` must be provided. Defaults to None.
* **model** (`str`, *optional*) — The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. Will override the model defined at the instance level. Defaults to None.
* **task** (`str`, *optional*) — The task to perform on the inference. Used only to default to a recommended model if `model` is not provided. At least `model` or `task` must be provided. Defaults to None.
* **stream** (`bool`, *optional*) — Whether to iterate over streaming APIs.

Returns

bytes

The raw bytes returned by the server.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `aiohttp.ClientResponseError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `aiohttp.ClientResponseError` — If the request fails with an HTTP error status code other than HTTP 503.

Make a POST request to the inference server.

**question\_answering**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L899)

( question: strcontext: strmodel: typing.Optional\[str] = None ) → `Dict`

Parameters

* **question** (`str`) — Question to be answered.
* **context** (`str`) — The context of the question.
* **model** (`str`) — The model to use for the question answering task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint.

Returns

`Dict`

a dictionary of question answering output containing the score, start index, end index, and answer.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `aiohttp.ClientResponseError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `aiohttp.ClientResponseError` — If the request fails with an HTTP error status code other than HTTP 503.

Retrieve the answer to a question from a given text.

Example:

Copied

```
# Must be run in an async context
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()
>>> await client.question_answering(question="What's my name?", context="My name is Clara and I live in Berkeley.")
{'score': 0.9326562285423279, 'start': 11, 'end': 16, 'answer': 'Clara'}
```

**sentence\_similarity**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L941)

( sentence: strother\_sentences: typing.List\[str]model: typing.Optional\[str] = None ) → `List[float]`

Parameters

* **sentence** (`str`) — The main sentence to compare to others.
* **other\_sentences** (`List[str]`) — The list of sentences to compare to.
* **model** (`str`, *optional*) — The model to use for the conversational task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended conversational model will be used. Defaults to None.

Returns

`List[float]`

The embedding representing the input text.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `aiohttp.ClientResponseError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `aiohttp.ClientResponseError` — If the request fails with an HTTP error status code other than HTTP 503.

Compute the semantic similarity between a sentence and a list of other sentences by comparing their embeddings.

Example:

Copied

```
# Must be run in an async context
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()
>>> await client.sentence_similarity(
...     "Machine learning is so easy.",
...     other_sentences=[
...         "Deep learning is so straightforward.",
...         "This is so difficult, like rocket science.",
...         "I can't believe how much I struggled with this.",
...     ],
... )
[0.7785726189613342, 0.45876261591911316, 0.2906220555305481]
```

**summarization**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L989)

( text: strparameters: typing.Union\[typing.Dict\[str, typing.Any], NoneType] = Nonemodel: typing.Optional\[str] = None ) → `str`

Parameters

* **text** (`str`) — The input text to summarize.
* **parameters** (`Dict[str, Any]`, *optional*) — Additional parameters for summarization. Check out this [page](https://huggingface.co/docs/api-inference/detailed_parameters#summarization-task) for more details.
* **model** (`str`, *optional*) — The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None.

Returns

`str`

The generated summary text.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `aiohttp.ClientResponseError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `aiohttp.ClientResponseError` — If the request fails with an HTTP error status code other than HTTP 503.

Generate a summary of a given text using a specified model.

Example:

Copied

```
# Must be run in an async context
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()
>>> await client.summarization("The Eiffel tower...")
'The Eiffel tower is one of the most famous landmarks in the world....'
```

**table\_question\_answering**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L1033)

( table: typing.Dict\[str, typing.Any]query: strmodel: typing.Optional\[str] = None ) → `Dict`

Parameters

* **table** (`str`) — A table of data represented as a dict of lists where entries are headers and the lists are all the values, all lists must have the same size.
* **query** (`str`) — The query in plain text that you want to ask the table.
* **model** (`str`) — The model to use for the table-question-answering task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint.

Returns

`Dict`

a dictionary of table question answering output containing the answer, coordinates, cells and the aggregator used.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `aiohttp.ClientResponseError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `aiohttp.ClientResponseError` — If the request fails with an HTTP error status code other than HTTP 503.

Retrieve the answer to a question from information given in a table.

Example:

Copied

```
# Must be run in an async context
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()
>>> query = "How many stars does the transformers repository have?"
>>> table = {"Repository": ["Transformers", "Datasets", "Tokenizers"], "Stars": ["36542", "4512", "3934"]}
>>> await client.table_question_answering(table, query, model="google/tapas-base-finetuned-wtq")
{'answer': 'AVERAGE > 36542', 'coordinates': [[0, 1]], 'cells': ['36542'], 'aggregator': 'AVERAGE'}
```

**tabular\_classification**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L1079)

( table: typing.Dict\[str, typing.Any]model: str ) → `List`

Parameters

* **table** (`Dict[str, Any]`) — Set of attributes to classify.
* **model** (`str`) — The model to use for the tabular-classification task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint.

Returns

`List`

a list of labels, one per row in the initial table.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `aiohttp.ClientResponseError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `aiohttp.ClientResponseError` — If the request fails with an HTTP error status code other than HTTP 503.

Classifying a target category (a group) based on a set of attributes.

Example:

Copied

```
# Must be run in an async context
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()
>>> table = {
...     "fixed_acidity": ["7.4", "7.8", "10.3"],
...     "volatile_acidity": ["0.7", "0.88", "0.32"],
...     "citric_acid": ["0", "0", "0.45"],
...     "residual_sugar": ["1.9", "2.6", "6.4"],
...     "chlorides": ["0.076", "0.098", "0.073"],
...     "free_sulfur_dioxide": ["11", "25", "5"],
...     "total_sulfur_dioxide": ["34", "67", "13"],
...     "density": ["0.9978", "0.9968", "0.9976"],
...     "pH": ["3.51", "3.2", "3.23"],
...     "sulphates": ["0.56", "0.68", "0.82"],
...     "alcohol": ["9.4", "9.8", "12.6"],
... }
>>> await client.tabular_classification(table=table, model="julien-c/wine-quality")
["5", "5", "5"]
```

**tabular\_regression**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L1124)

( table: typing.Dict\[str, typing.Any]model: str ) → `List`

Parameters

* **table** (`Dict[str, Any]`) — Set of attributes stored in a table. The attributes used to predict the target can be both numerical and categorical.
* **model** (`str`) — The model to use for the tabular-regression task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint.

Returns

`List`

a list of predicted numerical target values.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `aiohttp.ClientResponseError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `aiohttp.ClientResponseError` — If the request fails with an HTTP error status code other than HTTP 503.

Predicting a numerical target value given a set of attributes/features in a table.

Example:

Copied

```
# Must be run in an async context
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()
>>> table = {
...     "Height": ["11.52", "12.48", "12.3778"],
...     "Length1": ["23.2", "24", "23.9"],
...     "Length2": ["25.4", "26.3", "26.5"],
...     "Length3": ["30", "31.2", "31.1"],
...     "Species": ["Bream", "Bream", "Bream"],
...     "Width": ["4.02", "4.3056", "4.6961"],
... }
>>> await client.tabular_regression(table, model="scikit-learn/Fish-Weight")
[110, 120, 130]
```

**text\_classification**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L1164)

( text: strmodel: typing.Optional\[str] = None ) → `List[Dict]`

Parameters

* **text** (`str`) — A string to be classified.
* **model** (`str`, *optional*) — The model to use for the text classification task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended text classification model will be used. Defaults to None.

Returns

`List[Dict]`

a list of dictionaries containing the predicted label and associated probability.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `aiohttp.ClientResponseError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `aiohttp.ClientResponseError` — If the request fails with an HTTP error status code other than HTTP 503.

Perform text classification (e.g. sentiment-analysis) on the given text.

Example:

Copied

```
# Must be run in an async context
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()
>>> await client.text_classification("I like you")
[{'label': 'POSITIVE', 'score': 0.9998695850372314}, {'label': 'NEGATIVE', 'score': 0.0001304351753788069}]
```

**text\_generation**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L1293)

( prompt: strdetails: bool = Falsestream: bool = Falsemodel: typing.Optional\[str] = Nonedo\_sample: bool = Falsemax\_new\_tokens: int = 20best\_of: typing.Optional\[int] = Nonerepetition\_penalty: typing.Optional\[float] = Nonereturn\_full\_text: bool = Falseseed: typing.Optional\[int] = Nonestop\_sequences: typing.Optional\[typing.List\[str]] = Nonetemperature: typing.Optional\[float] = Nonetop\_k: typing.Optional\[int] = Nonetop\_p: typing.Optional\[float] = Nonetruncate: typing.Optional\[int] = Nonetypical\_p: typing.Optional\[float] = Nonewatermark: bool = Falsedecoder\_input\_details: bool = False ) → `Union[str, TextGenerationResponse, Iterable[str], Iterable[TextGenerationStreamResponse]]`

Parameters

* **prompt** (`str`) — Input text.
* **details** (`bool`, *optional*) — By default, text\_generation returns a string. Pass `details=True` if you want a detailed output (tokens, probabilities, seed, finish reason, etc.). Only available for models running on with the `text-generation-inference` backend.
* **stream** (`bool`, *optional*) — By default, text\_generation returns the full generated text. Pass `stream=True` if you want a stream of tokens to be returned. Only available for models running on with the `text-generation-inference` backend.
* **model** (`str`, *optional*) — The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None.
* **do\_sample** (`bool`) — Activate logits sampling
* **max\_new\_tokens** (`int`) — Maximum number of generated tokens
* **best\_of** (`int`) — Generate best\_of sequences and return the one if the highest token logprobs
* **repetition\_penalty** (`float`) — The parameter for repetition penalty. 1.0 means no penalty. See [this paper](https://arxiv.org/pdf/1909.05858.pdf) for more details.
* **return\_full\_text** (`bool`) — Whether to prepend the prompt to the generated text
* **seed** (`int`) — Random sampling seed
* **stop\_sequences** (`List[str]`) — Stop generating tokens if a member of `stop_sequences` is generated
* **temperature** (`float`) — The value used to module the logits distribution.
* **top\_k** (`int`) — The number of highest probability vocabulary tokens to keep for top-k-filtering.
* **top\_p** (`float`) — If set to < 1, only the smallest set of most probable tokens with probabilities that add up to `top_p` or higher are kept for generation.
* **truncate** (`int`) — Truncate inputs tokens to the given size
* **typical\_p** (`float`) — Typical Decoding mass See [Typical Decoding for Natural Language Generation](https://arxiv.org/abs/2202.00666) for more information
* **watermark** (`bool`) — Watermarking with [A Watermark for Large Language Models](https://arxiv.org/abs/2301.10226)
* **decoder\_input\_details** (`bool`) — Return the decoder input token logprobs and ids. You must set `details=True` as well for it to be taken into account. Defaults to `False`.

Returns

`Union[str, TextGenerationResponse, Iterable[str], Iterable[TextGenerationStreamResponse]]`

Generated text returned from the server:

* if `stream=False` and `details=False`, the generated text is returned as a `str` (default)
* if `stream=True` and `details=False`, the generated text is returned token by token as a `Iterable[str]`
* if `stream=False` and `details=True`, the generated text is returned with more details as a [TextGenerationResponse](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.inference._text_generation.TextGenerationResponse)
* if `details=True` and `stream=True`, the generated text is returned token by token as a iterable of [TextGenerationStreamResponse](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.inference._text_generation.TextGenerationStreamResponse)

Raises

`ValidationError` or [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `aiohttp.ClientResponseError`

* `ValidationError` — If input values are not valid. No HTTP call is made to the server.
* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `aiohttp.ClientResponseError` — If the request fails with an HTTP error status code other than HTTP 503.

Given a prompt, generate the following text.

It is recommended to have Pydantic installed in order to get inputs validated. This is preferable as it allow early failures.

API endpoint is supposed to run with the `text-generation-inference` backend (TGI). This backend is the go-to solution to run large language models at scale. However, for some smaller models (e.g. “gpt2”) the default `transformers` + `api-inference` solution is still in use. Both approaches have very similar APIs, but not exactly the same. This method is compatible with both approaches but some parameters are only available for `text-generation-inference`. If some parameters are ignored, a warning message is triggered but the process continues correctly.

To learn more about the TGI project, please refer to <https://github.com/huggingface/text-generation-inference>.

Example:

Copied

```
# Must be run in an async context
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()

# Case 1: generate text
>>> await client.text_generation("The huggingface_hub library is ", max_new_tokens=12)
'100% open source and built to be easy to use.'

# Case 2: iterate over the generated tokens. Useful async for large generation.
>>> async for token in await client.text_generation("The huggingface_hub library is ", max_new_tokens=12, stream=True):
...     print(token)
100
%
open
source
and
built
to
be
easy
to
use
.

# Case 3: get more details about the generation process.
>>> await client.text_generation("The huggingface_hub library is ", max_new_tokens=12, details=True)
TextGenerationResponse(
    generated_text='100% open source and built to be easy to use.',
    details=Details(
        finish_reason=<FinishReason.Length: 'length'>,
        generated_tokens=12,
        seed=None,
        prefill=[
            InputToken(id=487, text='The', logprob=None),
            InputToken(id=53789, text=' hugging', logprob=-13.171875),
            (...)
            InputToken(id=204, text=' ', logprob=-7.0390625)
        ],
        tokens=[
            Token(id=1425, text='100', logprob=-1.0175781, special=False),
            Token(id=16, text='%', logprob=-0.0463562, special=False),
            (...)
            Token(id=25, text='.', logprob=-0.5703125, special=False)
        ],
        best_of_sequences=None
    )
)

# Case 4: iterate over the generated tokens with more details.
# Last object is more complete, containing the full generated text and the finish reason.
>>> async for details in await client.text_generation("The huggingface_hub library is ", max_new_tokens=12, details=True, stream=True):
...     print(details)
...
TextGenerationStreamResponse(token=Token(id=1425, text='100', logprob=-1.0175781, special=False), generated_text=None, details=None)
TextGenerationStreamResponse(token=Token(id=16, text='%', logprob=-0.0463562, special=False), generated_text=None, details=None)
TextGenerationStreamResponse(token=Token(id=1314, text=' open', logprob=-1.3359375, special=False), generated_text=None, details=None)
TextGenerationStreamResponse(token=Token(id=3178, text=' source', logprob=-0.28100586, special=False), generated_text=None, details=None)
TextGenerationStreamResponse(token=Token(id=273, text=' and', logprob=-0.5961914, special=False), generated_text=None, details=None)
TextGenerationStreamResponse(token=Token(id=3426, text=' built', logprob=-1.9423828, special=False), generated_text=None, details=None)
TextGenerationStreamResponse(token=Token(id=271, text=' to', logprob=-1.4121094, special=False), generated_text=None, details=None)
TextGenerationStreamResponse(token=Token(id=314, text=' be', logprob=-1.5224609, special=False), generated_text=None, details=None)
TextGenerationStreamResponse(token=Token(id=1833, text=' easy', logprob=-2.1132812, special=False), generated_text=None, details=None)
TextGenerationStreamResponse(token=Token(id=271, text=' to', logprob=-0.08520508, special=False), generated_text=None, details=None)
TextGenerationStreamResponse(token=Token(id=745, text=' use', logprob=-0.39453125, special=False), generated_text=None, details=None)
TextGenerationStreamResponse(token=Token(
    id=25,
    text='.',
    logprob=-0.5703125,
    special=False),
    generated_text='100% open source and built to be easy to use.',
    details=StreamDetails(finish_reason=<FinishReason.Length: 'length'>, generated_tokens=12, seed=None)
)
```

**text\_to\_image**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L1562)

( prompt: strnegative\_prompt: typing.Optional\[str] = Noneheight: typing.Optional\[float] = Nonewidth: typing.Optional\[float] = Nonenum\_inference\_steps: typing.Optional\[float] = Noneguidance\_scale: typing.Optional\[float] = Nonemodel: typing.Optional\[str] = None\*\*kwargs ) → `Image`

Parameters

* **prompt** (`str`) — The prompt to generate an image from.
* **negative\_prompt** (`str`, *optional*) — An optional negative prompt for the image generation.
* **height** (`float`, *optional*) — The height in pixels of the image to generate.
* **width** (`float`, *optional*) — The width in pixels of the image to generate.
* **num\_inference\_steps** (`int`, *optional*) — The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference.
* **guidance\_scale** (`float`, *optional*) — Higher guidance scale encourages to generate images that are closely linked to the text `prompt`, usually at the expense of lower image quality.
* **model** (`str`, *optional*) — The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None.

Returns

`Image`

The generated image.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `aiohttp.ClientResponseError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `aiohttp.ClientResponseError` — If the request fails with an HTTP error status code other than HTTP 503.

Generate an image based on a given text using a specified model.

You must have `PIL` installed if you want to work with images (`pip install Pillow`).

Example:

Copied

```
# Must be run in an async context
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()

>>> image = await client.text_to_image("An astronaut riding a horse on the moon.")
>>> image.save("astronaut.png")

>>> image = await client.text_to_image(
...     "An astronaut riding a horse on the moon.",
...     negative_prompt="low resolution, blurry",
...     model="stabilityai/stable-diffusion-2-1",
... )
>>> image.save("better_astronaut.png")
```

**text\_to\_speech**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L1643)

( text: strmodel: typing.Optional\[str] = None ) → `bytes`

Parameters

* **text** (`str`) — The text to synthesize.
* **model** (`str`, *optional*) — The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None.

Returns

`bytes`

The generated audio.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `aiohttp.ClientResponseError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `aiohttp.ClientResponseError` — If the request fails with an HTTP error status code other than HTTP 503.

Synthesize an audio of a voice pronouncing a given text.

Example:

Copied

```
# Must be run in an async context
>>> from pathlib import Path
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()

>>> audio = await client.text_to_speech("Hello world")
>>> Path("hello_world.flac").write_bytes(audio)
```

**token\_classification**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L1676)

( text: strmodel: typing.Optional\[str] = None ) → `List[Dict]`

Parameters

* **text** (`str`) — A string to be classified.
* **model** (`str`, *optional*) — The model to use for the token classification task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended token classification model will be used. Defaults to None.

Returns

`List[Dict]`

List of token classification outputs containing the entity group, confidence score, word, start and end index.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `aiohttp.ClientResponseError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `aiohttp.ClientResponseError` — If the request fails with an HTTP error status code other than HTTP 503.

Perform token classification on the given text. Usually used for sentence parsing, either grammatical, or Named Entity Recognition (NER) to understand keywords contained within text.

Example:

Copied

```
# Must be run in an async context
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()
>>> await client.token_classification("My name is Sarah Jessica Parker but you can call me Jessica")
[{'entity_group': 'PER',
'score': 0.9971321225166321,
'word': 'Sarah Jessica Parker',
'start': 11,
'end': 31},
{'entity_group': 'PER',
'score': 0.9773476123809814,
'word': 'Jessica',
'start': 52,
'end': 59}]
```

**translation**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L1724)

( text: strmodel: typing.Optional\[str] = None ) → `str`

Parameters

* **text** (`str`) — A string to be translated.
* **model** (`str`, *optional*) — The model to use for the translation task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended translation model will be used. Defaults to None.

Returns

`str`

The generated translated text.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `aiohttp.ClientResponseError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `aiohttp.ClientResponseError` — If the request fails with an HTTP error status code other than HTTP 503.

Convert text from one language to another.

Check out <https://huggingface.co/tasks/translation> for more information on how to choose the best model for your specific use case. Source and target languages usually depends on the model.

Example:

Copied

```
# Must be run in an async context
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()
>>> await client.translation("My name is Wolfgang and I live in Berlin")
'Mein Name ist Wolfgang und ich lebe in Berlin.'
>>> await client.translation("My name is Wolfgang and I live in Berlin", model="Helsinki-NLP/opus-mt-en-fr")
"Je m'appelle Wolfgang et je vis à Berlin."
```

**visual\_question\_answering**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L393)

( image: typing.Union\[bytes, typing.BinaryIO, str, pathlib.Path]question: strmodel: typing.Optional\[str] = None ) → `List[Dict]`

Parameters

* **image** (`Union[str, Path, bytes, BinaryIO]`) — The input image for the context. It can be raw bytes, an image file, or a URL to an online image.
* **question** (`str`) — Question to be answered.
* **model** (`str`, *optional*) — The model to use for the visual question answering task. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. If not provided, the default recommended visual question answering model will be used. Defaults to None.

Returns

`List[Dict]`

a list of dictionaries containing the predicted label and associated probability.

Raises

`InferenceTimeoutError` or `aiohttp.ClientResponseError`

* `InferenceTimeoutError` — If the model is unavailable or the request times out.
* `aiohttp.ClientResponseError` — If the request fails with an HTTP error status code other than HTTP 503.

Answering open-ended questions based on an image.

Example:

Copied

```
# Must be run in an async context
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()
>>> await client.visual_question_answering(
...     image="https://huggingface.co/datasets/mishig/sample_images/resolve/main/tiger.jpg",
...     question="What is the animal doing?"
... )
[{'score': 0.778609573841095, 'answer': 'laying down'},{'score': 0.6957435607910156, 'answer': 'sitting'}, ...]
```

**zero\_shot\_classification**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L1762)

( text: strlabels: typing.List\[str]multi\_label: bool = Falsemodel: typing.Optional\[str] = None ) → `List[Dict]`

Parameters

* **text** (`str`) — The input text to classify.
* **labels** (`List[str]`) — List of string possible labels. There must be at least 2 labels.
* **multi\_label** (`bool`) — Boolean that is set to True if classes can overlap.
* **model** (`str`, *optional*) — The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None.

Returns

`List[Dict]`

List of classification outputs containing the predicted labels and their confidence.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `aiohttp.ClientResponseError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `aiohttp.ClientResponseError` — If the request fails with an HTTP error status code other than HTTP 503.

Provide as input a text and a set of candidate labels to classify the input text.

Example:

Copied

```
# Must be run in an async context
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()
>>> text = (
...     "A new model offers an explanation async for how the Galilean satellites formed around the solar system's"
...     "largest world. Konstantin Batygin did not set out to solve one of the solar system's most puzzling"
...     " mysteries when he went async for a run up a hill in Nice, France."
... )
>>> labels = ["space & cosmos", "scientific discovery", "microbiology", "robots", "archeology"]
>>> await client.zero_shot_classification(text, labels)
[
    {"label": "scientific discovery", "score": 0.7961668968200684},
    {"label": "space & cosmos", "score": 0.18570658564567566},
    {"label": "microbiology", "score": 0.00730885099619627},
    {"label": "archeology", "score": 0.006258360575884581},
    {"label": "robots", "score": 0.004559356719255447},
]
>>> await client.zero_shot_classification(text, labels, multi_label=True)
[
    {"label": "scientific discovery", "score": 0.9829297661781311},
    {"label": "space & cosmos", "score": 0.755190908908844},
    {"label": "microbiology", "score": 0.0005462635890580714},
    {"label": "archeology", "score": 0.00047131875180639327},
    {"label": "robots", "score": 0.00030448526376858354},
]
```

**zero\_shot\_image\_classification**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_generated/_async_client.py#L1835)

( image: typing.Union\[bytes, typing.BinaryIO, str, pathlib.Path]labels: typing.List\[str]model: typing.Optional\[str] = None ) → `List[Dict]`

Parameters

* **image** (`Union[str, Path, bytes, BinaryIO]`) — The input image to caption. It can be raw bytes, an image file, or a URL to an online image.
* **labels** (`List[str]`) — List of string possible labels. There must be at least 2 labels.
* **model** (`str`, *optional*) — The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None.

Returns

`List[Dict]`

List of classification outputs containing the predicted labels and their confidence.

Raises

[InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) or `aiohttp.ClientResponseError`

* [InferenceTimeoutError](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceTimeoutError) — If the model is unavailable or the request times out.
* `aiohttp.ClientResponseError` — If the request fails with an HTTP error status code other than HTTP 503.

Provide input image and text labels to predict text labels for the image.

Example:

Copied

```
# Must be run in an async context
>>> from huggingface_hub import AsyncInferenceClient
>>> client = AsyncInferenceClient()

>>> await client.zero_shot_image_classification(
...     "https://upload.wikimedia.org/wikipedia/commons/thumb/4/43/Cute_dog.jpg/320px-Cute_dog.jpg",
...     labels=["dog", "cat", "horse"],
... )
[{"label": "dog", "score": 0.956}, ...]
```

#### InferenceTimeoutError

#### class huggingface\_hub.InferenceTimeoutError

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_common.py#L94)

( \*args\*\*kwargs )

Error raised when a model is unavailable or the request times out.

#### Return types

For most tasks, the return value has a built-in type (string, list, image…). Here is a list for the more complex types.

#### class huggingface\_hub.inference.\_types.ClassificationOutput

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_types.py#L22)

( \*args\*\*kwargs )

Parameters

* **label** (`str`) — The label predicted by the model.
* **score** (`float`) — The score of the label predicted by the model.

Dictionary containing the output of a [audio\_classification()](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceClient.audio_classification) and [image\_classification()](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceClient.image_classification) task.

#### class huggingface\_hub.inference.\_types.ConversationalOutputConversation

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_types.py#L36)

( \*args\*\*kwargs )

Parameters

* **generated\_responses** (`List[str]`) — A list of the responses from the model.
* **past\_user\_inputs** (`List[str]`) — A list of the inputs from the user. Must be the same length as `generated_responses`.

Dictionary containing the “conversation” part of a [conversational()](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceClient.conversational) task.

#### class huggingface\_hub.inference.\_types.ConversationalOutput

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_types.py#L50)

( \*args\*\*kwargs )

Parameters

* **generated\_text** (`str`) — The last response from the model.
* **conversation** (`ConversationalOutputConversation`) — The past conversation.
* **warnings** (`List[str]`) — A list of warnings associated with the process.

Dictionary containing the output of a [conversational()](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceClient.conversational) task.

#### class huggingface\_hub.inference.\_types.ImageSegmentationOutput

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_types.py#L87)

( \*args\*\*kwargs )

Parameters

* **label** (`str`) — The label corresponding to the mask.
* **mask** (`Image`) — An Image object representing the mask predicted by the model.
* **score** (`float`) — The score associated with the label for this mask.

Dictionary containing information about a [image\_segmentation()](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceClient.image_segmentation) task. In practice, image segmentation returns a list of `ImageSegmentationOutput` with 1 item per mask.

#### class huggingface\_hub.inference.\_types.TokenClassificationOutput

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_types.py#L163)

( \*args\*\*kwargs )

Parameters

* **entity\_group** (`str`) — The type for the entity being recognized (model specific).
* **score** (`float`) — The score of the label predicted by the model.
* **word** (`str`) — The string that was captured.
* **start** (`int`) — The offset stringwise where the answer is located. Useful to disambiguate if word occurs multiple times.
* **end** (`int`) — The offset stringwise where the answer is located. Useful to disambiguate if word occurs multiple times.

Dictionary containing the output of a [token\_classification()](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceClient.token_classification) task.

#### Text generation types

[text\_generation()](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceClient.text_generation) task has a greater support than other tasks in `InferenceClient`. In particular, user inputs and server outputs are validated using [Pydantic](https://docs.pydantic.dev/latest/) if this package is installed. Therefore, we recommend installing it (`pip install pydantic`) for a better user experience.

You can find below the dataclasses used to validate data and in particular [TextGenerationParameters](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.inference._text_generation.TextGenerationParameters) (input), [TextGenerationResponse](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.inference._text_generation.TextGenerationResponse) (output) and [TextGenerationStreamResponse](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.inference._text_generation.TextGenerationStreamResponse) (streaming output).

#### class huggingface\_hub.inference.\_text\_generation.TextGenerationParameters

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_text_generation.py#L47)

( do\_sample: bool = Falsemax\_new\_tokens: int = 20repetition\_penalty: typing.Optional\[float] = Nonereturn\_full\_text: bool = Falsestop: typing.List\[str] = \<factory>seed: typing.Optional\[int] = Nonetemperature: typing.Optional\[float] = Nonetop\_k: typing.Optional\[int] = Nonetop\_p: typing.Optional\[float] = Nonetruncate: typing.Optional\[int] = Nonetypical\_p: typing.Optional\[float] = Nonebest\_of: typing.Optional\[int] = Nonewatermark: bool = Falsedetails: bool = Falsedecoder\_input\_details: bool = False )

Parameters

* **do\_sample** (`bool`, *optional*) — Activate logits sampling. Defaults to False.
* **max\_new\_tokens** (`int`, *optional*) — Maximum number of generated tokens. Defaults to 20.
* **repetition\_penalty** (`Optional[float]`, *optional*) — The parameter for repetition penalty. A value of 1.0 means no penalty. See [this paper](https://arxiv.org/pdf/1909.05858.pdf) for more details. Defaults to None.
* **return\_full\_text** (`bool`, *optional*) — Whether to prepend the prompt to the generated text. Defaults to False.
* **stop** (`List[str]`, *optional*) — Stop generating tokens if a member of `stop_sequences` is generated. Defaults to an empty list.
* **seed** (`Optional[int]`, *optional*) — Random sampling seed. Defaults to None.
* **temperature** (`Optional[float]`, *optional*) — The value used to modulate the logits distribution. Defaults to None.
* **top\_k** (`Optional[int]`, *optional*) — The number of highest probability vocabulary tokens to keep for top-k-filtering. Defaults to None.
* **top\_p** (`Optional[float]`, *optional*) — If set to a value less than 1, only the smallest set of most probable tokens with probabilities that add up to `top_p` or higher are kept for generation. Defaults to None.
* **truncate** (`Optional[int]`, *optional*) — Truncate input tokens to the given size. Defaults to None.
* **typical\_p** (`Optional[float]`, *optional*) — Typical Decoding mass. See [Typical Decoding for Natural Language Generation](https://arxiv.org/abs/2202.00666) for more information. Defaults to None.
* **best\_of** (`Optional[int]`, *optional*) — Generate `best_of` sequences and return the one with the highest token logprobs. Defaults to None.
* **watermark** (`bool`, *optional*) — Watermarking with [A Watermark for Large Language Models](https://arxiv.org/abs/2301.10226). Defaults to False.
* **details** (`bool`, *optional*) — Get generation details. Defaults to False.
* **decoder\_input\_details** (`bool`, *optional*) — Get decoder input token logprobs and ids. Defaults to False.

Parameters for text generation.

#### class huggingface\_hub.inference.\_text\_generation.TextGenerationResponse

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_text_generation.py#L353)

( generated\_text: strdetails: typing.Optional\[huggingface\_hub.inference.\_text\_generation.Details] = None )

Parameters

* **generated\_text** (`str`) — The generated text.
* **details** (`Optional[Details]`) — Generation details. Returned only if `details=True` is sent to the server.

Represents a response for text generation.

Only returned when `details=True`, otherwise a string is returned.

#### class huggingface\_hub.inference.\_text\_generation.TextGenerationStreamResponse

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_text_generation.py#L397)

( token: Tokengenerated\_text: typing.Optional\[str] = Nonedetails: typing.Optional\[huggingface\_hub.inference.\_text\_generation.StreamDetails] = None )

Parameters

* **token** (`Token`) — The generated token.
* **generated\_text** (`Optional[str]`, *optional*) — The complete generated text. Only available when the generation is finished.
* **details** (`Optional[StreamDetails]`, *optional*) — Generation details. Only available when the generation is finished.

Represents a response for streaming text generation.

Only returned when `details=True` and `stream=True`.

#### class huggingface\_hub.inference.\_text\_generation.InputToken

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_text_generation.py#L220)

( id: inttext: strlogprob: typing.Optional\[float] = None )

Parameters

* **id** (`int`) — Token ID from the model tokenizer.
* **text** (`str`) — Token text.
* **logprob** (`float` or `None`) — Log probability of the token. Optional since the logprob of the first token cannot be computed.

Represents an input token.

#### class huggingface\_hub.inference.\_text\_generation.Token

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_text_generation.py#L244)

( id: inttext: strlogprob: floatspecial: bool )

Parameters

* **id** (`int`) — Token ID from the model tokenizer.
* **text** (`str`) — Token text.
* **logprob** (`float`) — Log probability of the token.
* **special** (`bool`) — Indicates whether the token is a special token. It can be used to ignore tokens when concatenating.

Represents a token.

#### class huggingface\_hub.inference.\_text\_generation.FinishReason

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_text_generation.py#L272)

( valuenames = Nonemodule = Nonequalname = Nonetype = Nonestart = 1 )

An enumeration.

#### class huggingface\_hub.inference.\_text\_generation.BestOfSequence

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_text_generation.py#L283)

( generated\_text: strfinish\_reason: FinishReasongenerated\_tokens: intseed: typing.Optional\[int] = Noneprefill: typing.List\[huggingface\_hub.inference.\_text\_generation.InputToken] = \<factory>tokens: typing.List\[huggingface\_hub.inference.\_text\_generation.Token] = \<factory> )

Parameters

* **generated\_text** (`str`) — The generated text.
* **finish\_reason** (`FinishReason`) — The reason for the generation to finish, represented by a `FinishReason` value.
* **generated\_tokens** (`int`) — The number of generated tokens in the sequence.
* **seed** (`Optional[int]`) — The sampling seed if sampling was activated.
* **prefill** (`List[InputToken]`) — The decoder input tokens. Empty if `decoder_input_details` is False. Defaults to an empty list.
* **tokens** (`List[Token]`) — The generated tokens. Defaults to an empty list.

Represents a best-of sequence generated during text generation.

#### class huggingface\_hub.inference.\_text\_generation.Details

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_text_generation.py#L318)

( finish\_reason: FinishReasongenerated\_tokens: intseed: typing.Optional\[int] = Noneprefill: typing.List\[huggingface\_hub.inference.\_text\_generation.InputToken] = \<factory>tokens: typing.List\[huggingface\_hub.inference.\_text\_generation.Token] = \<factory>best\_of\_sequences: typing.Optional\[typing.List\[huggingface\_hub.inference.\_text\_generation.BestOfSequence]] = None )

Parameters

* **finish\_reason** (`FinishReason`) — The reason for the generation to finish, represented by a `FinishReason` value.
* **generated\_tokens** (`int`) — The number of generated tokens.
* **seed** (`Optional[int]`) — The sampling seed if sampling was activated.
* **prefill** (`List[InputToken]`, *optional*) — The decoder input tokens. Empty if `decoder_input_details` is False. Defaults to an empty list.
* **tokens** (`List[Token]`) — The generated tokens. Defaults to an empty list.
* **best\_of\_sequences** (`Optional[List[BestOfSequence]]`) — Additional sequences when using the `best_of` parameter.

Represents details of a text generation.

#### class huggingface\_hub.inference.\_text\_generation.StreamDetails

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference/_text_generation.py#L374)

( finish\_reason: FinishReasongenerated\_tokens: intseed: typing.Optional\[int] = None )

Parameters

* **finish\_reason** (`FinishReason`) — The reason for the generation to finish, represented by a `FinishReason` value.
* **generated\_tokens** (`int`) — The number of generated tokens.
* **seed** (`Optional[int]`) — The sampling seed if sampling was activated.

Represents details of a text generation stream.

### InferenceAPI

`InferenceAPI` is the legacy way to call the Inference API. The interface is more simplistic and requires knowing the input parameters and output format for each task. It also lacks the ability to connect to other services like Inference Endpoints or AWS SageMaker. `InferenceAPI` will soon be deprecated so we recommend using [InferenceClient](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceClient) whenever possible. Check out [this guide](https://huggingface.co/docs/huggingface_hub/guides/inference#legacy-inferenceapi-client) to learn how to switch from `InferenceAPI` to [InferenceClient](https://huggingface.co/docs/huggingface_hub/v0.18.0.rc0/en/package_reference/inference_client#huggingface_hub.InferenceClient) in your scripts.

#### class huggingface\_hub.InferenceApi

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference_api.py#L46)

( repo\_id: strtask: typing.Optional\[str] = Nonetoken: typing.Optional\[str] = Nonegpu: bool = False )

Client to configure requests and make calls to the HuggingFace Inference API.

Example:

Copied

```
>>> from huggingface_hub.inference_api import InferenceApi

>>> # Mask-fill example
>>> inference = InferenceApi("bert-base-uncased")
>>> inference(inputs="The goal of life is [MASK].")
[{'sequence': 'the goal of life is life.', 'score': 0.10933292657136917, 'token': 2166, 'token_str': 'life'}]

>>> # Question Answering example
>>> inference = InferenceApi("deepset/roberta-base-squad2")
>>> inputs = {
...     "question": "What's my name?",
...     "context": "My name is Clara and I live in Berkeley.",
... }
>>> inference(inputs)
{'score': 0.9326569437980652, 'start': 11, 'end': 16, 'answer': 'Clara'}

>>> # Zero-shot example
>>> inference = InferenceApi("typeform/distilbert-base-uncased-mnli")
>>> inputs = "Hi, I recently bought a device from your company but it is not working as advertised and I would like to get reimbursed!"
>>> params = {"candidate_labels": ["refund", "legal", "faq"]}
>>> inference(inputs, params)
{'sequence': 'Hi, I recently bought a device from your company but it is not working as advertised and I would like to get reimbursed!', 'labels': ['refund', 'faq', 'legal'], 'scores': [0.9378499388694763, 0.04914155602455139, 0.013008488342165947]}

>>> # Overriding configured task
>>> inference = InferenceApi("bert-base-uncased", task="feature-extraction")

>>> # Text-to-image
>>> inference = InferenceApi("stabilityai/stable-diffusion-2-1")
>>> inference("cat")
<PIL.PngImagePlugin.PngImageFile image (...)>

>>> # Return as raw response to parse the output yourself
>>> inference = InferenceApi("mio/amadeus")
>>> response = inference("hello world", raw_response=True)
>>> response.headers
{"Content-Type": "audio/flac", ...}
>>> response.content # raw bytes from server
b'(...)'
```

**\_\_init\_\_**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference_api.py#L93)

( repo\_id: strtask: typing.Optional\[str] = Nonetoken: typing.Optional\[str] = Nonegpu: bool = False )

Parameters

* **repo\_id** (`str`) — Id of repository (e.g. *user/bert-base-uncased*).
* **task** (`str`, *optional*, defaults `None`) — Whether to force a task instead of using task specified in the repository.
* **token** (*str*, *optional*) — The API token to use as HTTP bearer authorization. This is not the authentication token. You can find the token in <https://huggingface.co/settings/token>. Alternatively, you can find both your organizations and personal API tokens using *HfApi().whoami(token)*.
* **gpu** (*bool*, *optional*, defaults *False*) — Whether to use GPU instead of CPU for inference(requires Startup plan at least).

Inits headers and API call information.

**\_\_call\_\_**

[\<source>](https://github.com/huggingface/huggingface_hub/blob/v0.18.0.rc0/src/huggingface_hub/inference_api.py#L158)

( inputs: typing.Union\[str, typing.Dict, typing.List\[str], typing.List\[typing.List\[str]], NoneType] = Noneparams: typing.Optional\[typing.Dict] = Nonedata: typing.Optional\[bytes] = Noneraw\_response: bool = False )

Parameters

* **inputs** (`str` or `Dict` or `List[str]` or `List[List[str]]`, *optional*) — Inputs for the prediction.
* **params** (`Dict`, *optional*) — Additional parameters for the models. Will be sent as `parameters` in the payload.
* **data** (`bytes`, *optional*) — Bytes content of the request. In this case, leave `inputs` and `params` empty.
* **raw\_response** (`bool`, defaults to `False`) — If `True`, the raw `Response` object is returned. You can parse its content as preferred. By default, the content is parsed into a more practical format (json dictionary or PIL Image for example).

Make a call to the Inference API.