Disposes of all the ONNX sessions that were created during inference.
Kind: instance method of PreTrainedModelReturns: Promise.<Array<unknown>> - An array of promises, one for each ONNX session that is being disposed.
Todo
This function merges multiple generation configs together to form a final generation config to be used by the model for text generation. It first creates an empty GenerationConfig object, then it applies the model’s own generation_config property to it. Finally, if a generation_config object was passed in the arguments, it overwrites the corresponding properties in the final config with those of the passed config object.
Kind: instance method of PreTrainedModelReturns: GenerationConfig - The final generation config object to be used by the model for text generation.
Param
Type
Description
generation_config
GenerationConfig
A GenerationConfig object containing generation parameters.
Instantiate one of the model classes of the library from a pretrained model.
The model class to instantiate is selected based on the model_type property of the config object (either passed as an argument or loaded from pretrained_model_name_or_path if possible)
Kind: static method of PreTrainedModelReturns: Promise.<PreTrainedModel> - A new instance of the PreTrainedModel class.
Param
Type
Description
pretrained_model_name_or_path
string
The name or path of the pretrained model. Can be either:
A string, the model id of a pretrained model hosted inside a model repo on boincai.com. Valid model ids can be located at the root-level, like bert-base-uncased, or namespaced under a user or organization name, like dbmdz/bert-base-german-cased.
A path to a directory containing model weights, e.g., ./my_model_directory/.
options
*
Additional options for loading the model.
models.BaseModelOutput
Base class for model’s outputs, with potential hidden states and attentions.
Kind: instance method of BertForMaskedLMReturns: Promise.<MaskedLMOutput> - An object containing the model’s output logits for masked language modeling.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.BertForSequenceClassification
BertForSequenceClassification is a class representing a BERT model for sequence classification.
Kind: instance method of BertForSequenceClassificationReturns: Promise.<SequenceClassifierOutput> - An object containing the model’s output logits for sequence classification.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.BertForTokenClassification
BertForTokenClassification is a class representing a BERT model for token classification.
Kind: instance method of BertForTokenClassificationReturns: Promise.<TokenClassifierOutput> - An object containing the model’s output logits for token classification.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.BertForQuestionAnswering
BertForQuestionAnswering is a class representing a BERT model for question answering.
Kind: instance method of BertForQuestionAnsweringReturns: Promise.<QuestionAnsweringModelOutput> - An object containing the model’s output logits for question answering.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.CamembertModel
The bare CamemBERT Model transformer outputting raw hidden-states without any specific head on top.
Kind: instance method of CamembertForMaskedLMReturns: Promise.<MaskedLMOutput> - An object containing the model’s output logits for masked language modeling.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.CamembertForSequenceClassification
CamemBERT Model transformer with a sequence classification/regression head on top (a linear layer on top of the pooled output) e.g. for GLUE tasks.
Kind: instance method of CamembertForSequenceClassificationReturns: Promise.<SequenceClassifierOutput> - An object containing the model’s output logits for sequence classification.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.CamembertForTokenClassification
CamemBERT Model with a token classification head on top (a linear layer on top of the hidden-states output) e.g. for Named-Entity-Recognition (NER) tasks.
Kind: instance method of CamembertForTokenClassificationReturns: Promise.<TokenClassifierOutput> - An object containing the model’s output logits for token classification.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.CamembertForQuestionAnswering
CamemBERT Model with a span classification head on top for extractive question-answering tasks
Kind: instance method of CamembertForQuestionAnsweringReturns: Promise.<QuestionAnsweringModelOutput> - An object containing the model’s output logits for question answering.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.DebertaModel
The bare DeBERTa Model transformer outputting raw hidden-states without any specific head on top.
Kind: instance method of DebertaForMaskedLMReturns: Promise.<MaskedLMOutput> - An object containing the model’s output logits for masked language modeling.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.DebertaForSequenceClassification
DeBERTa Model transformer with a sequence classification/regression head on top (a linear layer on top of the pooled output)
Kind: instance method of DebertaForSequenceClassificationReturns: Promise.<SequenceClassifierOutput> - An object containing the model’s output logits for sequence classification.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.DebertaForTokenClassification
DeBERTa Model with a token classification head on top (a linear layer on top of the hidden-states output) e.g. for Named-Entity-Recognition (NER) tasks.
Kind: instance method of DebertaForTokenClassificationReturns: Promise.<TokenClassifierOutput> - An object containing the model’s output logits for token classification.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.DebertaForQuestionAnswering
DeBERTa Model with a span classification head on top for extractive question-answering tasks like SQuAD (a linear layers on top of the hidden-states output to compute span start logits and span end logits).
Kind: instance method of DebertaForQuestionAnsweringReturns: Promise.<QuestionAnsweringModelOutput> - An object containing the model’s output logits for question answering.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.DebertaV2Model
The bare DeBERTa-V2 Model transformer outputting raw hidden-states without any specific head on top.
Kind: instance method of DebertaV2ForMaskedLMReturns: Promise.<MaskedLMOutput> - An object containing the model’s output logits for masked language modeling.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.DebertaV2ForSequenceClassification
DeBERTa-V2 Model transformer with a sequence classification/regression head on top (a linear layer on top of the pooled output)
Kind: instance method of DebertaV2ForSequenceClassificationReturns: Promise.<SequenceClassifierOutput> - An object containing the model’s output logits for sequence classification.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.DebertaV2ForTokenClassification
DeBERTa-V2 Model with a token classification head on top (a linear layer on top of the hidden-states output) e.g. for Named-Entity-Recognition (NER) tasks.
Kind: instance method of DebertaV2ForTokenClassificationReturns: Promise.<TokenClassifierOutput> - An object containing the model’s output logits for token classification.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.DebertaV2ForQuestionAnswering
DeBERTa-V2 Model with a span classification head on top for extractive question-answering tasks like SQuAD (a linear layers on top of the hidden-states output to compute span start logits and span end logits).
Kind: instance method of DebertaV2ForQuestionAnsweringReturns: Promise.<QuestionAnsweringModelOutput> - An object containing the model’s output logits for question answering.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.DistilBertForSequenceClassification
DistilBertForSequenceClassification is a class representing a DistilBERT model for sequence classification.
Kind: instance method of DistilBertForSequenceClassificationReturns: Promise.<SequenceClassifierOutput> - An object containing the model’s output logits for sequence classification.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.DistilBertForTokenClassification
DistilBertForTokenClassification is a class representing a DistilBERT model for token classification.
Kind: instance method of DistilBertForTokenClassificationReturns: Promise.<TokenClassifierOutput> - An object containing the model’s output logits for token classification.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.DistilBertForQuestionAnswering
DistilBertForQuestionAnswering is a class representing a DistilBERT model for question answering.
Kind: instance method of DistilBertForQuestionAnsweringReturns: Promise.<QuestionAnsweringModelOutput> - An object containing the model’s output logits for question answering.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.DistilBertForMaskedLM
DistilBertForMaskedLM is a class representing a DistilBERT model for masking task.
Kind: instance method of MPNetForMaskedLMReturns: Promise.<MaskedLMOutput> - An object containing the model’s output logits for masked language modeling.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.MPNetForSequenceClassification
MPNetForSequenceClassification is a class representing a MPNet model for sequence classification.
Kind: instance method of MPNetForSequenceClassificationReturns: Promise.<SequenceClassifierOutput> - An object containing the model’s output logits for sequence classification.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.MPNetForTokenClassification
MPNetForTokenClassification is a class representing a MPNet model for token classification.
Kind: instance method of MPNetForTokenClassificationReturns: Promise.<TokenClassifierOutput> - An object containing the model’s output logits for token classification.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.MPNetForQuestionAnswering
MPNetForQuestionAnswering is a class representing a MPNet model for question answering.
Kind: instance method of MPNetForQuestionAnsweringReturns: Promise.<QuestionAnsweringModelOutput> - An object containing the model’s output logits for question answering.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.T5ForConditionalGeneration
T5Model is a class representing a T5 model for conditional generation.
Kind: instance method of BartForSequenceClassificationReturns: Promise.<SequenceClassifierOutput> - An object containing the model’s output logits for sequence classification.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.MBartModel
The bare MBART Model outputting raw hidden-states without any specific head on top.
Kind: instance method of MBartForSequenceClassificationReturns: Promise.<SequenceClassifierOutput> - An object containing the model’s output logits for sequence classification.
Kind: instance method of RobertaForTokenClassificationReturns: Promise.<TokenClassifierOutput> - An object containing the model’s output logits for token classification.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.RobertaForQuestionAnswering
RobertaForQuestionAnswering class for performing question answering on Roberta models.
Kind: instance method of XLMForTokenClassificationReturns: Promise.<TokenClassifierOutput> - An object containing the model’s output logits for token classification.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.XLMForQuestionAnswering
XLM Model with a span classification head on top for extractive question-answering tasks
Kind: instance method of XLMRobertaForTokenClassificationReturns: Promise.<TokenClassifierOutput> - An object containing the model’s output logits for token classification.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.XLMRobertaForQuestionAnswering
XLMRobertaForQuestionAnswering class for performing question answering on XLMRoberta models.
Calculates token-level timestamps using the encoder-decoder cross-attentions and dynamic time-warping (DTW) to map each output token to a position in the input audio.
Kind: instance method of WhisperForConditionalGenerationReturns: Tensor - tensor containing the timestamps in seconds for each predicted token
Param
Type
Default
Description
generate_outputs
Object
Outputs generated by the model
generate_outputs.cross_attentions
Array.<Array<Array<Tensor>>>
The cross attentions output by the model
generate_outputs.decoder_attentions
Array.<Array<Array<Tensor>>>
The decoder attentions output by the model
generate_outputs.sequences
Array.<Array<number>>
The sequences output by the model
alignment_heads
Array.<Array<number>>
Alignment heads of the model
[num_frames]
number
Number of frames in the input audio.
[time_precision]
number
0.02
Precision of the timestamps in seconds
models.VisionEncoderDecoderModel
Vision Encoder-Decoder model based on OpenAI’s GPT architecture for image captioning and other vision tasks
Classification logits (including no-object) for all queries.
output.pred_boxes
Tensor
Normalized boxes coordinates for all queries, represented as (center_x, center_y, width, height). These values are normalized in [0, 1], relative to the size of each individual image in the batch (disregarding possible padding).
Classification logits (including no-object) for all queries.
output.pred_boxes
Tensor
Normalized boxes coordinates for all queries, represented as (center_x, center_y, width, height). These values are normalized in [0, 1], relative to the size of each individual image in the batch (disregarding possible padding).
Kind: instance method of WavLMForSequenceClassificationReturns: Promise.<SequenceClassifierOutput> - An object containing the model’s output logits for sequence classification.
Param
Type
Description
model_inputs
Object
The inputs to the model.
models.SpeechT5PreTrainedModel
An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models.
Converts a sequence of input tokens into a sequence of mel spectrograms, which are subsequently turned into a speech waveform using a vocoder.
Kind: instance method of SpeechT5ForTextToSpeechReturns: Promise.<SpeechOutput> - A promise which resolves to an object containing the spectrogram, waveform, and cross-attention tensors.
Param
Type
Default
Description
input_values
Tensor
Indices of input sequence tokens in the vocabulary.
speaker_embeddings
Tensor
Tensor containing the speaker embeddings.
options
Object
Optional parameters for generating speech.
[options.threshold]
number
0.5
The generated sequence ends when the predicted stop token probability exceeds this value.
[options.minlenratio]
number
0.0
Used to calculate the minimum required length for the output sequence.
[options.maxlenratio]
number
20.0
Used to calculate the maximum allowed length for the output sequence.
[options.vocoder]
Object
The vocoder that converts the mel spectrogram into a speech waveform. If null, the output is the mel spectrogram.
[options.output_cross_attentions]
boolean
false
Whether or not to return the attentions tensors of the decoder's cross-attention layers.
Helper class which is used to instantiate pretrained models with the from_pretrained function. The chosen model class is determined by the type specified in the model config.
Helper class which is used to instantiate pretrained sequence classification models with the from_pretrained function. The chosen model class is determined by the type specified in the model config.
Helper class which is used to instantiate pretrained token classification models with the from_pretrained function. The chosen model class is determined by the type specified in the model config.
Helper class which is used to instantiate pretrained sequence-to-sequence models with the from_pretrained function. The chosen model class is determined by the type specified in the model config.
Helper class which is used to instantiate pretrained sequence-to-sequence speech-to-text models with the from_pretrained function. The chosen model class is determined by the type specified in the model config.
Helper class which is used to instantiate pretrained sequence-to-sequence text-to-spectrogram models with the from_pretrained function. The chosen model class is determined by the type specified in the model config.
Helper class which is used to instantiate pretrained causal language models with the from_pretrained function. The chosen model class is determined by the type specified in the model config.
Helper class which is used to instantiate pretrained masked language models with the from_pretrained function. The chosen model class is determined by the type specified in the model config.
Helper class which is used to instantiate pretrained question answering models with the from_pretrained function. The chosen model class is determined by the type specified in the model config.
Helper class which is used to instantiate pretrained vision-to-sequence models with the from_pretrained function. The chosen model class is determined by the type specified in the model config.
Helper class which is used to instantiate pretrained image classification models with the from_pretrained function. The chosen model class is determined by the type specified in the model config.
Helper class which is used to instantiate pretrained image segmentation models with the from_pretrained function. The chosen model class is determined by the type specified in the model config.
Helper class which is used to instantiate pretrained object detection models with the from_pretrained function. The chosen model class is determined by the type specified in the model config.
Helper class which is used to instantiate pretrained object detection models with the from_pretrained function. The chosen model class is determined by the type specified in the model config.
An tensor of key/value pairs that represent the previous state of the model.
output.encoder_outputs
Tensor
The output of the encoder in a sequence-to-sequence model.
[output.decoder_attentions]
Tensor
Attentions weights of the decoder, after the attention softmax, used to compute the weighted average in the self-attention heads.
[output.cross_attentions]
Tensor
Attentions weights of the decoder's cross-attention layer, after the attention softmax, used to compute the weighted average in the cross-attention heads.
models.SequenceClassifierOutput
Base class for outputs of sentence classification models.
Prediction scores of the language modeling head (scores for each vocabulary token before softmax).
output.past_key_values
Tensor
Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see past_key_values input) to speed up sequential decoding.
Generates text based on the given inputs and generation configuration using the model.
Kind: inner typedef of modelsReturns: Promise.<(Array<Array<number>>|EncoderDecoderOutput|DecoderOutput)> - An array of generated output sequences, where each sequence is an array of token IDs.
Throws:
Error Throws an error if the inputs array is empty.
Param
Type
Default
Description
inputs
Tensor | Array | TypedArray
An array of input token IDs.
generation_config
Object | GenerationConfig | null
The generation configuration to use. If null, default configuration will be used.
logits_processor
Object | null
An optional logits processor to use. If null, a new LogitsProcessorList instance will be created.
Kind: inner typedef of modelsExtends: GenerationConfigProperties
Name
Type
Default
Description
[return_timestamps]
boolean
Whether to return the timestamps with the text. This enables the WhisperTimestampsLogitsProcessor.
[return_token_timestamps]
boolean
Whether to return token-level timestamps with the text. This can be used with or without the return_timestamps option. To get word-level timestamps, use the tokenizer to group the tokens into words.
[num_frames]
number
The number of audio frames available in this chunk. This is only used generating word-level timestamps.
The predicted log-mel spectrogram of shape (output_sequence_length, config.num_mel_bins). Returned when no vocoder is provided
[waveform]
Tensor
The predicted waveform of shape (num_frames,). Returned when a vocoder is provided.
[cross_attentions]
Tensor
The outputs of the decoder's cross-attention layers of shape (config.decoder_layers, config.decoder_attention_heads, output_sequence_length, input_sequence_length). returned when output_cross_attentions is true.
import { AutoModelForSeq2SeqLM, AutoTokenizer } from '@xenova/transformers';
let tokenizer = await AutoTokenizer.from_pretrained('Xenova/t5-small');
let model = await AutoModelForSeq2SeqLM.from_pretrained('Xenova/t5-small');
let { input_ids } = await tokenizer('translate English to German: I love transformers!');
let outputs = await model.generate(input_ids);
let decoded = tokenizer.decode(outputs[0], { skip_special_tokens: true });
// 'Ich liebe Transformatoren!'
import { AutoTokenizer, AutoProcessor, CLIPModel, RawImage } from '@xenova/transformers';
// Load tokenizer, processor, and model
let tokenizer = await AutoTokenizer.from_pretrained('Xenova/clip-vit-base-patch16');
let processor = await AutoProcessor.from_pretrained('Xenova/clip-vit-base-patch16');
let model = await CLIPModel.from_pretrained('Xenova/clip-vit-base-patch16');
// Run tokenization
let texts = ['a photo of a car', 'a photo of a football match']
let text_inputs = tokenizer(texts, { padding: true, truncation: true });
// Read image and run processor
let image = await RawImage.read('https://boincai.com/datasets/Xenova/transformers.js-docs/resolve/main/football-match.jpg');
let image_inputs = await processor(image);
// Run model with both text and pixel inputs
let output = await model({ ...text_inputs, ...image_inputs });
// {
// logits_per_image: Tensor {
// dims: [ 1, 2 ],
// data: Float32Array(2) [ 18.579734802246094, 24.31830596923828 ],
// },
// logits_per_text: Tensor {
// dims: [ 2, 1 ],
// data: Float32Array(2) [ 18.579734802246094, 24.31830596923828 ],
// },
// text_embeds: Tensor {
// dims: [ 2, 512 ],
// data: Float32Array(1024) [ ... ],
// },
// image_embeds: Tensor {
// dims: [ 1, 512 ],
// data: Float32Array(512) [ ... ],
// }
// }
import { AutoTokenizer, CLIPTextModelWithProjection } from '@xenova/transformers';
// Load tokenizer and text model
const tokenizer = await AutoTokenizer.from_pretrained('Xenova/clip-vit-base-patch16');
const text_model = await CLIPTextModelWithProjection.from_pretrained('Xenova/clip-vit-base-patch16');
// Run tokenization
let texts = ['a photo of a car', 'a photo of a football match'];
let text_inputs = tokenizer(texts, { padding: true, truncation: true });
// Compute embeddings
const { text_embeds } = await text_model(text_inputs);
// Tensor {
// dims: [ 2, 512 ],
// type: 'float32',
// data: Float32Array(1024) [ ... ],
// size: 1024
// }
