Run Inference
Last updated
Last updated
This section shows how to run inference-only workloads on Gaudi. For more advanced information about how to speed up inference, check out .
You can find below a template to perform inference with a GaudiTrainer
instance where we want to compute the accuracy over the given dataset:
Copied
Copied
The variable my_args
should contain some inference-specific arguments, you can take a look to see the arguments that can be interesting to set for inference.
All contain instructions for running inference with a given model on a given dataset. The reasoning is the same for every example: run the example script with --do_eval
and --per_device_eval_batch_size
and without --do_train
. A simple template is the following: