Skip to main content
Batch API FAQ

Batch API endpoint for asynchronous batch processing

Updated this week

How does the Batch API work?

The Batch API endpoint, as documented here, allows users to submit requests for asynchronous batch processing. We will process these requests within 24 hours. The details of each request will be read from a pre-uploaded file, and the responses will be written to an output file. You can query the batch object for status updates and results.

What is the pricing for the Batch API?

Each model will be offered at 50% cost discount vs. the synchronous APIs. Here's more information regarding our API pricing.

What models can I batch?

The following models will be available for batching:

  1. gpt-4o

  2. gpt-4o-mini

  3. gpt-4o-2024-05-13

  4. gpt-3.5-turbo

  5. gpt-3.5-turbo-16k

  6. gpt-4

  7. gpt-4-32k

  8. gpt-4-turbo-preview

  9. gpt-4-vision-preview

  10. gpt-4-turbo

  11. gpt-3.5-turbo-0301

  12. gpt-3.5-turbo-16k-0613

  13. gpt-3.5-turbo-1106

  14. gpt-3.5-turbo-0613

  15. gpt-4-0314

  16. gpt-4-turbo-2024-04-09

  17. gpt-4-32k-0314

  18. gpt-4-32k-0613

  19. text-embedding-3-large

  20. text-embedding-3-small

  21. text-embedding-ada-002

What happens if the API doesn’t complete my request within the promised time?

We aim to get all requests within 24 hours. If a batch expires (i.e. it could not be completed within the SLA time window), then remaining work is cancelled and any already completed work is returned. Developers will be charged for any completed work.

What’s the limit of how many requests I can batch?

There is no fixed limit on the number of requests you can batch; however, each usage tier has an associated batch rate limit. Your batch rate limit includes the maximum number of input tokens you have enqueued at one time. You can find your rate limits here.

Once your batch request is completed, your batch rate limit is reset, as your input tokens are cleared. The limit depends on the number of global requests in the queue. If the Batch API queue processes your batches quickly, your batch rate limit is reset more quickly.

Can I change the time it takes to hear back from the Batch API?

Our current specified time window is 24 hours. We currently cannot change this time period.

Is streaming supported on the Batch API?

No, streaming is not supported on the Batch API.

Are images supported on the Batch API?

Yes, images are supported on the Batch API.

I got an error message “The URL provided for this request does not prefix-match the batch endpoint”. What should I do?

If you get this error, this means your URL is incorrectly formatted for the Batch API endpoint. Please refer to our documentation here to make sure you’re calling the endpoint correctly.

Does usage of the Batch API count against my other rate limits?

Batch API rate limits are completely separate from existing limits.

What happens if a batch is cancelled?

If a batch is manually cancelled then whatever results have already been completed are returned. Developers will be charged for any completed work.

What happens if a batch expires?

If a batch expires (i.e. it could not be completed within the SLA time window), then remaining work is cancelled and any already completed work is returned. Developers will be charged for any completed work.

What statuses can I expect on batch jobs?

The batch job can have any of the following statuses:

  • Validating: Validation of the uploaded file is in progress before the batch can begin.

  • Failed: The file has failed the validation process.

  • In Progress: The file was successfully validated and the batch process is underway.

  • Finalizing: The batch job has completed and the results are being prepared.

  • Completed: The batch job is complete and the results are ready.

  • Expired: The batch could not be completed within the SLA time window.

  • Canceling: Cancellation of the batch job has been initiated.

  • Canceled: The batch job was canceled.

How can I check which batch jobs I've already created?

You can get a list of all your batch jobs using the API call specified here.

Is zero data retention supported on this endpoint?

If you have zero data retention enabled for your org, please note that zero data retention is not supported on this endpoint.

Did this answer your question?