Introduction

​Since releasing the Classifications endpoint in beta last year, we’ve developed new methods that achieve better results for this task. As a result, we’ll be removing the Classifications endpoints from our documentation and removing access to this endpoint on December 3, 2022. New accounts created after today will not have access to this endpoint.

While applications currently using the endpoint won’t be immediately impacted by this change, the endpoint won’t be actively maintained going forward. We strongly encourage developers to switch over to newer techniques which produce better results, outlined below.

Current documentation

https://beta.openai.com/docs/guides/classifications

https://beta.openai.com/docs/api-reference/classifications

Options

As a quick review, here are the high level steps of the current Classifications endpoint:

All of these options are also outlined here.

Option 1: Transition to fine-tuning (recommended)

We believe that most use cases will be better served by moving to a fine tuned model. The major reason for this is that our current system used a bigram filter to narrow down the scope of candidates whereas our fine tuned system can take in an arbitrary amount of data and learn more nuance between examples. For more on creating a fine tuned model, check out our guide

Option 2: Transition to Embeddings-based search

Another possible option, especially if your classification labels change frequently, is to use embeddings. If you’re not familiar with this, you can learn more by visiting our guide to embeddings.

If you’re using a small dataset (<10,000 documents), consider using the techniques described in that guide to find the best documents to construct a prompt similar to this. Then, you can just submit that prompt to our Completions endpoint.

If you have a larger dataset, consider using a vector search engine like Pinecone or Weaviate to power that search.

Option 3: Reimplement existing functionality

If you’d like to recreate the functionality of the Classifications endpoint, here’s how we did it. This functionality is also mostly replicated in this script.

At a high level, there are two main ways you can use the classifications endpoint: you can source the data from an uploaded file or send it in with the request.

Document

There’s only one step if you provide the documents in the Classifications API call.

Here’s roughly the steps we used:

  • Construct the prompt with this format.

  • Gather all of the provided documents. If they fit in the prompt, just use all of them.

  • Do an Openai search (also being deprecated. Please see its transition guide) where the documents are the user provided documents and the query is the query from above. Rank the documents by score.

  • In order of score, attempt to add Elasticsearch documents until you run out of space in the context. Try to maximize the number of distinct labels as that will help the model understand the different labels that are available.

  • Request a completion with the provided parameters (logit_bias, n, stop, etc)

Throughout all of this, you’ll need to check that the prompt’s length doesn’t exceed the model's token limit. To assess the number of tokens present in a prompt, we recommend https://huggingface.co/docs/transformers/model_doc/gpt2#transformers.GPT2TokenizerFast.

File

Step 1: upload a jsonl file

Behind the scenes, we upload new files meant for classifications to an Elasticsearch. Each line of the jsonl is then submitted as a document.

In each line we require a “text” field, a “label” field, and an optional “metadata” field

These are the Elasticsearch settings and mappings for our index:

Elasticsearching mapping:

{

"properties": {
"document": {"type": "text", "analyzer": "standard_bigram_analyzer"}, -> the “text” field
"label": {"type": "text", "analyzer": "standard_bigram_analyzer"},
"metadata": {"type": "object", "enabled": False}, -> the “metadata” field
}
}

Elasticsearch analyzer:

{
"analysis": {
"analyzer": {
"standard_bigram_analyzer": {
"type": "custom",
"tokenizer": "standard",
"filter": ["lowercase", "english_stop", "shingle"],
}
},
"filter": {"english_stop": {"type": "stop", "stopwords": "_english_"}},
}
}

After that, we performed standard Elasticsearch search calls and used `max_examples` to determine the number of documents to return from Elasticsearch.

Step 2: Search

Here’s roughly the steps we used. Our end goal is to create a Completions request with this format. It will look very similar to Documents

From there, our steps are:

  • Start with the `experimental_alternative_question` or, if that's not provided, what’s in the `question` field. Call that the query.

  • Query Elasticsearch for `max_examples` documents with query as the search param.

  • Take those documents and do an OpenAI search on them where the entries from Elasticsearch are the docs, and the query is the query that you used above. Use the score from the search to rank the documents.

  • In order of score, attempt to add Elasticsearch documents until you run out of space in the prompt. Try to maximize the number of distinct labels as that will help the model understand the different labels that are available.

  • Request an Openai completion with the provided parameters (logit_bias, n, stop, etc). Return that generation to the user.

Completion Prompt

{{ an optional instruction }}

Text: {{example 1 text}}
Category: {{example 1 label}}
---
Text: {{example 2 text}}
Category: {{example 2 label}}
---
Text: {{question}}
Category:

Did this answer your question?