Stop Sequences

Stop Sequences are used to make the model stop at a desired point, such as the end of a sentence or a list. Generally, the return key will work well as a Stop Sequence for single line completions.The Stop Sequence is an optional setting that tells the API when to stop generating tokens. The completion will not contain the stop sequence and you can pass up to four stop sequences. If nothing is passed, it defaults to the token <|endoftext|>. This token represents a probable stopping point in the text.

Chat Example:

In the Chat example, three Stop Sequences are used: A new line, the value “Human:”, and the value “AI:”. Our goal is to generate only a single line of text that corresponds to the current speaker. The chosen three stop sequences create the needed constraints to achieve that end:

  • The completion can’t go to a new line.

  • The completion can’t change the speaker.

  • The completion won’t allow the speaker to speak twice in a row

Q&A Example:

In the Q&A example, the pattern of separating each Q&A pair with a new line gives us confidence that the return key(new line) will work well as our stop sequence. You can see below that the completion, following “A:”, will stop after a single line because the return key is used as a Stop Sequence.

List Example:

You can also use Stop Sequences to generate a list with a specific number of items. For example, by using “11.” as a stop sequence, you can generate a list with 10 items, since the completion will stop when “11.” is reached. This can be seen in the science fiction prompt.

Did this answer your question?