What Are LLM Presence and Frequency Penalties?

Presence and frequency penalty parameters are two of the least glamorous knobs in the sampling stack, and what they do is simple enough: they edit the logits, the raw scores a model assigns each candidate token, so that the model is less likely to say the same thing twice. Both work off what has already appeared in the context window. They shift the logit distribution before the final softmax, which steers the autoregressive model away from degenerate loops and pushes the generated sequence toward higher entropy. Order matters here, because the penalties land after the model produces raw logit scores but before truncation filters such as Top-P, so that the sampler correctly drops the tokens it has penalized.

Without them, models fall into highest-probability trails where certain phrases become statistically self-reinforcing. That is why the parameters matter most for long-form generation, where the local coherence of a sequence is exactly what can leave the model stuck in a recursive loop. What they buy you is a tax on redundant tokens, charged at the logit level, before the model makes its next selection.

Two knobs reshaping a language model's token probability distribution to stop it repeating itself

What are presence and frequency penalties?

In the OpenAI API specification and derivative runtimes like vLLM, both presence and frequency penalties are defined within a range of −2.0 to 2.0, with a default value of 0, so out of the box they do nothing at all. Positive values apply a tax to the raw logits and so decrease the likelihood of the model repeating tokens, while negative values encourage repetition, which is rarely what you want in production inference.

vLLM and the OpenAI specification both implement these as additive transformations: the penalty value is subtracted from the logit associated with a specific token, and that happens before the softmax function converts those scores into probabilities. The raw scores for repetitive tokens are suppressed, so the sampler picks from the remaining valid distribution.

The arithmetic on the logit distribution is this short:

python

logits -= frequency_penalties * output_bin_counts
logits -= presence_penalties * output_mask

Here output_bin_counts is an integer, the number of times a specific token has already been generated in the current response, and the output_mask is a boolean indicator — 1 if the token has appeared at least once, 0 otherwise. Almost everything else in this article follows from that one difference, because the presence penalty is a flat, one-off tax applied once a token enters the context, whereas the frequency penalty scales linearly with each subsequent occurrence of that token.

Why models repeat themselves

Neural text degeneration happens mostly because decoding follows the highest-probability trail. During autoregressive sampling the model picks the next token from current probabilities, and once a specific phrase has started, its local coherence and statistical momentum can keep it the most probable choice. That feedback loop ends in entropy collapse, a state where low-temperature decoding narrows the distribution until the model is trapped in a repetitive sequence.

A self-reinforcing loop where a phrase the model just produced makes itself more probable next time

The training data reinforces those loops. Web-scraped datasets are heavy with ritual phrases and templates — "in conclusion", "therefore", "it is important to note". Models over-optimize for these patterns because the patterns are statistically safe transitions. So when the model loses a specific reasoning path, or lacks enough information to keep generating something unique, it falls back on those over-represented templates to maintain high token-level likelihood.

Bad prompts contribute their share as well. If a prompt is vague or lacks a defined scope, the model may try to sound authoritative by paraphrasing its own earlier segments, and without a hard output contract or structural limits it has no internal mechanism to stop itself returning to ideas it has already established. That is the rambling behavior anyone recognizes from a poorly constrained session.

The ugliest version turns up in structured tasks, where a single token can trigger a probability collapse on its own: Markdown table pipes (|), or empty JSON brackets. Once the model assigns an extremely high probability to a repetitive character sequence, the entropy of the distribution collapses completely, and without logit-level manipulation the sampler cannot escape the local maximum of that repetitive character. What you get instead is an infinite loop of empty structural symbols.

How the two penalties differ

The frequency penalty is a tax on reuse, and it scales in proportion to the token count. Because it is integer-scaled on output_bin_counts, the penalty for a specific token increases every time that token is used, which makes it a surgical tool for verbal tics and over-reliance on specific adjectives. If a model overuses a word like "efficient", the frequency penalty makes it progressively more expensive to choose that exact token again, so the model has to go looking through the rest of its vocabulary.

The frequency penalty growing with every repeat of a token, beside the presence penalty as a flat one-off charge

The presence penalty is blunter, and deliberately so, because it applies a flat penalty regardless of how many times a token has appeared. The check is boolean, so if a token exists anywhere in the generated history it receives the full penalty once, which pushes the model toward entirely new vocabulary and new concepts. That is what you want for brainstorming or creative writing, where the point is to expand into new dimensions rather than circle the same ideas in different wordings.

The frequency penalty targets specific word over-use, or lexical diversity; the presence penalty stops the model lingering on a single idea, which is thematic diversity.

In a technical report, a high frequency penalty prevents the model from starting every paragraph with the same transition word, while a high presence penalty would instead force it to stop discussing cloud infrastructure entirely and move on to security protocols or cost management. One knob changes how you say things, and the other changes what you talk about.

You can run both at once, but they have to be balanced or the signals fight each other. Frequency manages local vocabulary density, so that a single concept gets a diverse choice of words, while presence manages the thematic breadth of the output. Turn both up high and you starve the model of available tokens, which is where the over-penalization failure modes start.

Repetition penalty: the third knob people confuse with these

The repetition penalty is an open-source classic, found in Hugging Face Transformers and llama.cpp. Unlike the additive OpenAI-style penalties, it is implemented as a multiplier, typically ranging from 1.15 to 1.30. The goal is identical — preventing the model from repeating itself — but the underlying mechanic is more aggressive.

It works by dividing the logit of any token that has already been generated, so if a token is in the context, its raw logit score comes back divided by the penalty value. Because that is a multiplicative operation, it can severely suppress tokens that are essential for grammatical structure, and it cannot distinguish between the legitimate reuse of a comma or the word "the" and a degenerate repetition loop. Blunt instrument is the fair description. The scope differs too, because the repetition penalty applies over a mask that combines the prompt tokens and the output tokens, while the OpenAI-style penalties consider only what has been generated.

An additive penalty reading only generated text, versus a multiplicative one reading the prompt and generated text together

The Hugging Face implementation applies the penalty at most once per token, and for decoder-only models the considered tokens include the prompt by default. The original CTRL paper that introduced the technique recommends a value of around 1.2 as a balance between truthful generation and lack of repetition.

Modern inference stacks have largely moved toward alternatives such as the DRY sampler. Unlike the standard repetition penalty, DRY evaluates sequences of tokens rather than isolated ones. By analyzing n-gram statistics and sequence lengths, DRY can prevent catastrophic loops — a repeating Markdown table, say — without punishing the single-word grammar that coherent sentences need.

What values to actually set

Adjust in increments of 0.1, because these knobs are touchier than their wide-looking range suggests. For long reports, a frequency penalty of 0.5 to 0.8 paired with a presence penalty of 0.2 to 0.5 is enough to prevent padding without forcing awkward phrasing. For marketing copy, where high variety is wanted, the frequency penalty range rises to 0.9 to 1.2 with a presence penalty of 0.1 to 0.4. Customer support sits lower, at 0.3 to 0.6 frequency and 0.2 to 0.5 presence, to keep the tone natural and consistent. For open-source models using the multiplicative knob instead, 1.15 to 1.30 is the working band.

The impact of these penalties depends on the penalty window, configured in llama.cpp via repeat-last-n. This defines how many tokens the model considers when applying the tax: 0 disables it, a positive integer limits the lookback to that many tokens, and −1 examines the entire context window. Short structured outputs do well with −1. Long-form creative writing benefits from a larger but bounded window of 256 to 1024 tokens, which lets the model reuse vocabulary it needs.

Penalties must be applied before truncation samplers like Top-P, and the default pipeline does exactly that.

Reversing the order truncates the pool to, say, 10 tokens, penalizes 8 of them, and forces the model to choose between 2 terrible remaining options. Applying penalties to the raw logits first means the truncation sampler is working on a distribution that has already been reshaped.

Order in the sampling pipeline: raw logits, then penalties, then Top-P truncation, then the token is chosen

When the penalty is too strong: broken grammar and invented words

Push too hard and the grammar goes first. Structural words such as punctuation, "the", and "a" are naturally frequent in English, so a frequency penalty set too high punishes the model for using basic syntax. The output stops repeating itself and stops being readable at the same time, because the model is building sentences without the linguistic glue they need.

A degenerate cycle where a token repeats until its penalty bans it, then the model jumps to another and repeats

Then there is the thesaurus effect, which is subtler and in some ways worse. When a model is heavily penalized for repeating a natural word like "coffee", it reaches for increasingly obscure synonyms to dodge the tax, and what comes back is unnatural, pretentious phrasing: "this caffeinated beverage", "a dark bean infusion". Technically the model has avoided repeating "coffee". It has also destroyed the clarity of the response, which was presumably the thing you were trying to protect.

In reasoning models, excessive penalties trigger a degenerate cycle instead of stopping one. A token repeats until the cumulative penalty reaches a threshold that effectively bans it, at which point the model jumps to a new token and starts a fresh repetitive loop with that one. A real excerpt from QwQ-32B at a frequency penalty of 0.3 and temperature 0 shows the pattern exactly: a long run of "Non!", then a run of "This!", then a run of "Third!". The penalty is doing its job on each individual token and the output is still garbage.

The structural reason is that both the frequency and repetition penalties work on single-token statistics and cannot forget. Their penalty on a token never decays with distance, so over a long reasoning trace they eventually suppress the punctuation and spacing that the output depends on. The LZ penalty proposed as an alternative does forget, weakening with distance until a token falls outside its lookback window entirely.

Why new reasoning models dropped both parameters

The latest frontier reasoning models — the o-series and the GPT-5 series — no longer support presence_penalty or frequency_penalty. Other parameters familiar from chat completions are on the same unsupported list: temperature, top_p, logprobs, top_logprobs, logit_bias, and max_tokens.

Repetition control moving from an external logit-level penalty into the model's own reasoning space

These models manage their own internal entropy and repetition through reasoning tokens that never appear in the final output. They rely on a different set of controls, most notably reasoning_effort, whose supported values are none, minimal, low, medium, high, xhigh, and max. The parameter tells the model how much to think internally before answering. By managing the reasoning process through internal chain-of-thought, the model identifies and abandons repetitive paths itself, which makes external logit-level penalties redundant.

The newer models add controls for the same problem at a different level. In long-running or tool-heavy workflows, gpt-5.5 and gpt-5.4 let you mark each assistant message in the Responses API with a phase value — commentary for a preamble the model produces before a tool call, final_answer for the completed response. The parameter is optional, but omitting it can cause the model to treat a preamble as the final answer and stop early. Repetition control moved from the sampler into the protocol.

When to reach for them, and when to leave them alone

My own order of operations is to fix the prompt first and reach for the penalties second. Many repetition problems are resolved by defining a clear output format, setting word counts, or explicitly listing the phrases you do not want to see. An instruction like "each bullet point must introduce a new, distinct technical dimension" is usually more effective, and far less destructive, than raw mathematical logit suppression.

These penalties are a multiplier on good instructions rather than a replacement for them, and they earn their keep when you are tuning the feel of a model's output or breaking a specific loop in a non-reasoning task. In most production settings the right move is to leave them at their defaults, or to make conservative 0.1 to 0.3 adjustments and stop there. Push past that and you are trading grammar for novelty, one token at a time.