Skip to content

fix: title generation fails with OpenAI reasoning models#2412

Open
trungutt wants to merge 2 commits intodocker:mainfrom
trungutt:fix/2318-gpt5-title-generation
Open

fix: title generation fails with OpenAI reasoning models#2412
trungutt wants to merge 2 commits intodocker:mainfrom
trungutt:fix/2318-gpt5-title-generation

Conversation

@trungutt
Copy link
Copy Markdown
Contributor

@trungutt trungutt commented Apr 14, 2026

  • Title generation with OpenAI reasoning models (gpt-5, o-series) fails with empty title output from model because these models always reason internally, even when no reasoning configuration is provided. The hidden reasoning tokens consume the entire output token budget, leaving nothing for the visible title text.

  • When the code intended to "skip reasoning", it simply omitted the reasoning parameter from the API request. This does not disable reasoning - the model defaults to its standard reasoning behavior, which can easily exhaust a small token budget.

  • The output token budget (20 tokens) was sized for non-reasoning models and far too small once reasoning overhead is factored in.

Fix

  • Explicitly request the lowest reasoning effort supported across all OpenAI reasoning models so the model minimizes internal reasoning token usage.
  • Increase the output token budget to accommodate the reasoning overhead that reasoning models always incur.

How to test

mise build

# The original failing command from the issue
./bin/docker-agent debug title ./examples/rag.yaml "What can you do?"

# Explicitly test reasoning models
./bin/docker-agent debug title ./examples/rag.yaml --model=openai/gpt-5 "What can you do?"
./bin/docker-agent debug title ./examples/rag.yaml --model=openai/o3-mini "What can you do?"
./bin/docker-agent debug title ./examples/rag.yaml --model=openai/o4-mini "What can you do?"

# Non-reasoning models should still work as before
./bin/docker-agent debug title ./examples/rag.yaml --model=openai/gpt-4o "What can you do?"
./bin/docker-agent debug title ./examples/rag.yaml --model=anthropic/claude-haiku-4-5 "What can you do?"

Each command should print a short title to stdout without errors.

Fixes #2318

OpenAI reasoning models (o-series, gpt-5) always reason internally;
omitting the reasoning parameter does not disable reasoning -- it uses
the model's default effort. Those hidden reasoning tokens count against
max_output_tokens, so with a small budget (e.g. title generation) the
model can exhaust all tokens on reasoning and return empty visible text.

When NoThinking is set, explicitly send reasoning_effort: low so the
model spends as few output tokens as possible on reasoning. We use
"low" rather than "minimal" because older models (o3-mini, o1) only
accept low/medium/high.

Also normalize the reasoning field in the VCR request matcher so
cassettes recorded with or without reasoning config still match.

Fixes docker#2318
The previous budget of 20 max_output_tokens was sufficient for
non-reasoning models, but reasoning models (o-series, gpt-5) include
hidden reasoning tokens in that same budget. Even with low effort,
20 tokens is too tight -- the model can exhaust them on reasoning and
produce no visible title text.

Raise the budget to 256, which gives reasoning models enough room for
low reasoning overhead plus a short title.

Refs docker#2318
@trungutt trungutt marked this pull request as ready for review April 14, 2026 13:45
@trungutt trungutt requested a review from a team as a code owner April 14, 2026 13:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Title generation with gpt-5 fails

1 participant