fix: title generation fails with OpenAI reasoning models#2412
Open
trungutt wants to merge 2 commits intodocker:mainfrom
Open
fix: title generation fails with OpenAI reasoning models#2412trungutt wants to merge 2 commits intodocker:mainfrom
trungutt wants to merge 2 commits intodocker:mainfrom
Conversation
OpenAI reasoning models (o-series, gpt-5) always reason internally; omitting the reasoning parameter does not disable reasoning -- it uses the model's default effort. Those hidden reasoning tokens count against max_output_tokens, so with a small budget (e.g. title generation) the model can exhaust all tokens on reasoning and return empty visible text. When NoThinking is set, explicitly send reasoning_effort: low so the model spends as few output tokens as possible on reasoning. We use "low" rather than "minimal" because older models (o3-mini, o1) only accept low/medium/high. Also normalize the reasoning field in the VCR request matcher so cassettes recorded with or without reasoning config still match. Fixes docker#2318
The previous budget of 20 max_output_tokens was sufficient for non-reasoning models, but reasoning models (o-series, gpt-5) include hidden reasoning tokens in that same budget. Even with low effort, 20 tokens is too tight -- the model can exhaust them on reasoning and produce no visible title text. Raise the budget to 256, which gives reasoning models enough room for low reasoning overhead plus a short title. Refs docker#2318
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Title generation with OpenAI reasoning models (gpt-5, o-series) fails with
empty title output from modelbecause these models always reason internally, even when no reasoning configuration is provided. The hidden reasoning tokens consume the entire output token budget, leaving nothing for the visible title text.When the code intended to "skip reasoning", it simply omitted the reasoning parameter from the API request. This does not disable reasoning - the model defaults to its standard reasoning behavior, which can easily exhaust a small token budget.
The output token budget (20 tokens) was sized for non-reasoning models and far too small once reasoning overhead is factored in.
Fix
How to test
Each command should print a short title to stdout without errors.
Fixes #2318