Skip to content

fix: fallback invalid attached file with gemini#5472

Open
Lin-Nikaido wants to merge 7 commits intogoogle:mainfrom
Lin-Nikaido:feat/fallback-invalid-attached-file-with-gemini
Open

fix: fallback invalid attached file with gemini#5472
Lin-Nikaido wants to merge 7 commits intogoogle:mainfrom
Lin-Nikaido:feat/fallback-invalid-attached-file-with-gemini

Conversation

@Lin-Nikaido
Copy link
Copy Markdown
Contributor

@Lin-Nikaido Lin-Nikaido commented Apr 24, 2026

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

1. Link to an existing issue (if applicable):

Problem:
gemini crash when invalid attached file type. raise 400 Error.
No fallback.

Solution:
Add fallback same as LiteLlm

Testing Plan

Please describe the tests that you ran to verify your changes. This is required
for all PRs that are not small documentation or typo fixes.

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.

Add 2 tests

  • tests.unittests.models.test_google_llm.test_preprocess_request_unsupported_mime_type
  • tests.unittests.models.test_google_llm.test_preprocess_request_supported_mime_type

And also passed tests

Manual End-to-End (E2E) Tests:
send and ask detail with attach file sample.pptx

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules.

Additional context

Lin-Nikaido added 5 commits April 24, 2026 23:51
…ile-with-gemini' into feat/fallback-invalid-attached-file-with-gemini
@adk-bot adk-bot added the models [Component] Issues related to model support label Apr 24, 2026
@Lin-Nikaido Lin-Nikaido changed the title Feat/fallback invalid attached file with gemini fix: fallback invalid attached file with gemini Apr 24, 2026
@rohityan rohityan self-assigned this Apr 24, 2026
@mynewestgitaccount
Copy link
Copy Markdown

Is silently not processing the attachment like this really better than raising an error? This fallback path you're adding doesn't provide the LLM with the data from the invalid attached file. That will lead to hallucinations when the model sees a file reference and invents data for it.

Also, you are not correct that this is what is done for LiteLlm. For LiteLlm, an error is raised if the mime type is not supported:

raise ValueError(
"LiteLlm(BaseLlm) does not support content part with MIME type "
f"{part.inline_data.mime_type}."
)

@Lin-Nikaido
Copy link
Copy Markdown
Contributor Author

Lin-Nikaido commented Apr 25, 2026

@mynewestgitaccount
Thank you. I misunderstood the current behavior.
In my case, the request with .pptx file in part.file and anthropic provided llm do fallback in this line

if _requires_file_uri_fallback(provider, model, part.file_data.file_uri):
logger.debug(
"File URI %s not supported for provider %s, using text fallback",
_redact_file_uri_for_log(
part.file_data.file_uri,
display_name=part.file_data.display_name,
),
provider,
)
identifier = part.file_data.display_name or part.file_data.file_uri
content_objects.append({
"type": "text",
"text": f'[File reference: "{identifier}"]',
})
continue

In this case, the fallback happens because it is anthropic model, not invalid MIME-type (it is my missunderstanding).
For unsupported inline_data MIME types, LiteLLM currently raises a ValueError, as you pointed out.

So today there are two different behaviors for attachment failures:

  1. provider/file URI incompatibility → silent fallback to text reference
  2. unsupported MIME type → explicit error

Both cases result in a file not being actually processed, but the behavior differs. That inconsistency was what confused me, and it may confuse other developers as well.

I think we should make this behavior consistent in one direction:

  • either always raise an explicit error when the file cannot actually be processed
  • or always fallback intentionally

Given your hallucination concern, I agree that explicitly raising an error is probably the safer behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

models [Component] Issues related to model support

Projects

None yet

Development

Successfully merging this pull request may close these issues.

google gemini raises a raw API error when invalid attached file type google gemini crash when invalid attached file type

4 participants