fix: fallback invalid attached file with gemini#5472
fix: fallback invalid attached file with gemini#5472Lin-Nikaido wants to merge 7 commits intogoogle:mainfrom
Conversation
|
Is silently not processing the attachment like this really better than raising an error? This fallback path you're adding doesn't provide the LLM with the data from the invalid attached file. That will lead to hallucinations when the model sees a file reference and invents data for it. Also, you are not correct that this is what is done for LiteLlm. For LiteLlm, an error is raised if the mime type is not supported: adk-python/src/google/adk/models/lite_llm.py Lines 1091 to 1094 in 2d61cb6 |
|
@mynewestgitaccount adk-python/src/google/adk/models/lite_llm.py Lines 1112 to 1126 in fc45fa6 In this case, the fallback happens because it is anthropic model, not invalid MIME-type (it is my missunderstanding). So today there are two different behaviors for attachment failures:
Both cases result in a file not being actually processed, but the behavior differs. That inconsistency was what confused me, and it may confuse other developers as well. I think we should make this behavior consistent in one direction:
Given your hallucination concern, I agree that explicitly raising an error is probably the safer behavior. |
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
1. Link to an existing issue (if applicable):
Problem:
gemini crash when invalid attached file type. raise 400 Error.
No fallback.
Solution:
Add fallback same as LiteLlm
Testing Plan
Please describe the tests that you ran to verify your changes. This is required
for all PRs that are not small documentation or typo fixes.
Unit Tests:
Add 2 tests
tests.unittests.models.test_google_llm.test_preprocess_request_unsupported_mime_typetests.unittests.models.test_google_llm.test_preprocess_request_supported_mime_typeAnd also passed tests
Manual End-to-End (E2E) Tests:
send and ask detail with attach file
sample.pptxChecklist
Additional context