Skip to content

fix(lite_llm): propagate Gemini grounding metadata from ModelResponse#5661

Open
1wos wants to merge 3 commits into
google:mainfrom
1wos:fix-litellm-grounding
Open

fix(lite_llm): propagate Gemini grounding metadata from ModelResponse#5661
1wos wants to merge 3 commits into
google:mainfrom
1wos:fix-litellm-grounding

Conversation

@1wos
Copy link
Copy Markdown

@1wos 1wos commented May 11, 2026

Link to Issue

Problem

LiteLLM exposes Gemini's grounding metadata on ModelResponse.vertex_ai_grounding_metadata rather than inside the message, and _model_response_to_generate_content_response in lite_llm.py never reads it. As a result LlmResponse.grounding_metadata is always None when Gemini is called through LiteLlm, breaking event.grounding_metadata, after_model_callback, and citation pipelines for the entire LiteLlm path. The native Gemini() path picks grounding up from candidate.grounding_metadata (llm_response.py:190).

Solution

Pull vertex_ai_grounding_metadata off the ModelResponse after the message is converted, and attach it to LlmResponse.grounding_metadata. Three shapes are accepted:

  • types.GroundingMetadata instance → used as-is.
  • dict → validated via types.GroundingMetadata.model_validate.
  • list (one entry per candidate) → first entry, then the same dispatch.

If validation fails the handler logs a warning and leaves grounding unset, so a single malformed payload doesn't break the rest of the response.

Testing Plan

Unit Tests in tests/unittests/models/test_litellm.py:

  • test_model_response_to_generate_content_response_grounding_metadata_dict — dict payload is propagated.
  • test_model_response_to_generate_content_response_grounding_metadata_list — list payload uses the first entry.
  • test_model_response_to_generate_content_response_no_grounding_metadata — missing attribute leaves grounding as None (no regression for non-Gemini LiteLlm models).

Full test_litellm.py suite passes locally:

259 passed in 9.36s

Manual E2E: Live Gemini API verification was not possible locally (no API key configured). The unit tests cover the conversion function the bug report points to.

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end. (See above — requires live Gemini access.)

@google-cla
Copy link
Copy Markdown

google-cla Bot commented May 11, 2026

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@rohityan rohityan self-assigned this May 12, 2026
@rohityan rohityan added the models [Component] Issues related to model support label May 12, 2026
@rohityan rohityan added the request clarification [Status] The maintainer need clarification or more information from the author label May 12, 2026
@rohityan
Copy link
Copy Markdown
Collaborator

Hi @1wos , Thank you for your contribution! We appreciate you taking the time to submit this pull request. can you please fix the failing test by running autoformat.sh

@1wos 1wos force-pushed the fix-litellm-grounding branch from 22e1e91 to 57851ed Compare May 12, 2026 08:42
@1wos
Copy link
Copy Markdown
Author

1wos commented May 12, 2026

@rohityan rohityanDone. Thanks for the review!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

models [Component] Issues related to model support request clarification [Status] The maintainer need clarification or more information from the author

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LiteLlm drops Gemini grounding_metadata: _model_response_to_generate_content_response ignores response.vertex_ai_grounding_metadata

2 participants