Skip to content

genai : Extract and include grounding_supports, grounding_chunks, and web_sea… #941

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

SYED-M-HUSSAIN
Copy link
Contributor

PR Description

This pull request fixes an issue where grounding metadata from the Google search tool responses was not preserved. It ensures that fields such as grounding_supports, grounding_chunks, and web_search_queries are extracted and included in response_metadata, enabling citation tracking and improved traceability in search-augmented responses.


Relevant issues

Fixes #907


Type

🐛 Bug Fix


Changes (optional)

  • Added _extract_grounding_metadata() helper function to safely extract grounding data.
  • Updated _response_to_result() to include grounding_supports, grounding_chunks, and web_search_queries in response_metadata.
  • Preserves backward compatibility with the existing response structure.

Testing (optional)

  • Manually verified that metadata is preserved across multiple Google search tool responses.
  • Confirmed presence of citation information in augmented responses.

Note (optional)

This fix improves the reliability of citations and source tracking in responses using Google search, supporting downstream systems relying on structured grounding metadata.

@SYED-M-HUSSAIN
Copy link
Contributor Author

Example Code

from google.ai.generativelanguage_v1beta.types import Tool as GenAITool
from langchain_google_genai import ChatGoogleGenerativeAI

# Define the LLM with Gemini 2.5 Flash preview model
llm = ChatGoogleGenerativeAI(model="gemini-2.5-flash-preview-04-17")

# Invoke the LLM with Google Search Tool enabled
resp = llm.invoke(
    "When is the next total solar eclipse in US?",
    tools=[GenAITool(google_search={})],
)

print(resp)

🔎 Before My Change

The model returned a response without grounded sources :

content='Okay, let\'s clarify based on the recent eclipse:\n\n1.  **The most recent total solar eclipse visible across a large portion of the contiguous US was on April 8, 2024.**\n\n2.  **The *absolute next* total solar eclipse visible in *any* US territory will be on March 30, 2033.** This one will be visible primarily in **Alaska**.\n\n3.  **The *next major* total solar eclipse to cross a significant portion of the *contiguous* United States (lower 48 states) will be on August 12, 2045.** This eclipse will have a path stretching from California to Florida.\n\nSo, depending on what you mean by "in US" (any territory vs. contiguous states), the answer is either **March 30, 2033 (Alaska)** or **August 12, 2045 (Contiguous US)**.' additional_kwargs={} response_metadata={'prompt_feedback': {'block_reason': 0, 'safety_ratings': []}, 'finish_reason': 'STOP', 'safety_ratings': []} id='run--d434c211-31c7-4dd5-9687-d61a55cadae9-0' usage_metadata={'input_tokens': 11, 'output_tokens': 203, 'total_tokens': 1102, 'input_token_details': {'cache_read': 0}}

🚀 After My Change

After enabling the google_search tool, the model response was grounded with real-time verified web sources and reflected improved factual accuracy:

content='The next total solar eclipse visible in the United States will occur on March 30, 2033, which will be visible over Alaska.\n\nFollowing the 2033 event, the contiguous United States will experience a total solar eclipse on August 23, 2044, with the path of totality crossing over Montana, North Dakota, and South Dakota. Another total solar eclipse with a coast-to-coast path across the US is anticipated on August 12, 2045, stretching from California to Florida.' additional_kwargs={} response_metadata={'prompt_feedback': {'block_reason': 0, 'safety_ratings': []}, 'finish_reason': 'STOP', 'model_name': 'models/gemini-2.5-flash-preview-04-17', 'safety_ratings': [], 'grounding_metadata': {'grounding_chunks': [{'web': {'uri': 'https://vertexaisearch.cloud.google.com/grounding-api-redirect/AbF9wXE1kr7ISat-otQwxzwCWm30cUU8DDPmFm8OF6oCTGaG0Y9CdRfH4L-ytp9B1cuj_9MFmpdd5yMJZsabEQTe8-DXeEL44aohfemCPElLpGdughHxse0FgkhNzkGIIg4hb_t4LWnLuTCXXmwcbuf1-wXHXMI=', 'title': 'wikipedia.org'}}, {'web': {'uri': 'https://vertexaisearch.cloud.google.com/grounding-api-redirect/AbF9wXGc7QDYHzpfGW7ZM1-fa3I297vfQZ1UBm6aQ5PmgJktm0ocY60MP21bjUNiB9sFTX_iZKCWIu8oy3iD8IkWOK7U-uFVMbIMALp1cpBS_TlS-Is6', 'title': 'nationaleclipse.com'}}, {'web': {'uri': 'https://vertexaisearch.cloud.google.com/grounding-api-redirect/AbF9wXHthFOEknQ_ubJRC31rMux4QVDyWtV2T1I9_UMnoLqb-ncjZ_NI3FxLMvZT8oXL9nTl4UnoR_AJAevje4DAJax7oilRv8_Aox2plp3HkD30Pg0KzOzRUMZHImMBsIrfR8QpEHsM8Q8xrsjD9jxCIp9j9kRK4sc7', 'title': 'cbsnews.com'}}], 'grounding_supports': [{'segment': {'start_index': 0, 'end_index': 122, 'text': 'The next total solar eclipse visible in the United States will occur on March 30, 2033, which will be visible over Alaska.'}, 'grounding_chunk_indices': [0, 1], 'confidence_scores': [1.0, 1.0]}, {'segment': {'start_index': 124, 'end_index': 317, 'text': 'Following the 2033 event, the contiguous United States will experience a total solar eclipse on August 23, 2044, with the path of totality crossing over Montana, North Dakota, and South Dakota.'}, 'grounding_chunk_indices': [2, 0, 1], 'confidence_scores': [1.0, 1.0, 1.0]}, {'segment': {'start_index': 318, 'end_index': 460, 'text': 'Another total solar eclipse with a coast-to-coast path across the US is anticipated on August 12, 2045, stretching from California to Florida.'}, 'grounding_chunk_indices': [2, 1], 'confidence_scores': [1.0, 1.0]}], 'web_search_queries': ['next total solar eclipse in US']}} id='run--8628b5bf-a2e7-4fd2-8a62-acf84c38c058-0' usage_metadata={'input_tokens': 11, 'output_tokens': 131, 'total_tokens': 343, 'input_token_details': {'cache_read': 0}, 'output_token_details': {'reasoning': 151}}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Native Tools Response metadata missing
1 participant