explain feedback trace_id= arg (#815)

Author: baskaryan

docs/evaluation/how_to_guides/attach_user_feedback.mdx

@@ -4,47 +4,84 @@ sidebar_position: 1
 
 import {
   CodeTabs,
-  PythonBlock,
-  TypeScriptBlock,
+  python,
+  typescript,
 } from "@site/src/components/InstructionsWithCode";
 
-# Log user feedback
+# Log user feedback using the SDK
 
-:::tip Recommended Reading
-Before diving into this content, it might be helpful to read the following:
+:::tip Key concepts
 
 - [Conceptual guide on tracing and feedback](../../../observability/concepts)
 - [Reference guide on feedback data format](/reference/data_formats/feedback_data_format)
 
 :::
 
-In many applications, but even more so for LLM applications, it is important to collect user feedback to understand how your application is performing in real-world scenarios.
-The ability to observe user feedback along with trace data can be very powerful to drill down into the most interesting datapoints, then send those datapoints for further review, automatic evaluation, or even datasets.
-To learn more about how to filter traces based on various attributes, including user feedback, see [this guide](../../../observability/how_to_guides/filter_traces_in_application)
+LangSmith makes it easy to attach feedback to traces. 
+This feedback can come from users, annotators, automated evaluators, etc., and is crucial for monitoring and evaluating applications.
 
-LangSmith makes it easy to attach user feedback to traces.
-It's often helpful to expose a simple mechanism (such as a thumbs-up, thumbs-down button) to collect user feedback for your application responses. You can then use the LangSmith SDK or API to send feedback for a trace. To get the `run_id` of a logged run, see [this guide](../../../observability/how_to_guides/access_current_span).
+## Use [create_feedback()](https://docs.smith.langchain.com/reference/python/client/langsmith.client.Client#langsmith.client.Client.create_feedback) / [createFeedback()](https://docs.smith.langchain.com/reference/js/classes/client.Client#createfeedback)
 
-:::note
+Here we'll walk through how to log feedback using the SDK.
 
-You can attach user feedback to ANY intermediate run (span) of the trace, not just the root span.
-This is useful for critiquing specific parts of the LLM application, such as the retrieval step or generation step of the RAG pipeline.
+:::info Child runs
+
+You can attach user feedback to ANY child run of a trace, not just the trace (root run) itself.
+This is useful for critiquing specific steps of the LLM application, such as the retrieval step or generation step of a RAG pipeline.
+
+:::
+
+:::tip Non-blocking creation (Python only)
+
+The Python client will automatically background feedback creation if you pass `trace_id=` to [create_feedback()](https://docs.smith.langchain.com/reference/python/client/langsmith.client.Client#langsmith.client.Client.create_feedback).
+This is essential for low-latency environments, where you want to make sure your application isn't blocked on feedback creation.
 
 :::
 
 <CodeTabs
   tabs={[
-    PythonBlock(`from langsmith import Client\n
-client = Client()\n
-# ... Run your application and get the run_id...
-# This information can be the result of a user-facing feedback form\n
-client.create_feedback(
-    run_id,
-    key="feedback-key",
-    score=1.0,
-    comment="comment",
-)`),
-    TypeScriptBlock(`import { Client } from "langsmith";
+    python({caption: "Requires `langsmith >= 0.3.43`"})
+`
+      from langsmith import trace, traceable, Client
+
+      @traceable
+      def foo(x):
+          return {"y": x * 2}
+
+      @traceable
+      def bar(y):
+          return {"z": y - 1}
+
+      client = Client()
+
+      inputs = {"x": 1}
+      with trace(name="foobar", inputs=inputs) as root_run:
+          result = foo(**inputs)
+          result = bar(**result)
+          root_run.outputs = result
+          trace_id = root_run.id
+          child_runs = root_run.child_runs
+
+      # Provide feedback for a trace (a.k.a. a root run)
+      client.create_feedback(
+          key="user_feedback",
+          score=1,
+          trace_id=trace_id,
+          comment="the user said that ..."
+      )
+
+      # Provide feedback for a child run
+      foo_run_id = [run for run in child_runs if run.name == "foo"][0].id
+      client.create_feedback(
+          key="correctness",
+          score=0,
+          run_id=foo_run_id,
+          # trace_id= is optional but recommended to enable batched and backgrounded 
+          # feedback ingestion.
+          trace_id=trace_id,
+      )
+    `,
+    typescript({})`import { Client } from "langsmith";
 const client = new Client();\n
 // ... Run your application and get the run_id...
 // This information can be the result of a user-facing feedback form\n
@@ -55,7 +92,11 @@ await client.createFeedback(
         score: 1.0,
         comment: "comment",
     }
-);`),
+);`,
   ]}
   groupId="client-language"
 />
+
+You can even log feedback for in-progress runs using `create_feedback() / createFeedback()`. See [this guide](../../../observability/how_to_guides/access_current_span) for how to get the run ID of an in-progress run.
+
+To learn more about how to filter traces based on various attributes, including user feedback, see [this guide](../../../observability/how_to_guides/filter_traces_in_application).

docs/observability/how_to_guides/index.md

@@ -73,6 +73,6 @@ Leverage LangSmith's powerful monitoring, automation, and online evaluation feat
 
 ## Human feedback
 
-- [Log user feedback](../evaluation/how_to_guides/attach_user_feedback)
+- [Log user feedback using the SDK](../evaluation/how_to_guides/attach_user_feedback)
 - [Set up a new feedback criteria](../evaluation/how_to_guides/set_up_feedback_criteria)
 - [Annotate traces inline in the UI](../evaluation/how_to_guides/annotate_traces_inline)