docs: multimodal #777
docs: multimodal #777
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
|
You'll need to add to the "Run an evaluation with multimodal content" page because the tabs mess up the table of contents :( |
|
|
||
|  | ||
|
|
||
| ## Define custom evaluators |
There was a problem hiding this comment.
I'd move this section up above the update section and follow the same section header as the SDK. So have Run evaluations as the header and then Create a multimodal prompt, define custom evaluators underneath
would also help to give an examples of what IS possible. Eg:
Even without multimodal support in your evaluators, you can still run text‐only evaluations. For example:
- OCR → text correction: Use a vision model to extract text from a document, then evaluate the accuracy of the extracted output.
- Speech‑to‑text → transcription quality: Use a voice model to transcribe audio to text, then evaluate the transcription against your reference.
| Some applications are based around multimodal content, like a chatbot that can answer questions about a PDF or image. | ||
| In these cases, you'll want to include multimodal content in your prompt and test the model's ability to answer questions about the content. | ||
|
|
||
| There are two ways to include multimodal content in a prompt: |
There was a problem hiding this comment.
any diff between the two that we should call out? how does a user know which approach to pick?
|
This is a really cool capability!! I would also make a tutorial about this (docs version of catherine's video) |
Co-authored-by: Tanushree <[email protected]>
Co-authored-by: Tanushree <[email protected]>
Co-authored-by: Tanushree <[email protected]>
Co-authored-by: Tanushree <[email protected]>
Co-authored-by: Tanushree <[email protected]>
…docs into maddy/multimodal
Co-authored-by: Tanushree <[email protected]>
Co-authored-by: Tanushree <[email protected]>
…docs into maddy/multimodal
No description provided.