Skip to content

AWS Bedrock Converse API Prompt Caching not supported #8158

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
5 tasks done
isaac-j-miller opened this issue May 11, 2025 · 1 comment
Open
5 tasks done

AWS Bedrock Converse API Prompt Caching not supported #8158

isaac-j-miller opened this issue May 11, 2025 · 1 comment
Labels
auto:bug Related to a bug, vulnerability, unexpected error with an existing feature

Comments

@isaac-j-miller
Copy link

Checked other resources

  • I added a very descriptive title to this issue.
  • I searched the LangChain.js documentation with the integrated search.
  • I used the GitHub search to find a similar question and didn't find it.
  • I am sure that this is a bug in LangChain.js rather than my code.
  • The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

the following code:

import { BedrockRuntimeClient } from '@aws-sdk/client-bedrock-runtime';
import { ChatBedrockConverse } from '@langchain/aws';
import {
    SystemMessage,
    HumanMessage
} from '@langchain/core/messages';

const bedrock = new BedrockRuntimeClient({region:"us-east-1"});
const chatBedrockConverse = new ChatBedrockConverse({
    model: "us.amazon.nova-lite-v1:0",
    client: bedrock
});

const messages = [
  new SystemMessage({content:[
    {
        type: "text",
        text: "my system prompt"
    },
    {
        cachePoint: {
        type: "default"
        }
    }
  ]
}),
new HumanMessage("foo bar")
]

await chatBedrockConverse.invoke(messages);
// throws "System message content must be either a string, or a content array containing a single text object."
// from https://github.com/langchain-ai/langchainjs/blob/main/libs/langchain-aws/src/common.ts:355

Error Message and Stack Trace (if applicable)

"System message content must be either a string, or a content array containing a single text object."
from https://github.com/langchain-ai/langchainjs/blob/main/libs/langchain-aws/src/common.ts:355

Description

I'm trying to use prompt caching, per https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html, with the Converse API. It works with langchain in python.

System Info

[email protected] | MIT | deps: 12 | versions: 319
Typescript bindings for langchain
https://github.com/langchain-ai/langchainjs/tree/main/langchain/

keywords: llm, ai, gpt3, chain, prompt, prompt engineering, chatgpt, machine learning, ml, openai, embeddings, vectorstores

dist
.tarball: https://registry.npmjs.org/langchain/-/langchain-0.3.24.tgz
.shasum: 24c5e16f7a43be8ce54d32079ea8dc8087c56777
.integrity: sha512-BTjiYkUCpWFAmufK8J5zMqc5aUs4eEnAXPWtPe2+R4ZPP+U7bXJSBHAcrB40rQ3VeTdRgMvgDjekOOgCMWut6Q==
.unpackedSize: 4.8 MB

dependencies:
@langchain/openai: >=0.1.0 <0.6.0 js-yaml: ^4.1.0 openapi-types: ^12.1.3 yaml: ^2.2.1
@langchain/textsplitters: >=0.0.0 <0.2.0 jsonpointer: ^5.0.1 p-retry: 4 zod-to-json-schema: ^3.22.3
js-tiktoken: ^1.0.12 langsmith: ^0.3.16 uuid: ^10.0.0 zod: ^3.22.4

maintainers:

dist-tags:
latest: 0.3.24 next: 0.3.2-rc.0 tag-for-publishing-older-releases: 0.2.20

published 2 weeks ago by basproul [email protected]

Using Node.js v22, on Fedora

@dosubot dosubot bot added the auto:bug Related to a bug, vulnerability, unexpected error with an existing feature label May 11, 2025
@kishan0725
Copy link

kishan0725 commented May 23, 2025

Any update on this? @benjamincburns

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto:bug Related to a bug, vulnerability, unexpected error with an existing feature
Projects
None yet
Development

No branches or pull requests

2 participants