Skip to content

feat: support AWS bedrock base models #25

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 46 commits into
base: main
Choose a base branch
from

Conversation

detunjiSamuel
Copy link

@detunjiSamuel detunjiSamuel commented Jan 9, 2025

AWS Bedrock Provider Integration

Added support for AWS Bedrock as a new LLM provider:

  • Anthropic Claude models
  • Amazon Titan models (text & embeddings)
  • AI21 Jurassic models

Key Changes

Added Bedrock provider implementation with model-specific handlers:

  • Anthropic: Reused existing Anthropic provider code with Bedrock-specific modifications ( no actual changes)
  • Titan: Added chat completion and embedding support
  • AI21: Added completion and chat completion support
  • Request/response mapping between Hub formats and Bedrock formats

Testing Notes

All tests pass using AWS credentials in us-east-1/2 regions
Verified error handling for invalid credentials/models
Tested non-streaming responses ( models in Bedrock don't seem to have streaming types )

Review notes

The model ID from AWS link does not work consistently.
Instead, use the Inference profile ARN or Inference profile ID from the cross-region reference tab as your model_id.

Issue: #20

/claim #20

@CLAassistant
Copy link

CLAassistant commented Jan 9, 2025

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ detunjiSamuel
❌ nirga
You have signed the CLA already but the status is still pending? Let us recheck it.

@detunjiSamuel detunjiSamuel marked this pull request as draft January 9, 2025 03:18
@detunjiSamuel detunjiSamuel changed the title Feat : Support AWS bedrock base models [ DRAFT ] Feat : Support AWS bedrock base models Jan 9, 2025
@detunjiSamuel
Copy link
Author

Hey @nirga ,

Quick question about integrating Stability AI models in our Hub. I'm looking at AWS Bedrock's Stable Diffusion 3.5 integration (from their model catalog).

Not sure about the best format to implement this:

  1. Should we use the existing chat_completions since it takes a similar prompt format?
  2. Or would it make more sense to create a new Content type specifically for image generation?

Would appreciate your thoughts on this.

Thanks!

@nirga
Copy link
Member

nirga commented Jan 9, 2025

I think it should be in a new api @detunjiSamuel
But we can do it in a separate PR

@detunjiSamuel detunjiSamuel marked this pull request as ready for review January 10, 2025 23:11
@detunjiSamuel
Copy link
Author

Hey @nirga , I have completed this PR and would appreciate your review.
I've verified the changes through:

  • External Python file testing
  • Manual console verification
  • Updated test suite
  • Results exported to Traceloop from the Pipeline.

Please let me know if any additional information or changes are needed.

Thank you!

Copy link

algora-pbc bot commented Jan 10, 2025

💵 To receive payouts, sign up on Algora, link your Github account and connect with Stripe.

detunjiSamuel and others added 25 commits April 1, 2025 14:16
… into support_aws_bedrock_as_provider

# Conflicts:
#	Cargo.toml
#	src/providers/bedrock/logs/amazon_titan_embed_text_v2_0_embeddings.json
#	src/providers/bedrock/logs/anthropic_claude_3_haiku_20240307_v1_0_chat_completion.json
#	src/providers/bedrock/logs/us_amazon_nova_lite_v1_0_chat_completion.json
#	src/providers/bedrock/mod.rs
#	src/providers/bedrock/models.rs
#	src/providers/bedrock/test.rs
@nirga nirga changed the title Feat : Support AWS bedrock base models feat: support AWS bedrock base models May 1, 2025
@nirga
Copy link
Member

nirga commented May 1, 2025

Hey @detunjiSamuel - I want to merge this PR, can you sign the CLA?

@nirga
Copy link
Member

nirga commented May 7, 2025

Ping @detunjiSamuel - can you sign the CLA?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants