feat: Add Application configuration

To use the project, Cognito user need a Cognito group (for example admin, user).
Chatbot design was replaced to use CloudScape design pattern.

- Support Amazon Nova LLMs Integration.
- Add bedrock guardrails to all models in the langchain interface.
- Add document support to chatbot sessions.
- Add custom system prompts for non-bedrock models.
- Add happy path selenium tests.
- Update chatbot to use latest cloudscape components.
- Create configurable applications with frontend implementation.

- Bug fixes:
	- Fixed video understanding input.
	- Fixed integration tests following appsec findings.
	- Fixed input validation & presign URL role verification.
	- Fixed federation + application update.
	- Fixed RSS Feed, Application validation, RDS permissions.
	- Fixed semantic search validation.
	- Fixed import and unit test snapshots.
- Security improvements:
	- Disabled Appsync introspection.
	- Updated security documentation.
- Access control improvements:
	- Added access control to APIs.
	- Added unauthorized unit tests.
	- Implemented sendQuery access control.
	- Added role API.
	- Hide application attributes for non-admin users.

Co-authored-by: Nikolai Grinko <[email protected]>
Co-authored-by: Charles Marion <[email protected]>

Author: 3 people

NOTICE

@@ -258,7 +258,7 @@ Author: Leonard Richardson
 
 The following Python packages may be included in this product:
 
-- numpy==1.26.0
+- numpy==1.26.4
 
 These packages each contain the following license and notice below:
 

README.md

@@ -1,4 +1,4 @@
-# Deploying a Multi-Model and Multi-RAG Powered Chatbot Using AWS CDK on AWS
+# Building RAG use cases with GenAI Chatbot on AWS
 
 [![Release Notes](https://img.shields.io/github/v/release/aws-samples/aws-genai-llm-chatbot)](https://github.com/aws-samples/aws-genai-llm-chatbot/releases)
 [![GitHub star chart](https://img.shields.io/github/stars/aws-samples/aws-genai-llm-chatbot?style=social)](https://star-history.com/#aws-samples/aws-genai-llm-chatbot)
@@ -8,7 +8,7 @@
 
 [![Full Documentation](https://img.shields.io/badge/Full%20Documentation-blue?style=for-the-badge&logo=Vite&logoColor=white)](https://aws-samples.github.io/aws-genai-llm-chatbot/)
 
-![sample](docs/about/assets/chabot-sample.gif "AWS GenAI Chatbot")
+![sample](docs/about/assets/chabot-sample.gif "GenAI Chatbot on AWS")
 
 
 ## 🚀 NEW! Support for new Amazon Nova Models 🚀
@@ -35,7 +35,7 @@ This solution provides ready-to-use code so you can start **experimenting with a
 
 Supported model providers:
 
-- [Amazon Bedrock](https://aws.amazon.com/bedrock/) which supports a wide range of models from AWS, Anthropic, Cohere and Mistral including the lastst models from Amazon Nova. See [Recent announcements](#) for more details.
+- [Amazon Bedrock](https://aws.amazon.com/bedrock/) which supports a wide range of models from AWS, Anthropic, Cohere and Mistral including the latest models from Amazon Nova. See [Recent announcements](https://aws.amazon.com/blogs/aws/introducing-amazon-nova-frontier-intelligence-and-industry-leading-price-performance/) for more details.
 - [Amazon SageMaker](https://aws.amazon.com/sagemaker/) self-hosted models from Foundation, Jumpstart and HuggingFace.
 - Third-party providers via API such as Anthropic, Cohere, AI21 Labs, OpenAI, etc. [See available langchain integrations](https://python.langchain.com/docs/integrations/llms/) for a comprehensive list.
 

boto3-1.35.58-py3-none-any.whl

@@ -354,7 +354,9 @@ async function processCreateOptions(options: any): Promise<void> {
     {
       type: "confirm",
       name: "guardrailsEnable",
-      message: "Do you want to enable Bedrock Guardrails",
+      message:
+        "Do you want to enable Bedrock Guardrails? This is a recommended feature to build responsible AI applications." +
+        " (Supported by all models except Idefics via SageMaker. If enabled, streaming will only work with Bedrock)",
       initial: options.guardrailsEnable ?? false,
     },
     {
@@ -381,7 +383,7 @@ async function processCreateOptions(options: any): Promise<void> {
     {
       type: "confirm",
       name: "enableSagemakerModels",
-      message: "Do you want to use any text generation Sagemaker Models",
+      message: "Do you want to use any text generation SageMaker Models",
       initial: options.enableSagemakerModels || false,
     },
     {

botocore-1.35.58-py3-none-any.whl

@@ -2,8 +2,8 @@ import { defineConfig } from 'vitepress'
 
 // https://vitepress.dev/reference/site-config
 export default defineConfig({
-  title: "AWS GenAI LLM Chatbot",
-  description: "Deploying a Multi-Model and Multi-RAG Powered Chatbot Using AWS CDK on AWS",
+  title: "GenAI Chatbot on AWS",
+  description: "Building RAG use cases with GenAI Chatbot on AWS",
   base: "/aws-genai-llm-chatbot/",
   themeConfig: {
     // https://vitepress.dev/reference/default-theme-config
@@ -44,13 +44,16 @@ export default defineConfig({
         text: 'Guide',
         items: [
           { text: 'Deploy', link: '/guide/deploy' },
+          { text: 'Configuration', link: '/guide/config' },
           { text: 'Developer Guide', link: '/guide/developers' },
           { text: 'Development Prioritization', link: '/guide/prioritization' },
         ]
       },
       {
         text: 'Documentation',
         items: [
+          { text: 'Access Control', link: '/documentation/access-control' },
+          { text: 'Applications', link: '/documentation/applications' },
           { text: 'AppSync', link: '/documentation/appsync' },
           { text: 'CloudFront Geo Restriction', link: '/documentation/cf-geo-restriction' },
           {
@@ -67,7 +70,7 @@ export default defineConfig({
           { text: 'Precautions', link: '/documentation/precautions' },
           { text: 'Private Chatbot', link: '/documentation/private-chatbot' },
           { text: 'SageMaker Schedule', link: '/documentation/sagemaker-schedule' },
-          { text: 'Security', link: '/documentation/vulnerability-scanning' },
+          { text: 'Security', link: '/documentation/security' },
           { text: 'Self-hosted models', link: '/documentation/self-hosted-models' },
         ]
       }

cli/magic-config.ts

@@ -16,6 +16,8 @@ Please note: This is list is considered a best effort approach to helping you un
 * **User Pool** [**Required**]
 * **User Pool Client** [**Required**]
     * Attach an existing Cognito Domain for Federation, adds Permission for User Pool to leverage existing Cognito Domain / Creates OIDC or SAML Identity Provider in Cognito [*Optional*]
+* **User Groups** [**Required**]
+    * Admin and workspace_manager roles allow applications (configurable chatbots) and workspace management retrospectively.
 
 ## Retrieval Augmented Generation (RAG) Engines
 This section describes the RAG engines that house and return stored data for use with Generative AI. Additionally this section includes resources deployed to support data ingestion and processing for RAG.

docs/.vitepress/config.mts

@@ -14,7 +14,7 @@ Supported model providers:
 
 Deploy [IDEFICS](https://huggingface.co/blog/idefics) models on [Amazon SageMaker](https://aws.amazon.com/sagemaker/) and see how the chatbot can answer questions about images, describe visual content, generate text grounded in multiple images.
 
-![sample](./assets/multimodal-sample.gif "AWS GenAI Chatbot")
+![sample](./assets/multimodal-sample.gif "GenAI Chatbot on AWS")
 
 Currently, the following multimodal models are supported:
 
@@ -36,13 +36,13 @@ and select one of the supported models (press Space to select/deselect) from the
 
 Send the same query to 2 to 4 separate models at once and see how each one responds based on its own learned history, context and access to the same powerful document retriever, so all requests can pull from the same up-to-date knowledge.
 
-![sample](./assets/multichat-sample.gif "AWS GenAI Chatbot")
+![sample](./assets/multichat-sample.gif "GenAI Chatbot on AWS")
 
 ## Experiment with multiple RAG options with Workspaces
 
 A workspace is a logical namespace where you can upload files for indexing and storage in one of the vector databases. You can select the embeddings model and text-splitting configuration of your choice.
 
-![sample](./assets/create-workspace-sample.gif "AWS GenAI Chatbot")
+![sample](./assets/create-workspace-sample.gif "GenAI Chatbot on AWS")
 
 ## Unlock RAG potentials with Workspaces Debugging Tools
 
@@ -52,7 +52,7 @@ The solution comes with several debugging tools to help you debug RAG scenarios:
 - Test different embeddings models directly in the UI
 - Test cross encoders and analyse distances from different functions between sentences.
 
-![sample](./assets/workspace-debug-sample.gif "AWS GenAI Chatbot")
+![sample](./assets/workspace-debug-sample.gif "GenAI Chatbot on AWS")
 
 ## Full-fledged User Interface
 
@@ -63,3 +63,11 @@ Protected with [Amazon Cognito Authentication](https://aws.amazon.com/cognito/)
 The interface layer between the UI and backend is built with [AppSync](https://docs.aws.amazon.com/appsync/latest/devguide/what-is-appsync.html) for management requests and for realtime interaction with chatbot (messages and responses) using GraphQL subscriptions.
 
 Design system provided by [AWS Cloudscape Design System](https://cloudscape.design/).
+
+## Configure and customize chatbots for different domains
+
+The solution supports creating configurable chatbots called **applications**. Admin users can define applications by selecting a model, model settings, workspace, and system prompts. Configured applications are then available for regular users based on their assigned role, allowing controlled access to pre-configured chat experiences. 
+
+Roles determine which applications a user can access - for example, a "customer support" role may only be able to use applications tailored for handling support inquiries from users. Additional roles can be created as needed to cater applications to specific user groups or departments.
+
+![sample](./assets/application.gif "GenAI Chatbot on AWS")

docs/about/assets/application.gif

@@ -2,7 +2,7 @@
 layout: doc
 ---
 
-# Deploying a Multi-Model and Multi-RAG Powered Chatbot Using AWS CDK on AWS
+# Building RAG use cases with GenAI Chatbot on AWS
 
 [![Release Notes](https://img.shields.io/github/v/release/aws-samples/aws-genai-llm-chatbot)](https://github.com/aws-samples/aws-genai-llm-chatbot/releases)
 
@@ -12,9 +12,9 @@ layout: doc
 
 [![Deploy with GitHub Codespaces](https://github.com/codespaces/badge.svg)](#deploy-with-github-codespaces)
 
-The AWS GenAI LLM Chatbot provides ready-to-use code so you can start experimenting with a variety of Large Language Models and Multimodal Language Models, settings and prompts in your own AWS account.
+The GenAI Chatbot on AWS provides ready-to-use code so you can start experimenting with a variety of Large Language Models and Multimodal Language Models, settings and prompts in your own AWS account.
 
-![sample](./assets/chabot-sample.gif "AWS GenAI Chatbot")
+![sample](./assets/chabot-sample.gif "GenAI Chatbot on AWS")
 
 Want to find out more? Continue to [Features](./features).
 

docs/about/aws-resources-deployed.md

@@ -0,0 +1,132 @@
+# Access Control
+
+**NOTE**
+
+You need to assign the admin role to a user to be able to use the project after the first deployment revision with this feature. Users without a role will not be able to use the Chatbot.
+
+
+Users can have one or more roles defining their permissions. It can be used to restrict access to the functionalities as defined below.
+
+
+Rubicon has 3 pre-defined roles. They are created during deployment by CDK code. 
+1. admin - _full access_
+2. workspace_manager - _full access except the admin pages such as managing applications_
+3. user - _access to assigned applications_
+
+Instead of using pre-defined `user` role you can create new roles in Cognito and assign them to applications.
+
+
+## Create users and Roles
+
+Users are created in Cognito user pool - [Create new users](https://docs.aws.amazon.com/cognito/latest/developerguide/how-to-create-user-accounts.html). If Cognito federation is used then users should be created in third-party identity provider. See [Cognito Federation](./cognito/overview.md) configuration examples. 
+
+User roles are defined using [Cognito user group](https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-pools-user-groups.html). When using federation with Cognito, the user's role is assigned on user log in reading the "custom:chatbot_role" user attribute. 
+
+
+## Page Access by Role
+
+* **admin**
+    * ✅ Home (documentation)
+    * ✅ Chatbot - Playground
+    * ✅ Chatbot - Multi-chat playground
+    * ✅ Chatbot - Sessions
+    * ✅ Chatbot - Models
+    * ✅ RAG - Dashboard
+    * ✅ RAG - Semantic search
+    * ✅ RAG - Workspaces
+    * ✅ RAG - Embeddings
+    * ✅ RAG - Engines
+    * ✅ Admin - Applications
+    * ✅ Applications - End user view
+
+* **workspaces_manager**
+    * ✅ Home (documentation)
+    * ✅ Chatbot - Playground
+    * ✅ Chatbot - Multi-chat playground
+    * ✅ Chatbot - Sessions
+    * ✅ Chatbot - Models
+    * ✅ RAG - Dashboard
+    * ✅ RAG - Semantic search
+    * ✅ RAG - Workspaces
+    * ✅ RAG - Embeddings
+    * ✅ RAG - Engines
+    * ⛔️ Admin - Applications
+    * ✅ Applications - End user view
+
+* **user**: 
+It could be a role with any name which is associated with an application.
+    * ⛔️ Home (documentation)
+    * ⛔️ Chatbot - Playground
+    * ⛔️ Chatbot - Multi-chat playground
+    * ⛔️ Chatbot - Sessions
+    * ⛔️ Chatbot - Models
+    * ⛔️ RAG - Dashboard
+    * ⛔️ RAG - Semantic search
+    * ⛔️ RAG - Workspaces
+    * ⛔️ RAG - Embeddings
+    * ⛔️ RAG - Engines
+    * ⛔️ Admin - Applications
+    * ✅ Applications - End user view
+
+
+## GraphQL operations access by Role
+
+* **Workspace**
+    * createKendraWorkspace - _admin, workspace_manager_
+    * startKendraDataSync - _admin, workspace_manager_
+    * isKendraDataSynching - _admin, workspace_manager_
+    * createBedrockKBWorkspace - _admin, workspace_manager_
+    * createOpenSearchWorkspace - _admin, workspace_manager_
+    * createAuroraWorkspace - _admin, workspace_manager_
+    * performSemanticSearch - _admin, workspace_manager_
+    * listWorkspaces - _admin, workspace_manager_
+    * listKendraIndexes - _admin, workspace_manager_
+    * listBedrockKnowledgeBases - _admin, workspace_manager_
+    * listRagEngines - _admin, workspace_manager_
+    * getWorkspace - _admin, workspace_manager_
+    * deleteWorkspace - _admin, workspace_manager_
+
+* **Document**
+    * addWebsite - _admin, workspace_manager_
+    * addRssFeed - _admin, workspace_manager_
+    * updateRssFeed - _admin, workspace_manager_
+    * setDocumentSubscriptionStatus - _admin, workspace_manager_
+    * addQnADocument - _admin, workspace_manager_
+    * addTextDocument - _admin, workspace_manager_
+    * getUploadFileURL - _any authenticated user_
+    * getDocument - _admin, workspace_manager_
+    * getRSSPosts - _admin, workspace_manager_
+    * listDocuments - _admin, workspace_manager_
+    * deleteDocument - _admin, workspace_manager_
+
+* **Session**
+    * addUserFeedback - _any authenticated user_
+    * deleteSession - _any authenticated user_
+    * deleteUserSessions - _any authenticated user_
+    * listSessions - _any authenticated user_
+    * getSession - _any authenticated user_
+
+* **Model**
+    * listModels - _admin, workspace_manager_
+    * listEmbeddingModels - _admin, workspace_manager_
+    * listCrossEncoders - _admin, workspace_manager_
+    * rankPassages - _admin, workspace_manager_
+    * calculateEmbeddings - _admin, workspace_manager_
+    * sendQuery - _any authenticated user_
+    * receiveMessages - _any authenticated user_
+
+* **Application**
+    * createApplication - _admin_
+    * updateApplication - _admin_
+    * deleteApplication - _admin_
+    * getApplication - _admin, workspace_manager, any other role that is assigned to the application_
+    * listApplications - _admin, workspace_manager, any other role that is assigned to the application_
+
+* **Role**
+    * listRoles - _admin_
+
+* **Internal**
+    * publishResponse - _IAM Can only be called by the Lambda forwarding the chatbot responses_
+
+* **Other**
+    * checkHealth - _any authenticated user_

docs/about/features.md

@@ -0,0 +1,38 @@
+# Applications
+
+An application is a pre-configured chatbot which could be embedded to a website. Admin users can define applications by selecting a model, model settings, workspace, and system prompts. Configured applications are then available for regular users based on their assigned role, allowing controlled access to pre-configured chat experiences. 
+
+Roles determine which applications a user can access - for example, a "customer support" role may only be able to use applications tailored for handling support inquiries from users. Admin and workspace_manager roles are created during deployment. Additional roles can be created by creating [Cognito groups](https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-pools-user-groups.html). Applications can then be assigned to one or more roles restricting what model and workspace an end user can use.
+
+
+## Application creation and usage
+![sample](../about/assets/application.gif "GenAI Chatbot on AWS")
+
+
+## Iframe integration
+1. Deploy Rubicon.
+2. Copy `GenAIChatBotStack.UserInterfaceUserInterfaceDomainName` value from the deployment output. It has `dxxxxxxxxxxxxx.cloudfront.net` format.
+3. Open UI dxxxxxxxxxxxxx.cloudfront.net.
+4. Create an application and open it (the URL will be used in the next step).
+5. Create an Iframe with the application URL as a source.
+
+```html
+<!DOCTYPE html>
+<html>
+<head>
+    <title>Chat Embed Test</title>
+</head>
+<body>
+    <iframe 
+        src="https://dxxxxxxxxxxxxx.cloudfront.net/application/408a4758-018b-4147-ac76-ee9c169ba775"
+        width="400"
+        height="480"
+        allow="microphone"
+        style="border: 1px solid #ccc;"
+    ></iframe>
+</body>
+</html>
+```
+
+### Example
+![Application](./assets/end-user-application.png "End-user chatbot")

docs/about/welcome.md

@@ -1,22 +1,9 @@
 # Using AppSync
 
-Define or change the schema in `./lib/chatbot-api/schema`.
+The project relies on [AWS AppSync](https://docs.aws.amazon.com/appsync/latest/devguide/what-is-appsync.html) which creates serverless GraphQL and Pub/Sub APIs that simplify application development through a single endpoint to securely query, update, or publish data. One of the key advantages is [GraphQL subscriptions](https://docs.aws.amazon.com/appsync/latest/devguide/aws-appsync-real-time-data.html) which is used to receive the chatbot responses in real time using streaming.
 
-At the moment we only use the `schema-ws.graphql` to define the real-time API. The REST API might be replaced by AppSync in the future.
+### Schema
+The GraphQL Schema defining the possible operatiosn can be found in `lib/chatbot-api/schema/schema.graphql`. Once deployed, AWS AppSync provides a Querying tool on the [AWS Console](https://us-east-1.console.aws.amazon.com/appsync/home?region=us-east-1) which can be used to explore and test the APIs.
 
-If you modified the definition for the schema, you can regenerate the client code using
-
-```bash
-cd lib/user-interface/react-app
-npx @npx @aws-amplify/cli codegen add --apiId <api_id> --region <region>
-```
-
-Accept all the defaults.
-
-If you use a None data source, you need to modify `src/API.ts` adding:
-
-```ts
-export type NoneQueryVariables = {
-  none?: string | null;
-};
-```
+### Authentication
+AWS AppSync supports built in authentication modes. This project relies on the [@aws_cognito_user_pools one](https://docs.aws.amazon.com/appsync/latest/devguide/security-authz.html#using-additional-authorization-modes) to authenticate Amazon Cognito users.