Improve the Chartsmith VS Code Extension (#91)

* improve the chartsmith extension

* clean up debug

* clean up debug

* re-add tsconfig

* fix missing extension config

* re-add tsconfig

* remove changelog

* re-add main.go

* fix apiURL in fileContent module

* remove demo content generation in dev mode

* remove www endpoint and just use api endpoint

* Revert "remove www endpoint and just use api endpoint"

This reverts commit 6a3bf73.

* update DEVELOPMENT.MD

* fix description for apiEndpoint

* remove unneeded commands from the palette

* make first user an admin user

* remove individual file content route

* remove run-postgres make target

* use env var for postgres uri for schemahero

* update ARCHITECTURE.md

* cleanup makefile

* remove validateBearerToken use findSession

* Remove deprecated individual file endpoint functions

* Remove deprecated individual file endpoint functions

* remove mock tests

* consolidate endpoints to api endpoint

* fix missing diff buttons

* remove debug code

* disable check-env

* disable check-env

Author: diamonwiggins

.gitignore

@@ -3,3 +3,4 @@ bin
 .aider*
 test-results/
 .envrc
+.specstory/

ARCHITECTURE.md

@@ -9,6 +9,14 @@ It's made for both the developer working on it and for AI models to read and app
 - We have a Postres/pgvector database and Centrifugo for realtime notifications.
 - The intent is to keep this system design and avoid new databases, queues, components. Simplicity matters.
 
+## API Design Principles
+- Prefer consolidated data endpoints over granular ones to minimize API calls and database load
+- Structure API routes using Next.js's file-based routing with resource-oriented paths
+- Implement consistent authentication and error handling patterns across endpoints
+- Return complete data objects rather than fragments to reduce follow-up requests
+- Prioritize server-side data processing over client-side assembly of multiple API calls
+
+
 # Subprojects
 - See chartsmith-app/ARCHITECTURE.md for the architecture principles for the front end.
 

CONTRIBUTING.md

@@ -82,7 +82,18 @@ NEXT_PUBLIC_API_ENDPOINT=http://localhost:3000/api
    ```
    This deploys the Schemahero migrations to set up the database schema.
 
-6. **Terminal 4: Claude Integration**
+6. **Terminal 4: Bootstrap Chart Data**
+   ```bash
+   make bootstrap
+   ```
+   This is a **critical step** that initializes the chart data in the database. Without this step, the application won't have the necessary template data to function properly.
+
+7. **Admin Access**
+   
+   The first user to log in will automatically be granted admin privileges and bypass the waitlist.
+   You can log in at: http://localhost:3000/login?test-auth=true
+
+8. **Terminal 5: Claude Integration (optional)**
    ```bash
    # Use Claude for development assistance
    ```
@@ -94,7 +105,6 @@ NEXT_PUBLIC_API_ENDPOINT=http://localhost:3000/api
   make run-worker
   ```
 
-
 ### Troubleshooting
 
 If you encounter any issues:
@@ -103,6 +113,15 @@ If you encounter any issues:
 2. Verify all required secrets are properly configured
 3. Check that Schemahero is installed and accessible in your PATH
 4. Make sure all dependencies are installed (both Go and npm packages)
+5. If you get an error `ERROR: type "vector" does not exist` when running `make schema`, you can manually enable the PGVector extension:
+   ```bash
+   docker exec -it chartsmith-dev-postgres-1 psql -U postgres -d chartsmith -c "CREATE EXTENSION IF NOT EXISTS vector;"
+   ```
+   Or simply run the Make target that handles this:
+   ```bash
+   make pgvector
+   ```
+   After enabling the extension, run `make schema` again (though it now automatically runs the pgvector target as a prerequisite).
 
 ### Development Workflow
 
@@ -118,6 +137,17 @@ If you encounter any issues:
 - The frontend runs on the default Next.js port
 - The worker runs on a separate process
 
+## VS Code Extension Development
+
+For detailed instructions on developing the VS Code extension, see [chartsmith-extension/DEVELOPMENT.md](chartsmith-extension/DEVELOPMENT.md). 
+
+This guide covers:
+- Building and installing the extension from a VSIX file
+- Configuring endpoints for local development
+- Enabling development mode
+- Debugging with the developer console
+- Testing extension features with built-in commands
+
 ## Release
 
 All releases are automated using various Dagger functions.

Makefile

@@ -1,36 +1,110 @@
-
 WORKER_BINARY_NAME=chartsmith-worker
 WORKER_BUILD_DIR=bin
 
 GOOS?=$(shell go env GOOS)
 GOARCH?=$(shell go env GOARCH)
 
-.PHONY: run-postgres
-run-postgres:
-	docker run -d --name chartsmith-postgres -e POSTGRES_PASSWORD=postgres -e POSTGRES_DB=chartsmith -p 5432:5432 postgres:16
+# =============================================================================
+# REQUIRED ENVIRONMENT VARIABLES (must be exported by the user)
+# =============================================================================
+# The following variables MUST be exported before running commands like:
+# - make run-worker
+# - make bootstrap
+# - make run-debug-console
+#
+# Required variables:
+#   - ANTHROPIC_API_KEY - API key for Anthropic services
+#   - GROQ_API_KEY - API key for Groq services
+#   - VOYAGE_API_KEY - API key for Voyage services
+#   - CHARTSMITH_PG_URI - PostgreSQL connection string
+#   - CHARTSMITH_CENTRIFUGO_ADDRESS - Centrifugo service address
+#   - CHARTSMITH_CENTRIFUGO_API_KEY - API key for Centrifugo
+#   - GOOGLE_CLIENT_ID - Google OAuth client ID
+#   - GOOGLE_CLIENT_SECRET - Google OAuth client secret
+#
+# Example:
+#   export ANTHROPIC_API_KEY=your-key
+#   export GROQ_API_KEY=your-key
+#   export VOYAGE_API_KEY=your-key
+#   export CHARTSMITH_PG_URI=postgresql://postgres:password@localhost:5432/chartsmith?sslmode=disable
+#   export CHARTSMITH_CENTRIFUGO_ADDRESS=http://localhost:8000/api
+#   export CHARTSMITH_CENTRIFUGO_API_KEY=api_key
+#   export GOOGLE_CLIENT_ID=your-id
+#   export GOOGLE_CLIENT_SECRET=your-secret
+#   make run-worker
+# =============================================================================
+
+# Environment variables checker helper function
+define check_env_var
+	@if [ -z "$(shell printenv $(1))" ]; then \
+		echo "Error: $(1) environment variable is not set"; \
+		echo "Please set this in your shell environment:"; \
+		echo "  export $(1)='your-$(1)-value'"; \
+		echo ""; \
+		echo "See CONTRIBUTING.md for more information about required environment variables."; \
+		exit 1; \
+	fi
+endef
+
+# Check required environment variables
+.PHONY: check-env
+check-env:
+	$(call check_env_var,ANTHROPIC_API_KEY)
+	$(call check_env_var,GROQ_API_KEY)
+	$(call check_env_var,VOYAGE_API_KEY)
+	$(call check_env_var,CHARTSMITH_PG_URI)
+	$(call check_env_var,CHARTSMITH_CENTRIFUGO_ADDRESS)
+	$(call check_env_var,CHARTSMITH_CENTRIFUGO_API_KEY)
+	$(call check_env_var,GOOGLE_CLIENT_ID)
+	$(call check_env_var,GOOGLE_CLIENT_SECRET)
+	@echo "All required environment variables are set"
+
+# =============================================================================
+# DATABASE COMMANDS
+# =============================================================================
+
+.PHONY: pgvector
+pgvector:
+	@echo "Ensuring pgvector extension is enabled..."
+	@PG_CONTAINER=$$(docker ps --format '{{.Names}}' | grep postgres | head -n1); \
+	if [ -z "$$PG_CONTAINER" ]; then \
+		echo "Error: No running Postgres container found"; \
+		echo "Make sure to start the development environment first:"; \
+		echo "  cd hack/chartsmith-dev && docker compose up -d"; \
+		exit 1; \
+	fi; \
+	echo "Using Postgres container: $$PG_CONTAINER"; \
+	docker exec -i $$PG_CONTAINER psql -U postgres -d chartsmith -c "CREATE EXTENSION IF NOT EXISTS vector;"; \
+	echo "PGVector extension enabled"
 
 .PHONY: schema
-schema:
+schema: pgvector
+	@echo "Running schema commands..."
 	rm -rf ./db/generated-schema
 	mkdir -p ./db/generated-schema/tables
-	schemahero plan --driver postgres --uri $(CHARTSMITH_PG_URI) --spec-file ./db/schema/tables --spec-type table --out ./db/generated-schema/tables
-	schemahero apply --driver postgres --uri $(CHARTSMITH_PG_URI) --ddl ./db/generated-schema/tables
+	schemahero plan --driver postgres --uri "$(CHARTSMITH_PG_URI)" --spec-file ./db/schema/tables --spec-type table --out ./db/generated-schema/tables
+	schemahero apply --driver postgres --uri "$(CHARTSMITH_PG_URI)" --ddl ./db/generated-schema/tables
+
+# =============================================================================
+# DEVELOPMENT COMMANDS
+# =============================================================================
 
 .PHONY: build
 build:
 	@echo "Building $(WORKER_BINARY_NAME)..."
 	@mkdir -p $(WORKER_BUILD_DIR)
 	@go build -o $(WORKER_BUILD_DIR)/$(WORKER_BINARY_NAME) main.go
 
+# Requires: ANTHROPIC_API_KEY, GROQ_API_KEY, VOYAGE_API_KEY
 .PHONY: run-worker
 run-worker: build
-	@echo "Running $(WORKER_BINARY_NAME)..."
-	@./$(WORKER_BUILD_DIR)/$(WORKER_BINARY_NAME) run --
+	@echo "Running $(WORKER_BINARY_NAME) with environment variables from shell..."
+	./$(WORKER_BUILD_DIR)/$(WORKER_BINARY_NAME) run --
 
 .PHONY: bootstrap
 bootstrap: build
 	@echo "Bootstrapping chart..."
-	@./$(WORKER_BUILD_DIR)/$(WORKER_BINARY_NAME) bootstrap \
+	./$(WORKER_BUILD_DIR)/$(WORKER_BINARY_NAME) bootstrap \
 		--force
 
 .PHONY: test-data
@@ -49,22 +123,30 @@ integration-test: build
 	@echo "Running integration tests..."
 	@./$(WORKER_BUILD_DIR)/$(WORKER_BINARY_NAME) integration
 
+# =============================================================================
+# CI/CD AND DEPLOYMENT COMMANDS
+# =============================================================================
+
 .PHONY: validate
 validate:
 	dagger call validate \
 		--op-service-account env:OP_SERVICE_ACCOUNT \
 		--progress plain
 
-.POHNY: okteto-dev
+.PHONY: okteto-dev
 okteto-dev:
 	@go mod download -x
 	@make build
 	@printf "\n\n To build and run this project, run: \n\n   # make run-worker\n   # make run-debug-console\n\n"
 
+# Requires: ANTHROPIC_API_KEY, GROQ_API_KEY, VOYAGE_API_KEY
 .PHONY: run-debug-console
 run-debug-console:
-	DB_URI="postgres://postgres:password@localhost:5432/chartsmith?sslmode=disable" go run main.go debug-console
+	@echo "Running debug console with environment variables from shell..."
+	@# We set DB_URI to maintain compatibility with existing code
+	export DB_URI=$(CHARTSMITH_PG_URI) && go run main.go debug-console
 
+# Requires: GITHUB_TOKEN, OP_SERVICE_ACCOUNT_PRODUCTION
 .PHONY: release
 release:
 	dagger call release \
@@ -73,6 +155,7 @@ release:
 		--op-service-account env:OP_SERVICE_ACCOUNT_PRODUCTION \
 		--progress plain
 
+# Requires: GITHUB_TOKEN, OP_SERVICE_ACCOUNT_PRODUCTION
 .PHONY: replicated
 replicated:
 	dagger call release \
@@ -85,6 +168,7 @@ replicated:
 		--op-service-account env:OP_SERVICE_ACCOUNT_PRODUCTION \
 		--progress plain
 
+# Requires: REPLICATED_API_TOKEN
 .PHONY: replicated-dev
 replicated-dev:
 	dagger call release-dev-replicated \
@@ -93,6 +177,7 @@ replicated-dev:
 		--api-token env:REPLICATED_API_TOKEN \
 		--progress plain
 
+# Requires: GITHUB_TOKEN, OP_SERVICE_ACCOUNT_PRODUCTION
 .PHONY: production
 production:
 	dagger call release \

README.md

@@ -6,3 +6,18 @@ Build Better Helm Charts
 
 Chartsmith is an AI-powered tool that helps you build better Helm charts.
 
+## Development
+
+See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup instructions.
+
+## Authentication
+
+### Extension Authentication
+
+The VS Code extension authenticates using a token-based mechanism:
+
+1. When a user clicks "Login" in the extension, it opens a browser window to the authentication page
+2. After successful authentication, the app generates an extension token and sends it to the extension
+3. The extension stores this token and uses it for API requests with a Bearer token header
+4. Token validation happens via the `/api/auth/status` endpoint
+

chartsmith-app/README.md

@@ -34,3 +34,20 @@ You can check out [the Next.js GitHub repository](https://github.com/vercel/next
 The easiest way to deploy your Next.js app is to use the [Vercel Platform](https://vercel.com/new?utm_medium=default-template&filter=next.js&utm_source=create-next-app&utm_campaign=create-next-app-readme) from the creators of Next.js.
 
 Check out our [Next.js deployment documentation](https://nextjs.org/docs/app/building-your-application/deploying) for more details.
+
+## Authentication
+
+### Extension Authentication
+
+The VS Code extension authenticates using a token-based mechanism:
+
+1. When a user clicks "Login" in the extension, it opens a browser window to the authentication page
+2. After successful authentication, the app generates an extension token and sends it to the extension
+3. The extension stores this token and uses it for API requests with a Bearer token header
+4. Token validation happens via the `/api/auth/status` endpoint
+
+To test extension authentication:
+```bash
+# Run the auth status test script
+./test-auth-status.sh <your-token>
+```

chartsmith-app/app/api/auth/status/route.ts

@@ -0,0 +1,29 @@
+import { NextResponse } from 'next/server';
+import { findSession } from '@/lib/auth/session';
+
+export async function GET(request: Request) {
+  try {
+    // Get token from authorization header
+    const authHeader = request.headers.get('authorization');
+    if (!authHeader || !authHeader.startsWith('Bearer ')) {
+      return NextResponse.json({ error: 'Unauthorized' }, { status: 401 });
+    }
+
+    const token = authHeader.substring(7); // Remove 'Bearer ' prefix
+    
+    // Find session using the token
+    const session = await findSession(token);
+    if (!session) {
+      return NextResponse.redirect(new URL('/login', request.url));
+    }
+
+    // Return complete user info and success status
+    return NextResponse.json({ 
+      user: session.user,
+      isAuthenticated: true
+    });
+  } catch (error) {
+    console.error('Error in auth status endpoint:', error);
+    return NextResponse.redirect(new URL('/login', request.url));
+  }
+} 

chartsmith-app/app/login-with-test-auth/page.tsx

@@ -69,4 +69,4 @@ export default function TestAuthPage() {
       </Card>
     </div>
   );
-}
+}