Skip to content

Commit 278ba5a

Browse files
committed
update readme
1 parent aa6c3f5 commit 278ba5a

File tree

1 file changed

+136
-1
lines changed

1 file changed

+136
-1
lines changed

README.md

Lines changed: 136 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,4 +7,139 @@ Manages Nginx for reverse proxy to multiple LLMs, with TLS & Bearer Auth tokens.
77
- Uses Let's Encrypt for TLS certificates
88
- Uses certbot for certificate issuance and renewal
99
- Uses Nginx as a public-domain reverse proxy to add TLS
10-
- Uses JWT for bearer authentication
10+
- Uses JWT for bearer authentication
11+
- Auth & nginx enpoints are IP restricted.
12+
13+
***All requests to `/v1/*` are proxied to the LLM APIs except for `/v1/models`***
14+
15+
`/v1/models` is a special endpoint that returns the list of models available from all LLM APIs.
16+
17+
## How to use
18+
19+
Docker compose is going to be the easiest way to get up and running, but you could also manually run the docker image. Before you do anything else, if you don't have a cloudflare account, sign up now - it's free. You will need to create an API Token with the "Zone", "DNS", "Edit" permissions. This will be used when issuing your certs to verify that you own the domain you want to use for TLS. After you have your key, set up a new DNS record to point to your IP address. This can be proxied on the cloudflare end.
20+
21+
After this, you'll set up your files, start the container, hit a few routes and you'll be good to go!
22+
23+
#### **Don't forget to forward port 443 on your router!**
24+
25+
I'll use `localhost`, `192.168.1.100` or `your.domain.com` as an examples, but fill these in with your domain or IP address.
26+
27+
### Files
28+
29+
Here's what you'll need in your docker-compose file:
30+
```yaml
31+
version: '3.6'
32+
33+
services:
34+
llmp:
35+
image: ghcr.io/j4ys0n/llm-proxy:1.1.5
36+
container_name: llmp
37+
hostname: llmp
38+
restart: unless-stopped
39+
ports:
40+
- 8080:8080
41+
- 443:443
42+
volumes:
43+
- .env:/app/.env # environment variables
44+
- ./data:/app/data # any data that the app needs to persist
45+
- ./cloudflare_credentials:/opt/cloudflare/credentials # cloudflare api token
46+
- ./nginx:/etc/nginx/conf.d # nginx configs
47+
- ./certs:/etc/letsencrypt # tsl certificates
48+
```
49+
50+
Here's what your `.env` file should look like:
51+
```bash
52+
PORT=8080 # node.js listen port. right now nginx is hard coded, so don't change this.
53+
TARGET_URLS=http://localhost:1234,http://192.168.1.100:1234 # list of api endpoints (don't add /v1)
54+
JWT_SECRET=randomly_generated_secret # secret for JWT token generation, change this!
55+
AUTH_USERNAME=admin
56+
AUTH_PASSWORD=secure_password # super basic auth credentials for the admin interface
57+
```
58+
59+
Here's what your cloudflare_credendials file should look like
60+
```bash
61+
dns_cloudflare_api_token = your_token_here
62+
```
63+
64+
### Routes
65+
66+
You'll need to use the local, unsecured endpoints to get set up initially. The `/auth/token` endpoint is the only endpoint that does't need an Authorization header and token.
67+
68+
Generate tokens.
69+
70+
`POST http://192.168.1.100:8080/auth/token`
71+
```json
72+
{
73+
"username": "admin",
74+
"password": "secure_password"
75+
}
76+
```
77+
response:
78+
```json
79+
{
80+
"token": "generated_token_here"
81+
}
82+
```
83+
84+
#### All of the routes below need a bearer token in the Authorization header.
85+
`Authorization: Bearer generated_token_here`
86+
87+
Get TLS certificates.
88+
89+
`POST http://192.168.1.100:8080/nginx/certificates/obtain`
90+
```json
91+
{
92+
"domains": ["your.domain.com"]
93+
}
94+
```
95+
response:
96+
```json
97+
{
98+
"success": true,
99+
"message": "Certificates obtained successfully."
100+
}
101+
```
102+
103+
Write default config with your domain. (this should be sufficient for you, fill in your domain and cider groups)
104+
105+
Note: you can add multiple CIDR groups if you have multiple internal IP ranges you want admin functions to be accessible to. This is all of the routes that start with `/auth` or `/nginx`.
106+
107+
Hint: `192.168.1.0/24` will allow all IPs from `192.168.1.1` - `192.168.1.254`. `192.168.1.111/32` will only allow `192.168.1.111`.
108+
109+
`POST http://192.168.1.100:8080/nginx/config/write-default`
110+
```json
111+
{
112+
"domain": "your.domain.com",
113+
"cidrGroups": ["192.168.1.0/24"]
114+
}
115+
```
116+
response:
117+
```json
118+
{
119+
"success": true,
120+
"message": "Default config written successfully"
121+
}
122+
```
123+
124+
Reload nginx to apply changes.
125+
`GET http://192.168.1.100:8080/nginx/reload`
126+
response:
127+
```json
128+
{
129+
"success": true,
130+
"message": "Nginx configuration reloaded successfully."
131+
}
132+
```
133+
134+
***If you made it here, you should be good to go!***
135+
136+
Other available endpoints (these will be documented better in the future)
137+
138+
`GET /nginx/config/get` - get current nginx config as a string.
139+
140+
`POST /nginx/config/update` - update the nginx config with a custom domain.
141+
Body: `{ "config": string }`
142+
143+
`GET /nginx/config/get-default` - get default nginx config template.
144+
145+
`GET /nginx/certificates/renew` - renew certificates for your domains.

0 commit comments

Comments
 (0)