Skip to content

Commit 9170191

Browse files
TimPietruskyTimPietrusky
authored
feat: added FLUX.1 schnell & dev
1 parent 4dd5d29 commit 9170191

File tree

7 files changed

+354
-35
lines changed

7 files changed

+354
-35
lines changed

‎.github/workflows/dev.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ on:
88

99
jobs:
1010
dev:
11-
runs-on: ubuntu-latest-m
11+
runs-on: ubuntu-latest-l
1212
steps:
1313
- name: Checkout
1414
uses: actions/checkout@v3
@@ -55,4 +55,4 @@ jobs:
5555
*.args.DOCKERHUB_REPO=${{ env.DOCKERHUB_REPO }}
5656
*.args.DOCKERHUB_IMG=${{ env.DOCKERHUB_IMG }}
5757
*.args.RELEASE_VERSION=${{ env.RELEASE_VERSION }}
58-
sd3.args.HUGGINGFACE_ACCESS_TOKEN=${{ env.HUGGINGFACE_ACCESS_TOKEN }}
58+
*.args.HUGGINGFACE_ACCESS_TOKEN=${{ env.HUGGINGFACE_ACCESS_TOKEN }}

‎Dockerfile

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,16 @@ RUN if [ "$MODEL_TYPE" = "sdxl" ]; then \
5757
wget -O models/vae/sdxl-vae-fp16-fix.safetensors https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/resolve/main/sdxl_vae.safetensors; \
5858
elif [ "$MODEL_TYPE" = "sd3" ]; then \
5959
wget --header="Authorization: Bearer ${HUGGINGFACE_ACCESS_TOKEN}" -O models/checkpoints/sd3_medium_incl_clips_t5xxlfp8.safetensors https://huggingface.co/stabilityai/stable-diffusion-3-medium/resolve/main/sd3_medium_incl_clips_t5xxlfp8.safetensors; \
60+
elif [ "$MODEL_TYPE" = "flux1-schnell" ]; then \
61+
wget -O models/unet/flux1-schnell.safetensors https://huggingface.co/black-forest-labs/FLUX.1-schnell/resolve/main/flux1-schnell.safetensors && \
62+
wget -O models/clip/clip_l.safetensors https://huggingface.co/comfyanonymous/flux_text_encoders/resolve/main/clip_l.safetensors && \
63+
wget -O models/clip/t5xxl_fp8_e4m3fn.safetensors https://huggingface.co/comfyanonymous/flux_text_encoders/resolve/main/t5xxl_fp8_e4m3fn.safetensors && \
64+
wget -O models/vae/ae.safetensors https://huggingface.co/black-forest-labs/FLUX.1-schnell/resolve/main/ae.safetensors; \
65+
elif [ "$MODEL_TYPE" = "flux1-dev" ]; then \
66+
wget --header="Authorization: Bearer ${HUGGINGFACE_ACCESS_TOKEN}" -O models/unet/flux1-dev.safetensors https://huggingface.co/black-forest-labs/FLUX.1-dev/resolve/main/flux1-dev.safetensors && \
67+
wget -O models/clip/clip_l.safetensors https://huggingface.co/comfyanonymous/flux_text_encoders/resolve/main/clip_l.safetensors && \
68+
wget -O models/clip/t5xxl_fp8_e4m3fn.safetensors https://huggingface.co/comfyanonymous/flux_text_encoders/resolve/main/t5xxl_fp8_e4m3fn.safetensors && \
69+
wget --header="Authorization: Bearer ${HUGGINGFACE_ACCESS_TOKEN}" -O models/vae/ae.safetensors https://huggingface.co/black-forest-labs/FLUX.1-dev/resolve/main/ae.safetensors; \
6070
fi
6171

6272
# Stage 3: Final image

‎README.md

Lines changed: 55 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -19,27 +19,30 @@ Read our article here: https://blib.la/blog/comfyui-on-runpod
1919
- [Quickstart](#quickstart)
2020
- [Features](#features)
2121
- [Config](#config)
22-
- [Upload image to AWS S3](#upload-image-to-aws-s3)
22+
* [Upload image to AWS S3](#upload-image-to-aws-s3)
2323
- [Use the Docker image on RunPod](#use-the-docker-image-on-runpod)
24+
* [Create your template (optional)](#create-your-template-optional)
25+
* [Create your endpoint](#create-your-endpoint)
26+
* [GPU recommendations](#gpu-recommendations)
2427
- [API specification](#api-specification)
25-
- [JSON Request Body](#json-request-body)
26-
- [Fields](#fields)
27-
- ["input.images"](#inputimages)
28+
* [JSON Request Body](#json-request-body)
29+
* [Fields](#fields)
30+
+ ["input.images"](#inputimages)
2831
- [Interact with your RunPod API](#interact-with-your-runpod-api)
29-
- [Health status](#health-status)
30-
- [Generate an image](#generate-an-image)
31-
- [Example request with cURL](#example-request-with-curl)
32+
* [Health status](#health-status)
33+
* [Generate an image](#generate-an-image)
34+
+ [Example request for SDXL with cURL](#example-request-for-sdxl-with-curl)
3235
- [How to get the workflow from ComfyUI?](#how-to-get-the-workflow-from-comfyui)
3336
- [Bring Your Own Models and Nodes](#bring-your-own-models-and-nodes)
34-
- [Network Volume](#network-volume)
35-
- [Custom Docker Image](#custom-docker-image)
37+
* [Network Volume](#network-volume)
38+
* [Custom Docker Image](#custom-docker-image)
3639
- [Local testing](#local-testing)
37-
- [Setup](#setup)
38-
- [Setup for Windows](#setup-for-windows)
39-
- [Testing the RunPod handler](#testing-the-runpod-handler)
40-
- [Local API](#local-api)
41-
- [Access the local Worker API](#access-the-local-worker-api)
42-
- [Access local ComfyUI](#access-local-comfyui)
40+
* [Setup](#setup)
41+
+ [Setup for Windows](#setup-for-windows)
42+
* [Testing the RunPod handler](#testing-the-runpod-handler)
43+
* [Local API](#local-api)
44+
+ [Access the local Worker API](#access-the-local-worker-api)
45+
+ [Access local ComfyUI](#access-local-comfyui)
4346
- [Automatically deploy to Docker hub with GitHub Actions](#automatically-deploy-to-docker-hub-with-github-actions)
4447
- [Acknowledgments](#acknowledgments)
4548

@@ -49,15 +52,14 @@ Read our article here: https://blib.la/blog/comfyui-on-runpod
4952

5053
## Quickstart
5154

52-
- 🐳 Choose one of the three available images for your serverless endpoint:
53-
- `timpietruskyblibla/runpod-worker-comfy:3.0.0-base`: doesn't contain any checkpoints, just a clean ComfyUI
54-
- `timpietruskyblibla/runpod-worker-comfy:3.0.0-sdxl`: contains the checkpoints and VAE for Stable Diffusion XL
55-
- Checkpoint: [sd_xl_base_1.0.safetensors](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0)
56-
- VAEs:
57-
- [sdxl_vae.safetensors](https://huggingface.co/stabilityai/sdxl-vae/)
58-
- [sdxl-vae-fp16-fix](https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/)
59-
- `timpietruskyblibla/runpod-worker-comfy:3.0.0-sd3`: contains the [sd3_medium_incl_clips_t5xxlfp8.safetensors](https://huggingface.co/stabilityai/stable-diffusion-3-medium) checkpoint for Stable Diffusion 3
55+
- 🐳 Choose one of the five available images for your serverless endpoint:
56+
- `timpietruskyblibla/runpod-worker-comfy:3.1.0-base`: doesn't contain anything, just a clean ComfyUI
57+
- `timpietruskyblibla/runpod-worker-comfy:3.1.0-flux1-schnell`: contains the checkpoint, text encoders and VAE for [FLUX.1 schnell](https://huggingface.co/black-forest-labs/FLUX.1-schnell)
58+
- `timpietruskyblibla/runpod-worker-comfy:3.1.0-flux1-dev`: contains the checkpoint, text encoders and VAE for [FLUX.1 dev](https://huggingface.co/black-forest-labs/FLUX.1-dev)
59+
- `timpietruskyblibla/runpod-worker-comfy:3.1.0-sdxl`: contains the checkpoint and VAE for [Stable Diffusion XL](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0)
60+
- `timpietruskyblibla/runpod-worker-comfy:3.1.0-sd3`: contains the checkpoint for [Stable Diffusion 3 medium](https://huggingface.co/stabilityai/stable-diffusion-3-medium)
6061
- ℹ️ [Use the Docker image on RunPod](#use-the-docker-image-on-runpod)
62+
- 🧪 Pick an [example workflow](./test_resources/workflows/) & [send it to your deployed endpoint](#interact-with-your-runpod-api)
6163

6264
## Features
6365

@@ -66,10 +68,15 @@ Read our article here: https://blib.la/blog/comfyui-on-runpod
6668
- The generated image is either:
6769
- Returned as base64-encoded string (default)
6870
- Uploaded to AWS S3 ([if AWS S3 is configured](#upload-image-to-aws-s3))
69-
- There are three different Docker images to choose from:
70-
- `timpietruskyblibla/runpod-worker-comfy:3.0.0-base`: doesn't contain anything, just a clean ComfyUI
71-
- `timpietruskyblibla/runpod-worker-comfy:3.0.0-sdxl`: contains the checkpoint and VAE for Stable Diffusion XL
72-
- `timpietruskyblibla/runpod-worker-comfy:3.0.0-sd3`: contains the checkpoint for Stable Diffusion 3
71+
- There are a few different Docker images to choose from:
72+
- `timpietruskyblibla/runpod-worker-comfy:3.1.0-flux1-schnell`: contains the [flux1-schnell.safetensors](https://huggingface.co/black-forest-labs/FLUX.1-schnell) checkpoint, the [clip_l.safetensors](https://huggingface.co/comfyanonymous/flux_text_encoders/resolve/main/clip_l.safetensors) + [t5xxl_fp8_e4m3fn.safetensors](https://huggingface.co/comfyanonymous/flux_text_encoders/resolve/main/t5xxl_fp8_e4m3fn.safetensors) text encoders and [ae.safetensors](https://huggingface.co/black-forest-labs/FLUX.1-schnell/resolve/main/ae.safetensors) VAE for FLUX.1-schnell
73+
- `timpietruskyblibla/runpod-worker-comfy:3.1.0-flux1-dev`: contains the [flux1-dev.safetensors](https://huggingface.co/black-forest-labs/FLUX.1-dev) checkpoint, the [clip_l.safetensors](https://huggingface.co/comfyanonymous/flux_text_encoders/resolve/main/clip_l.safetensors) + [t5xxl_fp8_e4m3fn.safetensors](https://huggingface.co/comfyanonymous/flux_text_encoders/resolve/main/t5xxl_fp8_e4m3fn.safetensors) text encoders and [ae.safetensors](https://huggingface.co/black-forest-labs/FLUX.1-dev/resolve/main/ae.safetensors) VAE for FLUX.1-dev
74+
- `timpietruskyblibla/runpod-worker-comfy:3.1.0-sdxl`: contains the checkpoints and VAE for Stable Diffusion XL
75+
- Checkpoint: [sd_xl_base_1.0.safetensors](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0)
76+
- VAEs:
77+
- [sdxl_vae.safetensors](https://huggingface.co/stabilityai/sdxl-vae/)
78+
- [sdxl-vae-fp16-fix](https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/)
79+
- `timpietruskyblibla/runpod-worker-comfy:3.1.0-sd3`: contains the [sd3_medium_incl_clips_t5xxlfp8.safetensors](https://huggingface.co/stabilityai/stable-diffusion-3-medium) checkpoint for Stable Diffusion 3 medium
7380
- [Bring your own models](#bring-your-own-models)
7481
- Based on [Ubuntu + NVIDIA CUDA](https://hub.docker.com/r/nvidia/cuda)
7582

@@ -99,30 +106,46 @@ This is only needed if you want to upload the generated picture to AWS S3. If yo
99106

100107
## Use the Docker image on RunPod
101108

109+
### Create your template (optional)
110+
102111
- Create a [new template](https://runpod.io/console/serverless/user/templates) by clicking on `New Template`
103112
- In the dialog, configure:
104113
- Template Name: `runpod-worker-comfy` (it can be anything you want)
105114
- Template Type: serverless (change template type to "serverless")
106-
- Container Image: `<dockerhub_username>/<repository_name>:tag`, in this case: `timpietruskyblibla/runpod-worker-comfy:3.0.0-sd3` (or `-base` for a clean image or `-sdxl` for Stable Diffusion XL)
115+
- Container Image: `<dockerhub_username>/<repository_name>:tag`, in this case: `timpietruskyblibla/runpod-worker-comfy:3.1.0-sd3` (or `-base` for a clean image or `-sdxl` for Stable Diffusion XL or `-flex1-schnell` for FLUX.1 schnell)
107116
- Container Registry Credentials: You can leave everything as it is, as this repo is public
108117
- Container Disk: `20 GB`
109118
- (optional) Environment Variables: [Configure S3](#upload-image-to-aws-s3)
110119
- Note: You can also not configure it, the images will then stay in the worker. In order to have them stored permanently, [we have to add the network volume](https://github.com/blib-la/runpod-worker-comfy/issues/1)
111120
- Click on `Save Template`
121+
122+
### Create your endpoint
123+
112124
- Navigate to [`Serverless > Endpoints`](https://www.runpod.io/console/serverless/user/endpoints) and click on `New Endpoint`
113125
- In the dialog, configure:
126+
114127
- Endpoint Name: `comfy`
115-
- Select Template: `runpod-worker-comfy` (or whatever name you gave your template)
128+
- Worker configuration: Select a GPU that can run the model you have chosen (see [GPU recommendations](#gpu-recommendations))
116129
- Active Workers: `0` (whatever makes sense for you)
117130
- Max Workers: `3` (whatever makes sense for you)
131+
- GPUs/Worker: `1`
118132
- Idle Timeout: `5` (you can leave the default)
119133
- Flash Boot: `enabled` (doesn't cost more, but provides faster boot of our worker, which is good)
134+
- Select Template: `runpod-worker-comfy` (or whatever name you gave your template)
120135
- (optional) Advanced: If you are using a Network Volume, select it under `Select Network Volume`. Otherwise leave the defaults.
121-
- Select a GPU that has some availability
122-
- GPUs/Worker: `1`
136+
123137
- Click `deploy`
124138
- Your endpoint will be created, you can click on it to see the dashboard
125139

140+
### GPU recommendations
141+
142+
| Model | Image | Minimum VRAM Required | Container Size |
143+
| ------------------------- | --------------- | --------------------- | -------------- |
144+
| Stable Diffusion XL | `sdxl` | 8 GB | 15 GB |
145+
| Stable Diffusion 3 Medium | `sd3` | 5 GB | 20 GB |
146+
| FLUX.1 Schnell | `flux1-schnell` | 24 GB | 30 GB |
147+
| FLUX.1 dev | `flux1-dev` | 24 GB | 30 GB |
148+
126149
## API specification
127150

128151
The following describes which fields exist when doing requests to the API. We only describe the fields that are sent via `input` as those are needed by the worker itself. For a full list of fields, please take a look at the [official documentation](https://docs.runpod.io/docs/serverless-usage).
@@ -193,7 +216,7 @@ The API expects a [JSON in this form](#json-request-body), where `workflow` is t
193216

194217
Please also take a look at the [test_input.json](./test_input.json) to see how the API input should look like.
195218

196-
#### Example request with cURL
219+
#### Example request for SDXL with cURL
197220

198221
```bash
199222
curl -X POST -H "Authorization: Bearer <api_key>" -H "Content-Type: application/json" -d '{"input":{"workflow":{"3":{"inputs":{"seed":1337,"steps":20,"cfg":8,"sampler_name":"euler","scheduler":"normal","denoise":1,"model":["4",0],"positive":["6",0],"negative":["7",0],"latent_image":["5",0]},"class_type":"KSampler"},"4":{"inputs":{"ckpt_name":"sd_xl_base_1.0.safetensors"},"class_type":"CheckpointLoaderSimple"},"5":{"inputs":{"width":512,"height":512,"batch_size":1},"class_type":"EmptyLatentImage"},"6":{"inputs":{"text":"beautiful scenery nature glass bottle landscape, purple galaxy bottle,","clip":["4",1]},"class_type":"CLIPTextEncode"},"7":{"inputs":{"text":"text, watermark","clip":["4",1]},"class_type":"CLIPTextEncode"},"8":{"inputs":{"samples":["3",0],"vae":["4",2]},"class_type":"VAEDecode"},"9":{"inputs":{"filename_prefix":"ComfyUI","images":["8",0]},"class_type":"SaveImage"}}}}' https://api.runpod.ai/v2/<endpoint_id>/runsync

‎docker-bake.hcl

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ variable "HUGGINGFACE_ACCESS_TOKEN" {
1515
}
1616

1717
group "default" {
18-
targets = ["base", "sdxl", "sd3"]
18+
targets = ["base", "sdxl", "sd3", "flux1-schnell", "flux1-dev"]
1919
}
2020

2121
target "base" {
@@ -47,3 +47,28 @@ target "sd3" {
4747
tags = ["${DOCKERHUB_REPO}/${DOCKERHUB_IMG}:${RELEASE_VERSION}-sd3"]
4848
inherits = ["base"]
4949
}
50+
51+
target "flux1-schnell" {
52+
context = "."
53+
dockerfile = "Dockerfile"
54+
target = "final"
55+
args = {
56+
MODEL_TYPE = "flux1-schnell"
57+
HUGGINGFACE_ACCESS_TOKEN = "${HUGGINGFACE_ACCESS_TOKEN}"
58+
}
59+
tags = ["${DOCKERHUB_REPO}/${DOCKERHUB_IMG}:${RELEASE_VERSION}-flux1-schnell"]
60+
inherits = ["base"]
61+
}
62+
63+
target "flux1-dev" {
64+
context = "."
65+
dockerfile = "Dockerfile"
66+
target = "final"
67+
args = {
68+
MODEL_TYPE = "flux1-dev"
69+
HUGGINGFACE_ACCESS_TOKEN = "${HUGGINGFACE_ACCESS_TOKEN}"
70+
}
71+
tags = ["${DOCKERHUB_REPO}/${DOCKERHUB_IMG}:${RELEASE_VERSION}-flux1-dev"]
72+
inherits = ["base"]
73+
}
74+

‎src/extra_model_paths.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,3 +9,4 @@ runpod_worker_comfy:
99
loras: models/loras/
1010
upscale_models: models/upscale_models/
1111
vae: models/vae/
12+
unet: models/unet/

‎test_resources/workflows/workflow_flux1_dev.json

Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
{
2+
"input": {
3+
"workflow": {
4+
"5": {
5+
"inputs": {
6+
"width": 1024,
7+
"height": 1024,
8+
"batch_size": 1
9+
},
10+
"class_type": "EmptyLatentImage",
11+
"_meta": {
12+
"title": "Empty Latent Image"
13+
}
14+
},
15+
"6": {
16+
"inputs": {
17+
"text": "grey cat wearing a harry potter hat and programming in javascript in its ultramodern computer",
18+
"clip": ["11", 0]
19+
},
20+
"class_type": "CLIPTextEncode",
21+
"_meta": {
22+
"title": "CLIP Text Encode (Prompt)"
23+
}
24+
},
25+
"8": {
26+
"inputs": {
27+
"samples": ["13", 0],
28+
"vae": ["10", 0]
29+
},
30+
"class_type": "VAEDecode",
31+
"_meta": {
32+
"title": "VAE Decode"
33+
}
34+
},
35+
"9": {
36+
"inputs": {
37+
"filename_prefix": "ComfyUI",
38+
"images": ["8", 0]
39+
},
40+
"class_type": "SaveImage",
41+
"_meta": {
42+
"title": "Save Image"
43+
}
44+
},
45+
"10": {
46+
"inputs": {
47+
"vae_name": "ae.safetensors"
48+
},
49+
"class_type": "VAELoader",
50+
"_meta": {
51+
"title": "Load VAE"
52+
}
53+
},
54+
"11": {
55+
"inputs": {
56+
"clip_name1": "t5xxl_fp8_e4m3fn.safetensors",
57+
"clip_name2": "clip_l.safetensors",
58+
"type": "flux"
59+
},
60+
"class_type": "DualCLIPLoader",
61+
"_meta": {
62+
"title": "DualCLIPLoader"
63+
}
64+
},
65+
"12": {
66+
"inputs": {
67+
"unet_name": "flux1-dev.safetensors",
68+
"weight_dtype": "fp8_e4m3fn"
69+
},
70+
"class_type": "UNETLoader",
71+
"_meta": {
72+
"title": "Load Diffusion Model"
73+
}
74+
},
75+
"13": {
76+
"inputs": {
77+
"noise": ["25", 0],
78+
"guider": ["22", 0],
79+
"sampler": ["16", 0],
80+
"sigmas": ["17", 0],
81+
"latent_image": ["5", 0]
82+
},
83+
"class_type": "SamplerCustomAdvanced",
84+
"_meta": {
85+
"title": "SamplerCustomAdvanced"
86+
}
87+
},
88+
"16": {
89+
"inputs": {
90+
"sampler_name": "euler"
91+
},
92+
"class_type": "KSamplerSelect",
93+
"_meta": {
94+
"title": "KSamplerSelect"
95+
}
96+
},
97+
"17": {
98+
"inputs": {
99+
"scheduler": "sgm_uniform",
100+
"steps": 4,
101+
"denoise": 1,
102+
"model": ["12", 0]
103+
},
104+
"class_type": "BasicScheduler",
105+
"_meta": {
106+
"title": "BasicScheduler"
107+
}
108+
},
109+
"22": {
110+
"inputs": {
111+
"model": ["12", 0],
112+
"conditioning": ["6", 0]
113+
},
114+
"class_type": "BasicGuider",
115+
"_meta": {
116+
"title": "BasicGuider"
117+
}
118+
},
119+
"25": {
120+
"inputs": {
121+
"noise_seed": 108076821791990
122+
},
123+
"class_type": "RandomNoise",
124+
"_meta": {
125+
"title": "RandomNoise"
126+
}
127+
}
128+
}
129+
}
130+
}

0 commit comments

Comments
 (0)