Skip to content

Commit 3c284cf

Browse files
dtrawinsdtrawins
authored
Freeze rel (#2863)
* freeze requriements and docker image * fixes
1 parent dbe6bb0 commit 3c284cf

File tree

5 files changed

+123
-40
lines changed

5 files changed

+123
-40
lines changed

‎demos/common/export_models/requirements.txt

Lines changed: 101 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,102 @@
11
--extra-index-url "https://download.pytorch.org/whl/cpu"
2-
--extra-index-url "https://storage.openvinotoolkit.org/simple/wheels/nightly"
3-
--pre
4-
optimum-intel@git+https://github.com/huggingface/optimum-intel.git
5-
openvino-tokenizers[transformers]==2024.5.*
6-
openvino==2024.5.*
7-
nncf>=2.11.0
8-
sentence_transformers==3.1.1
9-
openai
10-
transformers<4.45
11-
einops
2+
about-time==4.2.1
3+
aiohappyeyeballs==2.4.3
4+
aiohttp==3.11.6
5+
aiosignal==1.3.1
6+
alive-progress==3.2.0
7+
annotated-types==0.7.0
8+
anyio==4.6.2.post1
9+
async-timeout==5.0.1
10+
attrs==24.2.0
11+
autograd==1.7.0
12+
certifi==2024.8.30
13+
charset-normalizer==3.4.0
14+
cma==3.2.2
15+
coloredlogs==15.0.1
16+
contourpy==1.3.1
17+
cycler==0.12.1
18+
datasets==3.1.0
19+
Deprecated==1.2.15
20+
dill==0.3.8
21+
distro==1.9.0
22+
einops==0.8.0
23+
exceptiongroup==1.2.2
24+
filelock==3.16.1
25+
fonttools==4.55.0
26+
frozenlist==1.5.0
27+
fsspec==2024.9.0
28+
grapheme==0.6.0
29+
h11==0.14.0
30+
httpcore==1.0.7
31+
httpx==0.27.2
32+
huggingface-hub==0.26.2
33+
humanfriendly==10.0
34+
idna==3.10
35+
Jinja2==3.1.4
36+
jiter==0.7.1
37+
joblib==1.4.2
38+
jsonschema==4.23.0
39+
jsonschema-specifications==2024.10.1
40+
jstyleson==0.0.2
41+
kiwisolver==1.4.7
42+
markdown-it-py==3.0.0
43+
MarkupSafe==3.0.2
44+
matplotlib==3.9.2
45+
mdurl==0.1.2
46+
mpmath==1.3.0
47+
multidict==6.1.0
48+
multiprocess==0.70.16
49+
natsort==8.4.0
50+
networkx==3.3
51+
ninja==1.11.1.1
52+
nncf==2.13.0
53+
numpy==1.26.4
54+
onnx==1.17.0
55+
openai==1.54.5
56+
openvino==2024.5.0
57+
openvino-telemetry==2024.5.0
58+
openvino-tokenizers==2024.5.0.0
59+
optimum==1.23.3
60+
optimum-intel @ git+https://github.com/huggingface/optimum-intel.git@e3031f058fff4763a9fd917464e26aab9994449f
61+
packaging==24.2
62+
pandas==2.2.3
63+
pillow==11.0.0
64+
propcache==0.2.0
65+
protobuf==5.28.3
66+
psutil==6.1.0
67+
pyarrow==18.0.0
68+
pydantic==2.9.2
69+
pydantic_core==2.23.4
70+
pydot==2.0.0
71+
Pygments==2.18.0
72+
pymoo==0.6.1.3
73+
pyparsing==3.2.0
74+
python-dateutil==2.9.0.post0
75+
pytz==2024.2
76+
PyYAML==6.0.2
77+
referencing==0.35.1
78+
regex==2024.11.6
79+
requests==2.32.3
80+
rich==13.9.4
81+
rpds-py==0.21.0
82+
safetensors==0.4.5
83+
scikit-learn==1.5.2
84+
scipy==1.14.1
85+
sentence-transformers==3.1.1
86+
sentencepiece==0.2.0
87+
six==1.16.0
88+
sniffio==1.3.1
89+
sympy==1.13.1
90+
tabulate==0.9.0
91+
threadpoolctl==3.5.0
92+
tiktoken==0.8.0
93+
tokenizers==0.19.1
94+
torch==2.5.1+cpu
95+
tqdm==4.67.0
96+
transformers==4.44.2
97+
typing_extensions==4.12.2
98+
tzdata==2024.2
99+
urllib3==2.2.3
100+
wrapt==1.16.0
101+
xxhash==3.5.0
102+
yarl==1.17.2

‎demos/continuous_batching/README.md

Lines changed: 8 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -7,16 +7,14 @@ That makes it easy to use and efficient especially on on Intel® Xeon® processo
77
88
## Get the docker image
99

10-
Build the image from source to try the latest enhancements in this feature.
10+
Pull the image from Dockerhub with CPU support:
1111
```bash
12-
git clone https://github.com/openvinotoolkit/model_server.git
13-
cd model_server
14-
make release_image GPU=1
12+
docker pull openvino/model_server:2024.5
13+
```
14+
or if you want to include also the support for GPU execution:
15+
```bash
16+
docker pull openvino/model_server:2024.5-gpu
1517
```
16-
It will create an image called `openvino/model_server:latest`.
17-
> **Note:** This operation might take 40min or more depending on your build host.
18-
> **Note:** `GPU` parameter in image build command is needed to include dependencies for GPU device.
19-
> **Note:** The public image from the last release might be not compatible with models exported using the the latest export script. Check the [demo version from the last release](https://github.com/openvinotoolkit/model_server/tree/releases/2024/4/demos/continuous_batching) to use the public docker image.
2018

2119
## Model preparation
2220
> **Note** Python 3.9 or higher is need for that step
@@ -69,14 +67,14 @@ Check the [LLM calculator documentation](../../docs/llm/reference.md) to learn a
6967

7068
Running this command starts the container with CPU only target device:
7169
```bash
72-
docker run -d --rm -p 8000:8000 -v $(pwd)/models:/workspace:ro openvino/model_server:latest --rest_port 8000 --config_path /workspace/config.json
70+
docker run -d --rm -p 8000:8000 -v $(pwd)/models:/workspace:ro openvino/model_server:2024.5 --rest_port 8000 --config_path /workspace/config.json
7371
```
7472
### GPU
7573

7674
In case you want to use GPU device to run the generation, add extra docker parameters `--device /dev/dri --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1)`
7775
to `docker run` command, use the image with GPU support. Export the models with precision matching the GPU capacity and adjust pipeline configuration.
7876
It can be applied using the commands below:
79-
```
77+
```bash
8078
python demos/common/export_models/export_model.py text_generation --source_model meta-llama/Meta-Llama-3-8B-Instruct --weight-format int4 --target_device GPU --cache_size 2 --config_file_path models/config.json --model_repository_path models --overwrite_models
8179

8280
docker run -d --rm -p 8000:8000 --device /dev/dri --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1) -v $(pwd)/models:/workspace:ro openvino/model_server:latest-gpu --rest_port 8000 --config_path /workspace/config.json

‎demos/embeddings/README.md

Lines changed: 8 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -4,15 +4,13 @@ Text generation use case is exposed via OpenAI API `embeddings` endpoint.
44

55
## Get the docker image
66

7-
Build the image from source to try this new feature. It will be included in the public image in the coming version 2024.5.
7+
Pull the image from Dockerhub with CPU support:
88
```bash
9-
git clone https://github.com/openvinotoolkit/model_server.git
10-
cd model_server
11-
make release_image GPU=1
9+
docker pull openvino/model_server:2024.5
1210
```
13-
It will create an image called `openvino/model_server:latest`.
14-
> **Note:** This operation might take 40min or more depending on your build host.
15-
> **Note:** `GPU` parameter in image build command is needed to include dependencies for GPU device.
11+
or if you want to include also the support for GPU execution:
12+
```bash
13+
docker pull openvino/model_server:2024.5-gpu
1614

1715
## Model preparation
1816
> **Note** Python 3.9 or higher is needed for that step
@@ -77,17 +75,17 @@ All models supported by [optimum-intel](https://github.com/huggingface/optimum-i
7775
### CPU
7876

7977
```bash
80-
docker run -d --rm -p 8000:8000 -v $(pwd)/models:/workspace:ro openvino/model_server:latest --port 9000 --rest_port 8000 --config_path /workspace/config.json
78+
docker run -d --rm -p 8000:8000 -v $(pwd)/models:/workspace:ro openvino/model_server:2024.5 --port 9000 --rest_port 8000 --config_path /workspace/config.json
8179
```
8280
### GPU
8381

8482
In case you want to use GPU device to run the embeddings model, add extra docker parameters `--device /dev/dri --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1)`
8583
to `docker run` command, use the image with GPU support and make sure set the target_device in subconfig.json to GPU. Also make sure the export model quantization level and cache size fit to the GPU memory. All of that can be applied with the commands:
8684

87-
```
85+
```bash
8886
python demos/common/export_models/export_model.py embeddings --source_model Alibaba-NLP/gte-large-en-v1.5 --weight-format int8 --target_device GPU --config_file_path models/config.json --model_repository_path models
8987
90-
docker run -d --rm -p 8000:8000 --device /dev/dri --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1) -v $(pwd)/models:/workspace:ro openvino/model_server:latest-gpu --rest_port 8000 --config_path /workspace/config.json
88+
docker run -d --rm -p 8000:8000 --device /dev/dri --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1) -v $(pwd)/models:/workspace:ro openvino/model_server:2024.5-gpu --rest_port 8000 --config_path /workspace/config.json
9189
```
9290
### Check readiness
9391

‎demos/python_demos/clip_image_classification/download_model_requirements.txt

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,4 @@
11
--extra-index-url "https://download.pytorch.org/whl/cpu"
2-
--extra-index-url "https://storage.openvinotoolkit.org/simple/wheels/nightly"
3-
--pre
42
openvino==2024.5.*
53
numpy<2.0
64
transformers==4.40.2

‎demos/rerank/README.md

Lines changed: 6 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -2,15 +2,13 @@
22

33
## Get the docker image
44

5-
Build the image from source to try this new feature. It will be included in the public image in the coming version 2024.5.
5+
Pull the image from Dockerhub with CPU support:
66
```bash
7-
git clone https://github.com/openvinotoolkit/model_server.git
8-
cd model_server
9-
make release_image GPU=1
7+
docker pull openvino/model_server:2024.5
108
```
11-
It will create an image called `openvino/model_server:latest`.
12-
> **Note:** This operation might take 40min or more depending on your build host.
13-
> **Note:** `GPU` parameter in image build command is needed to include dependencies for GPU device.
9+
or if you want to include also the support for GPU execution:
10+
```bash
11+
docker pull openvino/model_server:2024.5-gpu
1412

1513
## Model preparation
1614
> **Note** Python 3.9 or higher is needed for that step
@@ -53,7 +51,7 @@ models
5351
## Deployment
5452

5553
```bash
56-
docker run -d --rm -p 8000:8000 -v $(pwd)/models:/workspace:ro openvino/model_server:latest --port 9000 --rest_port 8000 --config_path /workspace/config.json
54+
docker run -d --rm -p 8000:8000 -v $(pwd)/models:/workspace:ro openvino/model_server:2024.5 --port 9000 --rest_port 8000 --config_path /workspace/config.json
5755
```
5856

5957
Readiness of the model can be reported with a simple curl command.

0 commit comments

Comments
 (0)