Skip to content

Commit 4e8aa61

Browse files
committed
docs: add ollama option
1 parent 3dd6108 commit 4e8aa61

File tree

1 file changed

+38
-0
lines changed

1 file changed

+38
-0
lines changed

docs/sections/qdrant/02.1-additional-setup.md

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,10 @@ QDRANT_URL=http://localhost:6333
2323

2424
<div data-hidden="$$proxy$$">
2525

26+
Now you either have to deploy an Azure Open AI service to use the OpenAI API, or you can use a local emulator based on Ollama and an open-source LLM.
27+
28+
#### Using Azure Open AI
29+
2630
You first need to deploy an Azure Open AI service to use the OpenAI API.
2731

2832
Before moving to the next section, go to the **Azure setup** section (either on the left or using the "hamburger" menu depending of your device) to deploy the necessary resources and create your `.env` file needed.
@@ -41,4 +45,38 @@ With:
4145
QDRANT_URL="http://localhost:6333"
4246
```
4347

48+
#### (Optional) Using Ollama and ollamazure
49+
50+
If you have a machine with enough resources, you can run this workshop entirely locally without using any cloud resources. To do that, you first have to install [Ollama](https://ollama.com) and then run the following commands to download the models on your machine:
51+
52+
```bash
53+
ollama pull phi3
54+
ollama pull all-minilm:l6-v2
55+
```
56+
57+
<div class="info" data-title="Note">
58+
59+
> The `phi3` model with download a few gigabytes of data, so it can take some time depending on your internet connection.
60+
61+
</div>
62+
63+
<div class="important" data-title="Important">
64+
65+
> Ollama work in GitHub Codespaces, but runs very slow currently. If you want to use the Ollama option, it will work best if you are working on the workshop on your local machine directly.
66+
67+
</div>
68+
69+
Once the model are downloaded, create a `.env` file at the root of the project, and add the following content:
70+
71+
```
72+
AZURE_OPENAI_API_ENDPOINT=http://localhost:4041
73+
QDRANT_URL=http://localhost:6333
74+
```
75+
76+
Finally, run this command to start the Azure OpenAI emulator and leave it running in the background:
77+
78+
```bash
79+
npx -y ollamazure
80+
```
81+
4482
</div>

0 commit comments

Comments
 (0)