Skip to content

Commit 9572eb9

Browse files
author
xusenlin
committed
refactor project
1 parent 5ca94e3 commit 9572eb9

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

62 files changed

+1878
-5101
lines changed

‎README.md

+28-39
Original file line numberDiff line numberDiff line change
@@ -20,28 +20,25 @@
2020

2121
## 📢 新闻
2222

23-
+ 【2024.06.05】 已支持 `QWEN2` 模型，修改环境变量 `MODEL_NAME=qwen2` `PROMPT_NAME=qwen2`
23+
+ 【2024.06.12】 重构项目代码
2424

2525

26-
+ 【2024.06.05】 支持 `GLM4` 模型，修改环境变量 `MODEL_NAME=chatglm4` `PROMPT_NAME=chatglm4`
27-
28-
29-
+ 【2024.04.18】 支持 `Code Qwen` 模型，[sql问答demo](https://github.com/xusenlinzy/api-for-open-llm/tree/master/streamlit-demo/streamlit_gallery/components/sql_chat)
26+
+ 【2024.06.08】 已支持 `QWEN2` 模型，修改环境变量 `MODEL_NAME=qwen2` `PROMPT_NAME=qwen2`
3027

3128

32-
+ 【2024.04.16】 支持 `Rerank` 重排序模型，[使用方式](./docs/RAG.md)
29+
+ 【2024.06.08】 已支持 `QWEN2` 模型，修改环境变量 `MODEL_NAME=qwen2` `PROMPT_NAME=qwen2`
3330

3431

35-
+ 【2024.02.26】 `QWEN1.5` 模型需要修改环境变量 `MODEL_NAME=qwen2` `PROMPT_NAME=qwen2`
32+
+ 【2024.06.05】 支持 `GLM4` 模型，修改环境变量 `MODEL_NAME=chatglm4` `PROMPT_NAME=chatglm4`
3633

3734

38-
+ 【2024.01.19】 添加 [InternLM2](https://github.com/InternLM/InternLM) 模型支持，[启动方式](https://github.com/xusenlinzy/api-for-open-llm/blob/master/docs/SCRIPT.md#internlm2)
35+
+ 【2024.04.18】 支持 `Code Qwen` 模型，[sql问答demo](https://github.com/xusenlinzy/api-for-open-llm/tree/master/streamlit-demo/streamlit_gallery/components/sql_chat)
3936

4037

41-
+ 【2023.12.21】 添加 [TGI](https://github.com/huggingface/text-generation-inference) 生成接口转发和 [TEI](https://github.com/huggingface/text-embeddings-inference) embedding 接口转发
38+
+ 【2024.04.16】 支持 `Rerank` 重排序模型，[使用方式](./docs/RAG.md)
4239

4340

44-
+ 【2023.12.06】 添加 [SUS-Chat-34B](https://huggingface.co/SUSTech/SUS-Chat-34B) 模型支持，[启动方式链接](https://github.com/xusenlinzy/api-for-open-llm/blob/master/docs/SCRIPT.md#suschat)
41+
+ 【2024.02.26】 `QWEN1.5` 模型需要修改环境变量 `MODEL_NAME=qwen2` `PROMPT_NAME=qwen2`
4542

4643

4744
更多新闻和历史请转至 [此处](https://github.com/xusenlinzy/api-for-open-llm/blob/master/docs/NEWS.md)
@@ -80,46 +77,38 @@
8077
| [💁🏻‍♂支持模型](https://github.com/xusenlinzy/api-for-open-llm#-支持模型) | 此项目支持的开源模型以及简要信息 |
8178
| [🚄启动方式](https://github.com/xusenlinzy/api-for-open-llm/blob/master/docs/SCRIPT.md) | 启动模型的环境配置和启动命令 |
8279
| [⚡vLLM启动方式](https://github.com/xusenlinzy/api-for-open-llm/blob/master/docs/VLLM_SCRIPT.md) | 使用 `vLLM` 启动模型的环境配置和启动命令 |
83-
| [🦙llama-cpp启动方式](https://github.com/xusenlinzy/api-for-open-llm/blob/master/docs/LLAMA_CPP.md) | 使用 `llama-cpp` 启动模型的环境配置和启动命令 |
8480
| [💻调用方式](https://github.com/xusenlinzy/api-for-open-llm#-使用方式) | 启动模型之后的调用方式 |
8581
| [❓常见问题](https://github.com/xusenlinzy/api-for-open-llm/blob/master/docs/FAQ.md) | 一些常见问题的回复 |
86-
| [📚相关资源](https://github.com/xusenlinzy/api-for-open-llm/blob/master/docs/RESOURCES.md) | 关于开源模型训练和推理的相关资源 |
8782

8883

8984
## 🐼 支持模型
9085

9186
**语言模型**
9287

93-
| 模型 | 基座模型 | 参数量 | 语言 | 模型权重链接 |
94-
|:---------------------------------------------------------------------:|:------------:|:--------:|:------:|:-----------------------------------------------------------------------------------------------------------:|
95-
| [baichuan2](https://github.com/baichuan-inc/Baichuan2) | Baichuan | 7/13 | en, zh | [baichuan-inc/Baichuan2-13B-Chat](https://huggingface.co/baichuan-inc/Baichuan2-13B-Chat) |
96-
| [codellama](https://github.com/facebookresearch/codellama) | LLaMA2 | 7/13/34B | multi | [codellama/CodeLlama-7b-Instruct-hf](https://huggingface.co/codellama/CodeLlama-7b-Instruct-hf) |
97-
| [xverse-13b-chat](https://github.com/xverse-ai/XVERSE-13B) | Xverse | 13B | multi | [xverse/XVERSE-13B-Chat](https://huggingface.co/xverse/XVERSE-13B-Chat) |
98-
| [qwen-7b-chat](https://github.com/QwenLM/Qwen-7B) | Qwen | 7B | en, zh | [Qwen/Qwen-7B-Chat](https://huggingface.co/baichuan-inc/Qwen/Qwen-7B-Chat) |
99-
| [baichuan-13b-chat](https://github.com/baichuan-inc/Baichuan-13B) | Baichuan | 13B | en, zh | [baichuan-inc/Baichuan-13B-Chat](https://huggingface.co/baichuan-inc/Baichuan-13B-Chat) |
100-
| [InternLM](https://github.com/InternLM/InternLM) | InternLM | 7B | en, zh | [internlm/internlm-chat-7b](https://huggingface.co/internlm/internlm-chat-7b) |
101-
| [InternLM2](https://github.com/InternLM/InternLM) | InternLM2 | 20B | en, zh | [internlm/internlm2-chat-20b](https://huggingface.co/internlm/internlm2-chat-20b) |
102-
| [ChatGLM2](https://github.com/THUDM/ChatGLM2-6B) | GLM | 6/130B | en, zh | [THUDM/chatglm2-6b](https://huggingface.co/THUDM/chatglm2-6b) |
103-
| [baichaun-7b](https://github.com/baichuan-inc/baichuan-7B) | Baichuan | 7B | en, zh | [baichuan-inc/baichuan-7B](https://huggingface.co/baichuan-inc/baichuan-7B) |
104-
| [Guanaco](https://github.com/artidoro/qlora/tree/main) | LLaMA | 7/33/65B | en | [timdettmers/guanaco-33b-merged](https://huggingface.co/timdettmers/guanaco-33b-merged) |
105-
| [YuLan-Chat](https://github.com/RUC-GSAI/YuLan-Chat) | LLaMA | 13/65B | en, zh | [RUCAIBox/YuLan-Chat-13b-delta](https://huggingface.co/RUCAIBox/YuLan-Chat-13b-delta) |
106-
| [TigerBot](https://github.com/TigerResearch/TigerBot) | BLOOMZ | 7/180B | en, zh | [TigerResearch/tigerbot-7b-sft](https://huggingface.co/TigerResearch/tigerbot-7b-sft) |
107-
| [OpenBuddy](https://github.com/OpenBuddy/OpenBuddy) | LLaMA、Falcon | 7B | multi | [OpenBuddy](https://huggingface.co/OpenBuddy) |
108-
| [MOSS](https://github.com/OpenLMLab/MOSS) | CodeGen | 16B | en, zh | [fnlp/moss-moon-003-sft-int4](https://huggingface.co/fnlp/moss-moon-003-sft-int4) |
109-
| [Phoenix](https://github.com/FreedomIntelligence/LLMZoo) | BLOOMZ | 7B | multi | [FreedomIntelligence/phoenix-inst-chat-7b](https://huggingface.co/FreedomIntelligence/phoenix-inst-chat-7b) |
110-
| [BAIZE](https://github.com/project-baize/baize-chatbot) | LLaMA | 7/13/30B | en | [project-baize/baize-lora-7B](https://huggingface.co/project-baize/baize-lora-7B) |
111-
| [Chinese-LLaMA-Alpaca](https://github.com/ymcui/Chinese-LLaMA-Alpaca) | LLaMA | 7/13B | en, zh | [ziqingyang/chinese-alpaca-plus-lora-7b](https://huggingface.co/ziqingyang/chinese-alpaca-plus-lora-7b) |
112-
| [BELLE](https://github.com/LianjiaTech/BELLE) | BLOOMZ | 7B | zh | [BelleGroup/BELLE-7B-2M](https://huggingface.co/BelleGroup/BELLE-7B-2M) |
113-
| [ChatGLM](https://github.com/THUDM/ChatGLM-6B) | GLM | 6B | en, zh | [THUDM/chatglm-6b](https://huggingface.co/THUDM/chatglm-6b) |
114-
88+
| 模型 | 模型参数大小 |
89+
|----------------------------------------------------|----------------------------------|
90+
| [Baichuan](https://huggingface.co/baichuan-inc) | 7B/13B |
91+
| [ChatGLM](https://huggingface.co/THUDM) | 6B |
92+
| [DeepSeek](https://huggingface.co/deepseek-ai) | 7B/16B/67B/236B |
93+
| [InternLM](https://huggingface.co/internlm) | 7B/20B |
94+
| [LLaMA](https://github.com/facebookresearch/llama) | 7B/13B/33B/65B |
95+
| [LLaMA-2](https://huggingface.co/meta-llama) | 7B/13B/70B |
96+
| [LLaMA-3](https://huggingface.co/meta-llama) | 8B/70B |
97+
| [Qwen](https://huggingface.co/Qwen) | 1.8B/7B/14B/72B |
98+
| [Qwen1.5](https://huggingface.co/Qwen) | 0.5B/1.8B/4B/7B/14B/32B/72B/110B |
99+
| [Qwen2](https://huggingface.co/Qwen) | 0.5B/1.5B/7B/57B/72B |
100+
| [Yi (1/1.5)](https://huggingface.co/01-ai) | 6B/9B/34B |
101+
102+
启动方式详见 [vLLM启动方式](https://github.com/xusenlinzy/api-for-open-llm/blob/master/docs/VLLM_SCRIPT.md)、[transformers启动方式](https://github.com/xusenlinzy/api-for-open-llm/blob/master/docs/SCRIPT.md)
115103

116104
**嵌入模型**
117105

118-
| 模型 | 维度 | 权重链接 |
119-
|:----------------------:|:----:|:-----------------------------------------------------------------------------------:|
120-
| bge-large-zh | 1024 | [bge-large-zh](https://huggingface.co/BAAI/bge-large-zh) |
121-
| m3e-large | 1024 | [moka-ai/m3e-large](https://huggingface.co/moka-ai/m3e-large) |
122-
| text2vec-large-chinese | 1024 | [text2vec-large-chinese](https://huggingface.co/GanymedeNil/text2vec-large-chinese) |
106+
| 模型 | 维度 | 权重链接 |
107+
|:-------------------------:|:----:|:-----------------------------------------------------------------------------------:|
108+
| bge-large-zh | 1024 | [bge-large-zh](https://huggingface.co/BAAI/bge-large-zh) |
109+
| m3e-large | 1024 | [moka-ai/m3e-large](https://huggingface.co/moka-ai/m3e-large) |
110+
| text2vec-large-chinese | 1024 | [text2vec-large-chinese](https://huggingface.co/GanymedeNil/text2vec-large-chinese) |
111+
| bce-embedding-base_v1（推荐） | 768 | [bce-embedding-base_v1](https://huggingface.co/maidalun1020/bce-embedding-base_v1) |
123112

124113

125114
## 🤖 使用方式

‎api/adapter/__init__.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
from api.adapter.template import get_prompt_adapter
1+
from api.adapter.loader import load_model_and_tokenizer

0 commit comments

Comments
 (0)