Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: mlc-ai/web-llm
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v0.2.0
Choose a base ref
...
head repository: mlc-ai/web-llm
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: main
Choose a head ref
Loading
Showing with 23,171 additions and 5,677 deletions.
  1. +2 −0 .eslintignore
  2. +14 −3 .eslintrc.cjs
  3. +38 −0 .github/workflows/build-site.yaml
  4. +27 −0 .github/workflows/linter.yaml
  5. +4 −0 .gitignore
  6. +1 −0 .husky/pre-commit
  7. +3 −0 .lintstagedrc.json
  8. +8 −0 .prettierignore
  9. +3 −0 .prettierrc
  10. +11 −1 LICENSE
  11. +388 −120 README.md
  12. +5 −0 SECURITY.md
  13. +30 −0 cleanup-index-js.sh
  14. +20 −0 docs/Makefile
  15. +30 −0 docs/README.md
  16. 0 {site/img/logo → docs/_static/img}/mlc-logo-with-text-landscape.svg
  17. +102 −0 docs/conf.py
  18. +6 −0 docs/developer/add_models.rst
  19. +35 −0 docs/developer/building_from_source.rst
  20. +35 −0 docs/index.rst
  21. +35 −0 docs/make.bat
  22. +8 −0 docs/requirements.txt
  23. +133 −0 docs/user/advanced_usage.rst
  24. +202 −0 docs/user/api_reference.rst
  25. +120 −0 docs/user/basic_usage.rst
  26. +75 −0 docs/user/get_started.rst
  27. +51 −4 examples/README.md
  28. +13 −0 examples/abort-reload/README.md
  29. +20 −0 examples/abort-reload/package.json
  30. +23 −0 examples/abort-reload/src/get_started.html
  31. +32 −0 examples/abort-reload/src/get_started.js
  32. +22 −0 examples/cache-usage/README.md
  33. +20 −0 examples/cache-usage/package.json
  34. +24 −0 examples/cache-usage/src/cache_usage.html
  35. +79 −0 examples/cache-usage/src/cache_usage.ts
  36. +29 −0 examples/chrome-extension-webgpu-service-worker/README.md
  37. +23 −0 examples/chrome-extension-webgpu-service-worker/package.json
  38. +14 −0 examples/chrome-extension-webgpu-service-worker/src/background.ts
  39. +6 −0 examples/chrome-extension-webgpu-service-worker/src/content.js
  40. +20 −0 examples/chrome-extension-webgpu-service-worker/src/example.html
  41. BIN examples/chrome-extension-webgpu-service-worker/src/icons/icon-128.png
  42. BIN examples/chrome-extension-webgpu-service-worker/src/icons/icon-16.png
  43. BIN examples/chrome-extension-webgpu-service-worker/src/icons/icon-32.png
  44. BIN examples/chrome-extension-webgpu-service-worker/src/icons/icon-64.png
  45. +30 −0 examples/chrome-extension-webgpu-service-worker/src/manifest.json
  46. +240 −0 examples/chrome-extension-webgpu-service-worker/src/popup.css
  47. +46 −0 examples/chrome-extension-webgpu-service-worker/src/popup.html
  48. +167 −0 examples/chrome-extension-webgpu-service-worker/src/popup.ts
  49. +12 −0 examples/chrome-extension/README.md
  50. +23 −0 examples/chrome-extension/package.json
  51. +6 −0 examples/chrome-extension/src/content.js
  52. +20 −0 examples/chrome-extension/src/example.html
  53. BIN examples/chrome-extension/src/icons/icon-128.png
  54. BIN examples/chrome-extension/src/icons/icon-16.png
  55. BIN examples/chrome-extension/src/icons/icon-32.png
  56. BIN examples/chrome-extension/src/icons/icon-64.png
  57. +27 −0 examples/chrome-extension/src/manifest.json
  58. +23 −0 examples/chrome-extension/src/manifest_v2.json
  59. +240 −0 examples/chrome-extension/src/popup.css
  60. +50 −0 examples/chrome-extension/src/popup.html
  61. +299 −0 examples/chrome-extension/src/popup.ts
  62. +14 −0 examples/embeddings/README.md
  63. +21 −0 examples/embeddings/package.json
  64. +23 −0 examples/embeddings/src/embeddings.html
  65. +211 −0 examples/embeddings/src/embeddings.ts
  66. +11 −0 examples/function-calling/README.md
  67. +8 −0 examples/function-calling/function-calling-manual/README.md
  68. +20 −0 examples/function-calling/function-calling-manual/package.json
  69. +17 −0 examples/function-calling/function-calling-manual/src/function_calling_manual.html
  70. +235 −0 examples/function-calling/function-calling-manual/src/function_calling_manual.ts
  71. +8 −0 examples/function-calling/function-calling-openai/README.md
  72. +20 −0 examples/function-calling/function-calling-openai/package.json
  73. +17 −0 examples/function-calling/function-calling-openai/src/function_calling_openai.html
  74. +80 −0 examples/function-calling/function-calling-openai/src/function_calling_openai.ts
  75. +18 −0 examples/get-started-web-worker/README.md
  76. +20 −0 examples/get-started-web-worker/package.json
  77. +6 −5 examples/{web-worker → get-started-web-worker}/src/get_started.html
  78. +102 −0 examples/get-started-web-worker/src/main.ts
  79. +7 −0 examples/get-started-web-worker/src/worker.ts
  80. +10 −17 examples/get-started/README.md
  81. +18 −15 examples/get-started/package.json
  82. +6 −5 examples/get-started/src/get_started.html
  83. +62 −17 examples/get-started/src/get_started.ts
  84. +8 −0 examples/json-mode/README.md
  85. +20 −0 examples/json-mode/package.json
  86. +16 −0 examples/json-mode/src/json_mode.html
  87. +44 −0 examples/json-mode/src/json_mode.ts
  88. +8 −0 examples/json-schema/README.md
  89. +20 −0 examples/json-schema/package.json
  90. +16 −0 examples/json-schema/src/json_schema.html
  91. +288 −0 examples/json-schema/src/json_schema.ts
  92. +23 −0 examples/logit-processor/README.md
  93. +20 −0 examples/logit-processor/package.json
  94. +16 −0 examples/logit-processor/src/logit_processor.html
  95. +77 −0 examples/logit-processor/src/logit_processor.ts
  96. +21 −0 examples/logit-processor/src/my_logit_processor.ts
  97. +15 −0 examples/logit-processor/src/worker.ts
  98. +14 −0 examples/multi-models/README.md
  99. +20 −0 examples/multi-models/package.json
  100. +137 −0 examples/multi-models/src/main.ts
  101. +30 −0 examples/multi-models/src/multi_models.html
  102. +7 −0 examples/multi-models/src/worker.ts
  103. +8 −0 examples/multi-round-chat/README.md
  104. +20 −0 examples/multi-round-chat/package.json
  105. +16 −0 examples/multi-round-chat/src/multi_round_chat.html
  106. +82 −0 examples/multi-round-chat/src/multi_round_chat.ts
  107. +35 −0 examples/next-simple-chat/.gitignore
  108. +17 −0 examples/next-simple-chat/README.md
  109. +21 −0 examples/next-simple-chat/next.config.js
  110. +26 −0 examples/next-simple-chat/package.json
  111. +6 −0 examples/next-simple-chat/postcss.config.js
  112. BIN examples/next-simple-chat/public/favicon.ico
  113. +1 −0 examples/next-simple-chat/public/next.svg
  114. +1 −0 examples/next-simple-chat/public/vercel.svg
  115. +6 −0 examples/next-simple-chat/src/pages/_app.tsx
  116. +13 −0 examples/next-simple-chat/src/pages/_document.tsx
  117. +13 −0 examples/next-simple-chat/src/pages/api/hello.ts
  118. +25 −0 examples/next-simple-chat/src/pages/index.tsx
  119. +156 −0 examples/next-simple-chat/src/styles/globals.css
  120. +96 −0 examples/next-simple-chat/src/utils/chat_component.tsx
  121. +155 −0 examples/next-simple-chat/src/utils/chat_ui.ts
  122. +18 −0 examples/next-simple-chat/tailwind.config.js
  123. +23 −0 examples/next-simple-chat/tsconfig.json
  124. +8 −0 examples/seed-to-reproduce/README.md
  125. +20 −0 examples/seed-to-reproduce/package.json
  126. +17 −0 examples/seed-to-reproduce/src/seed.html
  127. +64 −0 examples/seed-to-reproduce/src/seed.ts
  128. +8 −0 examples/service-worker/README.md
  129. +20 −0 examples/service-worker/package.json
  130. +23 −0 examples/service-worker/src/index.html
  131. +120 −0 examples/service-worker/src/main.ts
  132. +8 −0 examples/service-worker/src/sw.ts
  133. +106 −0 examples/simple-chat-js/index.css
  134. +30 −0 examples/simple-chat-js/index.html
  135. +150 −0 examples/simple-chat-js/index.js
  136. 0 examples/{simple-chat → simple-chat-ts}/.gitignore
  137. +43 −0 examples/simple-chat-ts/README.md
  138. +20 −0 examples/simple-chat-ts/package.json
  139. +6 −0 examples/simple-chat-ts/src/gh-config.js
  140. BIN examples/simple-chat-ts/src/img/plane.png
  141. BIN examples/simple-chat-ts/src/img/reset.png
  142. +188 −0 examples/simple-chat-ts/src/llm_chat.css
  143. +25 −0 examples/simple-chat-ts/src/llm_chat.html
  144. +350 −0 examples/simple-chat-ts/src/simple_chat.ts
  145. +7 −0 examples/simple-chat-ts/src/worker.ts
  146. +1 −0 examples/simple-chat-upload/.gitignore
  147. +43 −0 examples/simple-chat-upload/README.md
  148. +20 −0 examples/simple-chat-upload/package.json
  149. +6 −0 examples/simple-chat-upload/src/gh-config.js
  150. BIN examples/simple-chat-upload/src/img/plane.png
  151. BIN examples/simple-chat-upload/src/img/reset.png
  152. +188 −0 examples/simple-chat-upload/src/llm_chat.css
  153. +34 −0 examples/simple-chat-upload/src/llm_chat.html
  154. +433 −0 examples/simple-chat-upload/src/simple_chat.ts
  155. +7 −0 examples/simple-chat-upload/src/worker.ts
  156. +0 −21 examples/simple-chat/README.md
  157. +0 −20 examples/simple-chat/package.json
  158. +0 −23 examples/simple-chat/src/gh-config.js
  159. +0 −127 examples/simple-chat/src/llm_chat.css
  160. +0 −21 examples/simple-chat/src/llm_chat.html
  161. +0 −27 examples/simple-chat/src/mlc-local-config.js
  162. +0 −244 examples/simple-chat/src/simple_chat.ts
  163. +0 −8 examples/simple-chat/src/worker.ts
  164. +8 −0 examples/streaming/README.md
  165. +20 −0 examples/streaming/package.json
  166. +17 −0 examples/streaming/src/streaming.html
  167. +53 −0 examples/streaming/src/streaming.ts
  168. +14 −0 examples/text-completion/README.md
  169. +20 −0 examples/text-completion/package.json
  170. +23 −0 examples/text-completion/src/text_completion.html
  171. +58 −0 examples/text-completion/src/text_completion.ts
  172. +14 −0 examples/vision-model/README.md
  173. +20 −0 examples/vision-model/package.json
  174. +32 −0 examples/vision-model/src/utils.ts
  175. +23 −0 examples/vision-model/src/vision_model.html
  176. +115 −0 examples/vision-model/src/vision_model.ts
  177. +7 −0 examples/vision-model/src/worker.ts
  178. +0 −25 examples/web-worker/README.md
  179. +0 −17 examples/web-worker/package.json
  180. +0 −41 examples/web-worker/src/main.ts
  181. +0 −8 examples/web-worker/src/worker.ts
  182. +4 −0 jest.config.cjs
  183. +201 −0 licenses/license.openai_node.txt
  184. +2,124 −4,088 package-lock.json
  185. +25 −6 package.json
  186. +4 −2 rollup.config.js
  187. +5 −6 scripts/gh_deploy_site.sh
  188. +1 −1 scripts/local_deploy_site.sh
  189. +6 −9 scripts/prep_deps.sh
  190. +4 −6 site/_config.yml
  191. +21 −0 site/_includes/arrow.svg
  192. +8 −0 site/_includes/github.svg
  193. +10 −2 site/_includes/head.html
  194. +72 −0 site/_includes/hero.html
  195. +0 −56 site/_includes/llm_chat.html
  196. +235 −0 site/assets/css/hero.scss
  197. BIN site/assets/img/fig/Pittsburgh.png
  198. 0 site/{ → assets}/img/fig/web-llm.svg
  199. 0 site/{ → assets}/img/logo/catalyst.svg
  200. BIN site/{ → assets}/img/logo/cmuscs.png
  201. BIN site/assets/img/logo/mlc-logo-with-text-landscape.png
  202. +87 −0 site/assets/img/logo/mlc-logo-with-text-landscape.svg
  203. BIN site/{ → assets}/img/logo/octoml.png
  204. BIN site/{ → assets}/img/logo/sjtu.png
  205. BIN site/{ → assets}/img/logo/uw.jpg
  206. BIN site/assets/video/Code.mp4
  207. BIN site/assets/video/Code.webm
  208. BIN site/assets/video/Pittsburgh.mp4
  209. BIN site/assets/video/Pittsburgh.webm
  210. BIN site/img/fig/demo.gif
  211. BIN site/img/fig/pitts.png
  212. +12 −34 site/index.md
  213. +149 −0 src/cache_util.ts
  214. +0 −239 src/chat_module.ts
  215. +2,119 −34 src/config.ts
  216. +460 −79 src/conversation.ts
  217. +294 −0 src/embedding.ts
  218. +1,390 −0 src/engine.ts
  219. +603 −0 src/error.ts
  220. +195 −0 src/extension_service_worker.ts
  221. +40 −8 src/index.ts
  222. +1,156 −159 src/llm_chat.ts
  223. +163 −0 src/message.ts
  224. +1,162 −0 src/openai_api_protocols/chat_completion.ts
  225. +363 −0 src/openai_api_protocols/completion.ts
  226. +198 −0 src/openai_api_protocols/embedding.ts
  227. +68 −0 src/openai_api_protocols/index.ts
  228. +253 −0 src/service_worker.ts
  229. +443 −0 src/support.ts
  230. +192 −32 src/types.ts
  231. +147 −0 src/utils.ts
  232. +660 −145 src/web_worker.ts
  233. +221 −0 tests/constants.ts
  234. +272 −0 tests/conversation.test.ts
  235. +414 −0 tests/function_calling.test.ts
  236. +101 −0 tests/generation_config.test.ts
  237. +241 −0 tests/multi_round_chat.test.ts
  238. +484 −0 tests/openai_chat_completion.test.ts
  239. +110 −0 tests/openai_completion.test.ts
  240. +122 −0 tests/openai_embeddings.test.ts
  241. +425 −0 tests/util.test.ts
  242. +5 −2 tsconfig.json
  243. +1 −0 utils/.gitignore
  244. +1 −0 utils/vram_requirements/.gitignore
  245. +5 −0 utils/vram_requirements/README.md
  246. +25 −0 utils/vram_requirements/package.json
  247. +6 −0 utils/vram_requirements/src/gh-config.js
  248. +17 −0 utils/vram_requirements/src/vram_requirements.html
  249. +112 −0 utils/vram_requirements/src/vram_requirements.ts
2 changes: 2 additions & 0 deletions .eslintignore
Original file line number Diff line number Diff line change
@@ -3,4 +3,6 @@ debug
lib
build
node_modules
3rdparty
.eslintrc.cjs
**/.next
17 changes: 14 additions & 3 deletions .eslintrc.cjs
Original file line number Diff line number Diff line change
@@ -1,9 +1,20 @@
module.exports = {
extends: ['eslint:recommended', 'plugin:@typescript-eslint/recommended'],
extends: ['eslint:recommended', 'plugin:@typescript-eslint/recommended', 'plugin:prettier/recommended'],
parser: '@typescript-eslint/parser',
plugins: ['@typescript-eslint'],
root: true,
rules: {
"@typescript-eslint/no-explicit-any": "off"
}
"@typescript-eslint/no-explicit-any": "off",
"@typescript-eslint/no-empty-function": "off",
"@typescript-eslint/no-non-null-assertion": "off",
},
overrides: [
{
"files": ["examples/**/*.js", "examples/**/*.ts"],
"rules": {
"no-undef": "off",
"@typescript-eslint/no-unused-vars": "off"
}
}
]
};
38 changes: 38 additions & 0 deletions .github/workflows/build-site.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: Build site and push to gh-pages

on:
push:
branches:
- main

jobs:
build:
name: Build site
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2

- name: Configuring build Environment
run: |
sudo apt-get update
python -m pip install -U pip
- name: Setup Ruby
uses: ruby/setup-ruby@v1
with:
ruby-version: '3.0'

- name: Installing dependencies
run: |
python -m pip install -r docs/requirements.txt
gem install jekyll jekyll-remote-theme jekyll-sass-converter
- name: Build and deploy site
if: github.ref == 'refs/heads/main'
run: |
git remote set-url origin https://x-access-token:${{ secrets.MLC_GITHUB_TOKEN }}@github.com/$GITHUB_REPOSITORY
git config --global user.email "mlc-gh-actions-bot@nomail"
git config --global user.name "mlc-gh-actions-bot"
./scripts/gh_deploy_site.sh
27 changes: 27 additions & 0 deletions .github/workflows/linter.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
name: Linter

on:
push:
branches:
- main
pull_request:
branches:
- main

jobs:
lint:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v3

- name: Set up Node.js
uses: actions/setup-node@v3
with:
node-version: '16'

- name: Install dependencies
run: npm install

- name: Run lint
run: npm run lint
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
scratch/
dist/
params/
*.bak
@@ -322,3 +323,6 @@ tvm_home
node_modules
lib
.parcel-cache

examples/tests
**/.next
1 change: 1 addition & 0 deletions .husky/pre-commit
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
npx lint-staged
3 changes: 3 additions & 0 deletions .lintstagedrc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"./**/*.{js,ts,jsx,tsx,json}": ["eslint --fix", "prettier --write"]
}
8 changes: 8 additions & 0 deletions .prettierignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
dist
debug
lib
build
node_modules
3rdparty
.eslintrc.cjs
**/.next
3 changes: 3 additions & 0 deletions .prettierrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"trailingComma": "all"
}
12 changes: 11 additions & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -198,4 +198,14 @@
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
limitations under the License.

------------------------------------------------------------------------------------
This product bundles various third-party components under other open source licenses.
This section summarizes those components and their licenses. See licenses/
for text of these licenses.

Apache Software Foundation License 2.0
--------------------------------------

src/openai_api_protocols
Loading