Skip to content

Commit 327aec5

Browse files
wx-chevalierwx-chevalier
committed
feat:添加json-parser
1 parent 1479372 commit 327aec5

File tree

662 files changed

+42076
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

662 files changed

+42076
-0
lines changed

‎common-examples/json-parser/README.md

Whitespace-only changes.

‎common-examples/json-parser/go-pkg-json/.github/workflows/golangci-lint.yml

+17
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
name: golangci-lint
2+
on:
3+
push:
4+
branches: [main]
5+
pull_request: {}
6+
jobs:
7+
golangci:
8+
runs-on: ubuntu-latest
9+
steps:
10+
- uses: actions/setup-go@v4
11+
with:
12+
go-version: '1.21'
13+
check-latest: true
14+
- uses: actions/checkout@v4
15+
- uses: golangci/golangci-lint-action@v3
16+
with:
17+
version: latest

‎common-examples/json-parser/go-pkg-json/.github/workflows/push.yml

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
on:
2+
push:
3+
branches:
4+
- main
5+
pull_request:
6+
branches:
7+
- main
8+
name: Push
9+
jobs:
10+
test:
11+
runs-on: ubuntu-latest
12+
strategy:
13+
matrix:
14+
go-version: [1.21.x]
15+
steps:
16+
- name: Install Go
17+
uses: actions/setup-go@v4
18+
with:
19+
go-version: ${{ matrix.go-version }}
20+
- name: Checkout code
21+
uses: actions/checkout@v4
22+
- name: Test
23+
run: go test ./...
24+
- name: Vet
25+
run: go vet ./...

‎common-examples/json-parser/go-pkg-json/.golangci.yml

+83
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
run:
2+
go: '1.21'
3+
4+
linters:
5+
enable-all: true
6+
disable:
7+
- asasalint
8+
- bodyclose
9+
- containedctx
10+
- contextcheck
11+
- cyclop
12+
- deadcode # replaced with unused
13+
- depguard # too annoying
14+
- dupl
15+
- dupword
16+
- errcheck
17+
- errchkjson
18+
- errname
19+
- errorlint
20+
- exhaustivestruct # Replaced by exhaustruct.
21+
- exhaustruct
22+
- exhaustive # too annoying, too hard to disable with comments
23+
- forbidigo
24+
- funlen
25+
- gci
26+
- gochecknoglobals
27+
- gocognit
28+
- goconst
29+
- gocritic
30+
- gocyclo
31+
- godot
32+
- godox
33+
- goerr113
34+
- gofumpt
35+
- golint # Replaced by revive.
36+
- gomnd
37+
- ifshort
38+
- inamedparam # too opinionated
39+
- interfacer # deprecated
40+
- ireturn
41+
- lll
42+
- maintidx
43+
- maligned # Replaced by govet
44+
- musttag # flaky, fires on structs which aren't JSON marshalled
45+
- nilnil
46+
- nlreturn
47+
- noctx
48+
- nonamedreturns
49+
- nosnakecase
50+
- paralleltest
51+
- prealloc
52+
- scopelint # Replaced by exportloopref.
53+
- structcheck # replaced with unused
54+
- tagliatelle
55+
- tagalign # too fussy
56+
- testpackage # too annoying
57+
- thelper
58+
- usestdlibvars
59+
- varcheck # Replaced by unused.
60+
- varnamelen
61+
- whitespace
62+
- wrapcheck
63+
- wsl
64+
65+
linters-settings:
66+
gofumpt:
67+
module-path: github.com/pkg/json
68+
revive:
69+
rules:
70+
- name: var-naming
71+
disabled: true
72+
stylecheck:
73+
checks: [
74+
"all",
75+
"-ST1003",
76+
"-ST1012"
77+
]
78+
unparam:
79+
check-exported: false
80+
81+
issues:
82+
max-issues-per-linter: 0
83+
max-same-issues: 0

‎common-examples/json-parser/go-pkg-json/LICENSE

+23
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
Copyright (c) 2020, Dave Cheney <[email protected]>
2+
All rights reserved.
3+
4+
Redistribution and use in source and binary forms, with or without
5+
modification, are permitted provided that the following conditions are met:
6+
7+
* Redistributions of source code must retain the above copyright notice, this
8+
list of conditions and the following disclaimer.
9+
10+
* Redistributions in binary form must reproduce the above copyright notice,
11+
this list of conditions and the following disclaimer in the documentation
12+
and/or other materials provided with the distribution.
13+
14+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
15+
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
16+
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
17+
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
18+
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
19+
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
20+
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
21+
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
22+
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
23+
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

‎common-examples/json-parser/go-pkg-json/Makefile

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
BENCH_OPTS := \
2+
-test.run=xxx \
3+
-test.bench="DecoderToken" \
4+
-test.count=5
5+
6+
benchstat: old.txt new.txt
7+
benchstat {old,new}.txt
8+
9+
old.txt: json.old
10+
./$< $(BENCH_OPTS) | tee $@
11+
12+
new.txt: json.test
13+
./$< $(BENCH_OPTS) | tee $@

‎common-examples/json-parser/go-pkg-json/README.md

+83
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
# pkg/json [![GoDoc](https://godoc.org/github.com/pkg/json?status.svg)](https://godoc.org/github.com/pkg/json)
2+
3+
An alternative JSON decoder for Go.
4+
5+
## Features
6+
7+
`pkg/json` aims to be a drop in replacement for `encoding/json`.
8+
It features:
9+
10+
- `json.Scanner` which, when provided an external buffer, does not allocate.
11+
- `io.Reader` friendly; you don't need to buffer your input in memory.
12+
- `json.Decoder.Token()` replacement that is 2-3x faster than `encoding/json`.
13+
- `json.Decoder.NextToken()` is _almost_ allocation free.
14+
15+
## Is it faster than fastjson/ultrajson/megajson/fujson?
16+
17+
Honestly, I don't know.
18+
I have some benchmarks that show that `pkg/json` is faster than `encoding/json` for tokenisation, but this package isn't finished yet.
19+
20+
21+
The `Decoder.Token` API is between 2-3x faster than `encoding/json.Decoder.Token`:
22+
```
23+
BenchmarkDecoderToken/pkgjson/canada.json.gz-16 51 21387427 ns/op 105.25 MB/s 4402975 B/op 222279 allocs/op
24+
BenchmarkDecoderToken/encodingjson/canada.json.gz-16 18 63962471 ns/op 35.19 MB/s 17740379 B/op 889106 allocs/op
25+
BenchmarkDecoderToken/pkgjson/citm_catalog.json.gz-16 235 5110849 ns/op 337.95 MB/s 966008 B/op 81995 allocs/op
26+
BenchmarkDecoderToken/encodingjson/citm_catalog.json.gz-16 61 19635188 ns/op 87.96 MB/s 5665660 B/op 324799 allocs/op
27+
BenchmarkDecoderToken/pkgjson/twitter.json.gz-16 495 2424694 ns/op 260.45 MB/s 768370 B/op 38992 allocs/op
28+
BenchmarkDecoderToken/encodingjson/twitter.json.gz-16 98 12028296 ns/op 52.50 MB/s 3660269 B/op 187815 allocs/op
29+
BenchmarkDecoderToken/pkgjson/code.json.gz-16 75 16111648 ns/op 120.44 MB/s 4304252 B/op 320235 allocs/op
30+
BenchmarkDecoderToken/encodingjson/code.json.gz-16 15 75464079 ns/op 25.71 MB/s 23355964 B/op 1319125 allocs/op
31+
BenchmarkDecoderToken/pkgjson/example.json.gz-16 23210 51699 ns/op 251.92 MB/s 16032 B/op 914 allocs/op
32+
BenchmarkDecoderToken/encodingjson/example.json.gz-16 4767 257169 ns/op 50.64 MB/s 82416 B/op 4325 allocs/op
33+
BenchmarkDecoderToken/pkgjson/sample.json.gz-16 1411 849803 ns/op 809.00 MB/s 213776 B/op 5081 allocs/op
34+
BenchmarkDecoderToken/encodingjson/sample.json.gz-16 345 3550453 ns/op 193.63 MB/s 759686 B/op 26643 allocs/op
35+
```
36+
37+
The alternative `Decoder.NextToken` API is between 8-10x faster than `encoding/json.Decoder.Token` while producing virtually no allocations:
38+
```
39+
BenchmarkDecoderNextToken/pkgjson/canada.json.gz-16 222 5157299 ns/op 436.48 MB/s 152 B/op 3 allocs/op
40+
BenchmarkDecoderNextToken/encodingjson/canada.json.gz-16 18 67566098 ns/op 33.32 MB/s 17740510 B/op 889106 allocs/op
41+
BenchmarkDecoderNextToken/pkgjson/citm_catalog.json.gz-16 510 2260514 ns/op 764.08 MB/s 152 B/op 3 allocs/op
42+
BenchmarkDecoderNextToken/encodingjson/citm_catalog.json.gz-16 61 20026083 ns/op 86.25 MB/s 5665620 B/op 324799 allocs/op
43+
BenchmarkDecoderNextToken/pkgjson/twitter.json.gz-16 1032 1093466 ns/op 577.53 MB/s 168 B/op 4 allocs/op
44+
BenchmarkDecoderNextToken/encodingjson/twitter.json.gz-16 93 12548028 ns/op 50.33 MB/s 3660291 B/op 187815 allocs/op
45+
BenchmarkDecoderNextToken/pkgjson/code.json.gz-16 211 5322161 ns/op 364.60 MB/s 264 B/op 6 allocs/op
46+
BenchmarkDecoderNextToken/encodingjson/code.json.gz-16 14 78195551 ns/op 24.82 MB/s 23355961 B/op 1319125 allocs/op
47+
BenchmarkDecoderNextToken/pkgjson/example.json.gz-16 49245 22873 ns/op 569.41 MB/s 168 B/op 4 allocs/op
48+
BenchmarkDecoderNextToken/encodingjson/example.json.gz-16 4500 269313 ns/op 48.36 MB/s 82416 B/op 4325 allocs/op
49+
BenchmarkDecoderNextToken/pkgjson/sample.json.gz-16 2032 553729 ns/op 1241.57 MB/s 1160 B/op 9 allocs/op
50+
BenchmarkDecoderNextToken/encodingjson/sample.json.gz-16 332 3574484 ns/op 192.33 MB/s 759685 B/op 26643 allocs/op
51+
```
52+
53+
Decoding into an `interface{}` this package is 40-50% faster than `encoding/json`:
54+
```
55+
BenchmarkDecoderDecodeInterfaceAny/pkgjson/canada.json.gz-16 43 27181309 ns/op 82.82 MB/s 8747225 B/op 281408 allocs/op
56+
BenchmarkDecoderDecodeInterfaceAny/encodingjson/canada.json.gz-16 30 36930290 ns/op 60.95 MB/s 20647699 B/op 392553 allocs/op
57+
BenchmarkDecoderDecodeInterfaceAny/pkgjson/citm_catalog.json.gz-16 156 7635508 ns/op 226.21 MB/s 5197871 B/op 89673 allocs/op
58+
BenchmarkDecoderDecodeInterfaceAny/encodingjson/citm_catalog.json.gz-16 73 15766855 ns/op 109.55 MB/s 9410808 B/op 95496 allocs/op
59+
BenchmarkDecoderDecodeInterfaceAny/pkgjson/twitter.json.gz-16 327 3664618 ns/op 172.33 MB/s 2130822 B/op 30182 allocs/op
60+
BenchmarkDecoderDecodeInterfaceAny/encodingjson/twitter.json.gz-16 175 6800730 ns/op 92.86 MB/s 4359944 B/op 31775 allocs/op
61+
BenchmarkDecoderDecodeInterfaceAny/pkgjson/code.json.gz-16 64 17302914 ns/op 112.15 MB/s 7331655 B/op 232059 allocs/op
62+
BenchmarkDecoderDecodeInterfaceAny/encodingjson/code.json.gz-16 49 23759564 ns/op 81.67 MB/s 12332748 B/op 271292 allocs/op
63+
BenchmarkDecoderDecodeInterfaceAny/pkgjson/example.json.gz-16 15920 76561 ns/op 170.11 MB/s 50988 B/op 739 allocs/op
64+
BenchmarkDecoderDecodeInterfaceAny/encodingjson/example.json.gz-16 8670 139230 ns/op 93.54 MB/s 82840 B/op 782 allocs/op
65+
BenchmarkDecoderDecodeInterfaceAny/pkgjson/sample.json.gz-16 1084 1119477 ns/op 614.12 MB/s 399983 B/op 5542 allocs/op
66+
BenchmarkDecoderDecodeInterfaceAny/encodingjson/sample.json.gz-16 217 5418480 ns/op 126.88 MB/s 2697781 B/op 8075 allocs/op
67+
```
68+
69+
## Should I use this?
70+
71+
Right now? No.
72+
In the future, maybe.
73+
74+
## I've found a bug!
75+
76+
Great! Please follow this two step process:
77+
78+
1. Raise an issue
79+
2. Send a PR (unless I fix it first)
80+
81+
## Standing on the shoulders of giants
82+
83+
This project is heavily influenced by Steven Schveighoffer's [`iopipe`](https://www.youtube.com/watch?v=un-bZdyumog) and [Phil Pearl](https://philpearl.github.io/post/reader/).

0 commit comments

Comments
 (0)