Skip to content
This repository was archived by the owner on Jul 25, 2024. It is now read-only.

Commit 215acd1

Browse files
wilwadewilwade
authored
Update Readme and standardize (#75)
Updating the readme to the standard as well as standardizing a bunch of other things. - Readme updates - e2e tests now use the app running in docker - All setup scripts should work again - Add webhook client utility to easily see the output of the webhook - Jest unit tests uses the env.template instead of .env - Dockerfiles updated to standards - Formatting Closes #32
1 parent 5c68bf7 commit 215acd1

24 files changed

+907
-310
lines changed

‎.dockerignore

Lines changed: 34 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,35 @@
1-
Dockerfile
2-
.dockerignore
3-
node_modules
4-
npm-debug.log
5-
dist
6-
.env*
1+
# Include any files or directories that you don't want to be copied to your
2+
# container here (e.g., local build artifacts, temporary files, etc.).
3+
#
4+
# For more help, visit the .dockerignore file reference guide at
5+
# https://docs.docker.com/go/build-context-dockerignore/
6+
7+
**/.classpath
8+
**/.dockerignore
9+
**/.env
10+
**/.git
11+
**/.gitignore
12+
**/.project
13+
**/.settings
14+
**/.toolstarget
15+
**/.vs
16+
**/.vscode
17+
**/.next
18+
**/.cache
19+
**/*.*proj.user
20+
**/*.dbmdl
21+
**/*.jfm
22+
**/charts
23+
**/docker-compose*
24+
**/compose.y*ml
25+
**/Dockerfile*
26+
**/node_modules
27+
**/npm-debug.log
28+
**/obj
29+
**/secrets.dev.yaml
30+
**/values.dev.yaml
31+
**/build
32+
**/dist
33+
LICENSE
34+
README.md
735
env.template

‎.env.content-publishing-service

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
CHAIN_ENVIRONMENT=dev
2+
PROVIDER_ID=1
3+
PROVIDER_ACCOUNT_SEED_PHRASE=//Alice
4+
FILE_UPLOAD_MAX_SIZE_IN_BYTES=10000000 # ~10Mb
5+
ASSET_EXPIRATION_INTERVAL_SECONDS=300
6+
BATCH_INTERVAL_SECONDS=12
7+
BATCH_MAX_COUNT=1000
8+
ASSET_UPLOAD_VERIFICATION_DELAY_SECONDS=5
9+
CAPACITY_LIMIT={"type":"percentage", "value":80}
10+
REDIS_URL=redis://redis:6379
11+
FREQUENCY_URL=ws://frequency:9944
12+
IPFS_ENDPOINT=http://ipfs:5001

‎.env.dev

Lines changed: 0 additions & 19 deletions
This file was deleted.

‎.env.docker.dev

Lines changed: 35 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,42 @@
1-
IPFS_ENDPOINT=http://kubo_ipfs:5001
2-
IPFS_BASIC_AUTH_USER=
3-
IPFS_BASIC_AUTH_SECRET=
4-
IPFS_GATEWAY_URL=http://kubo_ipfs:8080/ipfs/[CID]
1+
# Tweak values for local development
2+
# Content Publishing Service in Docker Compose will override these values with `.env.content-publishing-service`
53

6-
CHAIN_ENVIRONMENT=dev
4+
# URL to IPFS endpoint
5+
# IPFS_ENDPOINT="https://ipfs.infura.io:5001"
6+
IPFS_ENDPOINT="http://ipfs:5001"
7+
8+
# If using Infura with auth required for read access, put Project ID here, or leave blank for Kubo RPC
9+
# IPFS_BASIC_AUTH_USER=
10+
11+
# If using Infura with auth required for read access, put auth token here, or leave blank for Kubo RPC
12+
# IPFS_BASIC_AUTH_SECRET=
13+
14+
# IPFS gateway URL. '[CID]' is a token that will be replaced with an actual content ID
15+
# IPFS_GATEWAY_URL="https://ipfs.io/ipfs/[CID]"
16+
IPFS_GATEWAY_URL="http://ipfs:8080/ipfs/[CID]"
17+
18+
# Blockchain node address
719
FREQUENCY_URL=ws://frequency:9944
8-
PROVIDER_ID=1
20+
21+
# Block number from which the service will start scanning the chain
22+
STARTING_BLOCK=1
23+
24+
# Redis URL
925
REDIS_URL=redis://redis:6379
26+
27+
# How many minutes to delay between successive scans of the chain
28+
# for new accounts (after end of chain is reached)
1029
BLOCKCHAIN_SCAN_INTERVAL_MINUTES=1
30+
31+
# Max number of jobs allowed on the queue before
32+
# blockchain scan will be paused to allow queue to drain
1133
QUEUE_HIGH_WATER=1000
12-
PROVIDER_ACCOUNT_SEED_PHRASE="//Alice"
13-
WEBHOOK_FAILURE_THRESHOLD=3
34+
35+
# Number of retry attempts if a registered webhook call fails
36+
WEBHOOK_FAILURE_THRESHOLD=4
37+
38+
# Number of seconds between webhook retry attempts when failing
1439
WEBHOOK_RETRY_INTERVAL_SECONDS=10
15-
CAPACITY_LIMIT='{"type":"percentage", "value":80}'
16-
ENVIRONMENT=dev
17-
API_PORT=3000
1840

19-
FILE_UPLOAD_MAX_SIZE_IN_BYTES=2000000000
20-
ASSET_EXPIRATION_INTERVAL_SECONDS=300
21-
BATCH_INTERVAL_SECONDS=12
22-
BATCH_MAX_COUNT=1000
23-
ASSET_UPLOAD_VERIFICATION_DELAY_SECONDS=5
41+
# Port that the application REST endpoints listen on
42+
API_PORT=3000

‎.gitignore

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,10 @@
11
node_modules
22
dist
3-
.env
3+
.env*
4+
!.env.docker.dev
5+
!.env.content-publishing-service
46
.vscode
57
coverage
68
.idea
79
docs/*.html
10+
*.bkp

‎Dockerfile

Lines changed: 57 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,33 +1,72 @@
1-
# Use a multi-stage build for efficiency
2-
FROM node:20 AS builder
1+
# syntax=docker/dockerfile:1
32

3+
# Comments are provided throughout this file to help you get started.
4+
# If you need more help, visit the Dockerfile reference guide at
5+
# https://docs.docker.com/go/dockerfile-reference/
6+
7+
# Want to help us make this template better? Share your feedback here: https://forms.gle/ybq9Krt8jtBL3iCk7
8+
9+
ARG NODE_VERSION=20
10+
11+
################################################################################
12+
# Use node image for base image for all stages.
13+
FROM node:${NODE_VERSION}-alpine as base
14+
15+
# Set working directory for all build stages.
416
WORKDIR /app
517

6-
COPY package*.json ./
718

8-
RUN npm ci
19+
################################################################################
20+
# Create a stage for installing production dependecies.
21+
FROM base as deps
922

10-
COPY . .
23+
# Download dependencies as a separate step to take advantage of Docker's caching.
24+
# Leverage a cache mount to /root/.npm to speed up subsequent builds.
25+
# Leverage bind mounts to package.json and package-lock.json to avoid having to copy them
26+
# into this layer.
27+
RUN --mount=type=bind,source=package.json,target=package.json \
28+
--mount=type=bind,source=package-lock.json,target=package-lock.json \
29+
--mount=type=cache,target=/root/.npm \
30+
npm ci --omit=dev
1131

12-
# Build the application
32+
################################################################################
33+
# Create a stage for building the application.
34+
FROM deps as build
35+
36+
# Download additional development dependencies before building, as some projects require
37+
# "devDependencies" to be installed to build. If you don't need this, remove this step.
38+
RUN --mount=type=bind,source=package.json,target=package.json \
39+
--mount=type=bind,source=package-lock.json,target=package-lock.json \
40+
--mount=type=cache,target=/root/.npm \
41+
npm ci
42+
43+
# Copy the rest of the source files into the image.
44+
COPY . .
45+
# Run the build script.
1346
RUN npm run build
1447

15-
# Production stage
16-
FROM node:20
48+
################################################################################
49+
# Create a new stage to run the application with minimal runtime dependencies
50+
# where the necessary files are copied from the build stage.
51+
FROM base as final
1752

18-
WORKDIR /app
53+
# Use production node environment by default.
54+
ENV NODE_ENV production
55+
56+
# Run the application as a non-root user.
57+
USER node
1958

20-
COPY --from=builder /app/dist ./dist
21-
COPY package*.json ./
59+
# Copy package.json so that package manager commands can be used.
60+
COPY package.json .
2261

23-
RUN npm ci --omit=dev
62+
# Copy the production dependencies from the deps stage and also
63+
# the built application from the build stage into the image.
64+
COPY --from=deps /app/node_modules ./node_modules
65+
COPY --from=build /app/dist ./dist
2466

25-
# We want jq and curl in the final image, but we don't need the support files
26-
RUN apt-get update && \
27-
apt-get install -y jq curl tini && \
28-
apt-get clean && \
29-
rm -rf /usr/share/doc /usr/share/man /usr/share/zsh
3067

68+
# Expose the port that the application listens on.
3169
EXPOSE 3000
3270

33-
ENTRYPOINT ["/usr/bin/tini", "--", "npm", "run", "start:prod"]
71+
# Run the application.
72+
ENTRYPOINT ["/usr/bin/tini", "--", "node", "dist/apps/api/main.js"]

‎ENVIRONMENT.md

Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -2,17 +2,17 @@
22

33
This application recognizes the following environment variables:
44

5-
| Name | Description | Range/Type | Required? | Default |
6-
| ----------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | :--------------------------------: | :----------: | :-----: |
7-
|`API_PORT` | HTTP port that the application listens on | 1025 - 65535 | | 3000 |
8-
|`BLOCKCHAIN_SCAN_INTERVAL_MINUTES` | How many minutes to delay between successive scans of the chain for new accounts (after end of chain is reached) | > 0 | | 180 |
9-
|`FREQUENCY_URL` | Blockchain node address | http(s): or ws(s): URL | Y | |
10-
|`IPFS_BASIC_AUTH_SECRET`|If using Infura, put auth token here, or leave blank for Kubo RPC|string|N|blank|
11-
|`IPFS_BASIC_AUTH_USER`|If using Infura, put Project ID here, or leave blank for Kubo RPC|string|N|blank|
12-
|`IPFS_ENDPOINT`|URL to IPFS endpoint|URL|Y||
13-
|`IPFS_GATEWAY_URL`|IPFS gateway URL. '[CID]' is a token that will be replaced with an actual content ID|URL template|Y||
14-
|`QUEUE_HIGH_WATER` | Max number of jobs allowed on the 'graphUpdateQueue' before blockchain scan will be paused to allow queue to drain | >= 100 | | 1000 |
15-
|`REDIS_URL` | Connection URL for Redis | URL | Y |
16-
|`STARTING_BLOCK`|Block number from which the service will start scanning the chain|> 0||1|
17-
|`WEBHOOK_FAILURE_THRESHOLD` | Number of failures allowing in the provider webhook before the service is marked down | > 0 | | 3 |
18-
|`WEBHOOK_RETRY_INTERVAL_SECONDS` | Number of seconds between provider webhook retry attempts when failing | > 0 | | 10 |
5+
| Name | Description | Range/Type | Required? | Default |
6+
| ---------------------------------- | ------------------------------------------------------------------------------------------------------------------ | :--------------------: | :-------: | :-----: |
7+
| `API_PORT` | HTTP port that the application listens on | 1025 - 65535 | | 3000 |
8+
| `BLOCKCHAIN_SCAN_INTERVAL_MINUTES` | How many minutes to delay between successive scans of the chain for new accounts (after end of chain is reached) | > 0 | | 180 |
9+
| `FREQUENCY_URL` | Blockchain node address | http(s): or ws(s): URL | Y | |
10+
| `IPFS_BASIC_AUTH_SECRET` | If required for read requests, put Infura auth token here, or leave blank for default Kubo RPC | string | N | blank |
11+
| `IPFS_BASIC_AUTH_USER` | If required for read requests, put Infura Project ID here, or leave blank for default Kubo RPC | string | N | blank |
12+
| `IPFS_ENDPOINT` | URL to IPFS endpoint | URL | Y | |
13+
| `IPFS_GATEWAY_URL` | IPFS gateway URL. '[CID]' is a token that will be replaced with an actual content ID | URL template | Y | |
14+
| `QUEUE_HIGH_WATER` | Max number of jobs allowed on the '' before blockchain scan will be paused to allow queue to drain | >= 100 | | 1000 |
15+
| `REDIS_URL` | Connection URL for Redis | URL | Y |
16+
| `STARTING_BLOCK` | Block number from which the service will start scanning the chain | > 0 | | 1 |
17+
| `WEBHOOK_FAILURE_THRESHOLD` | Number of failures allowing in the provider webhook before the service is marked down | > 0 | | 3 |
18+
| `WEBHOOK_RETRY_INTERVAL_SECONDS` | Number of seconds between provider webhook retry attempts when failing | > 0 | | 10 |

‎INSTALLING.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,4 +62,4 @@ The following is a list of environment variables that may be set to control the
6262
|`STARTING_BLOCK`|**maybe**|Starting block for scanner to scan from|_none_|
6363
|`REDIS_URL`|**yes**|URL used to connect to Redis instance|_none_<br/>\*preset to the internal Redis URL in the standalone container|
6464
|`BLOCKCHAIN_SCAN_INTERVAL_MINUTES`|no|# of minutes to wait in between scans of the blockchain|180|
65-
|`QUEUE_HIGH_WATER`|no|# of pending graph scan queue entries to allow before pausing blockchain scanning until the next scan cycle|1000|
65+
|`QUEUE_HIGH_WATER`|no|# of pending queue entries to allow before pausing blockchain scanning until the next scan cycle|1000|

0 commit comments

Comments
 (0)