Skip to content
This repository was archived by the owner on Jul 25, 2024. It is now read-only.

Commit 7b2c502

Browse files
JoeCap08055JoeCap08055
authored
fix: standardize Docker containers (#35)
* Update the building of Docker containers per our standards [here](https://docs.google.com/document/d/1Iv5RDx6-OV4S0vnn8DG4b6-zia0qkQpXa0Y9m7ihG5Y/edit#heading=h.s8rj6z75559l) * Update launching of services for local testing in `docker-compose.dev.yaml` * Update launching instructions in README Closes #31
1 parent b90d64b commit 7b2c502

File tree

9 files changed

+194
-196
lines changed

9 files changed

+194
-196
lines changed

‎.env.cp.docker.dev renamed to ‎.env.docker.dev

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
1-
IPFS_ENDPOINT="http://kubo_ipfs:5001"
2-
IPFS_BASIC_AUTH_USER=""
3-
IPFS_BASIC_AUTH_SECRET=""
4-
IPFS_GATEWAY_URL="http://kubo_ipfs:8080/ipfs/[CID]"
1+
IPFS_ENDPOINT=http://kubo_ipfs:5001
2+
IPFS_BASIC_AUTH_USER=
3+
IPFS_BASIC_AUTH_SECRET=
4+
IPFS_GATEWAY_URL=http://kubo_ipfs:8080/ipfs/[CID]
55

6+
CHAIN_ENVIRONMENT=dev
67
FREQUENCY_URL=ws://frequency:9944
78
PROVIDER_ID=1
89
REDIS_URL=redis://redis:6379
@@ -12,8 +13,8 @@ PROVIDER_ACCOUNT_SEED_PHRASE="//Alice"
1213
WEBHOOK_FAILURE_THRESHOLD=3
1314
WEBHOOK_RETRY_INTERVAL_SECONDS=10
1415
CAPACITY_LIMIT='{"type":"percentage", "value":80}'
15-
ENVIRONMENT="dev"
16-
API_PORT=3001
16+
ENVIRONMENT=dev
17+
API_PORT=3000
1718

1819
FILE_UPLOAD_MAX_SIZE_IN_BYTES=2000000000
1920
ASSET_EXPIRATION_INTERVAL_SECONDS=300

‎.github/workflows/release.yml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,6 @@ jobs:
5151
with:
5252
flavor: |
5353
latest=auto
54-
prefix=api-,onlatest=true
5554
images: |
5655
${{env.DOCKER_HUB_PROFILE}}/${{env.IMAGE_NAME}}
5756
tags: |

‎Dockerfile

Lines changed: 15 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,26 +1,33 @@
11
# Use a multi-stage build for efficiency
2-
FROM node:18 AS builder
2+
FROM node:20 AS builder
33

4-
WORKDIR /usr/src/app
4+
WORKDIR /app
55

66
COPY package*.json ./
77

8-
RUN npm install
8+
RUN npm ci
99

1010
COPY . .
1111

1212
# Build the application
1313
RUN npm run build
1414

1515
# Production stage
16-
FROM node:18
16+
FROM node:20
1717

18-
WORKDIR /usr/src/app
18+
WORKDIR /app
1919

20-
COPY --from=builder /usr/src/app/dist ./dist
20+
COPY --from=builder /app/dist ./dist
2121
COPY package*.json ./
2222

23-
RUN npm install --only=production
23+
RUN npm ci --omit=dev
24+
25+
# We want jq and curl in the final image, but we don't need the support files
26+
RUN apt-get update && \
27+
apt-get install -y jq curl tini && \
28+
apt-get clean && \
29+
rm -rf /usr/share/doc /usr/share/man /usr/share/zsh
30+
2431
EXPOSE 3000
2532

26-
CMD ["sh", "-c", "npm run start:api:prod"]
33+
ENTRYPOINT ["/usr/bin/tini", "--", "npm", "run", "start:prod"]

‎README.md

Lines changed: 32 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -21,46 +21,55 @@ Before you begin, ensure you have met the following requirements:
2121

2222
Follow these steps to set up and run Content Watcher:
2323

24-
### Clone the Repository
24+
### 1. Clone the Repository
25+
Clone the Content Watcher repository to your local machine:
26+
```bash
27+
git clone https://github.com/amplicalabls/content-watcher-service.git
28+
```
2529

26-
1. Clone the Content Watcher repository to your local machine:
30+
### 2. Configure the app
31+
The application is configurable by means of the environment. Default environment files are provided with sane values. The available environment configuration variables are document [here](./ENVIRONMENT.md), and a sample file is located [here](./env.template)
2732

28-
```bash
29-
git clone https://github.com/amplicalabls/content-watcher-service.git
30-
```
33+
Since running a full stack supporting the content-watcher service necessitates having an instance of the content-publishing services, those services are included in the docker-compose profile. The content-publishing services are separately configured using [.env.cp.docker.dev](./.env.cp.docker.dev). Documentation for that configuration can be found [here](https://github.com/AmplicaLabs/content-publishing-service/blob/main/ENVIRONMENT.md)
3134

32-
### Configure the app
33-
The application is configurable by means of the environment. Default environment files are provided with sane values. The available environment configuration variables are document [here](./ENVIRONMENT.md), and a sample file is located [here](./env.template)
35+
### 3. Start the service:
36+
Run the following command to start the service:
37+
```bash
38+
docker-compose up -d
39+
```
3440

41+
## Testing
3542
### Run a Full End-to-End Test
36-
3743
1. Execute the following `make` command to deploy the entire stack:
44+
```bash
45+
make test-start-services
46+
```
3847

39-
```bash
40-
make test-start-services
41-
```
42-
43-
This command will set up the following services:
44-
48+
This command will set up the following services:
4549
- **Frequency:** A local instance of Frequency will be set up with the default instant sealing mode.
4650
- **Redis:** A local instance of Redis will be initiated and configured for use by content publishing and content watcher services.
4751
- **Kubo IPFS:** A local instance of IPFS will be initiated and configured for use in content publishing and retrieval.
4852
- **Content Publishing API:** A local instance of the content publishing API will be utilized to publish content to IPFS and Frequency for content watcher tests.
4953
- **Content Publishing Worker:** A local instance of the content publishing worker will be employed to publish content to IPFS and Frequency for content watcher tests via dedicated processors.
5054

51-
The following setup scenarios will be executed during the stack initialization:
55+
The following setup scenarios will be executed during the stack initialization:
5256

53-
- **Chain Setup Scenario:** A provider with MSA=1 will be created, with some user accounts, along with delegation to the provider. Capacity will be staked to MSA=1 to enable the provider to publish content on behalf of users.
54-
- **DSNP Schemas:** DSNP schemas will be registered on Frequency.
55-
- **Publish Some Example Content:** Example content will be published to IPFS and Frequency. Check the progress of content publishing at [Content Publishing BullBoard](http://0.0.0.0:3001/queues).
57+
- **Chain Setup Scenario:** A provider with MSA=1 will be created, with some user accounts, along with delegation to the provider. Capacity will be staked to MSA=1 to enable the provider to publish content on behalf of users.
58+
- **DSNP Schemas:** DSNP schemas will be registered on Frequency.
59+
- **Publish Some Example Content:** Example content will be published to IPFS and Frequency. Check the progress of content publishing at [Content Publishing BullBoard](http://0.0.0.0:3001/queues).
5660

5761
2. Run the following `make` command to execute the content watcher tests:
58-
5962
```bash
60-
make test-e2e
61-
```
63+
make test-e2e
64+
```
6265

6366
3. Alternatively, create a `.env` file, run `nest start api` to start the content watcher as a standalone service, register a webhook with the content watcher using [swagger](http://0.0.0.0:3000/api/docs/swagger#), and try the following scenarios:
6467

65-
- **Reset Scanner:** This action will reset the scanner to start from the beginning of the chain or whichever block is chosen to start with. Upon successful parsing, a respective announcement will be made to the webhook.
66-
- **Put a Search Request:** This action will put a search request on the queue. The request requires a start block and end block. Upon successful parsing, a respective announcement will be made to the webhook.
68+
- **Reset Scanner:** This action will reset the scanner to start from the beginning of the chain or whichever block is chosen to start with. Upon successful parsing, a respective announcement will be made to the webhook.
69+
- **Put a Search Request:** This action will put a search request on the queue. The request requires a start block and end block. Upon successful parsing, a respective announcement will be made to the webhook.
70+
71+
## Swagger UI
72+
Check out the Swagger UI hosted on the app instance at [\<base url>/api/docs/swagger](http://localhost:3000/api/docs/swagger) to view the API documentation and submit requests to the service.
73+
74+
## Queue Management
75+
You may also view and manage the application's queue at [\<base url>/queues](http://localhost:3000/queues).

‎dev.Dockerfile

Lines changed: 2 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,7 @@
1-
FROM node:18
1+
FROM node:20
22

33
WORKDIR /app
4-
COPY . .
54

6-
RUN npm install
75
EXPOSE 3000
8-
ENV START_PROCESS="api"
96

10-
CMD ["sh", "-c", "if [ \"$START_PROCESS\" = \"api\" ]; then npm run start:api; else npm run start:worker; fi"]
7+
ENTRYPOINT [ "npm", "run", "start:watch" ]

‎docker-compose.dev.yaml

Lines changed: 0 additions & 104 deletions
This file was deleted.

‎docker-compose.yaml

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
services:
2+
redis:
3+
image: redis:latest
4+
ports:
5+
- 6379:6379
6+
volumes:
7+
- redis_data:/data/redis
8+
networks:
9+
- content-watcher-service
10+
11+
frequency:
12+
image: dsnp/instant-seal-node-with-deployed-schemas:latest
13+
# We need to specify the platform because it's the only image
14+
# built by Frequency at the moment, and auto-pull won't work otherwise
15+
platform: linux/amd64
16+
# Uncomment SEALING_MODE and SEALING_INTERVAL if you want to use interval sealing.
17+
# Other options you may want to add depending on your test scenario.
18+
# environment:
19+
# - SEALING_MODE=interval
20+
# - SEALING_INTERVAL=3
21+
# - CREATE_EMPTY_BLOCKS=true
22+
# Uncomment below if you want to let the chain run and keep all of the historical blocks
23+
# command: --state-pruning=archive
24+
ports:
25+
- 9944:9944
26+
networks:
27+
- content-watcher-service
28+
container_name: frequency
29+
volumes:
30+
- chainstorage:/data
31+
32+
kubo_ipfs:
33+
image: ipfs/kubo:latest
34+
ports:
35+
- 4001:4001
36+
- 5001:5001
37+
- 8080:8080
38+
networks:
39+
- content-watcher-service
40+
volumes:
41+
- ipfs_data:/data/ipfs
42+
43+
content-publishing-service-api:
44+
image: amplicalabs/content-publishing-service:latest
45+
# For now, this is the only platform image published.
46+
platform: linux/amd64
47+
ports:
48+
- 3001:3000
49+
env_file:
50+
- .env.docker.dev
51+
environment:
52+
- START_PROCESS=api
53+
depends_on:
54+
- redis
55+
- frequency
56+
- kubo_ipfs
57+
networks:
58+
- content-watcher-service
59+
60+
content-publishing-service-worker:
61+
image: amplicalabs/content-publishing-service:latest
62+
# For now, this is the only platform image published.
63+
platform: linux/amd64
64+
env_file:
65+
- .env.docker.dev
66+
environment:
67+
- START_PROCESS=worker
68+
depends_on:
69+
- redis
70+
- frequency
71+
- kubo_ipfs
72+
networks:
73+
- content-watcher-service
74+
75+
content-watcher-service:
76+
pull_policy: never
77+
image: content-watcher-service
78+
build:
79+
context: .
80+
dockerfile: dev.Dockerfile
81+
tags:
82+
- content-watcher-service:latest
83+
env_file:
84+
- .env.docker.dev
85+
volumes:
86+
- ./:/app
87+
depends_on:
88+
- redis
89+
- frequency
90+
- kubo_ipfs
91+
- content-publishing-service-api
92+
- content-publishing-service-worker
93+
networks:
94+
- content-watcher-service
95+
96+
volumes:
97+
redis_data:
98+
ipfs_data:
99+
chainstorage:
100+
external: false
101+
102+
networks:
103+
content-watcher-service:

0 commit comments

Comments
 (0)