Skip to content

Fix Startup Race conditions when using local frequency node #175

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
1 task
mattheworris opened this issue Oct 24, 2024 · 3 comments
Open
1 task

Fix Startup Race conditions when using local frequency node #175

mattheworris opened this issue Oct 24, 2024 · 3 comments
Labels
bug Something isn't working

Comments

@mattheworris
Copy link
Contributor

mattheworris commented Oct 24, 2024

Description

When starting social-app-template with a local node, there is a race condition with the startup of the gateway services and the frequency node. If the services come up before the node, the provider seed phrase validation fails.

Steps to Reproduce

  1. Execute start.sh
  2. Accept all of the defaults
  3. Check docker logs to see that one or more of the workers have failed to verify the seed phrase and exited.

Solution

  • Use the gateway start.sh as a model and port over the startup logic such that the frequency container is started and setup before the rest of the gateway containers are started.
@mattheworris mattheworris added the bug Something isn't working label Oct 24, 2024
@JoeCap08055
Copy link
Contributor

I don't think the issue is that Frequency isn't running. docker-compose.local-frequency.yaml includes a dependency on frequency: service_healthy for all Gateway services.

The issue, I think, is that although the chain is up, the provider hasn't been provisioned yet when the Gateway services come up, so seed phrase validation fails.

Potential solution:

Define a custom HEALTHCHECK command for the local frequency service that runs a curl jsonrpc query to get the provider info for provider 1; when that returns with the the expected value, the service is healthy.

@demisx
Copy link

demisx commented Jan 16, 2025

I can confirm @JoeCap08055's assumption here. I've run into a similar issue when integrating Gateway Services with Custodial Wallet app. I had to manually create a provider on a brand new chain for the error to go away. This is the error I was running into:

https://github.com/ProjectLibertyLabs/custodial-wallet/issues/1099#issuecomment-2588770173

@mattheworris
Copy link
Contributor Author

Gateway has fixed this issue, see the following code snippet.

start.sh on gateway has been updated to use functions, which allows for a test framework, bats, to run some tests in CI.

The function relevant to fixing this startup race condition is in the function start_services(). The main difference being that the frequency container is started and runs the local_setup_tasks before the rest of the gateway service containers are started. This solves the issue that the gateway containers cannot validate the Provider key.

This is not an issue when configured to run on Testnet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants