mirror of
https://github.com/0glabs/0g-chain.git
synced 2024-11-10 10:05:18 +00:00
2ffb1edd1f
* fix go version in dockerfile * mix frequently occuring errors * add missed import * fix minor validator vesting sim bug * fix auction sim bug * fix docker build * add todo from sim failure * tidy up dockerfile * update docs, add dockerignore to speed up builds * Update simulations/README.md Co-authored-by: Kevin Davis <karzak@users.noreply.github.com> * add stack traces to logs for easier debugging * replace root func with sdk version Co-authored-by: Kevin Davis <karzak@users.noreply.github.com> Co-authored-by: karzak <kjydavis3@gmail.com>
44 lines
1.8 KiB
Markdown
44 lines
1.8 KiB
Markdown
# How To Run Sims In The Cloud
|
|
|
|
Sims run with AWS batch, with results uploaded to S3
|
|
|
|
## AWS Batch
|
|
|
|
In AWS batch you define:
|
|
|
|
- a "compute environment"--just how many machines you want (and of what kind)
|
|
- a "job queue"--just a place to put jobs (pairs them with a compute environment)
|
|
- a "job definition"--a template for jobs
|
|
|
|
Then to run stuff you create "jobs" and submit them to a job queue.
|
|
|
|
The number of machines running auto-scales to match the number of jobs. When there are no jobs there are no machines, so you don't pay for anything.
|
|
|
|
Jobs are defined as a docker image (assumed hosted on dockerhub) and a command string.
|
|
>e.g. `kava/kava-sim:version1`, `go test ./app`
|
|
|
|
This can run sims but doesn't collect the results. This is handled by a custom script.
|
|
|
|
## Running sims and uploading to S3
|
|
|
|
The dockerfile in this repo defines the docker image to run sims. It includes the kava source code, aws cli, and the custom simulation script.
|
|
|
|
The custom script reads some input args, runs a sim and uploads the stdout and stderr to a S3 bucket.
|
|
|
|
AWS Batch allows for "array jobs" which are a way of specifying many duplicates of a job, each with a different index passed in as an env var.
|
|
|
|
### Steps
|
|
|
|
- create and submit a new array job (based of the job definition) with
|
|
- image `kava/kava-sim:<some-version>`
|
|
- command `run-then-upload.sh TestFullAppSimulation <starting-seed> <num-blocks> <block-size>`
|
|
- array size of how many sims you want to run
|
|
- any changes needed to the code or script necessitates a rebuild:
|
|
- `docker build -f simulations/Dockerfile -t kava/kava-sim:<some-version> .`
|
|
- `docker push kava/kava-sim:<some-version>`
|
|
|
|
### Tips
|
|
|
|
- click on the compute environment name, to get details, then click the link ECS Cluster Name to get details on the actual machines running
|
|
- for array jobs, click the job name to get details of the individual jobs
|