mirror of https://github.com/0glabs/0g-chain.git synced 2024-11-10 10:05:18 +00:00

* fix go version in dockerfile

* mix frequently occuring errors

* add missed import

* fix minor validator vesting sim bug

* fix auction sim bug

* fix docker build

* add todo from sim failure

* tidy up dockerfile

* update docs, add dockerignore to speed up builds

* Update simulations/README.md

Co-authored-by: Kevin Davis <karzak@users.noreply.github.com>

* add stack traces to logs for easier debugging

* replace root func with sdk version

Co-authored-by: Kevin Davis <karzak@users.noreply.github.com>
Co-authored-by: karzak <kjydavis3@gmail.com>

2020-09-01 11:33:12 -04:00

1.8 KiB

Raw Blame History

How To Run Sims In The Cloud

Sims run with AWS batch, with results uploaded to S3

AWS Batch

In AWS batch you define:

a "compute environment"--just how many machines you want (and of what kind)
a "job queue"--just a place to put jobs (pairs them with a compute environment)
a "job definition"--a template for jobs

Then to run stuff you create "jobs" and submit them to a job queue.

The number of machines running auto-scales to match the number of jobs. When there are no jobs there are no machines, so you don't pay for anything.

Jobs are defined as a docker image (assumed hosted on dockerhub) and a command string.

e.g. kava/kava-sim:version1, go test ./app

This can run sims but doesn't collect the results. This is handled by a custom script.

Running sims and uploading to S3

The dockerfile in this repo defines the docker image to run sims. It includes the kava source code, aws cli, and the custom simulation script.

The custom script reads some input args, runs a sim and uploads the stdout and stderr to a S3 bucket.

AWS Batch allows for "array jobs" which are a way of specifying many duplicates of a job, each with a different index passed in as an env var.

Steps

create and submit a new array job (based of the job definition) with
- image kava/kava-sim:<some-version>
- command run-then-upload.sh TestFullAppSimulation <starting-seed> <num-blocks> <block-size>
- array size of how many sims you want to run
any changes needed to the code or script necessitates a rebuild:
- docker build -f simulations/Dockerfile -t kava/kava-sim:<some-version> .
- docker push kava/kava-sim:<some-version>

Tips

click on the compute environment name, to get details, then click the link ECS Cluster Name to get details on the actual machines running
for array jobs, click the job name to get details of the individual jobs

1.8 KiB Raw Blame History