Commit Graph

49 Commits

Author SHA1 Message Date
peilun-conflux
1c72607fbc
Set sync start index based on data in db. (#166)
Some checks failed
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* Set sync start index based on data in db.

* Fix test.

* nit.
2024-08-23 12:41:25 +08:00
peilun-conflux
d80e7e22ca
Prune no reward chunks. (#145)
* Prune no reward chunks.

* Add tests.

* Fix tests.

* Fix clippy.

* Revert test.

* Enable market in shard_sync_test.

* Add tx prune status.

* Fix tests.
2024-08-06 15:06:15 +08:00
peilun-conflux
dbd865fded
Change db_max_num_chunks to db_max_num_sectors. (#137)
* Change db_max_num_chunks to db_max_num_sectors.

* Update tests and config files.

* Revert contract change.
2024-07-29 22:31:19 +08:00
bruno-valante
7d73ccd1e1
Update with the most recent storage contract (#142)
* Update with the most recent storage contract

* fix nits
2024-07-24 13:01:30 +08:00
MiniFrenchBread
533bacb234
fix: admin_getFileLocation; test: sync test (#141)
* feat: add all_shards in admin_getFileLocation

* fix: admin_getFileLocation

* test: improve sync test

* fix: lint
2024-07-23 15:47:44 +08:00
bruno-valante
f0c3f2cfd0
Remove contract abi submodule (#139) 2024-07-22 17:32:51 +08:00
peilun-conflux
504a67fdd0
Handle chunk write of rpc in parallel. (#126)
* Handle chunk write of rpc in parallel.

* double mine period.

* Increase the log sync channel capacity.

* Increase capacity.
2024-07-15 15:50:51 +08:00
bruno-valante
cca14e246e
Support multiple mine submission (#124)
* Support multiple mine submission

* Update
2024-07-12 17:31:53 +08:00
Bo QIU
68b5d27728
Filter out file announcement of private ip (#113)
* Filter out file announcement of private ip

* add log for listen address
2024-07-08 18:45:55 +08:00
peilun-conflux
4eb2a50b0e
Use inner lock in storage and use async lock. (#92)
* Use inner lock in storage.

* Remove mut.

* Remove async lock for storage.

* Fix tests and warnings.

* Use spawn_blocking for storage task.

* Fix clippy.

* Finalize the new tx at last.

* Revert "Finalize the new tx at last."

This reverts commit b56ad5582d.

* Wait for old same-root txs to finalize.

* Use async storage in miner.

* Update rust version to 1.79.0.

* Use Vec to avoid stack overflow.

* Fix unused warning.

* Fix clippy.

* Fix test warning.

* Fix test.

* fmt.

* Use async storage in pruner.

* nit.
2024-06-29 17:08:02 +08:00
MiniFrenchBread
fa74a4b9c1
feat: contract refactor (#93)
* feat: use refactored contract

* chore: update submodule

* chore: update submodule
2024-06-25 18:53:28 +08:00
peilun-conflux
6383c7ddac
Update cli version and fix tests. (#81)
* fix: test

* Update cli version.

---------

Co-authored-by: MiniFrenchBread <103425574+MiniFrenchBread@users.noreply.github.com>
2024-06-10 22:54:39 +08:00
peilun-conflux
0f52325f67
Sync and finalize sharded data. (#79)
* Save test logs.

* Prepare to handle shard config in chunk pool.

* Allow files to be finalized with only sharded data.

* Handle shard config change.

* Only sync needed data in shard config.

* Fix clippy and tests.
2024-06-08 02:50:36 +08:00
peilun-conflux
c2c6e2d5fb
Store shard config for peers and choose sync peers accordingly. (#77)
* Implement Pruner.

* Put pruner in a crate.

* Fix clippy.

* Add rpc zgs_getShardConfig.

* Fix.

* Increase wait time.

* Add pruner_test and use max_num_chunks instead of size_limit.

* Store shard config for peers and choose sync peers accordingly.

* Add test and fix sync.

* Fix clippy and test.

* Fix some ut.

* Add AnnounceShardConfig gossip and fix tests.

* Add sharded tx finalize check in LogManager.

* Try,

* Rename.

* Longer timeout for mine_test.

* Save test logs.
2024-06-07 16:58:15 +08:00
peilun-conflux
ef82f64393
Implement Pruner to delete unwanted data. (#70)
* Implement Pruner.

* Put pruner in a crate.

* Fix clippy.

* Add rpc zgs_getShardConfig.

* Fix.

* Increase wait time.

* Add pruner_test and use max_num_chunks instead of size_limit.

* Add back shard config and fix test.

* fix: serde format

* Remove unneeded PORA_CHUNK_SIZE.

* Fix tests.

---------

Co-authored-by: MiniFrenchBread <103425574+MiniFrenchBread@users.noreply.github.com>
2024-05-31 13:11:06 +08:00
Bo QIU
b17fd117fd
Supports 0gchain in python test framework. (#66) 2024-05-17 16:29:23 +08:00
Chenxing Li
2262bc3fb9
Support mining on sharded storage. (#64)
* Change PoraHash compute

* Change padSeed compute

* Refactor

* Support mining on sharded storage

* Detect single core performance and set correct params for test

* Fix clippy

* Fix an overflow bug
2024-04-30 14:34:44 +08:00
Chenxing Li
193e154361
Change miner id logic & request miner id automatically (#60)
* Change miner id logic & request miner id automatically

* Not enable all features in the test workflow.

* Auto configurable mining period

* Adjust test params for ci
2024-04-27 11:15:57 +08:00
Bo QIU
5bcd3602b0
fix ci issues (#63) 2024-04-25 14:24:58 +08:00
Bo QIU
57090464ae
Supports to download blockchain binaries from github release (#61)
* build evmos before init genesis

* Supports to download conflux and evmos binary

* print unzip info

* download conflux windows binary in zip format

* unnecessary to install evmos when init node

* aaa

* bbb

* ccc

* ddd

* eee

* 111

* 2222

* aaa

* aaa

* aaa

* aaa
2024-04-25 11:25:05 +08:00
Chenxing Li
c85fe38c59
Remove cli build from server build workflow (#54)
* Remove cli build from server build workflow

* Build cli for a single test
2024-04-23 14:35:24 +08:00
Chenxing Li
95485c5c35
Enable data seal & merkle proof verification on mine (#59) 2024-04-23 14:34:46 +08:00
Chenxing Li
e5dc2d61cc
Configurable CPU usage on mine (#58)
* Configurable CPU usage on mine

* Cargo clippy
2024-04-23 12:52:39 +08:00
Bo QIU
0bd9ef0ed8
Automatically install dependent blockchain binary when run a single python test (#57)
* Separate script to build blockchain binaries

* auto remove http_proxy env if configured

* build bsc binary in run all process

* auto install binary when run a single test

* Add necessary output when build or download binary

* create tmp dir if absent
2024-04-19 20:24:50 +08:00
peilun-conflux
b9c6e43617
Enforce rear padding data check for segment proofs. (#55)
* Construct wrong proof.

* Enforce rear padding data check for segment proofs.
2024-04-17 18:46:04 +08:00
Bo QIU
6ca05090c3
update config (#56)
* update config

* use evmos testnet binary all the time
2024-04-16 21:24:50 +08:00
Chenxing Li
8cff5dabe7
Test mine on evmos (#48) 2024-04-15 01:38:33 +08:00
Bo QIU
4151375316
Supports flags for evmos start cli (#43) 2024-04-09 17:54:34 +08:00
Bo QIU
3e22a6a027
Supports evmos as blockchain fullnode in python tests (#40)
* refine blockchain type

* Supports evmos

* cp evmos to code dir

* improve timestamp
2024-04-09 15:45:02 +08:00
peilun-conflux
588bf39d7d
Fix a possible log sync break after a node restarts. (#34)
* Fix a possible log sync break after a node restarts.

* Skip `sync_blocks` in blockchain node setup.
2024-03-29 16:15:01 +08:00
Joel Liu
f9ce286909
replace filter with eth_getLogs (#32)
* replace filter with eth_getLogs

* update log

* store block history into db

* fix lints error

* add free disk step

* set max worker to 3

* resolve comments and refact code

* resolve comments

* set max-workers to 4

* set max-workers to 3
2024-03-27 13:54:06 +08:00
0g-peterzhb
06a335add2
@peter/doc (#22)
* add documentation
2024-03-24 22:37:46 +08:00
Chenxing Li
16dfc56437
Test mining with the simple market (#31)
* Test mining with the simple market

* Support reward donation
2024-03-19 14:09:17 +08:00
Chenxing Li
306c43c9dc
Migrate to the new contract interfaces (#29)
* Fix compile warning & Search contract by name automatically

Migrate to the new contract interfaces

* Fix compile

* Fix lint
2024-03-14 13:56:17 +08:00
0g-peterzhb
2e1d076460
@peter/update submodule (#25)
* update modules
2024-03-12 18:49:51 +08:00
peilun-conflux
310bf1c8dd
Save updated mpt nodes from proof to DB. (#20)
* Save updated mpt nodes from proof to DB.

* Free disk space for Github Actions.

* Use an alternative Rust cache action.

* Fix action usage.

* Do not free large packages.
2024-02-06 18:46:35 +08:00
Bo QIU
8f328a9eab
enhance py test framework to support more args (#19) 2024-02-04 15:07:42 +08:00
Bo QIU
80cebd4c70
refactor python run all (#18) 2024-02-01 15:54:31 +08:00
Bo QIU
012f5b34ec
hotfix for py test (#17) 2024-01-31 18:13:25 +08:00
Bo QIU
cb8c6ebf81
enhance test framework for extensibility (#16)
* enhance test framework for extensibility

* extract common tests dir
2024-01-31 16:56:13 +08:00
Bo QIU
c2b8b1cab3
Rpc enhancements (#15)
* add admin rpc to terminate file or chunks sync

* Disallow to change file sync goal when failed

* split admin rpc into separate endpoints

* Support to launch public and private rpc at one port

* use one rpc port for python test

* fix lint

* restart to sync file or chunks if sync already completed or failed
2024-01-30 16:50:35 +08:00
MiniFrenchBread
7667e02019
fix: contract, tests, clippy (#11) 2024-01-26 16:13:18 +08:00
peilun-conflux
0c123500c9
Update the merkle tree with proof data. (#9)
* Add proof data for chunk proof.

* Support file proof.

* Update with segment proof and fix issues.

* Fix more issues.

* Fix the process of file proof.

* Merge branch 'main' into fix_proof

* Enable sync_test.

* Fix wrongly updated submodule.

* Fix bsc node version.
2024-01-26 10:29:09 +08:00
Bo QIU
2c2dba8730
fix ci failure (#10)
* fix ci failure

* fix ci failures

* fix py test issue

* disable --all-features for code coverage CI

* do not fail ci if failed to upload code coverage
2024-01-25 19:00:00 +08:00
Bo QIU
8b225fde43
Fix python test (#8)
* Fix stdouterr issue when shutdown storage node

* Fix random failure for submission test

* add more info for failed rpc

* use single file instead of subprocess pipe to launch CLI

* Opt test_all for windows

* wait for log entry before file sync by admin
2024-01-24 20:42:25 +08:00
MiniFrenchBread
7479f56474
feat: write cached segments to storage on log synced (#7) 2024-01-23 18:47:14 +08:00
boqiu
99ead6423b add python test for chunks sync by rpc 2024-01-22 18:19:28 +08:00
Bo QIU
613c1a8eb5
add readme for python tests (#5) 2024-01-19 21:26:33 +08:00
Peilun Li
9058467be1 Initial commit 2024-01-04 18:05:32 +08:00