Bo QIU
64120399f1
refactor sync relevant python tests ( #301 )
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-12-19 16:36:53 +08:00
Bo QIU
d3a2118985
Sync file from neighbors by default ( #295 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* sync from neighbors by default
* change p2p protocol version
* minor fix for crash python test
* Fix pubsub message id issue
* fix python tests
2024-12-09 16:04:55 +08:00
Bo QIU
4bb2fac98f
Optional subscribe FIND_CHUNKS topic ( #291 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* sub annouce shard config msg
* Optional subscribe find_chunks topic
2024-12-05 14:04:57 +08:00
Bo QIU
b9e6431a4d
Add shard config in STATUS message and only dail to shard config matched peers ( #285 )
...
* Add shard config in status message
* verify shard config for status message
* Notify peer connected to sync layer after status message exchanged
* Do not dial to shard config mismatched peers
* Upgrade network protocol version
* disconnect peer instead of ban peer if shard config mismatch
* Add python test for TCP connection by shard config
2024-11-25 10:15:30 +08:00
0g-peterzhb
8f4dfff2f6
fix tx store ( #281 )
...
* fix tx store
* fix params
2024-11-20 18:02:43 +08:00
0g-peterzhb
0da3c374db
add snapshot test ( #276 )
...
* add snapshot test
2024-11-19 11:18:58 +08:00
Bo QIU
e912522386
Supports to sync historical files without NewFile gossip message ( #269 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Supports to randomly sync historical files
* Add name for random file sync batcher
* Remove sync store metrics since multiple random batcher created
* opt log
* ignore pruned or finalized historical file
* Add python tests for historical file sync
2024-11-15 10:00:58 +08:00
Bo QIU
baf0521c99
copy key file to bootnode folder in python tests ( #260 )
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-11-06 14:01:47 +08:00
Bo QIU
cae5b62440
Hotfix for python tests caused by unexpected file deletion ( #258 )
...
* Hotfix for python tests caused by unexpected file deletion
* add more info when launch blockchain node failed
* add stdout if blockchain launch failed
* seek stdout and err to 0 if failed to launch blockchain
* Improve zg chain port to avoid port conflict in parallel execution
* fix float issue
* Fix py failures
2024-11-05 13:49:58 +08:00
0g-peterzhb
9eea71e97d
separate data db from flow db ( #252 )
...
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* separate data db from flow db
2024-10-31 15:44:26 +08:00
Bo QIU
bb6e1457b7
Refuse network identity incompatible nodes in UDP discovery layer ( #253 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Add python test for UDP discovery
* Refuse nodes with incompatible ENR
2024-10-30 17:26:02 +08:00
Bo QIU
9b68a8b7d7
Implement file sync protocol V2 ( #249 )
...
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* Add new P2P protocol NewFile
* Publish NewFile message when any file finalized
* handle NewFile message in router
* handle NewFile in sync servic to write in db
* use propagation source to handle NewFile message
* Disable sequential sync and store new file in v2 sync store
* Add shard config in FindFile
* Add AnnounceFile RPC message in network layer
* do not propagate FindFile to whole network
* Mark peer connected if FileAnnouncement RPC message received
* fix unit test failures
* Change P2P protocol version
* Ignore py tests of sequential auto sync
* Add py test for auto sync v2
* fmt code
* remove dummy code in py test
* fix random test failure
* Add comments
* Enable file sync protocol v2 in config file by default
2024-10-28 14:56:08 +08:00
peilun-conflux
789eae5cc1
Start nodes sequentially to fix some random failure. ( #243 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
Currently we use the config `network_libp2p_nodes` to connect nodes
in the tests. This will not be retried, so if an early node starts
too slowly, other nodes may fail to connect to it.
2024-10-28 10:53:28 +08:00
peilun-conflux
506d234562
Use LRU to cache MPT nodes. ( #227 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Add trait.
* Update merkle tree trait.
* Use NodeManager.
* fix.
* Use LRU for cache.
* fix clippy.
* Save layer size.
* Initialize LogManager with NodeManager.
* Fix.
* Fix test.
* fix.
2024-10-27 12:52:06 +08:00
bruno-valante
b2a70501c2
Test flow root consistency ( #230 )
2024-10-15 14:24:56 +08:00
MiniFrenchBread
b131dc532f
test: update contracts, shard submission test ( #225 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* fix: pad tx based on the on-chain contract result.
* chore: update contracts
* test: shard submission
---------
Co-authored-by: Peilun Li <peilun.li@confluxnetwork.org>
2024-10-09 16:33:20 +08:00
MiniFrenchBread
07ac814e57
test: use fixed 0g binary ( #218 )
...
* test: use latest 0g binary
* feat: use bug fixed 0g binary
2024-09-30 14:56:38 +08:00
Bo QIU
ad80b22a1b
Optimize rpc config ( #213 )
2024-09-25 16:48:40 +08:00
Bo QIU
1dd7bf7734
Remove the 0gchain genesis init script for mac os ( #208 )
...
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* Remove the 0gchain genesis init script for mac os
* fix on mac
2024-09-20 19:16:53 +08:00
MiniFrenchBread
5849e9c2ba
fix: finalize file does not need to save ( #206 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* fix: finalize file does not need to save
* fix: rust fmt
2024-09-19 19:14:45 +08:00
Bo QIU
1d48cb1ea7
Use 0gchain to test mine tests ( #191 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Use 0gchain to test mine tests
* adjust zg block time
2024-09-11 10:13:31 +08:00
Bo QIU
052d2d781b
Change zg chain block time in python tests ( #181 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Change zg chain block time in python tests
2024-09-08 08:38:53 +08:00
bruno-valante
bf3694d138
Update mine test process to avoid random bugs on low-performance devices ( #184 )
2024-09-06 17:53:00 +08:00
Bo QIU
b6972b97af
Adjust default value for testnet configs ( #180 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Adjust default value for testnet configs
* Supports to disable sequential auto sync
* Add py tests for auto sync
* fix py file name
* rm dummy ;py code
* change default block confirm count
2024-09-05 10:09:29 +08:00
Bo QIU
f1a1c39332
adjust sync timeout in py test ( #171 )
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-08-27 09:52:00 +08:00
peilun-conflux
1c72607fbc
Set sync start index based on data in db. ( #166 )
...
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* Set sync start index based on data in db.
* Fix test.
* nit.
2024-08-23 12:41:25 +08:00
peilun-conflux
d80e7e22ca
Prune no reward chunks. ( #145 )
...
* Prune no reward chunks.
* Add tests.
* Fix tests.
* Fix clippy.
* Revert test.
* Enable market in shard_sync_test.
* Add tx prune status.
* Fix tests.
2024-08-06 15:06:15 +08:00
peilun-conflux
dbd865fded
Change db_max_num_chunks to db_max_num_sectors. ( #137 )
...
* Change db_max_num_chunks to db_max_num_sectors.
* Update tests and config files.
* Revert contract change.
2024-07-29 22:31:19 +08:00
bruno-valante
7d73ccd1e1
Update with the most recent storage contract ( #142 )
...
* Update with the most recent storage contract
* fix nits
2024-07-24 13:01:30 +08:00
MiniFrenchBread
533bacb234
fix: admin_getFileLocation; test: sync test ( #141 )
...
* feat: add all_shards in admin_getFileLocation
* fix: admin_getFileLocation
* test: improve sync test
* fix: lint
2024-07-23 15:47:44 +08:00
bruno-valante
f0c3f2cfd0
Remove contract abi submodule ( #139 )
2024-07-22 17:32:51 +08:00
peilun-conflux
504a67fdd0
Handle chunk write of rpc in parallel. ( #126 )
...
* Handle chunk write of rpc in parallel.
* double mine period.
* Increase the log sync channel capacity.
* Increase capacity.
2024-07-15 15:50:51 +08:00
bruno-valante
cca14e246e
Support multiple mine submission ( #124 )
...
* Support multiple mine submission
* Update
2024-07-12 17:31:53 +08:00
Bo QIU
68b5d27728
Filter out file announcement of private ip ( #113 )
...
* Filter out file announcement of private ip
* add log for listen address
2024-07-08 18:45:55 +08:00
peilun-conflux
4eb2a50b0e
Use inner lock in storage and use async lock. ( #92 )
...
* Use inner lock in storage.
* Remove mut.
* Remove async lock for storage.
* Fix tests and warnings.
* Use spawn_blocking for storage task.
* Fix clippy.
* Finalize the new tx at last.
* Revert "Finalize the new tx at last."
This reverts commit b56ad5582d
2024-06-29 17:08:02 +08:00
MiniFrenchBread
fa74a4b9c1
feat: contract refactor ( #93 )
...
* feat: use refactored contract
* chore: update submodule
* chore: update submodule
2024-06-25 18:53:28 +08:00
peilun-conflux
6383c7ddac
Update cli version and fix tests. ( #81 )
...
* fix: test
* Update cli version.
---------
Co-authored-by: MiniFrenchBread <103425574+MiniFrenchBread@users.noreply.github.com>
2024-06-10 22:54:39 +08:00
peilun-conflux
0f52325f67
Sync and finalize sharded data. ( #79 )
...
* Save test logs.
* Prepare to handle shard config in chunk pool.
* Allow files to be finalized with only sharded data.
* Handle shard config change.
* Only sync needed data in shard config.
* Fix clippy and tests.
2024-06-08 02:50:36 +08:00
peilun-conflux
c2c6e2d5fb
Store shard config for peers and choose sync peers accordingly. ( #77 )
...
* Implement Pruner.
* Put pruner in a crate.
* Fix clippy.
* Add rpc zgs_getShardConfig.
* Fix.
* Increase wait time.
* Add pruner_test and use max_num_chunks instead of size_limit.
* Store shard config for peers and choose sync peers accordingly.
* Add test and fix sync.
* Fix clippy and test.
* Fix some ut.
* Add AnnounceShardConfig gossip and fix tests.
* Add sharded tx finalize check in LogManager.
* Try,
* Rename.
* Longer timeout for mine_test.
* Save test logs.
2024-06-07 16:58:15 +08:00
peilun-conflux
ef82f64393
Implement Pruner to delete unwanted data. ( #70 )
...
* Implement Pruner.
* Put pruner in a crate.
* Fix clippy.
* Add rpc zgs_getShardConfig.
* Fix.
* Increase wait time.
* Add pruner_test and use max_num_chunks instead of size_limit.
* Add back shard config and fix test.
* fix: serde format
* Remove unneeded PORA_CHUNK_SIZE.
* Fix tests.
---------
Co-authored-by: MiniFrenchBread <103425574+MiniFrenchBread@users.noreply.github.com>
2024-05-31 13:11:06 +08:00
Bo QIU
b17fd117fd
Supports 0gchain in python test framework. ( #66 )
2024-05-17 16:29:23 +08:00
Chenxing Li
2262bc3fb9
Support mining on sharded storage. ( #64 )
...
* Change PoraHash compute
* Change padSeed compute
* Refactor
* Support mining on sharded storage
* Detect single core performance and set correct params for test
* Fix clippy
* Fix an overflow bug
2024-04-30 14:34:44 +08:00
Chenxing Li
193e154361
Change miner id logic & request miner id automatically ( #60 )
...
* Change miner id logic & request miner id automatically
* Not enable all features in the test workflow.
* Auto configurable mining period
* Adjust test params for ci
2024-04-27 11:15:57 +08:00
Bo QIU
5bcd3602b0
fix ci issues ( #63 )
2024-04-25 14:24:58 +08:00
Bo QIU
57090464ae
Supports to download blockchain binaries from github release ( #61 )
...
* build evmos before init genesis
* Supports to download conflux and evmos binary
* print unzip info
* download conflux windows binary in zip format
* unnecessary to install evmos when init node
* aaa
* bbb
* ccc
* ddd
* eee
* 111
* 2222
* aaa
* aaa
* aaa
* aaa
2024-04-25 11:25:05 +08:00
Chenxing Li
c85fe38c59
Remove cli build from server build workflow ( #54 )
...
* Remove cli build from server build workflow
* Build cli for a single test
2024-04-23 14:35:24 +08:00
Chenxing Li
95485c5c35
Enable data seal & merkle proof verification on mine ( #59 )
2024-04-23 14:34:46 +08:00
Chenxing Li
e5dc2d61cc
Configurable CPU usage on mine ( #58 )
...
* Configurable CPU usage on mine
* Cargo clippy
2024-04-23 12:52:39 +08:00
Bo QIU
0bd9ef0ed8
Automatically install dependent blockchain binary when run a single python test ( #57 )
...
* Separate script to build blockchain binaries
* auto remove http_proxy env if configured
* build bsc binary in run all process
* auto install binary when run a single test
* Add necessary output when build or download binary
* create tmp dir if absent
2024-04-19 20:24:50 +08:00
peilun-conflux
b9c6e43617
Enforce rear padding data check for segment proofs. ( #55 )
...
* Construct wrong proof.
* Enforce rear padding data check for segment proofs.
2024-04-17 18:46:04 +08:00
