Commit Graph

8 Commits

Author SHA1 Message Date
Peilun Li
6aa11b7b70 Remove async lock for storage. 2024-06-19 16:45:03 +08:00
peilun-conflux
0f52325f67
Sync and finalize sharded data. (#79)
* Save test logs.

* Prepare to handle shard config in chunk pool.

* Allow files to be finalized with only sharded data.

* Handle shard config change.

* Only sync needed data in shard config.

* Fix clippy and tests.
2024-06-08 02:50:36 +08:00
peilun-conflux
c2c6e2d5fb
Store shard config for peers and choose sync peers accordingly. (#77)
* Implement Pruner.

* Put pruner in a crate.

* Fix clippy.

* Add rpc zgs_getShardConfig.

* Fix.

* Increase wait time.

* Add pruner_test and use max_num_chunks instead of size_limit.

* Store shard config for peers and choose sync peers accordingly.

* Add test and fix sync.

* Fix clippy and test.

* Fix some ut.

* Add AnnounceShardConfig gossip and fix tests.

* Add sharded tx finalize check in LogManager.

* Try,

* Rename.

* Longer timeout for mine_test.

* Save test logs.
2024-06-07 16:58:15 +08:00
MiniFrenchBread
8b410f2fc8
Chore: expose RPC modules, update dependency (#35)
* feat: expose rpc clients

* chore: expose rpc types

* chore: update dependency
2024-04-02 21:00:26 +08:00
Bo QIU
a4abe2b2a4
fix build dependency isuse (#12)
* fix build dependency isuse

* cargo fmt

* fix lint failure in CI

* update toolchain
2024-01-26 19:48:40 +08:00
MiniFrenchBread
7479f56474
feat: write cached segments to storage on log synced (#7) 2024-01-23 18:47:14 +08:00
Bo QIU
9b4b0436c3
Supports to sync partial chunks (#4)
* refactor p2p signed message

* add new pubsub messages in network layer to find chunks

* handle find chunks pubsub message in router

* Supports to sync partial chunks

* add admin rpc to sync chunks

* limit number of chunks to sync at a time

* refactor code to sync file and chunks

* add more switches to trigger file sync

* fix ut failure

* refactor code
2024-01-19 14:04:59 +08:00
Peilun Li
9058467be1 Initial commit 2024-01-04 18:05:32 +08:00