Bo QIU
b9e6431a4d
Add shard config in STATUS message and only dail to shard config matched peers ( #285 )
...
* Add shard config in status message
* verify shard config for status message
* Notify peer connected to sync layer after status message exchanged
* Do not dial to shard config mismatched peers
* Upgrade network protocol version
* disconnect peer instead of ban peer if shard config mismatch
* Add python test for TCP connection by shard config
2024-11-25 10:15:30 +08:00
0g-peterzhb
96f846073f
increase db_max_num_sectors to be about 1000GB ( #272 )
2024-11-15 13:10:07 +08:00
Bo QIU
9b68a8b7d7
Implement file sync protocol V2 ( #249 )
...
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* Add new P2P protocol NewFile
* Publish NewFile message when any file finalized
* handle NewFile message in router
* handle NewFile in sync servic to write in db
* use propagation source to handle NewFile message
* Disable sequential sync and store new file in v2 sync store
* Add shard config in FindFile
* Add AnnounceFile RPC message in network layer
* do not propagate FindFile to whole network
* Mark peer connected if FileAnnouncement RPC message received
* fix unit test failures
* Change P2P protocol version
* Ignore py tests of sequential auto sync
* Add py test for auto sync v2
* fmt code
* remove dummy code in py test
* fix random test failure
* Add comments
* Enable file sync protocol v2 in config file by default
2024-10-28 14:56:08 +08:00
Bo QIU
8f17a7ad72
Fix metrics config deser ( #245 )
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-10-22 14:14:33 +08:00
0g-peterzhb
48868b60db
update ip ( #223 )
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-10-11 17:07:43 +08:00
Bo QIU
ae6ecfec96
reduce default sync threads ( #214 )
2024-09-26 14:55:00 +08:00
Bo QIU
ad80b22a1b
Optimize rpc config ( #213 )
2024-09-25 16:48:40 +08:00
Bo QIU
9cde84ae15
Refactor network peer db configs ( #209 )
2024-09-24 11:59:34 +08:00
Bo QIU
a9f5169c15
Allow user to configure network bandwidth for file sync ( #198 )
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-09-13 16:58:27 +08:00
Bo QIU
b6972b97af
Adjust default value for testnet configs ( #180 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Adjust default value for testnet configs
* Supports to disable sequential auto sync
* Add py tests for auto sync
* fix py file name
* rm dummy ;py code
* change default block confirm count
2024-09-05 10:09:29 +08:00
0g-peterzhb
508e787bd0
update boot nodes ( #175 )
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-08-30 10:38:19 +08:00
Bo QIU
2fd9712d59
Enhance P2P network protocol to support batch messages for performance concern ( #173 )
...
* Add p2p protocol version in network identity
* Cache annouce file pubsub messages to publish in batch
* fix file location cache
* opt sync metrics
* opt file location cache default configs
* publish files announcements in batch
* enhance announce file pubsub msg metrics
* opt metrics
* fix ci
* fix clippy
* fix batcher
* minor fix
* opt batcher: publish all if expired
2024-08-29 09:55:24 +08:00
bruno-valante
5757c98d2a
Config query context interval & Add config comments ( #156 )
2024-08-27 17:36:36 +08:00
Bo QIU
37344b9439
opt sync config and update config file ( #169 )
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-08-26 15:29:52 +08:00
Bo QIU
82fef674ed
Add more metrics for file sync ( #164 )
...
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* add metrics in log sync package
* udpate auto sync metrics
* Add metrics for completed file sync
* add more metrics for serial file sync
* adjust default timeout value for auto sync
* fix metrics rpc for Timer type
* add metrics for channel
* refactor channel metrics
* add timeout metrics for segment sync
* refactor channel receiver
2024-08-22 10:42:15 +08:00
Bo QIU
22ed8f5f91
opt router debug logs ( #160 )
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-08-16 10:34:28 +08:00
Bo QIU
03286ebd78
Supports HDD config file ( #158 )
...
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* Add debug log config
* Add config file for ssd and hdd
* Set most default value for ssd and hdd config file
2024-08-12 17:07:06 +08:00
Bo QIU
9189cabbb2
Supports batch randomly auto sync files ( #154 )
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-08-08 18:36:08 +08:00
peilun-conflux
dbd865fded
Change db_max_num_chunks to db_max_num_sectors. ( #137 )
...
* Change db_max_num_chunks to db_max_num_sectors.
* Update tests and config files.
* Revert contract change.
2024-07-29 22:31:19 +08:00
MiniFrenchBread
085c34beb0
feat: file location admin rpc; refactor: all_shards_available ( #134 )
...
* feat: find file rpc
* refactor: all_shards_available
* fix: fmt
* chore: remove rpc trace
2024-07-19 11:55:18 +08:00
Bo QIU
727a9b8bb6
enhance config and cli ( #116 )
...
* Update default value in config file
* Add config file for testnet
* Change default value for testnet config file
* Supports environment config source
* Add cli args
* add default value for cli args
* fix fmt
* Fix cli args default value
2024-07-10 15:36:10 +08:00
Bo QIU
821e4bd06e
Use public ip address to broadcast file announcement ( #115 )
...
* Use public ip address to broadcast file announcement
* auto detect public ip address for ENR address
* add more log
2024-07-09 17:39:14 +08:00
Bo QIU
84f455d1d4
Fix some issues in sync package ( #111 )
...
* Change default concurrent sync threads
* enhance log level in sync store
* Add rwlock in sync store to avoid concurrency issue
* enhance serial sync state machine log level
* wrap Instant for debug purpose
* add timestamp for connecting peers state
* Select random peer to sync chunks
* return sync peers states instead of connected peers only
* Use tokio async RwLock for sync store
2024-07-08 11:47:59 +08:00
Bo QIU
816353dfd4
Supports to concurrently sync files in sequence ( #108 )
...
* Supports to concurrently sync files in sequence
* add more comments
* refactor random auto sync
2024-07-04 14:04:17 +08:00
Bo QIU
5bc5bbd696
Enhance log level ( #103 )
...
* reduce default find peers timeout for auto sync
* opt log level to avoid too many noisy info logs
2024-07-02 11:52:52 +08:00
0g-peterzhb
cc44c9db66
remove mine id ( #98 )
2024-06-27 16:19:40 +08:00
Bo QIU
4ee782e4bc
enhance default config value ( #97 )
2024-06-27 14:47:33 +08:00
Bo QIU
d8d8c28c64
Add discv5 config and enlarge the default rate limiter ( #86 )
2024-06-14 18:04:35 +08:00
Bo QIU
dea518a0aa
Supports discv5 related configurations ( #84 )
...
* Supports discv5 related configurations
* fix fmt
2024-06-13 16:14:20 +08:00
peilun-conflux
489d973347
Update run/config.toml and version. ( #83 )
...
* Update run/config.toml.
* Update version
2024-06-11 13:17:32 +08:00
Bo QIU
6ca05090c3
update config ( #56 )
...
* update config
* use evmos testnet binary all the time
2024-04-16 21:24:50 +08:00
0g-peterzhb
6b9ee41457
update default config ( #52 )
2024-04-15 10:50:13 +08:00
0g-peterzhb
7bcd9b1988
update default config ( #44 )
2024-04-09 17:23:17 +08:00
0g-peterzhb
0131de26c8
update default config ( #42 )
2024-04-09 15:48:35 +08:00
0g-peterzhb
b07e6e776a
update boot node to static ip ( #38 )
2024-04-06 20:38:53 +08:00
Bo QIU
c2b8b1cab3
Rpc enhancements ( #15 )
...
* add admin rpc to terminate file or chunks sync
* Disallow to change file sync goal when failed
* split admin rpc into separate endpoints
* Support to launch public and private rpc at one port
* use one rpc port for python test
* fix lint
* restart to sync file or chunks if sync already completed or failed
2024-01-30 16:50:35 +08:00
Bo QIU
9b4b0436c3
Supports to sync partial chunks ( #4 )
...
* refactor p2p signed message
* add new pubsub messages in network layer to find chunks
* handle find chunks pubsub message in router
* Supports to sync partial chunks
* add admin rpc to sync chunks
* limit number of chunks to sync at a time
* refactor code to sync file and chunks
* add more switches to trigger file sync
* fix ut failure
* refactor code
2024-01-19 14:04:59 +08:00
Peilun Li
9058467be1
Initial commit
2024-01-04 18:05:32 +08:00
