Commit Graph

153 Commits

Author SHA1 Message Date
Joel Liu
43c2d5f788
Add configuration which can force sync from start_block_number ()
* queyr logs via LogQuery in wath loop
2024-09-08 08:34:59 +08:00
peilun-conflux
29fcc415a6
Add log for proof generation errors. () 2024-09-06 17:56:04 +08:00
bruno-valante
bf3694d138
Update mine test process to avoid random bugs on low-performance devices () 2024-09-06 17:53:00 +08:00
Joel Liu
041f5f12b6
Allow for retry attempts in the watch loop ()
* Allow for retry attempts in the watch loop
2024-09-06 15:08:13 +08:00
Bo QIU
b6972b97af
Adjust default value for testnet configs ()
* Adjust default value for testnet configs

* Supports to disable sequential auto sync

* Add py tests for auto sync

* fix py file name

* rm dummy ;py code

* change default block confirm count
2024-09-05 10:09:29 +08:00
Joel Liu
e20be63026
Query logs via LogQuery in watch loop ()
* queyr logs via LogQuery in wath loop

* fix lints error
2024-09-02 18:49:18 +08:00
boqiu
e7a562fa61 hotfix for json serde camel case 2024-09-02 16:12:01 +08:00
Bo QIU
4edd61b9d2
return network identity for status rpc () 2024-09-02 16:05:44 +08:00
0g-peterzhb
e3c199d361
check error messages () 2024-08-30 17:28:34 +08:00
0g-peterzhb
508e787bd0
update boot nodes () 2024-08-30 10:38:19 +08:00
Bo QIU
6685757a03
notify file announcement to sync layer only if shard config matches ()
* notify file announcement to sync layer only if shard config matches

* use FromStr trait
2024-08-29 18:35:35 +08:00
boqiu
75531a5878 Add metrics in file location cache 2024-08-29 11:55:00 +08:00
Bo QIU
2fd9712d59
Enhance P2P network protocol to support batch messages for performance concern ()
* Add p2p protocol version in network identity

* Cache annouce file pubsub messages to publish in batch

* fix file location cache

* opt sync metrics

* opt file location cache default configs

* publish files announcements in batch

* enhance announce file pubsub msg metrics

* opt metrics

* fix ci

* fix clippy

* fix batcher

* minor fix

* opt batcher: publish all if expired
2024-08-29 09:55:24 +08:00
bruno-valante
a79f7bbf12
Fix contract interface build cache () 2024-08-27 18:08:10 +08:00
bruno-valante
aaba4b5da0
Fix issue found in auditions () 2024-08-27 17:36:55 +08:00
bruno-valante
5757c98d2a
Config query context interval & Add config comments () 2024-08-27 17:36:36 +08:00
Bo QIU
f1a1c39332
adjust sync timeout in py test () 2024-08-27 09:52:00 +08:00
Bo QIU
37344b9439
opt sync config and update config file () 2024-08-26 15:29:52 +08:00
0g-peterzhb
780865b1b8
do not fail on prune error ()
* do not fail on prune error
2024-08-25 16:48:03 +08:00
peilun-conflux
1c72607fbc
Set sync start index based on data in db. ()
* Set sync start index based on data in db.

* Fix test.

* nit.
2024-08-23 12:41:25 +08:00
Bo QIU
f14a1b5975
add timestamp in p2p rpc request id for latency stat ()
* add timestamp in p2p rpc request id for latency stat

* fix test compilation error
2024-08-22 14:34:10 +08:00
Bo QIU
82fef674ed
Add more metrics for file sync ()
* add metrics in log sync package

* udpate auto sync metrics

* Add metrics for completed file sync

* add more metrics for serial file sync

* adjust default timeout value for auto sync

* fix metrics rpc for Timer type

* add metrics for channel

* refactor channel metrics

* add timeout metrics for segment sync

* refactor channel receiver
2024-08-22 10:42:15 +08:00
Bo QIU
c1f465e009
Do not verify announced ip address by default () 2024-08-20 14:39:58 +08:00
boqiu
c337ff90fb hotfix for admin rpc metrics 2024-08-19 17:07:57 +08:00
Bo QIU
908ee156ca
Add admin rpc to exports metrics () 2024-08-19 15:18:12 +08:00
Bo QIU
f9120b1e4a
Add metrics for router and auto sync ()
* Add metrics configurations

* Add metrics in router package

* Add catch up info in sync service state

* Add metrics for auto sync

* update cargo lock
2024-08-19 09:54:52 +08:00
Bo QIU
22ed8f5f91
opt router debug logs () 2024-08-16 10:34:28 +08:00
peilun-conflux
12d0c6b675
Check network id in status and ban incompatible peers. ()
* Check network id in status and ban incompatible peers.

* Revert debug changes.

* Request chain id from the blockchain server.
2024-08-14 11:35:48 +08:00
Bo QIU
03286ebd78
Supports HDD config file ()
* Add debug log config

* Add config file for ssd and hdd

* Set most default value for ssd and hdd config file
2024-08-12 17:07:06 +08:00
0g-peterzhb
f03f97c609
more robust provider ()
* more robust provider
2024-08-12 11:35:21 +08:00
peilun-conflux
53449e1faa
Half log_page_size when it queries too many logs. ()
* Half log_page_size when it queries too many logs.

* fmt.

* Increase log broadcast channel size.
2024-08-09 17:13:20 +08:00
Bo QIU
9189cabbb2
Supports batch randomly auto sync files () 2024-08-08 18:36:08 +08:00
Bo QIU
e1df2f0868
Change hardcoded const variables to configs () 2024-08-08 16:21:50 +08:00
rickiey
77d1b84974
fix:log config : invalid filter directive () 2024-08-06 18:46:53 +08:00
peilun-conflux
d80e7e22ca
Prune no reward chunks. ()
* Prune no reward chunks.

* Add tests.

* Fix tests.

* Fix clippy.

* Revert test.

* Enable market in shard_sync_test.

* Add tx prune status.

* Fix tests.
2024-08-06 15:06:15 +08:00
Bo QIU
6ade66c086
Add admin rpc to return sync service state () 2024-08-06 14:01:57 +08:00
Bo QIU
891e00fa80
Refactor auto sync to return sync status ()
* Refactor auto sync to return sync state

* Add auto sync manager to wrap multiple objects

* use auto sync manager

* fix python tests
2024-08-05 17:30:26 +08:00
peilun-conflux
dbd865fded
Change db_max_num_chunks to db_max_num_sectors. ()
* Change db_max_num_chunks to db_max_num_sectors.

* Update tests and config files.

* Revert contract change.
2024-07-29 22:31:19 +08:00
Bo QIU
ae9c52c0e6
Terminate file sync if connecting peers too long () 2024-07-27 21:05:44 +08:00
bruno-valante
7d73ccd1e1
Update with the most recent storage contract ()
* Update with the most recent storage contract

* fix nits
2024-07-24 13:01:30 +08:00
MiniFrenchBread
533bacb234
fix: admin_getFileLocation; test: sync test ()
* feat: add all_shards in admin_getFileLocation

* fix: admin_getFileLocation

* test: improve sync test

* fix: lint
2024-07-23 15:47:44 +08:00
bruno-valante
f0c3f2cfd0
Remove contract abi submodule () 2024-07-22 17:32:51 +08:00
dependabot[bot]
116046366a
Bump thread_local from 1.1.3 to 1.1.8 in /version-meld/discv5 ()
Bumps [thread_local](https://github.com/Amanieu/thread_local-rs) from 1.1.3 to 1.1.8.
- [Commits](https://github.com/Amanieu/thread_local-rs/compare/v1.1.3...1.1.8)

---
updated-dependencies:
- dependency-name: thread_local
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-22 15:51:45 +08:00
MiniFrenchBread
a782cf2d54
feat: add all_shards in admin_getFileLocation () 2024-07-22 11:28:05 +08:00
MiniFrenchBread
085c34beb0
feat: file location admin rpc; refactor: all_shards_available ()
* feat: find file rpc

* refactor: all_shards_available

* fix: fmt

* chore: remove rpc trace
2024-07-19 11:55:18 +08:00
Bo QIU
a5f95e2e7b
Verify announced IP with seen ip for pubsub msg ()
* Verify announced IP with seen ip for pubsub msg

* do not verify announced ip at local mode
2024-07-17 19:01:41 +08:00
Bo QIU
920efe0b59
enhance admin rpc to return peers instead of dump () 2024-07-16 14:57:36 +08:00
Bo QIU
0f9f1c7bf2
Add admin RPC to dump discovered peers ()
* add admin rpc to dump peers

* allow to specify file name

* return listen addreses in rpc

* add trace for admin rpc parameters
2024-07-15 18:19:30 +08:00
peilun-conflux
504a67fdd0
Handle chunk write of rpc in parallel. ()
* Handle chunk write of rpc in parallel.

* double mine period.

* Increase the log sync channel capacity.

* Increase capacity.
2024-07-15 15:50:51 +08:00
Bo QIU
b1c52c7ad6
disable auto sync until catched up () 2024-07-12 17:45:20 +08:00