Commit Graph

36 Commits

Author SHA1 Message Date
0g-peterzhb
f4d5228234
@peter/detailed metrics (#256)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* add detailed metrics for storage layer
2024-11-13 17:07:34 +08:00
Joel Liu
0c493880ee
add timeout for rpc connections (#263)
Some checks failed
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-11-09 13:37:09 +08:00
0g-peterzhb
fae2d5efb6
@peter/db split (#262)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* do not pad tx during sync phase

* add pad data store

* support async padding after sync phase

* split misc

* add sleep for the next loop
2024-11-08 22:06:45 +08:00
Joel Liu
2947cb7ac6
Optimizing recover perf by reducing sync progress events (#244)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Optimizing recover perf by reducing sync progress events

* add log

* add log
2024-10-21 21:24:50 +08:00
Joel Liu
9fe5a2c18b
Stop recovery when sending via the channel fails (#240)
Some checks failed
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-10-18 16:08:35 +08:00
0g-peterzhb
a4b02a21b7
add retry (#232) 2024-10-14 14:19:05 +08:00
peilun-conflux
45fa344564
Check the local flow root against the contract state. (#229)
* Check the local flow root against the contract state.

* Check zero contract root.

* Fix wrong root before the first segment.

* Update contracts.

* Fix proof insertion.
2024-10-12 16:50:31 +08:00
Joel Liu
649f6e5e9f
Fix the dead loop problem (#228)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Fix the dead loop problem

* keep parent block of log_latest_block_number block

* avoid overflow
2024-10-10 22:02:20 +08:00
0g-peterzhb
20266e0a6c
@peter/async padding (#219)
* support async padding
2024-10-08 13:13:32 +08:00
MiniFrenchBread
ee4123418f
fix: end_segment_index (#221)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* fix: end_segment_index
2024-10-05 14:31:12 +08:00
MiniFrenchBread
949462084a
fix: finalize check on tx sync (#220)
Some checks failed
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* fix: finalize

* fix: check

* fix: end_segment_index

* fix: fmt
2024-09-30 18:43:11 +08:00
MiniFrenchBread
5849e9c2ba
fix: finalize file does not need to save (#206)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* fix: finalize file does not need to save

* fix: rust fmt
2024-09-19 19:14:45 +08:00
bruno-valante
da0e2c7031
Fix issues found in auditions (#203) 2024-09-14 18:19:16 +08:00
Joel Liu
f878a4849c
Handle cases where sequence is not continuous during catch-up (#199)
* Handle cases where the sequence is not continuous during the catch-up process

* get block hash from rpc if not found in local cache
2024-09-14 09:05:11 +08:00
Joel Liu
702680f3a4
reset parent block hash when reset progress (#188)
Some checks failed
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* reset parent block hash when reset progress
2024-09-09 14:51:08 +08:00
Joel Liu
43c2d5f788
Add configuration which can force sync from start_block_number (#183)
* queyr logs via LogQuery in wath loop
2024-09-08 08:34:59 +08:00
Joel Liu
041f5f12b6
Allow for retry attempts in the watch loop (#179)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Allow for retry attempts in the watch loop
2024-09-06 15:08:13 +08:00
Joel Liu
e20be63026
Query logs via LogQuery in watch loop (#177)
Some checks failed
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* queyr logs via LogQuery in wath loop

* fix lints error
2024-09-02 18:49:18 +08:00
0g-peterzhb
e3c199d361
check error messages (#176)
Some checks failed
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-08-30 17:28:34 +08:00
Bo QIU
82fef674ed
Add more metrics for file sync (#164)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* add metrics in log sync package

* udpate auto sync metrics

* Add metrics for completed file sync

* add more metrics for serial file sync

* adjust default timeout value for auto sync

* fix metrics rpc for Timer type

* add metrics for channel

* refactor channel metrics

* add timeout metrics for segment sync

* refactor channel receiver
2024-08-22 10:42:15 +08:00
peilun-conflux
53449e1faa
Half log_page_size when it queries too many logs. (#152)
Some checks failed
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* Half log_page_size when it queries too many logs.

* fmt.

* Increase log broadcast channel size.
2024-08-09 17:13:20 +08:00
peilun-conflux
504a67fdd0
Handle chunk write of rpc in parallel. (#126)
* Handle chunk write of rpc in parallel.

* double mine period.

* Increase the log sync channel capacity.

* Increase capacity.
2024-07-15 15:50:51 +08:00
peilun-conflux
3e23e81ae6
Repeat log catch-up until it's close to the latest height. (#114)
* Repeat log catch-up until it's close to the latest height.

* Add an oneshot channel to send catch up end complete.

* Fix comments.

* Fix tests.

* Fix ut.
2024-07-11 14:07:03 +08:00
peilun-conflux
4eb2a50b0e
Use inner lock in storage and use async lock. (#92)
* Use inner lock in storage.

* Remove mut.

* Remove async lock for storage.

* Fix tests and warnings.

* Use spawn_blocking for storage task.

* Fix clippy.

* Finalize the new tx at last.

* Revert "Finalize the new tx at last."

This reverts commit b56ad5582d.

* Wait for old same-root txs to finalize.

* Use async storage in miner.

* Update rust version to 1.79.0.

* Use Vec to avoid stack overflow.

* Fix unused warning.

* Fix clippy.

* Fix test warning.

* Fix test.

* fmt.

* Use async storage in pruner.

* nit.
2024-06-29 17:08:02 +08:00
Joel Liu
6c1b0b35ec
Wait SyncedBlock to be processed for revert block (#45)
* wait SyncedBlock to be processed

* remove retry counter

* use default parent block hash for missing case
2024-04-11 18:13:52 +08:00
peilun-conflux
bd4ebee2da
Change some log level. (#46)
* Change some log level.

* Add log for flow root commit.
2024-04-11 17:15:05 +08:00
MiniFrenchBread
0047ab22d4
chore: update dependency (#39)
* chore: update dependency

* chore: update dependency
2024-04-08 13:54:01 +08:00
Joel Liu
b8a59e9222
query getLogs from batch blocks (#37)
* query getLogs from batch blocks

* resolve comments
2024-04-07 11:27:18 +08:00
peilun-conflux
588bf39d7d
Fix a possible log sync break after a node restarts. (#34)
* Fix a possible log sync break after a node restarts.

* Skip `sync_blocks` in blockchain node setup.
2024-03-29 16:15:01 +08:00
Joel Liu
f9ce286909
replace filter with eth_getLogs (#32)
* replace filter with eth_getLogs

* update log

* store block history into db

* fix lints error

* add free disk step

* set max worker to 3

* resolve comments and refact code

* resolve comments

* set max-workers to 4

* set max-workers to 3
2024-03-27 13:54:06 +08:00
peilun-conflux
0c123500c9
Update the merkle tree with proof data. (#9)
* Add proof data for chunk proof.

* Support file proof.

* Update with segment proof and fix issues.

* Fix more issues.

* Fix the process of file proof.

* Merge branch 'main' into fix_proof

* Enable sync_test.

* Fix wrongly updated submodule.

* Fix bsc node version.
2024-01-26 10:29:09 +08:00
peilun-conflux
877f484dd0
Merge pull request #2 from zero-gravity-labs/filter_address
Filter address in log sync.
2024-01-19 19:45:27 +08:00
Bo QIU
971d344acd
fmt code (#3) 2024-01-19 10:56:59 +08:00
Peilun Li
986bfa412c Add to start_recover. 2024-01-17 10:57:07 +08:00
Peilun Li
e236c2f70a Filter address in log sync. 2024-01-16 17:54:23 +08:00
Peilun Li
9058467be1 Initial commit 2024-01-04 18:05:32 +08:00