Commit Graph

227 Commits

Author SHA1 Message Date
0g-peterzhb
0da3c374db
add snapshot test (#276)
* add snapshot test
2024-11-19 11:18:58 +08:00
Bo QIU
2a24bbde18
Fix protocol version issue (#277) 2024-11-19 09:56:59 +08:00
Bo QIU
cfe05b6f00
Always init chunk pool (#275)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-11-18 16:08:56 +08:00
Bo QIU
c4845f9103
Simplify serde with untagged attribute (#274)
Some checks failed
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-11-15 18:43:12 +08:00
0g-peterzhb
ce2b97f3c1
update chain rpc returned error msg (#273) 2024-11-15 14:32:16 +08:00
0g-peterzhb
96f846073f
increase db_max_num_sectors to be about 1000GB (#272) 2024-11-15 13:10:07 +08:00
Bo QIU
e912522386
Supports to sync historical files without NewFile gossip message (#269)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Supports to randomly sync historical files

* Add name for random file sync batcher

* Remove sync store metrics since multiple random batcher created

* opt log

* ignore pruned or finalized historical file

* Add python tests for historical file sync
2024-11-15 10:00:58 +08:00
MiniFrenchBread
4566eadb3e
chore: expose chunk data (#270)
* chore: expose chunk data
2024-11-15 08:26:34 +08:00
0g-peterzhb
4b48d25fb4
track tx seq number time consumed (#268)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-11-14 16:54:42 +08:00
peilun-conflux
1046fed088
Wait for tx to be processed in pruner. (#267)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Wait for tx to be processed in pruner.

* Put FIRST_REWARDABLE_CHUNK_KEY in data db.
2024-11-14 02:58:59 +08:00
0g-peterzhb
f4d5228234
@peter/detailed metrics (#256)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* add detailed metrics for storage layer
2024-11-13 17:07:34 +08:00
Bo QIU
d93f453d50
RPC return file pruned info (#266)
* RPC return file pruned info

* return tx status in atomic manner

* fix clippy
2024-11-13 16:55:57 +08:00
leopardracer
16e70bde68
fix: typos in documentation files (#265)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Update README.md
2024-11-13 09:07:06 +08:00
Bo QIU
1de7afec14
Add more metrics for network unbounded channel (#264)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Add metrics for file finalization in chunk pool

* Add metrics for network unbounded channel
2024-11-12 17:25:49 +08:00
Joel Liu
0c493880ee
add timeout for rpc connections (#263)
Some checks failed
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-11-09 13:37:09 +08:00
0g-peterzhb
fae2d5efb6
@peter/db split (#262)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* do not pad tx during sync phase

* add pad data store

* support async padding after sync phase

* split misc

* add sleep for the next loop
2024-11-08 22:06:45 +08:00
Bo QIU
3fd800275a
Improve rate limit for UDP discovery (#261)
Some checks failed
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-11-06 18:25:08 +08:00
Bo QIU
baf0521c99
copy key file to bootnode folder in python tests (#260)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-11-06 14:01:47 +08:00
Bo QIU
bcbd8b3baa
Ban peer if failed to decode pubsub message (#259)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-11-05 15:10:44 +08:00
Bo QIU
cae5b62440
Hotfix for python tests caused by unexpected file deletion (#258)
* Hotfix for python tests caused by unexpected file deletion

* add more info when launch blockchain node failed

* add stdout if blockchain launch failed

* seek stdout and err to 0 if failed to launch blockchain

* Improve zg chain port to avoid port conflict in parallel execution

* fix float issue

* Fix py failures
2024-11-05 13:49:58 +08:00
0g-peterzhb
9eea71e97d
separate data db from flow db (#252)
Some checks failed
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* separate data db from flow db
2024-10-31 15:44:26 +08:00
Bo QIU
bb6e1457b7
Refuse network identity incompatible nodes in UDP discovery layer (#253)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Add python test for UDP discovery

* Refuse nodes with incompatible ENR
2024-10-30 17:26:02 +08:00
peilun-conflux
da2cdec8a1
Remove unused. (#251)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-10-29 23:23:53 +08:00
Bo QIU
9b68a8b7d7
Implement file sync protocol V2 (#249)
Some checks failed
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* Add new P2P protocol NewFile

* Publish NewFile message when any file finalized

* handle NewFile message in router

* handle NewFile in sync servic to write in db

* use propagation source to handle NewFile message

* Disable sequential sync and store new file in v2 sync store

* Add shard config in FindFile

* Add AnnounceFile RPC message in network layer

* do not propagate FindFile to whole network

* Mark peer connected if FileAnnouncement RPC message received

* fix unit test failures

* Change P2P protocol version

* Ignore py tests of sequential auto sync

* Add py test for auto sync v2

* fmt code

* remove dummy code in py test

* fix random test failure

* Add comments

* Enable file sync protocol v2 in config file by default
2024-10-28 14:56:08 +08:00
peilun-conflux
789eae5cc1
Start nodes sequentially to fix some random failure. (#243)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
Currently we use the config `network_libp2p_nodes` to connect nodes
in the tests. This will not be retried, so if an early node starts
too slowly, other nodes may fail to connect to it.
2024-10-28 10:53:28 +08:00
peilun-conflux
2f9960e8e7
Hardcode pad data segment root. (#250)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Hardcode pad data segment root.

* fix deref

---------

Co-authored-by: Peter Zhang <peter@0g.ai>
2024-10-27 20:58:03 +08:00
peilun-conflux
506d234562
Use LRU to cache MPT nodes. (#227)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Add trait.

* Update merkle tree trait.

* Use NodeManager.

* fix.

* Use LRU for cache.

* fix clippy.

* Save layer size.

* Initialize LogManager with NodeManager.

* Fix.

* Fix test.

* fix.
2024-10-27 12:52:06 +08:00
Bo QIU
8f17a7ad72
Fix metrics config deser (#245)
Some checks failed
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-10-22 14:14:33 +08:00
Joel Liu
2947cb7ac6
Optimizing recover perf by reducing sync progress events (#244)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Optimizing recover perf by reducing sync progress events

* add log

* add log
2024-10-21 21:24:50 +08:00
peilun-conflux
39efb721c5
Remove sender from contract call. (#242)
This allows the RPC services to cache the results.
2024-10-21 16:58:50 +08:00
Joel Liu
9fe5a2c18b
Stop recovery when sending via the channel fails (#240)
Some checks failed
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-10-18 16:08:35 +08:00
MiniFrenchBread
80b4d63cba
fix: error handling (#235)
Some checks failed
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-10-15 19:27:22 +08:00
bruno-valante
b2a70501c2
Test flow root consistency (#230) 2024-10-15 14:24:56 +08:00
Bo QIU
e701c8fdbd
Supports custom public ip to announce file (#233)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Supports custom public ip to announce file

* Fix comment
2024-10-14 14:57:42 +08:00
0g-peterzhb
a4b02a21b7
add retry (#232) 2024-10-14 14:19:05 +08:00
MiniFrenchBread
3fc1543fb4
chore: update abi (#234)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-10-14 12:38:13 +08:00
bruno-valante
82fd29968b
Support shard in case the mining is not enabled (#231)
Some checks failed
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-10-12 17:03:47 +08:00
peilun-conflux
45fa344564
Check the local flow root against the contract state. (#229)
* Check the local flow root against the contract state.

* Check zero contract root.

* Fix wrong root before the first segment.

* Update contracts.

* Fix proof insertion.
2024-10-12 16:50:31 +08:00
0g-peterzhb
48868b60db
update ip (#223)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-10-11 17:07:43 +08:00
Joel Liu
649f6e5e9f
Fix the dead loop problem (#228)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* Fix the dead loop problem

* keep parent block of log_latest_block_number block

* avoid overflow
2024-10-10 22:02:20 +08:00
MiniFrenchBread
79d960d4ea
feat: tx_seq rpc (#226)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-10-10 14:57:40 +08:00
MiniFrenchBread
b131dc532f
test: update contracts, shard submission test (#225)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* fix: pad tx based on the on-chain contract result.

* chore: update contracts

* test: shard submission

---------

Co-authored-by: Peilun Li <peilun.li@confluxnetwork.org>
2024-10-09 16:33:20 +08:00
peilun-conflux
4e2c5fa8a4
fix: pad tx based on the on-chain contract result. (#224) 2024-10-09 15:20:42 +08:00
peilun-conflux
b76fa7be9b
Add TopicScoreParams for gossipsubs. (#189)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
2024-10-08 14:32:23 +08:00
bruno-valante
27db2d6496
Avoid appending to_seal_set if sealer is not enabled (#212) 2024-10-08 14:26:35 +08:00
0g-peterzhb
20266e0a6c
@peter/async padding (#219)
* support async padding
2024-10-08 13:13:32 +08:00
peilun-conflux
395aeabde7
fix: do not finalize same-root tx with missing data. (#222)
Some checks failed
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
2024-10-05 22:16:33 +08:00
MiniFrenchBread
ee4123418f
fix: end_segment_index (#221)
Some checks are pending
abi-consistent-check / build-and-compare (push) Waiting to run
code-coverage / unittest-cov (push) Waiting to run
rust / check (push) Waiting to run
rust / test (push) Waiting to run
rust / lints (push) Waiting to run
functional-test / test (push) Waiting to run
* fix: end_segment_index
2024-10-05 14:31:12 +08:00
MiniFrenchBread
949462084a
fix: finalize check on tx sync (#220)
Some checks failed
abi-consistent-check / build-and-compare (push) Has been cancelled
code-coverage / unittest-cov (push) Has been cancelled
rust / check (push) Has been cancelled
rust / test (push) Has been cancelled
rust / lints (push) Has been cancelled
functional-test / test (push) Has been cancelled
* fix: finalize

* fix: check

* fix: end_segment_index

* fix: fmt
2024-09-30 18:43:11 +08:00
MiniFrenchBread
07ac814e57
test: use fixed 0g binary (#218)
* test: use latest 0g binary

* feat: use bug fixed 0g binary
2024-09-30 14:56:38 +08:00