Commit Graph

113 Commits

Author SHA1 Message Date
MiniFrenchBread
533bacb234
fix: admin_getFileLocation; test: sync test (#141)
* feat: add all_shards in admin_getFileLocation

* fix: admin_getFileLocation

* test: improve sync test

* fix: lint
2024-07-23 15:47:44 +08:00
bruno-valante
f0c3f2cfd0
Remove contract abi submodule (#139) 2024-07-22 17:32:51 +08:00
dependabot[bot]
116046366a
Bump thread_local from 1.1.3 to 1.1.8 in /version-meld/discv5 (#140)
Bumps [thread_local](https://github.com/Amanieu/thread_local-rs) from 1.1.3 to 1.1.8.
- [Commits](https://github.com/Amanieu/thread_local-rs/compare/v1.1.3...1.1.8)

---
updated-dependencies:
- dependency-name: thread_local
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-22 15:51:45 +08:00
MiniFrenchBread
a782cf2d54
feat: add all_shards in admin_getFileLocation (#138) 2024-07-22 11:28:05 +08:00
MiniFrenchBread
085c34beb0
feat: file location admin rpc; refactor: all_shards_available (#134)
* feat: find file rpc

* refactor: all_shards_available

* fix: fmt

* chore: remove rpc trace
2024-07-19 11:55:18 +08:00
Bo QIU
a5f95e2e7b
Verify announced IP with seen ip for pubsub msg (#130)
* Verify announced IP with seen ip for pubsub msg

* do not verify announced ip at local mode
2024-07-17 19:01:41 +08:00
Bo QIU
920efe0b59
enhance admin rpc to return peers instead of dump (#128) 2024-07-16 14:57:36 +08:00
Bo QIU
0f9f1c7bf2
Add admin RPC to dump discovered peers (#127)
* add admin rpc to dump peers

* allow to specify file name

* return listen addreses in rpc

* add trace for admin rpc parameters
2024-07-15 18:19:30 +08:00
peilun-conflux
504a67fdd0
Handle chunk write of rpc in parallel. (#126)
* Handle chunk write of rpc in parallel.

* double mine period.

* Increase the log sync channel capacity.

* Increase capacity.
2024-07-15 15:50:51 +08:00
Bo QIU
b1c52c7ad6
disable auto sync until catched up (#125) 2024-07-12 17:45:20 +08:00
bruno-valante
cca14e246e
Support multiple mine submission (#124)
* Support multiple mine submission

* Update
2024-07-12 17:31:53 +08:00
peilun-conflux
ba8d065e73
Generate sync proof with the given tx seq. (#123)
* Generate sync proof with the given tx seq.

* Fill HistoryTree with new data.

* Fix clippy.
2024-07-12 15:55:57 +08:00
Bo QIU
2bc402f94b
Enhance sync logs (#122)
* enhance sync logs

* fix lint
2024-07-12 14:26:23 +08:00
Bo QIU
cc5f8c2da4
Terminate file sync if failed (#121)
* terminate file sync if failed

* fix fmt

* always add log for file termination result
2024-07-11 16:54:04 +08:00
iwantanode
0d2caf9b76
Docs improvements (#82)
* Update README.md

* Update proof-of-random-access.md

* Update architecture.md

* Update introduction.md

* Update log-system.md

* Update run.md

* Update transaction-processing.md

* Update README.md
2024-07-11 14:10:57 +08:00
tom
fa2033eb30
Update introduction.md (#74) 2024-07-11 14:10:12 +08:00
peilun-conflux
e54b583b1c
Add zgs_getSectorProof. (#112) 2024-07-11 14:07:46 +08:00
peilun-conflux
0a69d0f56c
Remove a wrong double check when we fill the mpt with valid proof data. (#117) 2024-07-11 14:07:20 +08:00
peilun-conflux
3e23e81ae6
Repeat log catch-up until it's close to the latest height. (#114)
* Repeat log catch-up until it's close to the latest height.

* Add an oneshot channel to send catch up end complete.

* Fix comments.

* Fix tests.

* Fix ut.
2024-07-11 14:07:03 +08:00
Bo QIU
59d3a03487
Hotfix for cli default value (#118) 2024-07-11 11:54:13 +08:00
Bo QIU
727a9b8bb6
enhance config and cli (#116)
* Update default value in config file

* Add config file for testnet

* Change default value for testnet config file

* Supports environment config source

* Add cli args

* add default value for cli args

* fix fmt

* Fix cli args default value
2024-07-10 15:36:10 +08:00
Bo QIU
821e4bd06e
Use public ip address to broadcast file announcement (#115)
* Use public ip address to broadcast file announcement

* auto detect public ip address for ENR address

* add more log
2024-07-09 17:39:14 +08:00
Bo QIU
68b5d27728
Filter out file announcement of private ip (#113)
* Filter out file announcement of private ip

* add log for listen address
2024-07-08 18:45:55 +08:00
Bo QIU
84f455d1d4
Fix some issues in sync package (#111)
* Change default concurrent sync threads

* enhance log level in sync store

* Add rwlock in sync store to avoid concurrency issue

* enhance serial sync state machine log level

* wrap Instant for debug purpose

* add timestamp for connecting peers state

* Select random peer to sync chunks

* return sync peers states instead of connected peers only

* Use tokio async RwLock for sync store
2024-07-08 11:47:59 +08:00
Bo QIU
e44db25807
opt log level for file announcement relevant logs (#109) 2024-07-04 16:30:05 +08:00
Bo QIU
816353dfd4
Supports to concurrently sync files in sequence (#108)
* Supports to concurrently sync files in sequence

* add more comments

* refactor random auto sync
2024-07-04 14:04:17 +08:00
0g-peterzhb
d66fa10fe5
@peter/update config script (#106)
* update default config
2024-07-03 08:22:02 +08:00
0g-peterzhb
16d9beae15
update default config (#105) 2024-07-02 15:36:34 +08:00
0g-peterzhb
9e0d4deb47
update default config (#104) 2024-07-02 14:14:17 +08:00
Bo QIU
5bc5bbd696
Enhance log level (#103)
* reduce default find peers timeout for auto sync

* opt log level to avoid too many noisy info logs
2024-07-02 11:52:52 +08:00
Bo QIU
055d1fa32d
Fix sync state machine issue for sharding case (#102)
* Fix sync state machine issue for sharding case

* Fix fmt lint
2024-07-01 16:35:27 +08:00
0g-peterzhb
2f3361fafc
fix pruning (#101)
* fix pruning
2024-06-30 08:31:06 +08:00
peilun-conflux
4eb2a50b0e
Use inner lock in storage and use async lock. (#92)
* Use inner lock in storage.

* Remove mut.

* Remove async lock for storage.

* Fix tests and warnings.

* Use spawn_blocking for storage task.

* Fix clippy.

* Finalize the new tx at last.

* Revert "Finalize the new tx at last."

This reverts commit b56ad5582d.

* Wait for old same-root txs to finalize.

* Use async storage in miner.

* Update rust version to 1.79.0.

* Use Vec to avoid stack overflow.

* Fix unused warning.

* Fix clippy.

* Fix test warning.

* Fix test.

* fmt.

* Use async storage in pruner.

* nit.
2024-06-29 17:08:02 +08:00
0g-peterzhb
cc44c9db66
remove mine id (#98) 2024-06-27 16:19:40 +08:00
Bo QIU
4ee782e4bc
enhance default config value (#97) 2024-06-27 14:47:33 +08:00
Bo QIU
439314372b
Remove file location cache for connection timeout peer (#96) 2024-06-27 01:51:37 +08:00
MiniFrenchBread
fa74a4b9c1
feat: contract refactor (#93)
* feat: use refactored contract

* chore: update submodule

* chore: update submodule
2024-06-25 18:53:28 +08:00
Bo QIU
c6325f7643
Fix file sync pending due to peer connection timeout (#95)
* enhance logs for sync module

* report peer if connection timeout

* Use origin timestamp of sync state to determine if sync too long to terminate

* fix compilation error
2024-06-25 17:47:25 +08:00
Bo QIU
d8d8c28c64
Add discv5 config and enlarge the default rate limiter (#86) 2024-06-14 18:04:35 +08:00
Bo QIU
dea518a0aa
Supports discv5 related configurations (#84)
* Supports discv5 related configurations

* fix fmt
2024-06-13 16:14:20 +08:00
peilun-conflux
489d973347
Update run/config.toml and version. (#83)
* Update run/config.toml.

* Update version
2024-06-11 13:17:32 +08:00
peilun-conflux
6383c7ddac
Update cli version and fix tests. (#81)
* fix: test

* Update cli version.

---------

Co-authored-by: MiniFrenchBread <103425574+MiniFrenchBread@users.noreply.github.com>
2024-06-10 22:54:39 +08:00
peilun-conflux
0f52325f67
Sync and finalize sharded data. (#79)
* Save test logs.

* Prepare to handle shard config in chunk pool.

* Allow files to be finalized with only sharded data.

* Handle shard config change.

* Only sync needed data in shard config.

* Fix clippy and tests.
2024-06-08 02:50:36 +08:00
peilun-conflux
c2c6e2d5fb
Store shard config for peers and choose sync peers accordingly. (#77)
* Implement Pruner.

* Put pruner in a crate.

* Fix clippy.

* Add rpc zgs_getShardConfig.

* Fix.

* Increase wait time.

* Add pruner_test and use max_num_chunks instead of size_limit.

* Store shard config for peers and choose sync peers accordingly.

* Add test and fix sync.

* Fix clippy and test.

* Fix some ut.

* Add AnnounceShardConfig gossip and fix tests.

* Add sharded tx finalize check in LogManager.

* Try,

* Rename.

* Longer timeout for mine_test.

* Save test logs.
2024-06-07 16:58:15 +08:00
peilun-conflux
ef82f64393
Implement Pruner to delete unwanted data. (#70)
* Implement Pruner.

* Put pruner in a crate.

* Fix clippy.

* Add rpc zgs_getShardConfig.

* Fix.

* Increase wait time.

* Add pruner_test and use max_num_chunks instead of size_limit.

* Add back shard config and fix test.

* fix: serde format

* Remove unneeded PORA_CHUNK_SIZE.

* Fix tests.

---------

Co-authored-by: MiniFrenchBread <103425574+MiniFrenchBread@users.noreply.github.com>
2024-05-31 13:11:06 +08:00
Chenxing Li
c0f9d5cc31
Fix quality overflow (#75) 2024-05-30 21:49:47 +08:00
Bo QIU
0852b2355c
enhance rpc logs (#73) 2024-05-29 10:16:28 +08:00
Bo QIU
b17fd117fd
Supports 0gchain in python test framework. (#66) 2024-05-17 16:29:23 +08:00
Chenxing Li
2262bc3fb9
Support mining on sharded storage. (#64)
* Change PoraHash compute

* Change padSeed compute

* Refactor

* Support mining on sharded storage

* Detect single core performance and set correct params for test

* Fix clippy

* Fix an overflow bug
2024-04-30 14:34:44 +08:00
Chenxing Li
193e154361
Change miner id logic & request miner id automatically (#60)
* Change miner id logic & request miner id automatically

* Not enable all features in the test workflow.

* Auto configurable mining period

* Adjust test params for ci
2024-04-27 11:15:57 +08:00