0g-storage-node

mirror of https://github.com/0glabs/0g-storage-node.git synced 2024-11-14 20:15:17 +00:00

Author	SHA1	Message	Date
Bo QIU	9b68a8b7d7	Implement file sync protocol V2 (#249 ) Some checks failed abi-consistent-check / build-and-compare (push) Has been cancelled Details code-coverage / unittest-cov (push) Has been cancelled Details rust / check (push) Has been cancelled Details rust / test (push) Has been cancelled Details rust / lints (push) Has been cancelled Details functional-test / test (push) Has been cancelled Details * Add new P2P protocol NewFile * Publish NewFile message when any file finalized * handle NewFile message in router * handle NewFile in sync servic to write in db * use propagation source to handle NewFile message * Disable sequential sync and store new file in v2 sync store * Add shard config in FindFile * Add AnnounceFile RPC message in network layer * do not propagate FindFile to whole network * Mark peer connected if FileAnnouncement RPC message received * fix unit test failures * Change P2P protocol version * Ignore py tests of sequential auto sync * Add py test for auto sync v2 * fmt code * remove dummy code in py test * fix random test failure * Add comments * Enable file sync protocol v2 in config file by default	2024-10-28 14:56:08 +08:00
Bo QIU	8f17a7ad72	Fix metrics config deser (#245 ) Some checks failed abi-consistent-check / build-and-compare (push) Has been cancelled Details code-coverage / unittest-cov (push) Has been cancelled Details rust / check (push) Has been cancelled Details rust / test (push) Has been cancelled Details rust / lints (push) Has been cancelled Details functional-test / test (push) Has been cancelled Details	2024-10-22 14:14:33 +08:00
Bo QIU	ae6ecfec96	reduce default sync threads (#214 )	2024-09-26 14:55:00 +08:00
Bo QIU	ad80b22a1b	Optimize rpc config (#213 )	2024-09-25 16:48:40 +08:00
Bo QIU	9cde84ae15	Refactor network peer db configs (#209 )	2024-09-24 11:59:34 +08:00
Bo QIU	a9f5169c15	Allow user to configure network bandwidth for file sync (#198 ) Some checks are pending abi-consistent-check / build-and-compare (push) Waiting to run Details code-coverage / unittest-cov (push) Waiting to run Details rust / check (push) Waiting to run Details rust / test (push) Waiting to run Details rust / lints (push) Waiting to run Details functional-test / test (push) Waiting to run Details	2024-09-13 16:58:27 +08:00
Bo QIU	b6972b97af	Adjust default value for testnet configs (#180 ) Some checks are pending abi-consistent-check / build-and-compare (push) Waiting to run Details code-coverage / unittest-cov (push) Waiting to run Details rust / check (push) Waiting to run Details rust / test (push) Waiting to run Details rust / lints (push) Waiting to run Details functional-test / test (push) Waiting to run Details * Adjust default value for testnet configs * Supports to disable sequential auto sync * Add py tests for auto sync * fix py file name * rm dummy ;py code * change default block confirm count	2024-09-05 10:09:29 +08:00
Bo QIU	2fd9712d59	Enhance P2P network protocol to support batch messages for performance concern (#173 ) * Add p2p protocol version in network identity * Cache annouce file pubsub messages to publish in batch * fix file location cache * opt sync metrics * opt file location cache default configs * publish files announcements in batch * enhance announce file pubsub msg metrics * opt metrics * fix ci * fix clippy * fix batcher * minor fix * opt batcher: publish all if expired	2024-08-29 09:55:24 +08:00
bruno-valante	5757c98d2a	Config query context interval & Add config comments (#156 )	2024-08-27 17:36:36 +08:00
Bo QIU	37344b9439	opt sync config and update config file (#169 ) Some checks are pending abi-consistent-check / build-and-compare (push) Waiting to run Details code-coverage / unittest-cov (push) Waiting to run Details rust / check (push) Waiting to run Details rust / test (push) Waiting to run Details rust / lints (push) Waiting to run Details functional-test / test (push) Waiting to run Details	2024-08-26 15:29:52 +08:00
Bo QIU	82fef674ed	Add more metrics for file sync (#164 ) Some checks are pending abi-consistent-check / build-and-compare (push) Waiting to run Details code-coverage / unittest-cov (push) Waiting to run Details rust / check (push) Waiting to run Details rust / test (push) Waiting to run Details rust / lints (push) Waiting to run Details functional-test / test (push) Waiting to run Details * add metrics in log sync package * udpate auto sync metrics * Add metrics for completed file sync * add more metrics for serial file sync * adjust default timeout value for auto sync * fix metrics rpc for Timer type * add metrics for channel * refactor channel metrics * add timeout metrics for segment sync * refactor channel receiver	2024-08-22 10:42:15 +08:00
Bo QIU	9189cabbb2	Supports batch randomly auto sync files (#154 ) Some checks are pending abi-consistent-check / build-and-compare (push) Waiting to run Details code-coverage / unittest-cov (push) Waiting to run Details rust / check (push) Waiting to run Details rust / test (push) Waiting to run Details rust / lints (push) Waiting to run Details functional-test / test (push) Waiting to run Details	2024-08-08 18:36:08 +08:00
peilun-conflux	dbd865fded	Change db_max_num_chunks to db_max_num_sectors. (#137 ) * Change db_max_num_chunks to db_max_num_sectors. * Update tests and config files. * Revert contract change.	2024-07-29 22:31:19 +08:00
MiniFrenchBread	085c34beb0	feat: file location admin rpc; refactor: all_shards_available (#134 ) * feat: find file rpc * refactor: all_shards_available * fix: fmt * chore: remove rpc trace	2024-07-19 11:55:18 +08:00
Bo QIU	727a9b8bb6	enhance config and cli (#116 ) * Update default value in config file * Add config file for testnet * Change default value for testnet config file * Supports environment config source * Add cli args * add default value for cli args * fix fmt * Fix cli args default value	2024-07-10 15:36:10 +08:00
Bo QIU	821e4bd06e	Use public ip address to broadcast file announcement (#115 ) * Use public ip address to broadcast file announcement * auto detect public ip address for ENR address * add more log	2024-07-09 17:39:14 +08:00
Bo QIU	84f455d1d4	Fix some issues in sync package (#111 ) * Change default concurrent sync threads * enhance log level in sync store * Add rwlock in sync store to avoid concurrency issue * enhance serial sync state machine log level * wrap Instant for debug purpose * add timestamp for connecting peers state * Select random peer to sync chunks * return sync peers states instead of connected peers only * Use tokio async RwLock for sync store	2024-07-08 11:47:59 +08:00
Bo QIU	816353dfd4	Supports to concurrently sync files in sequence (#108 ) * Supports to concurrently sync files in sequence * add more comments * refactor random auto sync	2024-07-04 14:04:17 +08:00
Bo QIU	5bc5bbd696	Enhance log level (#103 ) * reduce default find peers timeout for auto sync * opt log level to avoid too many noisy info logs	2024-07-02 11:52:52 +08:00
0g-peterzhb	cc44c9db66	remove mine id (#98 )	2024-06-27 16:19:40 +08:00
Bo QIU	4ee782e4bc	enhance default config value (#97 )	2024-06-27 14:47:33 +08:00
Bo QIU	d8d8c28c64	Add discv5 config and enlarge the default rate limiter (#86 )	2024-06-14 18:04:35 +08:00
Bo QIU	dea518a0aa	Supports discv5 related configurations (#84 ) * Supports discv5 related configurations * fix fmt	2024-06-13 16:14:20 +08:00
peilun-conflux	489d973347	Update run/config.toml and version. (#83 ) * Update run/config.toml. * Update version	2024-06-11 13:17:32 +08:00
Bo QIU	6ca05090c3	update config (#56 ) * update config * use evmos testnet binary all the time	2024-04-16 21:24:50 +08:00
0g-peterzhb	6b9ee41457	update default config (#52 )	2024-04-15 10:50:13 +08:00
0g-peterzhb	7bcd9b1988	update default config (#44 )	2024-04-09 17:23:17 +08:00
0g-peterzhb	0131de26c8	update default config (#42 )	2024-04-09 15:48:35 +08:00
0g-peterzhb	b07e6e776a	update boot node to static ip (#38 )	2024-04-06 20:38:53 +08:00
Bo QIU	c2b8b1cab3	Rpc enhancements (#15 ) * add admin rpc to terminate file or chunks sync * Disallow to change file sync goal when failed * split admin rpc into separate endpoints * Support to launch public and private rpc at one port * use one rpc port for python test * fix lint * restart to sync file or chunks if sync already completed or failed	2024-01-30 16:50:35 +08:00
Bo QIU	9b4b0436c3	Supports to sync partial chunks (#4 ) * refactor p2p signed message * add new pubsub messages in network layer to find chunks * handle find chunks pubsub message in router * Supports to sync partial chunks * add admin rpc to sync chunks * limit number of chunks to sync at a time * refactor code to sync file and chunks * add more switches to trigger file sync * fix ut failure * refactor code	2024-01-19 14:04:59 +08:00
Peilun Li	9058467be1	Initial commit	2024-01-04 18:05:32 +08:00

32 Commits