RFP-003: Long-Term Block Data Storage
- Accepting one or multiple proposals: Multiple
- Submit by: Ongoing
- Selection by: Ongoing
- RFP Category:
infrastructure
- Link to RFP-003 and Application Instructions
- Learn more about the RFP Grants Program
Note
Use this space to discuss and ask questions about the RFP. We want this thread to facilitate a 2-way communication between the community and the team regarding this RFP.
Context/Background
Avail is a blockchain focused on “data availability” which means that it is designed to order and publish data (usually transaction data from other blockchains) and provide a high degree of confidence that the data was indeed published correctly.
Note that the focus is on ensuring that block data was published properly, not that it is stored long-term. The Avail validators/nodes currently do store all data generated by the network, because it is still new. Eventually, we plan to begin pruning block data older than some cutoff. To that end, Avail will benefit from a complementary solution designed specifically for data storage.
Project Description
A long-term data storage solution tailored to work with Avail would ideally have the following requirements:
- [P0] Observe the Avail network and detect when new blocks are finalized
- [P0] Automatically archive block data for finalized blocks
- [P0] Provide documentation for strong guarantees of continued storage
- [P0] Provide a method of retrieving the archived data
- [P1] Programmatic retrieval built into the Avail node, so that a node can be synced from genesis even after old blocks have been pruned from the other nodes on the network (via a command-line option/config)
- [P1] Provide a method for checking if archived data is still available
- [P2] Surface this info on a web-based status page or dashboard
Additional Information
Avail is based on Substrate, and much of the tooling available for Substrate/Polkadot will work with Avail with some tweaking. We have a library called avail-js that wraps polkadot.js and provides a way to access most chain functionality.
The block production and finality algorithms are GRANDPA/BABE. This is relevant because it makes sense to only store blocks once they have been finalized. The GRANDPA algorithm can be used to determine which blocks have been finalized, rather than using a trailing number of blocks or some other heuristic. For an overview, see here.
References
- Reference paper and repo links
- Avail documentation
- Communicating with Avail via RPC – see also:
- avail-js repository - library to connect to the Avail network
- polkadot.js documentation (generally works with Avail via avail-js)
- Communicating with Avail via RPC – see also:
- Embedding the Light Client
- Substrate
- GRANDPA finality overview
Read the entire RFP, Funding Milestones, and Application Instructions here.