Improving DevEx, Response Times, Throughput Limitations for Users through Turbo DA

[privacy_prophet]

When submitting data to Avail, users face the following challenges::

• Non-optimal wait times: Users often need to prove to someone else that data was submitted. To do that they need to wait until their data is posted and finalised on-chain. Given Avail’s 20 second block time, and with finalisation taking multiple blocks, means a 60-90s wait time.
• Block & tx size constraints: Apps with larger amounts of data have to slice their data into chunks that can fit into blocks.
• Error handling logic: RPCs can go down, return errors, etc. Standard best practice is to handle errors and retry — which means additional logic that needs to be implemented for every integration.
• Lack of predictable/guaranteed costs: Avail charges posting fees in AVAIL, which means other chains have to swap their tokens or use fiat to have AVAIL on hand. From their perspective, this adds an exchange rate risk. Additionally, Avail’s fees can go up/down, either as a result of congestion or governance. So there is no way for projects to get guaranteed pricing.
• AVAIL swap burden & strategic mis-alignment: Having to convert other tokens to AVAIL is often a manual operation. The alternative is to swap larger amounts less often, but reducing their own token’s account to keep some as another project’s token can feel like a strategic mis-alignment.

TurboDA solves all of these issues.

TurboDA is a service that posts data on the users’ behalf. Users can buy “credits” which allow them to post data via the service at a guaranteed rate. It handles all the error logic internally, and can respond immediately with a “pre-confirmation” even before the data is finalized on the Avail chain.

There are three main elements of the solution:

• Immediate soft confirmations
• High availability & service scalability
• Pre-paid “data credits”

Immediate soft confirmations

TurboDA will issue immediate (<250ms) soft confirmations as soon as data is posted to it (before it gets published on the Avail chain).

Those soft confirmations can be used as external commitments: they can be given to anyone, and in turn used to check with TurboDA to confirm what the data is, retrieve a full copy of it if necessary, and obtain details about whether it has been posted onto Avail yet (and if so, to which block).

High availability & service scalability

TurboDA is written to be very robust, and will guarantee high availability via error handling, RPC fallbacks, and ability to survive even sudden power outages without any data loss. Once a soft confirmation is issued, it has been committed to its database and can be relied upon.

These guarantees are not quite as strong as posting to Avail itself, but they are about as strong as possible for a centralized service.

Additionally, the service will be designed to allow our operations to effectively support many users and limit administrative work needed to operate the service. This includes:

• Automatically picking up transfers to user address (topping up their tokens)
• Automatically swapping spent tokens to AVAIL and bridging back to Avail to replenish the internal send wallet

Pre-paid “data credits”

In order to allow users to reserve blockspace and pay in any token, we will create a system of “data credits” that can be purchased in any ERC-20 token as well as native AVAIL. Once purchased, credits can be used to post data on chain.

This makes pricing more predictable. Although the price of the credits might fluctuate, once purchased they guarantee the ability to post the stated amount of data. For example, an account with 5MB of credits can post 5MB of data (or more, with discounts), no matter what happens to relative exchange rates, AVAIL fees, etc.

It also means that it allows any project to pay for DA in any token of their choosing, without having to keep AVAIL on hand.

Credits & Discounts

Accounts are funded with data credits. Credits are purchased with tokens, and represent a guarantee of being able to post 1KB of data each. In this way, they work similar to cell phone minutes: an account with 100KB of credits can spend those credits to post 100KB of data (or more, see below for how discounts are applied).

Purchasing credits

Data credits can be purchased on the dashboard at their current price. The current price per credit is determined by estimating the required AVAIL to post 1KB (the amount guaranteed by the data credit).

Payment is accepted in native AVAIL, or in a variety of other ERC-20 tokens (by calculating the equivalent to the AVAIL price). When payment is made in another token, TurboDA uses 3rd party exchange (e.g., a DEX such as Uniswap) to convert the tokens paid into AVAIL. Down the line, we may consider allowing them to be purchased with fiat currency as well (though not at this time).

For clarity, users can purchase credits (1KB each), so the price calculation would be: (current price for 1KB) * (number of credits being purchased)

Then translated to the token the user wants to pay with at the current exchange rate.

Purchased credits are tracked in the user’s TurboDA account, and can be used as payment for posting data as described in the next section.

Spending credits

Avail charges less per byte for larger transactions, and we want to pass on those savings to TurboDA users. Thus, although we standardize on 1KB credits, we offer discounts when users post more than 1KB.

When posting data, credits are spent with the following rules:

• A minimum of 1 credit is always spent, regardless of the amount of data posted (for example, if the user posts 100 bytes, we still charge 1KB / 1 credit).
• When the posted data is greater than 1KB, a bulk discount is applied:
  • (1KB_fee / data_posted_fee) * data_posted_amount = data_billed
  • Where:
    • 1KB_fee is the Avail fee required to post 1KB at the current time
    • data_posted_fee is the Avail fee required to post the amount of data being posted at the current time
    • data_posted_amount is the amount of data being posted
    • data_billed = amount of data which will be deducted from the account’s credit balance

This means that a minimum of 1KB is always billed, but if more data is posted at once, a bulk discount is applied. For example, the discount for posting 100KB of data is around 45%, so only ~55KB of data would be billed.

The goal is to approximate the actual Avail fees, though we may simplify the above algorithm (for example, with a hardcoded formula instead of using realtime fees).

Technical Design

• The Dashboard is used to login and register yourself to the service.
• From the same dashboard, you buy “Data Credits” from Avail in any enabled tokens.
• Turbo DA exposes two endpoints: submit_data and submit_raw_data, where former is used to submit data in json format, and the later as an octet-stream.
• Turbo DA acts as a mempool on top of Avail DA. It holds all your transactions and parallelized the data submission process to increase the overall throughput of any connected application. The degree of how many transaction can be parallelize is configurable, i.e. system is veritcally scalable too.
• The response time of the service ~250ms, since once submitted all requests are mapped to a submission id.
• There are other services in the system besides the above data submission:
  • Fallback service: To monitor any failed service and retry submitting it.
  • Funds monitor: Monitors credit buy requests, and on payment updates the user’s credit balance.
• Additional upcoming features include ability to post encrypted data to Avail, and ability to partially / completely decrypt the cipher text depending upon the decryption requirements based on MPC.

Conclusion

TurboDA is a service that solves several key challenges users face when submitting data to Avail:

1. Provides immediate soft confirmations (< 250ms) before data is published on Avail chain.
2. Offers high availability through robust error handling and RPC fallbacks.
3. Introduces pre-paid “data credits” system that:
   • Can be purchased in any ERC-20 token or native AVAIL
   • Guarantees ability to post specific amounts of data
   • Includes bulk discounts for larger data posts

This solution effectively addresses the previous issues of long wait times, size constraints, error handling, unpredictable costs, and AVAIL token swap operational overhead.