Introduction

This document discusses retention periods for data chunks in our data availability system. We define and explore the Minimal Chunk Retention Period (MCRP) and Extended Chunk Retention Period (ECRP). We analyze how these periods impact data availability verification, system reliability, and node incentives. The document also proposes a modification to the rewarding mechanism to ensure data persistence throughout the required retention period.

In the context of the Nomos DA the chunk term should be interpreted as a column.

Background

We begin by presenting a simplified architectural overview of the interaction between data availability and consensus protocols.

We differentiate node types by their functions, while in practice a node can provide multiple functions simultaneously—it's expected that a data availability node also serves as a consensus validator and performs a sampling as a light node.

1. When a blob is generated, it is encoded and divided into a set of chunks. Each chunk is sent to a dedicated subnetwork and stored by DA nodes assigned to that particular subnetwork. As a result of this DA interaction, a blob commitment—a unique identifier of the blob—is generated.
2. This blob commitment is used to generate a blob transaction that is added to the transactions mempool.
3. A block builder selects transactions from the mempool (one of which might be a blob transaction) and constructs a block, which is propagated to nodes participating in the consensus (validators, light nodes).
4. The consensus nodes validate the block, and if it contains a blob transaction, the node interacts with DA to verify the blob's availability. This is done by executing a blob data availability sampling mechanism that queries DA for proofs (requesting 20 chunks) confirming that the blob data is available.
5. A consensus node accepts a block's validity when all blobs are available and all transactions are correct. Otherwise, the block is rejected.

We can now define the Minimal Chunk Retention Period (MCRP) as the time a node must store the chunk to ensure data availability checks function properly. This is the period between blob encoding and propagation (step 1) and the acceptance of a block containing that blob (step 5). The MCRP provides sufficient time for all validators to validate and either accept or reject a block proposal containing one or more blob transactions.

We note that the MCRP time varies for each blob as the time since blob chunks are stored in DA and until the block is accepted is not deterministic. It depends on the content of the mempool and the transaction selection algorithm implemented by the block builder.

Discussion

However, the MCRP might be too short for light nodes that can spontaneously validate blocks (including past blocks). Therefore, we need to introduce the Extended Chunk Retention Period (ECRP) that provides sufficient time for block validation even for light nodes.

We assume that after block finalization there is no reason to validate the block; therefore, all chunks can be safely removed at that point. The ECRP must equal the MCRP plus the time necessary for block finalization.

Due to the nondeterminism of the ECRP (inherited from the MCRP), we cannot use it directly and must define a deterministic time period instead. We define the session length as the block finalization time, expressed in number of blocks. This way, a block is considered finalized after a "session number" of blocks following its acceptance.

Rewarding