Background

Evaluating how to efficiently handle DaShare and Block deserialization using zero copy techniques.

Shares are sometimes read from the database to be sent over the network, and sometimes received from the network and verified. Access to Proof and Commitment is only needed during verification. Similarly, blocks are sometimes deserialized for validation, and in other cases read from the database to be sent over the network.

This pattern suggest that we may completely avoid deserialisation on some paths but benefits may not be very big. Please see below.

Reviewed types

pub struct DaShare {
    pub column: Vec<Chunk>,
    pub share_idx: u16,
    pub combined_column_proof: Proof,
    pub rows_commitments: Vec<Commitment>,
}


pub struct Block<Tx, BlobCertificate> {
    header: Header,
    cl_transactions: IndexSet<Tx>,
    bl_blobs: IndexSet<BlobCertificate>,
}


pub struct Header {
    parent: HeaderId,
    slot: Slot,
    content_size: u32,
    content_id: ContentId,
    leader_proof: Risc0LeaderProof,
    orphaned_leader_proofs: Vec<Header>,
}

Options Reviewed

Approach Pros Cons
zerocopy Provides direct access to fixed-size fields without copying Limited to simple types; not compatible with Vec or cryptographic types
rkyv Allows reading full struct layout from buffer with zero allocation Returns archived types; constructing cryptographic types requires allocation
nom Supports selective parsing of individual fields Parsing logic must be written manually; constructing cryptographic types requires allocation
bytes Enables efficient slicing of raw buffers with minimal overhead Field layout and offsets must be tracked manually; no struct decoding

Performance Estimates (1000 structs per second)

Approach Time to construct from Bytes (1 KB struct) Notes on what the time includes
bincode ~2–5 ms Includes full deserialization into owned Rust structs
zerocopy ~0.1–0.3 ms Includes alignment checks and casting to typed references
rkyv ~0.5 ms Includes creating archived read-only view
nom ~1–2 ms Includes manual parsing and struct construction
bytes wrapper ~0 ms N/A

Limitations of rkyv(best candidate)

Conclusion

If we decide to go with a zero-copy approach, a simple and flexible way is to wrap Bytes inside a struct like this and deserialise when needed: