Refinement
Updated: 28 Aug 2024
Items in this category need to undergo additional requirements that must be met in order to show sufficient maturity for funding:
A set of well-scoped deliverables (whether for research or for production software) ready to be made into a work-package (which may define future phases/partitions as well as feasibility)
Acceptance of those deliverables by the relevant TWG and/or SIGs
For any further information, refer to the process flow here.
LSM-2. Increase Capability to store UTxOs on disk using LSM (Integration to Node)
This is a follow-up work from current inflight item LSM-1
With continued growth of the Cardano Blockchain Ledger, the need to move from an in-memory to a disk persisted solution is even more important. This achieves more scalability and assures SPOs and full node wallets of being able to operate on commodity hardware or retain a lower footprint on cloud . Implementation of LSM will use an innovative approach to persisting to disk (SSD) using a Log Structured Merge(LSM) tree, which will be built standalone and integrated in node.
The component will take the form of a Haskell library for managing tables of key-value data stored persistently on disk. This is to reduce the memory size of the UTXO set (ledger) that will allow us to potentially quadruple the transactions on the network without increasing memory cost, which is hugely beneficial to Stake Pool Operators. This is intended to be used later within the Cardano node for storing large parts of the ledger state on disk. The component is specified as a “stand alone” component, in isolation from Cardano, with detailed functional and non-functional requirements. The work is structured in such a way that after an initial design phase further ratification and community feedback will be sought prior to any implementation or production deployment.
Status | In draft This item is pending validation of business benefits. A SIG is being set up to gather community input to articulate its value. |
Drivers |
|
Documentation | |
Target State | Developed new Log Structured Merge Tree implementations on Cardano mainnet |
Definition of Done | LSM work integrated into mainnet |
Product Stage | Integration |
Functional Requirements | LSM work integrated into the node providing an in-memory backend and an on-disk backend that can be toggled via config and/or runtime flags |
Non-functional Requirements | TBC |
External Dependencies | Completion of LSM libraries by Well-Typed |
Communication Channel | Intersect Development Updates Intersect Knowledge Base |
Indicative Sizing | L (Will be several months of work) |
Context of Indicative Sizing | Much of consensus needs to be adapted to use the LSM libraries for reading/writing states from/to disk |
Dependencies & Dependants | TBC |
Hardfork (Yes,No, TBC) | Not required |
Security Review | Yes |
Code Audit | Part of Node audit |
Community Endorsement | TBC |
Feasibility Study Required? | No |
Reviewing Working Group | Ledger working group (Also involved Consensus & CLI) |
Core Infrastructure (Yes, No, TBC) | Yes |
Item Champion | IOE |
Last updated