L-Store, HTAP Database Engine Built from Scratch

Problem

Transactional and analytical workloads want opposite storage layouts. Row-oriented storage serves point updates well and scans badly; columnar storage does the reverse. An HTAP engine has to serve both without keeping two copies of the truth.

Approach

Lineage-based columnar storage splitting immutable base pages from append-only tail pages, with an indirection column resolving each record to its latest version at read time. Writes append rather than mutate, so readers never block on writers.

What it measured

Validated across 3 autograded milestones covering correctness, throughput, and concurrent transaction behavior.

Result

A working engine with an LRU bufferpool doing dirty-page write-back, a background merge compacting tail updates back into read-optimized base pages, and concurrent multithreaded transactions under record-level two-phase locking. Built with a 5-person team.

On the engineer resume

• Built an HTAP database engine in Python (5-person team): lineage-based columnar storage, an LRU bufferpool with dirty-page write-back, background merge, and concurrent multithreaded transactions under record-level two-phase locking, validated across 3 autograded milestones.