Notes from Architecture of Maria Storage Engine

Here is the quick notes from the session “Architecture of Maria” from Monty Widenius, one of my all time favorite developer and founder of MySQL.

• Goals
  • To create ACID complaint and Multi-version concurrency control (MVCC)  transactional storage engine for MySQL
  • To replace existing replacement for MyISAM and if possible the default transactional database
  • Maria can run in both transactional and non-transactional, so its easy to replace existing transactional and non-transactional engines
  • To create a storage engine which is good for data warehousing (DW) purposes
• Why Falcon and Maria
  • two different architectures
  • different in feature, compatibility, performance
  • Target is for data warehousing
• Project plan
  • 5.1 + Maria release
    • 1.0 – crash-safe, released in jan, 2008
    • 1.5 – concurrent insert/select, released in apr-may 2008
  • 2008 it will be merged with 6.0
  • 6.0 + Maria
    • 2.0 – transactional and ACID (Q4-2008)
    • 3.0 – high concurrency & online backup (Q1-2009)
    • 3.0 – GA by Q2-2009
  • 6.1 + Maria
    • 4.0 – Data warehousing Q3/2009
• Technical goals
  • MVCC
  • ACID
  • MyISAM replacement
  • Internal temporary tables replacement
  • All Indexes should have equal speed
  • Allow any length transactions to work
  • Log shipping (incremental backups, just by coping the logs)
  • Can be used as standalone library
  • copying of tables b/w different Maria servers
  • Better blob handling
  • Select count ( * )  and table checksums
  • Efficient storage on disk
  • Small footprint
  • behavior of transactional and non-transactional is dynamic
• Maria 1.0 is basically a crash-safe, non-transactional version of MyISAM with cacheable row format
• Primary goal is to make Maria of good quality
• Comparison to MyISAM
  • Mostly everything is compatible
  • repair, compression of rows
  • different row formats (fixed size, dynamic, compressed + pages)
  • fast count ( * )  and checksum
  • different index compress format
  • after a clean shutdown, one can copy logs and build a new server
• Advantages
  • Data, index and statistics are crash-safe
  • Can replay everything from log
  • include create/drop/rename/truncate tables
  • backup of Maria by just coping logs
  • Load index can skip index blocks for not wanted indexes
  • It has unit tests that we missed it for MyISAM
  • support both crash-safe, transactional
  • non-txn will use less space
  • supports all MyISAM row formats +PAGE format
• Differences to MyISAM
  • uses big log files (1G by default)
  • has log control file
  • Uses 8K pages by default (MyISAM 1K)
• Disadvantages to MyISAM
  • repair with many threads not yet
• Will not be fixed
  • No external locking
  • Maria has one page size for both index and data
  • Maria has only one page cache
  • Index number + checksums requires 5 extra bytes per page
  • It does not support RAID
  • Min data file size for BLOCK format is 16K
  • Storage for very small rows (< 25bytes) is not optimal
• TODO
  • Make indexes and statistics versioned
  • Make fulltext (boolean mode) and R-tree index crash safe
  • Parallel-recovery (rolling feature)
• Well documented from day-1 (architecture, faq, everything), maria@lists.mysql.com
• XDB indexes for Maria-4.0