IOTA IRI rocksdb data storage structure

This post also available at: https://blog.louie.lu/2017/10/31/iota-iri-rocksdb-data-storage-structure/

0x1 Overview

In IOTA IRI, it uses rocksdb as its database with column family to separate different data stored. Here is the introduction about the IRI rocksdb data storage structure and how to manipulate in Python.

0x2 Column Families

In IOTA rocksdb, just think column families as a sheet to separate data stored. No high technique inside here. Also, need to notice that some of the columns are using merge operation with string append operator in “,”. That is, a value may contain multiple items, and separate with “,”.

Above table is the column families list of IOTA IRI rocksdb:

0x3 Convert between trytes-string and bytes

A big issue between a trytes-string and the database is, in the database, it stores not the “trytes-string” with bytes, but the “trytes-string” convert to bytes. Sounds strange, ugh? (more exactly, is “trits” convert to bytes)

Without any implement details, IOTA using trytes-string as its represent method, and we need to save trytes-string into database, there is a mapping between trits and bytes that can convert trytes-string into bytes.

Given an example in Python, consider the transaction hash (in trytes-string):

UNUK99RCIWLUQ9WMUT9MPQSZCHTUMGN9IWOCOXWMNPICCCQKLLNIIE9UIFGKZLHRI9QAOEQXQJLL99999

its trits will be:

[0, 1, -1, -1, -1, -1, 0, 1, -1, -1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, -1, 0, 1, 0, 0, 0, 1, -1, -1, 0, 0, 1, 1, 0, 1, -1, -1, 0, -1, 0, 0, 0, -1, -1, 0, 1, 1, 1, 0, 1, -1, -1, 1, -1, 0, 0, 0, 1, 1, 1, 1, -1, -1, -1, 0, -1, 1, 0, -1, -1, 0, 0, 0, 1, 0, -1, 0, 1, -1, 1, -1, 0, 1, -1, 1, 1, 1, 1, -1, 1, -1, -1, -1, 0, 0, 0, 0, 0, 1, -1, -1, 0, 0, -1, -1, 0, 1, 0, 0, -1, -1, 0, -1, 0, -1, -1, 0, 1, 1, 1, -1, -1, -1, 1, -1, -1, 0, 0, 1, 0, 1, 0, 0,