MomoDB

Work in progress

• A key-value store, created purely for educational purposes.
• Built in C++. This is making use of C++20 features, so you will need a modern compiler to build this.
• Named after Momo (Wikipedia)

Commands

• SET <key> <value>: holds value <value> against key <key>
• GET <key>: returns value
• DEL <key>: deletes a key; essentially marks it as

Current capabilities

• Reading and writing to log files successfully.
• Storage using map.
• Rebuilding map based on log file.
• Added basic TCP server
  • Listens to port 9001 (hardcoded currently)
    • You can find processes using 9001 already using sudo ss -tulnp | grep :9001
  • Currently, can accept commands and responds with OK
  • EXIT command works to shut down the server
  • SET, GET work
  • DEL works by overwriting the value with an empty string.
  • Can be tested locally using nc localhost 9001
• Current performance (based on a toy script acting as a client):

Successfully completed: 10000/10000
Total time: 1.07 seconds
Performance: 9347.80 operations/sec

Branches

• master: Primary branch.
• single-client: Branch for testing single client persistence and performance.
• v0.1: version 0.1 of MomoDB. It closes connections after every request. Not recommended for production use, but kept as a reference.

TODOs

• Performance improvements can be made to a lot of aspects.
• Large scale tests are pending.
• Unit tests are pending.
• Better data structures and writing methods.
• Better network protocol is needed.

Known issues

• JSON values cause issues, since they have a :
  • switching to a newer format is required.
• Server closes TCP connection after every request.
  • This reduces the potential speed it can reach as TCP handshake has to be repeated.
  • Switching to epoll or io_uring is the potential next step.

What I'll look into next

• Improving networking
  • persistent connections and handling multiple clients properly
• Improving log format
  • Currently hitting limits
• Improving internal data structures.
  • map was a good start, but better data structures exist.
  • Switched to using unordered_map currently

Log format

For human readability and easier debugging, logs are plain-text only for now. They will be moved to a binary format once features are more stable.

Currently log format is:

timestamp:operation_type:key:value

Example:

1768059728273357375:1:userid:42

This has clear flaws, for example, keys and values cannot contain : themselves or everything breaks down, but this is a decent enough start for a system.

AI usage

0 or minimal AI code will be involved, as it is more of a learning exercise pursuing newer C++ features and performance.

Pull requests

Currently not accepting any.