ToyDB - A Simple Database Implementation in Go

Overview

ToyDB is an educational database implementation inspired by Let's Build a Simple Database, built in Go as a learning project. This implementation provides a hands-on exploration of database internals, from parsing SQL-like commands to managing data storage with B-trees.

Architecture

Frontend Components

• Tokenizer: Breaks input into meaningful tokens
• Parser: Analyzes token sequences to build abstract syntax trees
• Code Generator: Converts parsed statements into executable bytecode

Backend Components

• Virtual Machine: Executes generated bytecode operations
• B-tree: Efficient index structure for data organization
• Pager: Manages memory pages and disk I/O operations
• OS Interface: Abstracts operating system interactions

Getting Started

Prerequisites

• Docker and Docker Compose

Running ToyDB

# Build the development container
docker-compose build toydb-dev

# Start the database shell
docker-compose run --rm toydb-dev

Supported Operations

INSERT Statement

Add new records to the database:

db > insert 1 user1 user1@gmail.com
Executed.
db > insert 2 user2 user2@gmail.com
Executed.
db > insert 3 user3 user3@gmail.com
Executed.

SELECT Statement

Retrieve all records from the database:

db > select
(1, user1, user1@gmail.com)
(2, user2, user2@gmail.com)
(3, user3, user3@gmail.com)
Executed.

B-tree Inspection

View the internal B-tree structure:

db > .btree
Tree:
- leaf (size 3)
    - key 1
    - key 2
    - key 3

Learning Objectives

This project serves as a practical implementation for understanding:

• Database query parsing and execution
• B-tree data structures for indexing
• Memory management and paging systems
• Virtual machine design for database operations
• File I/O and persistence mechanisms

Development Status

Currently supports basic INSERT and SELECT operations. The project is actively developed as part of the learning journey in Go programming and database internals.

Contributing

This is primarily an educational project, but suggestions and improvements are welcome through GitHub issues or pull requests.

Author

JingHuang Su - LinkedIn

Open to new opportunities!