SSDiskDB - Persistent Disk Database for Node.js

What is SSDiskDB?

SSDiskDB is an open-source, persistent disk-based database for Node.js. It is a lightweight, embedded key-value store built from scratch for the JavaScript community as a disk-backed alternative to in-memory caches like Redis. In large-scale production environments, caching billions of records in memory becomes extremely expensive due to RAM pricing constraints.

By utilizing Google's LevelDB storage library, SSDiskDB stores data directly on disk (SSDs or persistent volumes) while maintaining an internal cache for fast read access, enabling near-memory speeds for hot keys while saving database infrastructure costs.

SSDiskDB is a local Node.js database that runs in-process — it does not require installing an external SSDB server process on your machine. It provides a Redis-like API with String, Hash, and Sorted Set data structures, optional client-side encryption, and persistent storage that survives process restarts.

Motivation & Inspiration

The creation of SSDiskDB is inspired by SSDB and tech-industry pioneers like Zerodha (India's largest stock broker), who document their use of disk-backed databases in their Zerodha Tech Stack.

How the Author Discovered Disk-Backed Caching from Zerodha

During research into cost-efficient, high-volume caching architectures, the project's creator, Manoj Gowda, came across the public engineering disclosures of Zerodha. In their technical stack documentation on zerodha.tech, Zerodha's engineering team detailed how they host SSDB (an open-source, disk-backed NoSQL database utilizing Google's LevelDB engine) as a key-value cache.

In massive production environments, storing billions of keys in memory-only databases like Redis becomes prohibitively expensive due to RAM costs. By reading Zerodha's technical posts, Manoj learned how they leveraged SSDB to write records directly to SSDs while keeping a highly optimized memory cache for hot data. This hybrid architecture allowed them to achieve near-Redis latencies at a fraction of the cost, saving massive amounts of RAM and scaling cost-effectively.

This discovery inspired Manoj to build SSDiskDB from scratch specifically for the JavaScript & Node.js community. Instead of running a separate C++ SSDB daemon process, SSDiskDB implements these identical design principles inside an embedded Node.js library—bundling Google's LevelDB storage engine with a Redis-like API, connection pooling, security access whitelisting, and secure remote proxies.

Architecture Comparison

Here is how SSDiskDB compares to standard databases:

How SSDiskDB builds on top of LevelDB

LevelDB is a primitive raw byte key-value engine. SSDiskDB abstracts this engine by providing structure, security, and networking:

- **Types Namespacing**: Maps high-level data types into flat keyspaces by appending type tags (e.g. s: for Strings, h: for Hashes, z: for Sorted Sets).
- **Envelope Encryption**: Generates a cryptographically strong initialization vector (IV) per entry, encrypts values using AES-256-CBC locally, and stores raw encrypted hex strings, keeping database hosts blind to client payloads.

Getting Started

SSDiskDB can be installed in two official ways:

Option 1: Native Installation (Linux/macOS)

One-command installer that sets up Node.js (if needed), installs SSDiskDB globally, and makes the ssdiskdb CLI available:

curl -fsSL https://ssdiskdb.js.org/install.sh | bash

After installation:

ssdiskb start

Dashboard: http://localhost:8971

Option 2: Docker Compose (Cross-platform)

Build the image locally from the GitHub repository—no Docker Hub pull required—and start with persistent storage:

git clone https://github.com/manojgowdain/ssdiskdb.git
cd ssdiskdb
docker compose up -d --build

Dashboard: http://localhost:8971

Note: SSDiskDB's Docker installation builds the image locally from the GitHub repository. It does not require Docker Hub.

Option 3: One-Line Docker Install (Linux/macOS)

If you prefer a single command that sets up the repository and starts Docker Compose:

curl -fsSL https://ssdiskdb.js.org/docker-install.sh | bash

NPM Package (for embedding in your Node.js application)

Install the library to use it programmatically in your own code:

npm install @manojgowdain/ssdiskdb

Then use npx ssdiskdb start or import it in your code. See Connection Options.

Connection Options

1. Embedded Local Database

Connect natively to a LevelDB instance inside your Node application:

const { connect } = require("@manojgowdain/ssdiskdb");

(async () => {
  const db = await connect({
    storagePath: "./my-custom-db",
    encryptionKey: "supersecretkey", // Enables transparent AES-256-CBC encryption
    startDashboard: true,            // Spawns Web Insights Dashboard
    dashboardPort: 8971
  });

  await db.set("welcome_message", "Hello World!");
  console.log(await db.get("welcome_message"));

  await db.close();
})();

2. Single URI Scheme Configurations

Connect programmatically using connection URIs:

// Standard plaintext connection
const db = await connect("ssdiskdb://ssdb_key123@127.0.0.1:8971/server-a");

// Secure connection with client-side envelope encryption key
const dbSecure = await connect("ssdiskdb+encry://ssdb_key123@127.0.0.1:8971/server-a?key=aes-key");

CLI Commands

SSDiskDB features a complete command-line interface to manage active servers, database nodes, credentials, and configurations:

# Start the local database engine and open the dashboard
npx ssdiskdb start --port 8971 --path ./my-db-dir

# Start a remote client directly pointing to a central cloud server
npx ssdiskdb start ssdiskdb://ssdb_key@localhost:8971/server-a

# Configure Admin user credentials
npx ssdiskdb credentials --username myuser --password mypassword

# Register a whitelisted server client
npx ssdiskdb server add web-server-1

# List all allowed client servers and active API keys
npx ssdiskdb server list

Environment Variables

The following environment variables can configure SSDiskDB without modifying code:

Native Example

SSDISKDB_PORT=9000 SSDISKDB_DATA_DIR=/data/db ssdiskdb start

Docker Example

docker run -d -p 8971:8971 \
  -e SSDISKDB_PORT=8971 \
  -e SSDISKDB_DATA_DIR=/data/ssdb-local-db \
  -e SSDISKDB_USERNAME=admin \
  -e SSDISKDB_PASSWORD=secret \
  -v ./ssdb-local-db:/data/ssdb-local-db \
  ssdiskdb

Cloud Deployment & Local Dashboard Setup

For secure production deployments, you should run your central database headlessly on the cloud (behind a VPC) and use a local dashboard instance to manage its assets securely via reverse proxy connection strings.

Step-by-Step Walkthrough

Step 1: Deploy SSDiskDB on your cloud VPS instance (using Docker Compose):

git clone https://github.com/manojgowdain/ssdiskdb.git
cd ssdiskdb
docker compose up -d --build

Step 2: Register a console profile on the cloud instance:

docker exec -it ssdb node dist/cjs/cli.js server add my-workstation

This generates an API Key. For example: ssdb_c4dee067d4a23dd35da3270ddd5b2cc5.

Step 3: Copy the remote connection string:

ssdiskdb://ssdb_c4dee067d4a23dd35da3270ddd5b2cc5@<YOUR_CLOUD_IP>:8971/my-workstation

Step 4: Launch local dashboard on your workstation to proxy cloud data:

1. Start a local server: npx ssdiskdb start --port 8971

2. Open http://localhost:8971 in your web browser.

3. Select the Remote Connection tab.

4. Input the copied URI from Step 3 and click **Sign In**.

The local dashboard now serves as a secure proxy console to view and modify your cloud database records safely without exposing admin screens publicly.

Available APIs

Here are the core methods exposed by the client interface:

Strings

await db.set("key", { user: "Manoj" });
await db.get("key"); // Returns { user: "Manoj" }
await db.exists("key"); // Returns true
await db.incr("counter", 5); // Increments by 5
await db.del("key");

Hashes (Maps)

await db.hset("user:100", "role", "admin");
await db.hget("user:100", "role"); // Returns "admin"
await db.hdel("user:100", "role");

Sorted Sets (ZSets)

await db.zset("highscores", "alex", 99.5);
await db.zget("highscores", "alex"); // Returns 99.5
await db.zdel("highscores", "alex");

TTL / Expiration

TTL (Time-To-Live) controls how long a key is available before SSDiskDB removes it automatically. TTL is expressed in milliseconds and is passed as the ttl option to db.set() (and to hset, zset, mset, and batch).

const { connect } = require("@manojgowdain/ssdiskdb");

(async () => {
  const db = await connect();

  // `ttl` is an option to db.set(). It is specified in milliseconds.
  // `5 * 1000` means 5 seconds.
  await db.set("welcome_message", "Hello World!", {
    ttl: 5 * 1000
  });

  console.log(await db.get("welcome_message")); // "Hello World!" (while alive)

  await db.close();
})();

How expiration works

• The absolute expiration timestamp (Date.now() + ttl) is stored together with the record, so a read can decide from its own primary lookup whether the key has expired.
• TTL is optional. If ttl is not provided, the key follows the normal default persistence behavior of SSDiskDB: it stays available until deleted.
• Reading an expired key returns undefined from get(), false from exists(), and -2 from ttl(). The expired record is removed on the read path (lazy expiration).
• Expiration is handled automatically. A single background worker walks the ordered ttl/<19-digit-expiresAt>/<fullDataKey> index in bounded batches and deletes records whose expiration has passed. It uses no per-key timers, runs at a fixed interval, and stops during close(). Expired records are also removed lazily by reads, so a key is never returned after its TTL expires.
• TTL data survives process restarts: the expiration timestamp and the cleanup index are persisted on disk.

TTL examples

const { connect } = require("@manojgowdain/ssdiskdb");

(async () => {
  const db = await connect();

  // 5 seconds
  await db.set("temp:5s", "value", { ttl: 5 * 1000 });

  // 1 minute
  await db.set("session", { userId: 123, authenticated: true }, { ttl: 60 * 1000 });

  // 1 hour
  await db.set("token", "abc123", { ttl: 60 * 60 * 1000 });

  // No TTL: persistent until deleted
  await db.set("config", { theme: "dark" });

  console.log(await db.get("session")); // { userId: 123, authenticated: true }

  // After 60 seconds the key has expired:
  console.log(await db.get("session")); // undefined

  await db.close();
})();

Checking a key before and after expiration

const { connect } = require("@manojgowdain/ssdiskdb");

(async () => {
  const db = await connect();

  await db.set("short", "value", { ttl: 2 * 1000 }); // 2 seconds

  console.log(await db.get("short"));   // "value"
  console.log(await db.exists("short")); // true
  console.log(await db.ttl("short"));    // >= 0 (remaining milliseconds)

  // Wait past the TTL...
  await new Promise((resolve) => setTimeout(resolve, 2500));

  console.log(await db.get("short"));    // undefined
  console.log(await db.exists("short")); // false
  console.log(await db.ttl("short"));    // -2 (absent or expired)

  await db.close();
})();

TTL reference table

TTL semantics

• ttl(key) === -2: key is absent or expired.
• ttl(key) === -1: key exists without expiration.
• ttl(key) >= 0: remaining TTL in milliseconds.
• ttl: 0: immediately expired; the key is not readable.
• Negative or non-finite values throw a RangeError.
• Replacing a record without a TTL removes its previous expiration.
• Old records without TTL metadata remain persistent and readable.
• set, hset, and zset accept per-record { ttl }.
• expire(key, ttlMs) sets or replaces a String key's TTL; persist(key) removes it. Hash fields and Sorted Set members receive TTL only through their hset/zset options and have no separate public TTL query method.

Configure cleanup on open

const db = await connect({
  storagePath: "./data",
  ttl: {
    cleanupInterval: 1_000, // milliseconds between cleanup passes
    cleanupBatchSize: 500, // maximum index rows processed per pass
  },
});

Best Practices & Production Hardening

1. Volume Mounts: When deploying via Docker, always map your storage volume (e.g. -v ssdb-volume:/data). LevelDB stores tables on-disk; omitting volume tags will result in full data loss when containers restart.

2. Reverse Proxy and SSL: Central database consoles should be placed behind a secure reverse proxy like **Nginx** or **Caddy** to handle SSL termination (HTTPS routing).

3. Single Process Locking: LevelDB places a persistent LOCK file in its storage directory. If a secondary Node process attempts to read/write the same directory locally, it will fail throwing LEVEL_LOCKED. For multi-application access, run a single central server process and connect other apps as remote clients over connection URIs.

4. SSD Storage Selection: Since LevelDB uses Log-Structured Merge (LSM) trees writing SST files on-disk, disk IOPS is the primary performance factor. Deploying SSDiskDB on NVMe SSDs provides outstanding transaction throughput.

References & Resources

Here are the key references and external resources related to SSDiskDB, including inspiration and technical dependencies:

• Manoj Gowda Portfolio - Official website of the author of SSDiskDB.
• Google LevelDB Storage Engine - The high-performance key-value store library that SSDiskDB builds on top of.
• Official SSDB Database - The fast NoSQL database server that inspired the design of this Node.js implementation.
• Zerodha Tech Stack - Official list of tools and technologies self-hosted by Zerodha, showcasing their production use of SSDB.
• Zerodha Tech Blog - Engineering insights, architecture patterns, and posts by Zerodha developers.
• SSDiskDB GitHub Repository - Direct link to the open-source repository for issues, contributions, and documentation.
• SSDiskDB on npm - Scoped Node.js package.
• SSDiskDB on JSR - JSR package and documentation.
• Legacy SSDiskDB NPM Package (ssdiskdb) - The previous unscoped package name.
• Source Code - Build the Docker image locally from the GitHub repository.