Snapshots

VortexDB’s snapshot system provides point-in-time backups of your entire database, including both the index topology and stored data.

Overview

A snapshot captures:

Index topology: The structure of your index (graph edges, tree nodes, etc.)
Index metadata: Configuration specific to the index type
Storage data: All vectors and their payloads
Database metadata: Dimensions, timestamps, version info

How It Works

Creating a Snapshot

The snapshot process involves two parallel operations:

Index Serialization

The index serializes its topology and metadata into binary format:

pub trait SerializableIndex {
    fn serialize_topology(&self) -> Result<Vec<u8>>;
    fn serialize_metadata(&self) -> Result<Vec<u8>>;
    fn snapshot(&self) -> Result<IndexSnapshot>;
}

Storage Checkpoint

The storage backend creates a consistent checkpoint:

fn checkpoint_at(&self, path: &Path) -> Result<StorageCheckpoint>;

For RocksDB, this uses the native checkpoint API for efficiency.

Manifest Generation

A manifest file is created with:

Snapshot UUID
Timestamp
Parser version (for compatibility)
SHA-256 checksums of all files

Archive Creation

All files are bundled into a compressed tarball (.tar.gz).

Restoring a Snapshot

Extract Archive

The tarball is extracted to a temporary directory.

Verify Checksums

All file checksums are verified against the manifest.

Version Check

The parser version is checked for compatibility.

Restore Storage

The storage checkpoint is restored first.

Rebuild Index

The index is deserialized from topology and metadata, then populated with vectors from storage.

Index-Specific Serialization

Each index type has its own serialization format:

Flat Index
KD-Tree
HNSW

Topology: List of point IDs in insertion order
Metadata: Minimal (just magic bytes)
Restore: O(n) - simply rebuild the vector list

Snapshot Data Structures

Snapshot Object

pub struct Snapshot {
    pub id: Uuid,                        // Unique identifier
    pub date: SystemTime,                // Creation timestamp
    pub sem_ver: Version,                // Parser version
    pub index_snapshot: IndexSnapshot,   // Index data
    pub storage_snapshot: StorageCheckpoint, // Storage data
    pub dimensions: usize,               // Vector dimensions
}

Index Snapshot

pub struct IndexSnapshot {
    pub index_type: IndexType,  // Flat, KDTree, or HNSW
    pub magic: Magic,           // 4-byte identifier
    pub topology_b: Vec<u8>,    // Serialized graph/tree
    pub metadata_b: Vec<u8>,    // Index-specific config
}

Storage Checkpoint

pub struct StorageCheckpoint {
    pub path: PathBuf,         // Path to checkpoint files
    pub storage_type: StorageType, // InMemory or RocksDB
}

Snapshot Engine

The SnapshotEngine manages snapshot creation and restoration:

Registry Interface

pub trait SnapshotRegistry: Send + Sync {
    // Add a new snapshot to the registry
    fn add_snapshot(&mut self, path: &Path) -> Result<Metadata>;
    
    // List snapshots with pagination
    fn list_snapshots(&mut self, limit: usize, offset: usize) -> Result<SnapshotMetaPage>;
    
    // Get the most recent snapshot
    fn get_latest_snapshot(&mut self) -> Result<Metadata>;
    
    // Get metadata for a specific snapshot
    fn get_metadata(&mut self, id: String) -> Result<Metadata>;
    
    // Remove a snapshot
    fn remove_snapshot(&mut self, id: String) -> Result<Metadata>;
    
    // Load and restore a snapshot
    fn load(&mut self, id: String, path: &Path) -> Result<VectorDbRestore>;
}

Manifest Format

The manifest (manifest.json) contains all metadata needed for restore:

{
  "snapshot_id": "550e8400-e29b-41d4-a716-446655440000",
  "created_at": "2026-03-31T12:00:00Z",
  "parser_version": "1.0.0",
  "dimensions": 384,
  "index_type": "HNSW",
  "files": {
    "index_metadata": {
      "filename": "hnsw-index-meta.bin",
      "checksum": "sha256:abc123..."
    },
    "index_topology": {
      "filename": "hnsw-index-topo.bin", 
      "checksum": "sha256:def456..."
    },
    "storage": {
      "filename": "rocksdb-checkpoint/",
      "checksum": "sha256:789ghi..."
    }
  }
}

Getting Started

Concepts

Snapshots

Snapshots

Overview

How It Works

Creating a Snapshot

Restoring a Snapshot

Index-Specific Serialization

Snapshot Data Structures

Snapshot Object

Index Snapshot

Storage Checkpoint

Snapshot Engine

Registry Interface

Manifest Format

Next Steps

Architecture

Indexers

​Snapshots

​Overview

​How It Works

​Creating a Snapshot

​Restoring a Snapshot

​Index-Specific Serialization

​Snapshot Data Structures

​Snapshot Object

​Index Snapshot

​Storage Checkpoint

​Snapshot Engine

​Registry Interface

​Manifest Format

​Next Steps

Architecture

Indexers

Snapshots

Overview

How It Works

Creating a Snapshot

Restoring a Snapshot

Index-Specific Serialization

Snapshot Data Structures

Snapshot Object

Index Snapshot

Storage Checkpoint

Snapshot Engine

Registry Interface

Manifest Format

Next Steps