FoldDB Atom & AtomRef System

What is the Atom & AtomRef System?

The Atom & AtomRef system provides the foundation for FoldDB's immutable data storage model. This system enables version tracking, immutable data storage, and efficient data access through a combination of Atoms (immutable data containers) and AtomRefs (version pointers).

Key Components

The system consists of three primary components that work together to provide immutable data storage with version control:

How the Atom & AtomRef System Works

Data Flow

When data is created or updated in FoldDB, it follows this sequence:

1. A new Atom is created with content and metadata
2. The Atom is assigned a unique UUID
3. An AtomRef is created to point to the Atom
4. When data is updated, a new Atom is created with a reference to the previous version
5. The AtomRef is updated to point to the new Atom
6. Previous versions remain accessible through the version chain

flowchart LR
    subgraph Creation
        C[Create Request] --> A1[Atom v1]
        A1 --> AR[AtomRef]
    end
    subgraph Update
        U[Update Request] --> A2[Atom v2]
        A2 --> |prev_atom_uuid| A1
        AR --> |update pointer| A2
    end
    subgraph Access
        AR --> |get_atom_uuid| A2
        A2 --> |prev_atom_uuid| A1
    end

Component Architecture

The components interact in this structure:

classDiagram
    class Atom {
        +uuid: String
        +source_schema_name: String
        +source_pub_key: String
        +created_at: DateTime
        +prev_atom_uuid: Option<String>
        +content: Value
        +status: AtomStatus
        +new()
        +with_prev_version()
        +content()
        +uuid()
    }
    class AtomRef {
        +uuid: String
        +atom_uuid: String
        +updated_at: DateTime
        +status: AtomRefStatus
        +update_history: Vec<AtomRefUpdate>
        +new()
        +set_atom_uuid()
        +get_atom_uuid()
    }
    class AtomRefCollection {
        +uuid: String
        +atom_uuids: HashMap<String, String>
        +updated_at: DateTime
        +status: AtomRefStatus
        +update_history: Vec<AtomRefUpdate>
        +new()
        +set_atom_uuid()
        +get_atom_uuid()
        +remove_atom_uuid()
    }
    class AtomRefBehavior {
        <<interface>>
        +uuid()
        +updated_at()
        +status()
        +set_status()
        +update_history()
    }
    AtomRef ..|> AtomRefBehavior
    AtomRefCollection ..|> AtomRefBehavior
    AtomRef --> Atom
    AtomRefCollection --> Atom

Implementation Features

1. Immutable Data Storage

The system stores data with these features:

• Immutable Atoms - Once created, Atom content cannot be modified
• Unique identifiers - Each Atom has a UUID for unique identification
• Metadata tracking - Stores schema name, creator, and timestamp
• Status tracking - Maintains Active or Deleted status
• Content storage - Stores JSON data in the content field

2. Version Control

The system provides version control through:

• Version chaining - Each Atom can reference a previous version
• AtomRef pointers - Point to the latest version of data
• Update history - Tracks all changes to references
• Timestamp recording - Maintains creation and update times
• Status tracking - Records Active or Deleted status

3. Reference Collections

The system supports organizing references through:

• Key-based access - References stored with string keys
• Collection management - Add, update, or remove references
• Shared behavior - Common interface through AtomRefBehavior
• Update tracking - Records all collection changes
• Status management - Maintains collection status

Data Structures

Atom Structure

The Atom structure contains these fields:

• uuid - Unique identifier for this Atom
• source_schema_name - Name of the schema that defines this Atom
• source_pub_key - Public key of the creator
• created_at - Timestamp when this Atom was created
• prev_atom_uuid - Optional reference to previous version
• content - The actual data content
• status - Current status (Active or Deleted)

AtomRef Structure

The AtomRef structure contains these fields:

• uuid - Unique identifier for this reference
• atom_uuid - UUID of the referenced Atom
• updated_at - Timestamp of the last update
• status - Current status (Active or Deleted)
• update_history - Record of all updates with timestamps

AtomRefCollection Structure

The AtomRefCollection structure contains these fields:

• uuid - Unique identifier for this collection
• atom_uuids - Map of keys to Atom UUIDs
• updated_at - Timestamp of the last update
• status - Current status (Active or Deleted)
• update_history - Record of all updates with timestamps

Common Operations

Creating Data

To create new data in the system:

1. Create a new Atom with content and metadata
2. Create an AtomRef pointing to the Atom
3. Store both the Atom and AtomRef

Updating Data

To update existing data in the system:

1. Create a new Atom with updated content
2. Set the prev_atom_uuid to reference the previous version
3. Update the AtomRef to point to the new Atom

Accessing Data

To access data in the system:

1. Retrieve the AtomRef for the desired data
2. Get the atom_uuid from the AtomRef
3. Retrieve the Atom using the atom_uuid
4. Access the content from the Atom

Managing Collections

To manage collections of references:

1. Create an AtomRefCollection
2. Add references using set_atom_uuid with keys
3. Retrieve references using get_atom_uuid with keys
4. Remove references using remove_atom_uuid with keys

Best Practices

Data Management

Follow these practices for data management:

1. Always use AtomRefs to access the latest version of data
2. Create new Atoms for each data change to maintain immutability
3. Link versions properly using prev_atom_uuid
4. Use appropriate status values (Active or Deleted)
5. Maintain proper update history

Version Control

Follow these practices for version control:

1. Always create a new Atom for changes instead of modifying existing ones
2. Update AtomRefs atomically to point to the latest version
3. Use the version chain to access historical versions when needed
4. Track status changes in update history
5. Record the source of each update in the history

Performance

Follow these practices for performance:

1. Use AtomRefs to access the latest version directly
2. Create AtomRefCollections for related data
3. Use appropriate keys in collections for efficient access
4. Implement caching for frequently accessed Atoms
5. Consider version pruning for very long chains