What is Sandbox?

This is a env where developers/AI agents can create isolated execution environments, execute commands, and connect to them through APIs/WebSockets

Requirements

1. Sandbox configurable with different CPU, memory, disk, network, and runtime/image; each sandbox must be strongly isolated from other tenants and the host.
2. Clients can execute commands, stream stdout/stderr, inspect process status, upload/download files, and maintain interactive terminal sessions over REST APIs and WebSockets
3. Horizontally scalable sandbox

High Level Design

flowchart LR
    Client["Developer / AI Agent"]

    GLB["Global LB"]

    subgraph Regions["Global Infrastructure"]
        RLB["Regional LB - Region 1"]
        RLB1["Regional LB - Region 2"]
    end

    subgraph AWS["AWS Region / Datacenter"]

        %% =========================
        %% CONTROL PLANE
        %% =========================
        subgraph CP["CONTROL PLANE (Sandbox creation, billing, auth)"]
            direction TB

            API["API Gateway
REST + WebSocket"]
            Auth["Auth + Rate Limiter"]
            SandboxAPI["Sandbox Control Service"]

            Queue["Sandbox Job Queue
Kafka/RabitMQ"]
            Scheduler["Scheduler
(See Below)"]

            DB[("Metadata DB")]
            Cache[("Redis / Session Store")]

            API --> Auth
            Auth --> SandboxAPI

            SandboxAPI --> DB
            SandboxAPI --> Cache
            SandboxAPI --> Queue

            Queue --> Scheduler
            Scheduler --> DB
        end

        %% =========================
        %% DATA PLANE
        %% =========================
        subgraph DP["DATA PLANE "]
            direction TB

            Router["Sandbox / Session Router"]

            subgraph Compute["Sandbox Compute Cluster"]
                direction LR

                Worker1["Worker Node 1
Worker Agent"]
                Worker2["Worker Node 2
Worker Agent"]
                WorkerN["Worker Node N
Worker Agent"]

                Runtime1["Sandbox Runtime 1
Container / MicroVM"]
                Runtime2["Sandbox Runtime 2
Container / MicroVM"]
                RuntimeN["Sandbox Runtime N
Container / MicroVM"]

                Worker1 --> Runtime1
                Worker2 --> Runtime2
                WorkerN --> RuntimeN
            end

            ObjectStore[("Object Storage
Files / Artifacts")]

            Router --> Worker1
            Router --> Worker2
            Router --> WorkerN

            Runtime1 --> ObjectStore
            Runtime2 --> ObjectStore
            RuntimeN --> ObjectStore
        end

    end

    %% =========================
    %% GLOBAL ROUTING
    %% =========================
    Client --> GLB
    GLB --> RLB
    GLB --> RLB1

    RLB --> API

    %% =========================
    %% CONTROL PLANE
    %% =========================
    Scheduler --> Worker1
    Scheduler --> Worker2
    Scheduler --> WorkerN

    %% =========================
    %% DATA PLANE
    %% =========================
    API -. "WebSocket
terminal / streaming I/O" .-> Router
    Router -. "Interactive I/O" .-> Runtime1
    Router -. "Interactive I/O" .-> Runtime2
    Router -. "Interactive I/O" .-> RuntimeN
Component Role
Scheduler Why not place Worker directly at end of Kafka?
Kafka directly gives jobs to workers(consumer groups), does not understand which worker has how much capacity
We can assign capacity to workers and scheduler can understand and route accordingly

                 Sandbox Request
                       │
                       ▼
                 Scheduler
                       │
          ┌────────────┼─────────────┐
          ▼            ▼             ▼
       Worker A     Worker B      Worker C
       2 CPU        16 CPU         8 CPU
       8 GB RAM     64 GB RAM      32 GB RAM
       no GPU       GPU            no GPU

Seqeunce Diagram

Whole architecture is divided into 2 planes

Control plane (Sandbox creation, billing, auth)

Responsible for: create sandbox, delete sandbox, authentication, quotas, scheduling, billing, metadata, lifecycle, worker health

Data Plane (Actual user commands)

Run code, stdin stdout stderr, file operations, terminal sessions, network traffic 

sequenceDiagram
    autonumber

    actor C as Developer / AI Agent

    box Control Plane
        participant API as API Gateway
        participant S as Sandbox Service
        participant Q as Sandbox Job Queue
        participant SCH as Scheduler
        participant DB as Metadata DB
    end

    box Data Plane
        participant W as Worker Agent
        participant R as Sandbox Runtime
        participant P as User Process
    end

    box Data Plane Routing
        participant SR as Sandbox / Session Router
    end

    %% =========================================
    %% CREATE SANDBOX - CONTROL PLANE
    %% =========================================

    Note over C: CREATE SANDBOX - Control Plane

    C->>API: POST /v1/sandboxes
    Note right of C: 
{
image: python:3.12,
cpu: 2,
memory_mb: 4096,
disk_gb: 20,
ttl_seconds: 3600}

    API->>S: Create sandbox request
    S->>DB: Create sandbox record
status = CREATING

    S->>Q: Enqueue CREATE_SANDBOX job

    Note over Q,SCH: Durable job waiting for capacity

    Q->>SCH: Consume sandbox job

    SCH->>DB: Find suitable worker
    DB-->>SCH: Worker Node 17 available

    SCH->>W: Deploy sandbox
sandbox_id + resource requirements

    %% =========================================
    %% SANDBOX CREATION - DATA PLANE
    %% =========================================

    Note over W,R: Sandbox creation - Data Plane

    W->>R: Create isolated runtime
    R->>R: Allocate CPU / RAM / Disk
    R->>R: Boot Container / MicroVM
    R->>R: Start sandbox-agent

    R-->>W: READY
runtime_id + endpoint

    W->>S: Sandbox READY
    S->>DB: Update status = READY
worker_id + runtime_id

    S-->>API: Sandbox details
    API-->>C: 201 Created
sandbox_id + status=READY

    %% =========================================
    %% INTERACTIVE EXECUTION - DATA PLANE
    %% =========================================

    Note over C,SR: EXECUTION - Data Plane

    C->>API: WebSocket /v1/sandboxes/{id}/terminal
    API->>SR: Establish session for sandbox_id

    SR->>DB: Resolve sandbox → worker/runtime
    DB-->>SR: worker-17 / runtime-42

    SR->>R: Establish terminal session
    R-->>SR: Session established
    SR-->>API: WebSocket connected
    API-->>C: WebSocket connected

    %% =========================================
    %% COMMAND / TERMINAL I/O
    %% =========================================

    C->>API: stdin: "python\n"
    API->>SR: Forward stdin
    SR->>R: stdin

    R->>P: Start Python process
    P-->>R: stdout: "Python 3.12..."

    R-->>SR: stdout
    SR-->>API: stdout
    API-->>C: Stream stdout

    C->>API: stdin: "print(2 + 3)\n"
    API->>SR: Forward stdin
    SR->>R: stdin
    R->>P: Execute command

    P-->>R: stdout: "5\n"
    R-->>SR: stdout
    SR-->>API: stdout
    API-->>C: Stream output

    %% =========================================
    %% CLOSE SESSION
    %% =========================================

    C->>API: Close WebSocket
    API->>SR: Close terminal session
    SR->>R: Close PTY/session
    R-->>SR: Session closed

    %% =========================================
    %% DELETE SANDBOX - CONTROL PLANE
    %% =========================================

    Note over C,API: DELETE SANDBOX - Control Plane

    C->>API: DELETE /v1/sandboxes/{id}
    API->>S: Delete sandbox request

    S->>DB: status = TERMINATING
    S->>Q: Enqueue DELETE_SANDBOX job

    Q->>SCH: Consume cleanup job
    SCH->>W: Destroy sandbox

    %% =========================================
    %% DESTROY - DATA PLANE
    %% =========================================

    W->>R: Destroy runtime
    R->>P: Terminate processes
    R-->>W: Resources released

    W->>S: Sandbox destroyed
    S->>DB: status = TERMINATED

    S-->>API: Cleanup complete
    API-->>C: 204 No Content