Skip to content

README.md

Latest commit

 

History

History
347 lines (237 loc) · 13.2 KB

README.md

File metadata and controls

347 lines (237 loc) · 13.2 KB
Raw

CoCalc-AI

CoCalc-AI is a complete rewrite of CoCalc. This repository is the codebase for the next generation of CoCalc products:

  • CoCalc Plus: a local, single-user CoCalc runtime
  • CoCalc Launchpad: a multi-user CoCalc with project hosts
  • CoCalc CLI: the operator and agent CLI used to drive projects, browsers, workspaces, and automation
  • CoCalc Rocket: the multi-bay scalable architecture built on the same core ideas as Launchpad

The relaunch is AI-native: Codex is integrated as a first-class agent runtime, browser automation is built into the platform, and the same project/workspace concepts are intended to work across local, self-hosted, and large-scale deployments.

Short illustrated guides for using CoCalc-AI are published at sagemathinc.github.io/cocalc-guides.

flowchart TD
    A["CoCalc-AI<br/>one codebase"]:::core

    A --> P["CoCalc Plus<br/>local, single-user"]
    A --> L["CoCalc Launchpad<br/>multi-user, project-host"]
    A --> C["CoCalc CLI<br/>browser + ops + automation"]
    A --> R["CoCalc Rocket<br/>multi-bay scale-out"]

    P --> S["Workspaces"]
    L --> S
    C --> S
    R --> S

    P --> G["Codex agents"]
    L --> G
    C --> G
    R --> G

    P --> T["Notebooks • Files • Terminals • Apps"]
    L --> T
    C --> T
    R --> T

    classDef core fill:#1f2937,color:#fff,stroke:#111827,stroke-width:2px;
Loading

What CoCalc-AI Is

At a product level, CoCalc-AI combines:

  • computational documents and notebooks
  • files, terminals, app servers, and browser-based IDE workflows
  • real-time collaboration
  • project-scoped agent workflows
  • durable project infrastructure: root filesystems, backups, snapshots, and project and host placement

Compared to the older CoCalc architecture, the important shift is this:

  • Plus is the fast local single-user path
  • Launchpad is the current multi-user project-host architecture
  • Rocket is the scale-out target
  • Codex and agent tooling are core part of the platform, not an add-on

Product Surface

CoCalc Plus

Single-user, local-first CoCalc. This is the lightweight path for running CoCalc on one machine without the full multi-host control plane.

CoCalc Launchpad

The current multi-user direction. A hub manages auth, routing, and orchestration while project-hosts run project workloads and own project storage, proxies, backups, and runtime state.

CoCalc CLI

The CLI is increasingly important. It is not just an admin tool; it also provides browser automation, workspace inspection, notebook execution helpers, and agent-facing control surfaces.

CoCalc Rocket

Rocket is the scalable form of Launchpad: many bays, many project hosts, and the same basic architecture extended to much larger deployments.

Naming Map

The repo has both product names and package/runtime names. The important mapping is:

  • lite <-> CoCalc Plus
  • hub <-> CoCalc Launchpad control plane
  • hub + multibay/project-host deployment <-> CoCalc Rocket

In concrete terms:

  • src/packages/lite is the local single-user runtime used by Plus-style development.
  • src/packages/hub is the multi-user control-plane server used by Launchpad-style development.
  • src/packages/project-host and related runtime packages are part of the Launchpad/Rocket host architecture, not the Lite runtime.

Current Architecture In One Page

The current architecture centers on:

  • a TypeScript-heavy pnpm monorepo under src/packages
  • Conat for typed RPC and streaming between frontend, hub, hosts, and project services
  • project-hosts that combine file server, runtime, proxying, quotas, snapshots, and backups
  • local Lite/Plus mode for fast single-user workflows
  • Codex app-server integration through ACP-style request/stream transport
  • browser automation and workspace-aware tooling as first-class capabilities

Useful architecture docs:

Also note that src/.agents/ contains many working design docs and implementation plans. Those files are valuable, but some describe target state or active rollout work rather than fully shipped behavior.

CoCalc-AI And Kubernetes

CoCalc-AI borrows some useful operational ideas from Kubernetes, but it is not a Kubernetes-native application and does not run on Kubernetes.

The similarities are conceptual:

  • CoCalc has a control plane that authorizes users, routes requests, schedules work, tracks health, and records operational state.
  • Project hosts are analogous to worker nodes: they run user workloads, own local runtime state, report heartbeats, expose metrics, and need safe rollout/repair workflows.
  • Operators need Kubernetes-like primitives: describe an object, view bounded logs, inspect events, see resource pressure, roll out software, retry failed long-running operations, and drain or repair capacity.
  • The system benefits from declarative desired state, durable operations, health checks, automated reconciliation, and audited operator actions.

The differences are important:

  • CoCalc-AI is built around bays, not one central Kubernetes cluster. A bay is a regional control-plane and database unit. Launchpad is the one-bay case; Rocket is the multi-bay scale-out form.
  • A production deployment is globally distributed across regions and can span multiple clouds. A project host belongs to exactly one bay, but users may connect from anywhere.
  • Users connect directly to project hosts for steady-state project traffic such as files, terminals, Jupyter, app servers, browser sessions, and Codex app-server traffic. The hub/bay control plane authorizes and routes, but it should not proxy normal project data-plane traffic.
  • Ingress is therefore deliberately distributed. Each project host has its own public routing/tunnel surface instead of all project traffic entering through one central ingress controller.
  • Runtime management uses a mix of systemd services, custom CoCalc services, Conat RPC, host-local SQLite, Postgres, cloud-provider APIs, Cloudflare tunnels, btrfs, rootless Podman, and project-specific runtime processes.
  • CoCalc-AI does not primarily deploy core services as OCI images. Most CoCalc components are built as ncc JavaScript bundles and installed/rolled out as software artifacts. OCI/rootfs/container tooling is used for project runtimes and related isolation, but it is not the main packaging model for the control plane or project-host daemon stack.
  • Project hosts run customer workloads and customer-facing services. Operational tooling must account for user-visible sessions, project storage, backups, snapshots, SSH, app servers, and long-lived compute processes.
  • The normal deployment target for a bay is single VM/systemd friendly. Kubernetes can be a future packaging or operations target for some environments, but correctness should not depend on Kubernetes APIs or cluster-specific service discovery.

We use Kubernetes as inspiration for operator ergonomics, not as an implementation substrate. CoCalc-AI exposes describe, logs, events, top, rollout, reconcile, and drain-style operations through audited CoCalc APIs and the cocalc CLI, whether the underlying deployment is a single VM, a VM fleet, or a partially Kubernetes-backed installation.

Repository Layout

Top-level:

  • docs/ - architecture notes, operational docs, and implementation references
  • src/ - the actual application monorepo
  • AGENTS.md - repo-specific guidance for coding agents and contributors

Inside src/:

  • package.json - the main build, test, Lite, and local dev scripts
  • workspaces.py - the workspace build/install helper used across packages
  • packages/ - the monorepo packages
  • scripts/dev/ - Lite and hub local dev helpers, smoke scripts, and local dev tooling
  • python/ - the Python API client and related Python build surface

Notable package areas:src/packages/frontend - main browser UI

  • src/packages/conat - RPC, persistence, and routing primitives

  • src/packages/lite - local single-user runtime

  • src/packages/hub - hub/control-plane server

  • src/packages/cli - cocalc CLI

  • src/packages/ai - agent and Codex integration

  • src/packages/file-server - project storage, quotas, snapshots, and backup plumbing

  • src/packages/cloud / src/packages/launchpad - cloud and Launchpad-specific functionality

Building From Source

If you are working on the codebase itself, almost everything starts in src/.

Prerequisites

  • Node.js 22+
  • a recent pnpm
  • Python 3
  • make for the Python API build

For full Launchpad / project-host work you will also want a Linux environment with the host/runtime tooling used by that stack. For general frontend, Lite, CLI, and many agent workflows, the local Lite path is enough.

First Build

git clone https://github.com/sagemathinc/cocalc-ai.git
cd cocalc-ai/src
pnpm build

That command installs package dependencies and builds the development bundles across the monorepo.

Running CoCalc-AI Locally

Lite / CoCalc Plus Style Development

This is the fastest way to get a local server running.

cd src
pnpm dev:lite:init
pnpm dev:lite:start
pnpm dev:lite:status

To load the matching environment in your current shell:

cd src
eval "$(pnpm -s dev:lite:env)"

That prints and exports the current Lite API URL, browser target, auth context, and helper paths. It is the recommended starting point for browser automation and local bug reproduction.

Hub / Launchpad Development

For the fuller control-plane path:

cd src
pnpm dev:hub:init
pnpm dev:hub:start
pnpm dev:hub:status

And load the corresponding shell environment:

cd src
eval "$(pnpm -s dev:hub:env)"

That environment matters for CLI commands, browser automation, host operations, and any live Launchpad control-plane testing.

Useful Development Commands

Run these from src/ unless stated otherwise.

Core Build And Validation

pnpm build:dev   # debug frontend - uses a lot more browser memory but better for some development
pnpm tsc
pnpm lint
pnpm version-check
pnpm test

Package-Scoped Work

cd src/packages/<package>
pnpm tsc --build
pnpm build

Formatting

pnpm prettier --write <file>

Lite / Browser Validation

pnpm dev:lite:status
pnpm lite:test:e2e
pnpm lite:test:e2e:headed

Smoke / Ops Tooling

pnpm smoke:self-host
pnpm smoke:cloud-host
pnpm smoke:codex-launchpad

Recommended Docs To Read First

If you are new to this repository, start here:

  1. docs/overview.md
    Fast entry point to the newer subsystem docs.
  2. docs/architecture.md
    Current project-host architecture.
  3. docs/agents.md
    How Codex/ACP fits into CoCalc today.
  4. docs/api.md
    Browser automation and agentic browser API.
  5. docs/browser-debugging.md
    How to debug real browser behavior when tests are not enough.
  6. docs/launchpad.md and docs/self-host.md
    Launchpad and self-hosted deployment direction.

Then browse src/.agents/ for deeper design notes on the specific subsystem you are touching.

What Has Changed From Older CoCalc

The old top-level description of CoCalc as a single hosted collaboration app is no longer enough.

This repo now includes:

  • local single-user runtime work
  • multi-user project-host orchestration
  • agent and Codex infrastructure
  • browser automation APIs
  • workspaces, app servers, and portability tooling
  • the early architecture for the Rocket scale-out path

So the right mental model is:

  • this is a product-family repository
  • src/ is the real monorepo
  • Lite and Launchpad are both first-class
  • many docs in docs/ and src/.agents/ are newer and more accurate than older public-facing descriptions

License And Commercial Use

This repository is source-available under the Microsoft Reference Source License. See LICENSE.md.

Important implication: this is not an ordinary permissive open source license. Read the license carefully before building, running, redistributing, or hosting CoCalc outside of an authorized context.

If you need:

  • a licensed self-hosted deployment
  • commercial support
  • permission to evaluate or deploy CoCalc in your organization

contact SageMath, Inc. through the commercial CoCalc channels:

Project Links

Acknowledgements

CoCalc has been developed over many years by SageMath, Inc. and a long list of contributors. See AUTHORS.md and the contributor history in GitHub for the broader picture.