|
| 1 | +# Cuttlefish Driver |
| 2 | + |
| 3 | +`jumpstarter-driver-cuttlefish` manages |
| 4 | +[Android Cuttlefish](https://source.android.com/docs/devices/cuttlefish) |
| 5 | +virtual devices through the |
| 6 | +[Host Orchestrator](https://github.com/google/android-cuttlefish) REST API. |
| 7 | +It provides full CVD (Cuttlefish Virtual Device) lifecycle management through |
| 8 | +standard Jumpstarter interfaces: `VirtualPowerInterface` for on/off/cycle, |
| 9 | +plus cuttlefish-specific operations |
| 10 | +(snapshot, powerwash, restart). |
| 11 | + |
| 12 | +## Installation |
| 13 | + |
| 14 | +```{code-block} console |
| 15 | +:substitutions: |
| 16 | +$ pip3 install --extra-index-url {{index_url}} jumpstarter-driver-cuttlefish |
| 17 | +``` |
| 18 | + |
| 19 | +### Prerequisites |
| 20 | + |
| 21 | +- A running Cuttlefish Host Orchestrator (port 2080 by default) |
| 22 | + |
| 23 | +## Host Setup |
| 24 | + |
| 25 | +A `cvd-images` named volume mounted at `/home/vsoc-01/fetch` persists |
| 26 | +fetched AOSP images across container restarts. Instance state (`/var/tmp/cvd`) |
| 27 | +is deliberately kept ephemeral — restarting the container gives you a clean |
| 28 | +slate with no orphaned instance directories. |
| 29 | + |
| 30 | +```bash |
| 31 | +# 1. Pull the orchestration image |
| 32 | +podman pull us-docker.pkg.dev/android-cuttlefish-artifacts/cuttlefish-orchestration/cuttlefish-orchestration:stable |
| 33 | + |
| 34 | +# 2. Create a named volume for AOSP images |
| 35 | +podman volume create cvd-images |
| 36 | + |
| 37 | +# 3. Start the container |
| 38 | +# --network=host: netsim and rootcanal bind to 127.0.0.1 inside the |
| 39 | +# container, so without host networking they'd be unreachable from |
| 40 | +# outside. Host networking shares the VM's network namespace directly. |
| 41 | +# |
| 42 | +# Security note: --privileged + --network=host gives the container full |
| 43 | +# access to the VM's network stack. HO, netsim, and rootcanal have no |
| 44 | +# auth — only deploy on dedicated, non-public hosts. |
| 45 | +podman run -d \ |
| 46 | + --name cuttlefish-orchestrator \ |
| 47 | + --restart=always \ |
| 48 | + --privileged \ |
| 49 | + --network=host \ |
| 50 | + -v cvd-images:/home/vsoc-01/fetch:Z \ |
| 51 | + -v /opt/cuttlefish:/opt/cuttlefish:Z \ |
| 52 | + us-docker.pkg.dev/android-cuttlefish-artifacts/cuttlefish-orchestration/cuttlefish-orchestration:stable |
| 53 | + |
| 54 | +# 4. Fix permissions |
| 55 | +podman exec cuttlefish-orchestrator chown -R httpcvd:httpcvd /home/vsoc-01/fetch |
| 56 | + |
| 57 | +# 5. Fetch AOSP images (one-time, ~2 minutes) |
| 58 | +podman exec cuttlefish-orchestrator cvd fetch \ |
| 59 | + --default_build=aosp-android-latest-release/aosp_cf_x86_64_auto-userdebug \ |
| 60 | + --target_directory=/home/vsoc-01/fetch |
| 61 | + |
| 62 | +# 6. Verify |
| 63 | +curl -s http://localhost:2080/_debug/statusz # should return 200 |
| 64 | +curl -s http://localhost:2080/cvds # should return {"cvds":[]} |
| 65 | +``` |
| 66 | + |
| 67 | +### Ports |
| 68 | + |
| 69 | +After a CVD boots, the following ports are available on the host. |
| 70 | +All per-instance ports use the same offset: `base + instance_num - 1`. |
| 71 | + |
| 72 | +| Service | Base port | Instance 1 | Instance 2 | |
| 73 | +|---------|-----------|------------|------------| |
| 74 | +| Host Orchestrator | 2080 | 2080 (fixed) | 2080 (fixed) | |
| 75 | +| ADB | 6520 | 6520 | 6521 | |
| 76 | +| Netsim REST | 7681 | 7681 | 7682 | |
| 77 | +| Rootcanal HCI | 7300 | 7300 | 7301 | |
| 78 | + |
| 79 | +When using `instance_num > 1`, update the netsim `port` and bt_peer |
| 80 | +`hci_port` in the exporter config to match. |
| 81 | + |
| 82 | +### SSH tunnel (local development only) |
| 83 | + |
| 84 | +For local development, when running the exporter on your workstation |
| 85 | +instead of as a pod, tunnel ports from the VM. This works because |
| 86 | +`--network=host` places netsim and rootcanal on the VM's loopback - |
| 87 | +the tunnel's `localhost` target reaches them directly. |
| 88 | + |
| 89 | +```bash |
| 90 | +ssh -L 2080:localhost:2080 \ |
| 91 | + -L 6520:localhost:6520 \ |
| 92 | + -L 7681:localhost:7681 \ |
| 93 | + -L 7300:localhost:7300 \ |
| 94 | + fedora@<vm-ip> -p 22000 -N |
| 95 | +``` |
| 96 | + |
| 97 | +In production, the exporter runs as a pod and the `host` config |
| 98 | +points to the cuttlefish VM's address directly - no tunnel needed. |
| 99 | + |
| 100 | +### Resetting stale state |
| 101 | + |
| 102 | +If CVDs get stuck or orphaned, clear stale state inside the container: |
| 103 | + |
| 104 | +```bash |
| 105 | +podman exec cuttlefish-orchestrator bash -c ' |
| 106 | + rm -rf /var/tmp/cvd/[0-9]* /var/tmp/cvd/lock/* /tmp/cf_avd_* /tmp/vsock_* |
| 107 | + chown -R httpcvd:httpcvd /var/tmp/cvd/ |
| 108 | +' |
| 109 | +``` |
| 110 | + |
| 111 | +Or restart the container - ephemeral `/var/tmp/cvd` means a restart is |
| 112 | +equivalent to a full reset. Fetched images in the `cvd-images` volume |
| 113 | +are preserved. |
| 114 | + |
| 115 | +### Teardown |
| 116 | + |
| 117 | +Delete CVDs and snapshots when done to avoid accumulation: |
| 118 | + |
| 119 | +```bash |
| 120 | +j power off --destroy # deletes the CVD |
| 121 | +j cuttlefish snapshot delete <id> # remove specific snapshots |
| 122 | +``` |
| 123 | + |
| 124 | +## Configuration |
| 125 | + |
| 126 | +Example exporter configuration: |
| 127 | + |
| 128 | +```yaml |
| 129 | +export: |
| 130 | + cuttlefish: |
| 131 | + type: jumpstarter_driver_cuttlefish.driver.Cuttlefish |
| 132 | + config: |
| 133 | + host: localhost |
| 134 | + port: 2080 |
| 135 | + instance_num: 1 |
| 136 | + env_config: |
| 137 | + instances: |
| 138 | + - disk: |
| 139 | + default_build: /home/vsoc-01/fetch |
| 140 | + vm: |
| 141 | + enable_virtiofs: false # required for snapshot support |
| 142 | + common: |
| 143 | + host_package: /home/vsoc-01/fetch |
| 144 | + gpu_mode: guest_swiftshader # required for snapshot support |
| 145 | + netsim: |
| 146 | + type: jumpstarter_driver_netsim.driver.Netsim |
| 147 | + config: |
| 148 | + host: localhost |
| 149 | + port: 7681 # 7681 + instance_num - 1 |
| 150 | + bt_peer: |
| 151 | + type: jumpstarter_driver_bt_peer.driver.BtPeer |
| 152 | + config: |
| 153 | + hci_host: 127.0.0.1 |
| 154 | + hci_port: 7300 # 7300 + instance_num - 1 |
| 155 | + power: |
| 156 | + ref: cuttlefish.power |
| 157 | + adb: |
| 158 | + ref: cuttlefish.adb |
| 159 | +``` |
| 160 | +
|
| 161 | +### Configuration Parameters |
| 162 | +
|
| 163 | +| Parameter | Description | Type | Required | Default | |
| 164 | +| --------------- | ----------------------------------- | ---- | -------- | ----------- | |
| 165 | +| host | Host Orchestrator hostname | str | no | "localhost" | |
| 166 | +| port | Host Orchestrator HTTP port | int | no | 2080 | |
| 167 | +| group | CVD group name passed to `cvd load`. HO auto-assigns a different group name (e.g. `cvd_1`); the driver tracks the assigned name internally. | str | no | "cvd" | |
| 168 | +| name | CVD instance name within the group | str | no | "1" | |
| 169 | +| instance_num | CVD instance number (determines ADB/netsim/HCI ports). Must match HO's assigned slot. Pinning avoids drift (see `env_config` example). | int | no | 1 | |
| 170 | +| adb_server_port | ADB server port on the exporter | int | no | 15037 | |
| 171 | +| boot_timeout | Seconds to wait for boot on power on| int | no | 300 | |
| 172 | +| env_config | Default env_config for CVD creation | dict | no | {} | |
| 173 | + |
| 174 | +This is a **composite driver** with three children: |
| 175 | +- **power** — `VirtualPowerInterface`: `j power on`, `j power off [--destroy]`, `j power cycle` |
| 176 | +- **storage** — `FlasherInterface`: not yet implemented (planned: HO artifact upload API) |
| 177 | +- **adb** — ADB server for device communication |
| 178 | + |
| 179 | +The exporter config also typically includes sibling drivers: |
| 180 | +- **netsim** (`jumpstarter-driver-netsim`) — virtual radio control (BLE, WiFi, UWB) via netsim REST API |
| 181 | +- **bt_peer** (`jumpstarter-driver-bt-peer`) — Bluetooth peer device via bumble + rootcanal HCI |
| 182 | + |
| 183 | +Use `ref:` entries in the exporter config to expose children at the top level. |
| 184 | + |
| 185 | +## Usage |
| 186 | + |
| 187 | +### CLI |
| 188 | + |
| 189 | +```bash |
| 190 | +# Power on (creates CVD if none exists, starts if stopped) |
| 191 | +j power on |
| 192 | +
|
| 193 | +# Power off (stops CVD, keeps state) |
| 194 | +j power off |
| 195 | +
|
| 196 | +# Power off and delete CVD entirely |
| 197 | +j power off --destroy |
| 198 | +
|
| 199 | +# Power cycle |
| 200 | +j power cycle |
| 201 | +
|
| 202 | +# Health check |
| 203 | +j cuttlefish status |
| 204 | +
|
| 205 | +# List all CVDs |
| 206 | +j cuttlefish list |
| 207 | +
|
| 208 | +# Get this CVD's details |
| 209 | +j cuttlefish get |
| 210 | +
|
| 211 | +# Restart the CVD |
| 212 | +j cuttlefish restart |
| 213 | +
|
| 214 | +# Factory reset |
| 215 | +j cuttlefish powerwash |
| 216 | +
|
| 217 | +# Simulate power button press |
| 218 | +j cuttlefish powerbtn |
| 219 | +
|
| 220 | +# List running operations |
| 221 | +j cuttlefish ops |
| 222 | +
|
| 223 | +# Snapshot management |
| 224 | +# Requires: x86_64 host, enable_virtiofs: false, gpu_mode: guest_swiftshader |
| 225 | +j cuttlefish snapshot create --id my-snapshot |
| 226 | +j cuttlefish snapshot delete <snapshot_id> |
| 227 | +``` |
| 228 | + |
| 229 | +### Python API |
| 230 | + |
| 231 | +```python |
| 232 | +from jumpstarter.common.utils import serve |
| 233 | +from jumpstarter_driver_cuttlefish.driver import Cuttlefish |
| 234 | +
|
| 235 | +driver = Cuttlefish( |
| 236 | + host="localhost", |
| 237 | + port=2080, |
| 238 | + env_config={ |
| 239 | + "instances": [{"disk": {"default_build": "/home/vsoc-01/fetch"}}], |
| 240 | + "common": {"host_package": "/home/vsoc-01/fetch"}, |
| 241 | + }, |
| 242 | +) |
| 243 | +with serve(driver) as client: |
| 244 | + # Check Host Orchestrator is reachable |
| 245 | + print(client.status()) # "OK" |
| 246 | +
|
| 247 | + # Power on (creates CVD from env_config) |
| 248 | + client.power.on() |
| 249 | +
|
| 250 | + # List CVDs |
| 251 | + cvds = client.list_cvds() |
| 252 | + print(cvds) |
| 253 | +
|
| 254 | + # Snapshots |
| 255 | + client.create_snapshot(snapshot_id="baseline") |
| 256 | +
|
| 257 | + # Cleanup |
| 258 | + client.power.off(destroy=True) |
| 259 | +``` |
| 260 | + |
| 261 | +## Architecture |
| 262 | + |
| 263 | +```text |
| 264 | +┌────────────┐ gRPC ┌────────────────┐ HTTP ┌──────────────────┐ |
| 265 | +│ jmp shell │──────────────►│ Exporter │────────────►│ Host │ |
| 266 | +│ (client) │ │ ├─ cuttlefish │ :2080 │ Orchestrator │ |
| 267 | +│ │ │ │ ├─ power │ │ │ |
| 268 | +│ │ │ │ ├─ storage │ │ cvd create/ │ |
| 269 | +│ │ │ │ └─ adb │ │ start/stop │ |
| 270 | +│ │ │ ├─ netsim ────│── :7681 ──►│ netsim REST │ |
| 271 | +│ │ │ └─ bt_peer ───│── :7300 ──►│ rootcanal HCI │ |
| 272 | +└────────────┘ └────────────────┘ └────────┬─────────┘ |
| 273 | + │ |
| 274 | + ▼ |
| 275 | + ┌──────────────────┐ |
| 276 | + │ Cuttlefish VM │ |
| 277 | + │ (Android guest) │ |
| 278 | + │ ADB :6520 │ |
| 279 | + └──────────────────┘ |
| 280 | +``` |
| 281 | + |
| 282 | +The driver is a thin REST client that translates Jumpstarter driver calls into |
| 283 | +Host Orchestrator API requests. Long-running operations (create, start, stop, |
| 284 | +delete) are handled asynchronously - the driver polls the `/operations/:wait` |
| 285 | +endpoint until completion or timeout. |
| 286 | + |
| 287 | +`power.on()` waits for full boot by default (`boot_timeout=300`). It polls |
| 288 | +`adb connect` + `adb devices` until the device is online, then waits for |
| 289 | +`sys.boot_completed=1`. Set `boot_timeout: 0` to skip the wait. |
| 290 | + |
| 291 | +### CVD Build Sources |
| 292 | + |
| 293 | +The `env_config` supports two build source formats in `disk.default_build`: |
| 294 | + |
| 295 | +- **Android CI**: `@ab/<branch>/<target>` - fetches images from Android Build servers. |
| 296 | + Example: `@ab/aosp-android-latest-release/aosp_cf_x86_64_auto-userdebug` (AAOS) |
| 297 | +- **Local path**: `/path/to/android/build` - uses pre-fetched images on the host. |
| 298 | + |
| 299 | +## API Reference |
| 300 | + |
| 301 | +### Driver |
| 302 | + |
| 303 | +```{eval-rst} |
| 304 | +.. autoclass:: jumpstarter_driver_cuttlefish.driver.Cuttlefish() |
| 305 | + :members: |
| 306 | +``` |
| 307 | + |
| 308 | +### Client |
| 309 | + |
| 310 | +```{eval-rst} |
| 311 | +.. autoclass:: jumpstarter_driver_cuttlefish.client.CuttlefishClient() |
| 312 | + :members: |
| 313 | +``` |
0 commit comments