|
1 | | -# WARNING: NOT READY FOR PRODUCTION USE |
| 1 | +# ReflectSync |
2 | 2 |
|
3 | | -November 15, 2025 Status: **THIS IS NOT READY FOR PRODUCTION USE. IT MAY DELETE YOUR FILES.** |
| 3 | +ReflectSync is a fast, fully observant, bidirectional file synchronizer built on rsync and SQLite. It targets large development trees, remote SSH roots, and long-running sessions (days to months) where you want to understand exactly why bytes move. |
4 | 4 |
|
5 | | -# ReflectSync |
| 5 | +## Why ReflectSync? |
| 6 | + |
| 7 | +- **SQLite everywhere.** Alpha, beta, and base databases capture hashes, metadata, and logical timestamps so merges are deterministic and auditable. |
| 8 | +- **Modern rsync pipeline.** Each session owns a persistent SSH ControlMaster, rsync batches are chunked, and remote scans stream NDJSON over stdin/stdout instead of temporary files. |
| 9 | +- **Watchers + restricted scans.** Hot paths flow through debounced watchers; forced scans read NUL-separated path lists from stdin. |
| 10 | +- **Filesystem-aware.** Every root is probed for case sensitivity, Unicode normalization, reflink support, and privilege level. Sessions record canonical winners for conflicts so macOS/Windows volumes cannot clobber distinct case variants that originated on Linux. |
| 11 | +- **Transparency-first.** Structured logs, copy_pending enforcement, and `reflect query …` let you inspect exactly what happened. Nothing is hidden inside opaque daemons. |
| 12 | + |
| 13 | +> Requires **Node.js 22+** and a recent rsync. |
6 | 14 |
|
7 | | -ReflectSync is a fast bidirectional file synchronization program built on rsync and sqlite, designed for very large trees, low RAM, and observability. |
| 15 | +## Quick start |
| 16 | + |
| 17 | +```bash |
| 18 | +pnpm install -g reflect-sync |
8 | 19 |
|
9 | | -- **Rsync transport** \(latest rsync recommended\) |
10 | | -- **SQLite indexes** per side \(alpha / beta\) \+ base snapshot for true 3\-way merges |
11 | | -- **Incremental scans** with content hashes only when needed \(based on ctime\) |
12 | | -- **Realtime restricted cycles** for hot files \(debounced, safe on partial edits\) |
13 | | -- **SSH\-friendly**: stream remote deltas over stdin |
14 | | -- **Supports mixed filesystem types**: that collapse case and normalize unicode (macos, windows) |
15 | | -- **Copy on Write (optional):** when both roots are local you can opt into `cp --reflink` copies for fast COW transfers (`reflect create … --enable-reflink`); the flag is off by default so nothing attempts reflinks automatically. |
16 | | -- **Lightweight:** The reflect-sync bundled Javascript distribution is about 100KB compressed, including all dependencies. |
| 20 | +# create a session between two paths (local or ssh-style) |
| 21 | +reflect create ~/alpha ~/beta |
17 | 22 |
|
18 | | -> Requires **Node.js 22+**. |
| 23 | +# run a sync cycle until copy_pending clears |
| 24 | +reflect sync 1 |
| 25 | + |
| 26 | +# follow structured progress logs |
| 27 | +reflect logs 1 --follow --message progress |
| 28 | + |
| 29 | +# inspect current state |
| 30 | +reflect status 1 |
| 31 | +reflect query recent 1 |
| 32 | +``` |
| 33 | + |
| 34 | +Remote endpoints use the familiar `user@host:/absolute/path` syntax (or `host:2222:/path` for custom ports). Reflect opens one SSH ControlMaster per session and refuses to run if the remote `reflect-sync` binary reports a different version, preventing protocol drift. |
| 35 | + |
| 36 | +## Architecture at a glance |
| 37 | + |
| 38 | +1. **Scheduler** – coordinates scans, watchers, merge planning, and rsync batches. It can run once via `reflect sync <id>` or stay resident under `reflect daemon`. |
| 39 | +2. **Scans & watchers** – `scan.ts` crawls filesystems (locally or over SSH) while `hotwatch.ts` listens for inotify/FSEvents. Updates flow into alpha/beta/base SQLite DBs. |
| 40 | +3. **Merge planner** – `three-way-merge.ts` compares alpha/beta/base rows, respects merge strategies (default `last-write-wins`), and emits rsync batches while honoring `copy_pending` and canonical conflict winners. |
| 41 | +4. **Transport** – `rsync.ts` and `ssh-control.ts` keep SSH sockets alive, capture transfer logs (mode/user/group), and retry if the ControlMaster dies. |
| 42 | + |
| 43 | +Because state lives in SQLite, you can reproduce or audit any run with normal SQL tooling. The CLI (`reflect query recent`, `reflect query size`, …) provides common summaries without learning the schema. |
| 44 | + |
| 45 | +## Trust & observability |
| 46 | + |
| 47 | +- **Structured logging everywhere.** Scans, merges, rsync batches, and assertions emit JSON logs consumable via `reflect logs … --json`. |
| 48 | +- **Environment probing.** Sessions fail fast if filesystem capability probes (case sensitivity, Unicode normalization, reflinks) or privilege checks fail—we never guess. |
| 49 | +- **Version enforcement.** Remote sessions run `reflect --version` over SSH and compare it to the local version before syncing. |
| 50 | +- **Explicit merge semantics.** The default `last-write-wins` strategy is documented, and logs call out why each path was copied, skipped, or flagged as a conflict. |
| 51 | +- **No black boxes.** Databases live under `~/.local/share/reflect-sync/`; copying that directory lets you replay or inspect every decision offline. |
19 | 52 |
|
20 | 53 | ## LICENSE |
21 | 54 |
|
22 | 55 | Open source under the MIT License. |
23 | 56 |
|
| 57 | + |
24 | 58 | ## Details |
25 | 59 |
|
26 | 60 | All the planned major functionality is now implemented and there is a test suite. |
|
0 commit comments