|
| 1 | +# CLAUDE.md |
| 2 | + |
| 3 | +Guidance for Claude Code (and other AI coding agents) when working in this repository. |
| 4 | + |
| 5 | +## Project overview |
| 6 | + |
| 7 | +This is the **Postman Insights Agent** — an open-source Go CLI that captures HTTP traffic from network interfaces (via libpcap), parses/witnesses requests and responses, and ships them to the Postman Insights backend so users can see endpoints, errors, latency, and volume in Postman. |
| 8 | + |
| 9 | +- **Language:** Go (see `go.mod` — `go 1.25.0`, toolchain `go1.25.11`). |
| 10 | +- **Module:** `github.com/postmanlabs/postman-insights-agent` |
| 11 | +- **Entry point:** `main.go` → `cmd.Execute()` (Cobra-based CLI). |
| 12 | +- **Binary name:** `postman-insights-agent` |
| 13 | +- **License:** See `LICENSE`. |
| 14 | +- **Origin:** Forked from / successor to the Akita Software agent — many internal packages still reference `akita*` and `akitasoftware/*` libs. |
| 15 | + |
| 16 | +Note: This open-source repo does **not** include type/format inference. That functionality only exists in the official distributed binary. |
| 17 | + |
| 18 | +## Build, test, run |
| 19 | + |
| 20 | +System prerequisites: |
| 21 | +- Go 1.25 (repo pins `go1.25.0` / toolchain `go1.25.11`). |
| 22 | +- `libpcap` |
| 23 | + - macOS: `brew install libpcap` |
| 24 | + - Debian/Ubuntu: `apt-get install libpcap-dev` |
| 25 | +- `mockgen` for tests: `go install github.com/golang/mock/mockgen@v1.5.0` |
| 26 | +- For eBPF features: `clang`, `llvm`, `linux-headers` (CI installs these automatically; see `.circleci/config.yml`). |
| 27 | + |
| 28 | +Common commands (from `Makefile`): |
| 29 | + |
| 30 | +| Command | What it does | |
| 31 | +| --------------- | ------------ | |
| 32 | +| `make` | `make build` — produces `bin/postman-insights-agent` | |
| 33 | +| `make build` | `go build -o bin/postman-insights-agent .` (runs `clean` first) | |
| 34 | +| `make clean` | `go clean` | |
| 35 | +| `make mock` | `go generate ./rest` (regenerates gomock mocks) | |
| 36 | +| `make test` | `make mock` then `go test ./...` | |
| 37 | +| `make docker-build` | Build via `build-scripts/Dockerfile`, output binary to `bin/` | |
| 38 | + |
| 39 | +CI (`.circleci/config.yml`) runs `make` and `gotestsum --junitfile ...` against all packages. |
| 40 | + |
| 41 | +## Repository layout |
| 42 | + |
| 43 | +Top-level CLI plumbing: |
| 44 | +- `main.go` — trivial entry point. |
| 45 | +- `cmd/` — Cobra command tree (`root.go`, `supervisor.go`). |
| 46 | +- `cmd/internal/` — subcommand implementations: |
| 47 | + - `apidump/` — `apidump` command: the core traffic-capture command. |
| 48 | + - `apidump-ebpf/` — hidden `apidump-ebpf` command: credential-free eBPF capture for local dev/debugging. |
| 49 | + - `ec2/` — `ec2 setup|remove`: install agent as a systemd service on EC2. |
| 50 | + - `ecs/` — `ecs add|remove|cf-fragment|task-def`: AWS ECS integration. |
| 51 | + - `kube/` — `kube inject|run|secret|helm-fragment|tf-fragment|webhook`: Kubernetes integration (DaemonSet, sidecar injection, manifests, mutating webhook). |
| 52 | + - `kube-webhook/` — `kube webhook` subcommand: HTTPS admission webhook server that auto-injects the Java agent into pods. |
| 53 | + - `legacy/` — legacy `specs` and other deprecated commands. |
| 54 | + - `ascii/`, `akiflag/`, `cmderr/`, `pluginloader/` — CLI utilities. |
| 55 | + |
| 56 | +Core domain packages: |
| 57 | +- `apidump/` — orchestrates a capture session: pcap → parse → trace → backend. |
| 58 | +- `pcap/` — libpcap wrappers, packet/stream reassembly, replay. |
| 59 | +- `learn/` — HTTP parsing (request/response → IR "witnesses"), JSON preprocessing, event-stream parsing, Luhn checks. |
| 60 | +- `trace/` — collectors (`backend_collector`, `dummy_collector`), rate limiting, stats, filters, reporting buffer. |
| 61 | +- `rest/` — HTTP clients for the Postman/Akita backend (`front_client`, `learn_client`, base client, auth, errors). Mocks are generated here via `go generate`. |
| 62 | +- `daemon/` — long-running daemon HTTP server. |
| 63 | +- `plugin/` — pluggable architecture (`interface.go`, `akita/` plugin). |
| 64 | +- `ebpf/` — eBPF-based HTTPS capture (libssl uprobes + Java TLS kprobe). Key sub-packages: |
| 65 | + - `ebpf/loader/` — loads BPF programs via `bpf2go`; generated `*_bpfel.go` files are gitignored (regenerated by CI). |
| 66 | + - `ebpf/events/` — Go-side event types, ring-buffer reader, HTTP/2 adapter. |
| 67 | + - `ebpf/programs/` — BPF C source (`libssl.bpf.c`, `java_tls.bpf.c`, `event.h`). |
| 68 | + - `ebpf/discovery/` — /proc-based PID watcher for auto-discovery of libssl processes. |
| 69 | + |
| 70 | +Integrations and platform helpers: |
| 71 | +- `integrations/cri_apis/` — Container Runtime Interface client (containerd/CRI-O). |
| 72 | +- `integrations/kube_apis/` — Kubernetes API access. |
| 73 | +- `integrations/nginx/` — nginx-related helpers. |
| 74 | +- `integrations/tests/` — integration tests. |
| 75 | +- `aws_utils/` — AWS SDK helpers. |
| 76 | +- `tcp_conn_tracker/`, `tls_conn_tracker/` — connection tracking. |
| 77 | +- `useragent/`, `location/`, `version/`, `setversion/`, `telemetry/`, `usage/`, `printer/`, `consts/`, `cfg/`, `env/`, `util/` — utilities. |
| 78 | + |
| 79 | +Other: |
| 80 | +- `data_masks/` — PII/sensitive data redaction. |
| 81 | +- `apispec/` — API spec generation/handling. |
| 82 | +- `architecture/architecture.go` — architecture metadata. |
| 83 | +- `docs/discovery-mode.md` — important user-facing doc explaining **Discovery Mode** vs **Workspace Mode** onboarding (Kubernetes). Read this before changing onboarding/k8s flows. |
| 84 | +- `charts/postman-insights-webhook/` — Helm chart for the mutating admission webhook (Java agent auto-injection). This is the canonical chart location; do not put charts under `deployment/`. |
| 85 | +- `deployment/` — Go package (`package deployment`) for deployment-environment detection (EC2, ECS, k8s, etc.). **Not** for chart artifacts. |
| 86 | +- `build-scripts/`, `ci/` — release & deployment scripts. `build-scripts/Dockerfile` uses `golang:1.25-alpine` for static binary builds (Alpine's libpcap has no D-Bus dependency). |
| 87 | + |
| 88 | +## CLI surface (high level) |
| 89 | + |
| 90 | +Root: `postman-insights-agent` (see `cmd/root.go`). Notable subcommands: |
| 91 | + |
| 92 | +- `apidump` — Capture requests/responses from network traffic (primary command). Supports `--enable-https-capture` (eBPF libssl) and `--enable-java-tls` (Java TLS kprobe). |
| 93 | +- `ec2 setup` / `ec2 remove` — Install/uninstall as systemd service on EC2. |
| 94 | +- `ecs add|remove|cf-fragment|task-def` — Manage agent on AWS ECS. |
| 95 | +- `kube inject|run|secret|helm-fragment|tf-fragment` — Kubernetes integration. `kube run` is the DaemonSet entrypoint (Linux only). |
| 96 | +- `kube webhook` — Run the mutating admission webhook that auto-injects the Postman Java agent into pods in opted-in namespaces. Runs as a separate Deployment in the cluster (not the DaemonSet); deployed via `charts/postman-insights-webhook/`. |
| 97 | +- `specs` (legacy) — Manage API specs. |
| 98 | +- `aki` — ASCII art easter egg. |
| 99 | + |
| 100 | +Test-only / hidden flags live on the root command (`testOnlyUseHTTPSFlag`, `dogfoodFlag`, `debugFlag`, profiling flags, etc.) — see `cmd/root.go`. |
| 101 | + |
| 102 | +## Conventions and gotchas |
| 103 | + |
| 104 | +- **Cobra + Viper + pflag** are used throughout. New subcommands should follow the pattern in `cmd/internal/<name>/<name>.go` and be wired into `cmd/root.go`. |
| 105 | +- **Mocks**: `rest/` uses gomock. Run `make mock` (or `go generate ./rest`) after changing interfaces in `rest/interface.go`. `make test` does this automatically. |
| 106 | +- **libpcap is required at build time** on every platform — `pcap/get_pcap_handle_linux.go` is Linux-only; the generic file covers other OSes. |
| 107 | +- **Linux-only features**: `kube run` (DaemonSet mode), eBPF capture, and some pcap paths assume Linux. Be mindful when adding code under build-tagged files. |
| 108 | +- **eBPF generated files**: `ebpf/loader/*_bpfel.go` and `*.o` are produced by `go generate ./ebpf/loader` (via `bpf2go`) and are gitignored — CI regenerates them from source. Do not commit them. |
| 109 | +- **Helm charts** live in `charts/`, not `deployment/`. The `deployment/` directory is a Go package for environment detection. |
| 110 | +- **Akita lineage**: Many package names, struct names, and dependencies still say `akita`. Do **not** rename these casually — they cross module boundaries (`github.com/akitasoftware/akita-ir`, `akita-libs`, `go-utils`). When in doubt, leave the name alone. |
| 111 | +- **Backend talk** goes through `rest/` clients. Avoid hitting external HTTP from other packages directly. |
| 112 | +- **Telemetry** is in `telemetry/`; usage events in `usage/`. Keep new event names consistent with existing ones. |
| 113 | +- **Discovery vs Workspace mode**: see `docs/discovery-mode.md`. New k8s onboarding work should support both and avoid resurrecting the legacy `--project` / `--collection` flags. |
| 114 | +- **Code owners**: see `.github/CODEOWNERS` before opening PRs. |
| 115 | +- **Versioning**: SemVer; bump via files under `version/` and `setversion/` as documented in `CONTRIBUTING.md`. |
| 116 | + |
| 117 | +## Running locally |
| 118 | + |
| 119 | +```bash |
| 120 | +make # builds bin/postman-insights-agent |
| 121 | +./bin/postman-insights-agent --help # explore commands |
| 122 | +./bin/postman-insights-agent apidump --help |
| 123 | +``` |
| 124 | + |
| 125 | +Capturing traffic typically requires root / `CAP_NET_RAW` (libpcap). Use `sudo` locally or run inside the provided container. |
| 126 | + |
| 127 | +## When making changes |
| 128 | + |
| 129 | +1. Prefer small, well-scoped edits. The codebase mixes legacy Akita code with newer Postman code — don't refactor broadly without reason. |
| 130 | +2. If you touch anything in `rest/`, regenerate mocks (`make mock`) and run `make test`. |
| 131 | +3. If you add a subcommand, add it under `cmd/internal/<name>/` and register it in `cmd/root.go`. |
| 132 | +4. If you change onboarding/k8s behavior, update `docs/discovery-mode.md`. |
| 133 | +5. Update `README.md` per `CONTRIBUTING.md` when changing user-visible interface (env vars, flags, ports). |
| 134 | +6. Run `make test` before claiming work is done. CI is `make` + `make mock` + `gotestsum`. |
0 commit comments