Skip to content

Commit 843f522

Browse files
authored
fix(client): a broken store answer is not an absent skill (#57)
Ports two Agent Skills store-answer fixes from launchdarkly/python-ai-sdk (`split/skills-materialization`, launchdarkly/python-ai-sdk#53) into the Node client. ## What changed **A non-object listing is a broken store, not an empty one** (`packages/client/src/skills-core.ts`). `allRawObjects` used to collapse anything `store.allObjects()` returned that was not an object down to `{}` with no error. A store that served nothing usable was therefore indistinguishable from a store holding no skills, which the `'*'` reconcile read as "every skill was revoked" and pruned accordingly. It now returns `{ objects, error }`, logs an error naming the returned type (`the skill store listed skills as null rather than an object`), and puts that answer on the same footing as a store that threw. A throwing store is caught in the same place with the same wording (`storeThrew`), so `allSkills` and the reconcile's `resolveAll` no longer each re-derive the log line and the message. `allSkills` still returns `[]` on either failure; `writeSkills('*')` reports an incomplete run and prunes nothing. Mirrors Python's `list_raw_objects` returning `(objects, error)`. **An answer under a different key is withheld** (`resolveFromStore`). Identity is read off the object itself, so a store answering under a different key used to hand the caller a `Skill` carrying someone else's key. The reconcile then wrote that other key's path and, since prune keys off the request, deleted it in the same pass while reporting `ok`. After verification, `skill.key !== key` is now withheld with `skill '<key>' is not available: the store answered under key '<skill.key>'`, using the same shape as the existing version-mismatch branch. **Tests.** Accessor side from Python `5817d89` (aliased key withheld by `getSkill` / `getSkillResult`; non-object listing reported via `allRawObjects` with the type named) and reconcile side from `4c6d965` (a non-object listing leaves every managed file and manifest entry alone; an aliased answer writes nothing, is reported against the requested key, and never reaches the other key's file). All six fail against the unpatched source. Commits mirror Python `5817d89` and `4c6d965`. ## Notes for review - **Reason token for the key mismatch.** Python's `Resolution` has no typed outcome, so there is nothing to port there. Here it reuses `wrong_version`, per the existing branch's shape: the store held an answer, but not the one that was asked for. If a dedicated token is wanted it is a cross-language vocabulary change and belongs in its own PR. - **`e2b54fd` (an unheld version pin is a miss, not an integrity failure) is deliberately not ported.** Python's `InMemorySkillStore` files several versions per key plus a version-less slot, and that commit changes which one a pinned miss falls back to. Node's `InMemorySkillStore` holds exactly one object per key and ignores the version argument, letting `resolveFromStore` refuse a wrong-version answer afterwards. There is no fallback slot to fix, so the change has no direct equivalent; whether Node should grow multi-version semantics is a separate decision. - **#50 (`xie/skills-fdv2-transport`) sits above this branch and will need a rebase once this lands**, since `allRawObjects` changed its return type and `resolveAll` / `allSkills` changed accordingly. Not rebased here. ## Verification - `vitest run` in `packages/client`: 652 passed, 2 skipped - `tsc --noEmit`: clean - `biome check`: clean 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > Hardens the Node client against **untrusted skill store** behavior (ported from the Python SDK): broken listings and key aliasing no longer look like “no skills” or succeed while corrupting disk state. > > **Non-object listings** — `allRawObjects` now returns `{ objects, error }` instead of silently `{}`. Throws and invalid list shapes (null, array, etc.) are logged with consistent wording via **`storeThrew`**, and **`writeSkills('*')`** treats a listing error as an **incomplete run** so **prune does not delete** managed files. **`allSkills`** still returns `[]` on failure but the error is available on the listing path. > > **Key aliasing** — After verification, **`resolveFromStore`** withholds answers where **`skill.key !==` the requested key**, with the same outcome shape as version mismatch (`wrong_version`). That blocks **`getSkill`** from returning the wrong identity and stops **`writeSkills`** from writing under one key and pruning under another in the same reconcile. > > **Tests** cover accessor and filesystem reconcile paths for both fixes. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit ee5d469. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
2 parents 888a616 + ee5d469 commit 843f522

5 files changed

Lines changed: 213 additions & 29 deletions

File tree

packages/client/src/__tests__/skills-fs.test.ts

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -719,6 +719,87 @@ describe('writeSkills resilience', () => {
719719
);
720720
});
721721

722+
it('never prunes on a non-object listing', async () => {
723+
// A store that cannot list is not a store holding nothing. A listing
724+
// collapsed to "no skills" is indistinguishable from every skill having
725+
// been revoked, and prune would then delete every managed file and report a
726+
// clean run. The listing failure has to reach the prune gate as an
727+
// incomplete run.
728+
const noListing: SkillStore = {
729+
getObject() {
730+
return null;
731+
},
732+
allObjects() {
733+
return null as unknown as Record<string, RawSkillObject>;
734+
},
735+
};
736+
const existing = await placeManaged(root, 'pdf-extraction', SKILL_BODY);
737+
_setStore(noListing);
738+
739+
const report = await writeSkills('*', root);
740+
741+
expect(report.ok).toBe(false);
742+
expect(await readFile(existing, 'utf-8')).toBe(SKILL_BODY);
743+
expect(report.actions.map((a) => a.action)).toEqual(['error']);
744+
expect(errorMessages(report).some((m) => m.includes('rather than an object'))).toBe(true);
745+
// The entry survives, so the next reconcile picks it up.
746+
expect(Object.keys((await readManifest(root)).entries as Record<string, unknown>)).toContain(
747+
`pdf-extraction/${SKILL_MD}`,
748+
);
749+
});
750+
751+
it('writes nothing for an answer served under another key', async () => {
752+
// The file is named after the key the object carries, so a store answering
753+
// under a different key would write one path and prune another. Left
754+
// unchecked, the run wrote the aliased key, then deleted it in the same
755+
// pass because prune keys off the request — and reported ok. The requested
756+
// key has to be the one the outcome is reported against.
757+
const aliasing: SkillStore = {
758+
getObject() {
759+
return rawSkill('other-key');
760+
},
761+
allObjects() {
762+
return {};
763+
},
764+
};
765+
_setStore(aliasing);
766+
767+
const report = await writeSkills(['requested-key'], root);
768+
769+
expect(report.ok).toBe(false);
770+
expect(report.actions.map((a) => a.action)).toEqual(['error']);
771+
// Reported against the key that was asked for, not the one served.
772+
expect(actionsByKey(report)['requested-key'].action).toBe('error');
773+
expect(await exists(path.join(root, 'other-key'))).toBe(false);
774+
expect((await readManifest(root)).entries).toEqual({});
775+
});
776+
777+
it('an answer served under another key never reaches that key’s file', async () => {
778+
// Both keys are requested here, so nothing is prunable and the write itself
779+
// is what is under test: unchecked, the object served under the alias is
780+
// written to the *other* key's path, clobbering the content that key's own
781+
// lookup resolved — and the run still reports ok.
782+
const aliased = 'aliased\n';
783+
const aliasing: SkillStore = {
784+
getObject(_kind, key) {
785+
return key === 'other-key' ? rawSkill('other-key') : rawSkill('other-key', 2, aliased);
786+
},
787+
allObjects() {
788+
return {};
789+
},
790+
};
791+
const existing = await placeManaged(root, 'other-key', SKILL_BODY);
792+
_setStore(aliasing);
793+
794+
// The alias is resolved last, so an unchecked write would land on top.
795+
const report = await writeSkills(['other-key', 'requested-key'], root);
796+
797+
expect(report.ok).toBe(false);
798+
expect(await readFile(existing, 'utf-8')).toBe(SKILL_BODY);
799+
expect(actionsByKey(report)['requested-key'].action).toBe('error');
800+
expect(actionsByKey(report)['other-key'].action).toBe('skipped_current');
801+
});
802+
722803
it('never corrupts the manifest on an unavailable run', async () => {
723804
await placeManaged(root, 'a', SKILL_BODY);
724805
const before = await readManifest(root);

packages/client/src/__tests__/skills.test.ts

Lines changed: 73 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ import {
4646
InMemorySkillStore,
4747
skillRefs,
4848
} from '../skills.js';
49-
import { MAX_SKILL_CONTENT_BYTES } from '../skills-core.js';
49+
import { allRawObjects, MAX_SKILL_CONTENT_BYTES, requireStore } from '../skills-core.js';
5050
import type { RawSkillObject, Skill, SkillOutcomeReason, SkillStore } from '../types.js';
5151
import {
5252
createReconcileAction,
@@ -603,6 +603,29 @@ describe('getSkill', () => {
603603
it('returns null for a missing key, never raising', async () => {
604604
expect(await getSkill('nope')).toBeNull();
605605
});
606+
607+
it('withholds a store answering under a different key', async () => {
608+
// The key needs the same post-fetch defense the version already has.
609+
// Identity is read off the object itself, and the store is untrusted. An
610+
// answer served under a different key would otherwise be handed back under
611+
// the key the caller asked for while carrying its own.
612+
const aliasing: SkillStore = {
613+
getObject() {
614+
return rawSkill({ key: 'other-key' });
615+
},
616+
allObjects() {
617+
return {};
618+
},
619+
};
620+
_setStore(aliasing);
621+
622+
expect(await getSkill('asked-for')).toBeNull();
623+
624+
const outcome = await getSkillResult('asked-for');
625+
expect(outcome.skill).toBeNull();
626+
expect(outcome.reason).toBe('wrong_version');
627+
expect(outcome.detail).toBe("skill 'asked-for' is not available: the store answered under key 'other-key'");
628+
});
606629
});
607630

608631
// ─── getSkills ─────────────────────────────────────────────────────────
@@ -680,6 +703,55 @@ describe('allSkills', () => {
680703
_setStore(new InMemorySkillStore());
681704
expect(await allSkills()).toEqual([]);
682705
});
706+
707+
it('reports a non-object listing as a broken store, not an empty one', async () => {
708+
// `allSkills` has no way to report the difference, so it returns an empty
709+
// list either way — but the reason has to reach the caller that does act on
710+
// it. Collapsing the answer to "no skills" reads downstream as "every skill
711+
// was revoked".
712+
const spy = vi.spyOn(console, 'error').mockImplementation(() => {});
713+
try {
714+
const brokenListing: SkillStore = {
715+
getObject() {
716+
return null;
717+
},
718+
allObjects() {
719+
return null as unknown as Record<string, RawSkillObject>;
720+
},
721+
};
722+
_setStore(brokenListing);
723+
724+
expect(await allSkills()).toEqual([]);
725+
726+
const { objects, error } = allRawObjects(requireStore());
727+
expect(objects).toEqual({});
728+
expect(error).toBe('the skill store listed skills as null rather than an object');
729+
expect(spy.mock.calls.map(([line]) => String(line))).toEqual([
730+
'[LaunchDarkly] Skill store listed skills as null rather than an object',
731+
'[LaunchDarkly] Skill store listed skills as null rather than an object',
732+
]);
733+
} finally {
734+
spy.mockRestore();
735+
}
736+
});
737+
738+
it('names the type a broken listing came back as', () => {
739+
const spy = vi.spyOn(console, 'error').mockImplementation(() => {});
740+
try {
741+
const listingAs = (value: unknown): SkillStore => ({
742+
getObject: () => null,
743+
allObjects: () => value as Record<string, RawSkillObject>,
744+
});
745+
expect(allRawObjects(listingAs([])).error).toBe('the skill store listed skills as array rather than an object');
746+
expect(allRawObjects(listingAs('x')).error).toBe('the skill store listed skills as string rather than an object');
747+
expect(allRawObjects(listingAs(undefined)).error).toBe(
748+
'the skill store listed skills as undefined rather than an object',
749+
);
750+
expect(allRawObjects(new InMemorySkillStore())).toEqual({ objects: {}, error: null });
751+
} finally {
752+
spy.mockRestore();
753+
}
754+
});
683755
});
684756

685757
// ─── Integrity verification ────────────────────────────────────────────

packages/client/src/skills-core.ts

Lines changed: 55 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -419,10 +419,49 @@ export function verifyRawSkill(raw: unknown): Skill | null {
419419
});
420420
}
421421

422-
/** Lists every raw object the store holds. Propagates whatever it throws. */
423-
export function allRawObjects(store: SkillStore): Record<string, RawSkillObject> {
424-
const objects = store.allObjects(SKILL_OBJECT_KIND);
425-
return typeof objects === 'object' && objects !== null && !Array.isArray(objects) ? objects : {};
422+
/** The one wording for "the store could not answer", used by every path. */
423+
export function storeThrew(error: unknown): string {
424+
const name = error instanceof Error ? error.constructor.name : 'unknown error';
425+
const message = error instanceof Error ? error.message : String(error);
426+
return `the skill store threw ${name}: ${message}`;
427+
}
428+
429+
/** Every raw object the store holds, or the reason it could not answer. */
430+
export type RawListing = {
431+
readonly objects: Record<string, RawSkillObject>;
432+
/** `null` when the store answered; otherwise why it could not. */
433+
readonly error: string | null;
434+
};
435+
436+
/**
437+
* Lists every raw object the store holds, or reports why it could not.
438+
*
439+
* A throwing store is caught here rather than propagated so that `allSkills` and
440+
* the `'*'` reconcile path log and word the failure identically. Letting the
441+
* exception out instead would make each of them re-derive the log line and the
442+
* message, which is the drift this module exists to prevent.
443+
*
444+
* An answer that is not an object is a broken store, on the same footing as one
445+
* that threw — **not** an empty one. Collapsing it to `{}` would make a store
446+
* that served nothing usable indistinguishable from a store that holds no
447+
* skills, which reads downstream as "every skill was revoked".
448+
*/
449+
export function allRawObjects(store: SkillStore): RawListing {
450+
let objects: unknown;
451+
try {
452+
objects = store.allObjects(SKILL_OBJECT_KIND);
453+
} catch (error) {
454+
// biome-ignore lint/suspicious/noConsole: this package has no logger abstraction; a failing store must be visible
455+
console.error(`[LaunchDarkly] Skill store threw while listing skills: ${storeThrew(error)}`);
456+
return { objects: {}, error: storeThrew(error) };
457+
}
458+
if (typeof objects !== 'object' || objects === null || Array.isArray(objects)) {
459+
const typeName = Array.isArray(objects) ? 'array' : objects === null ? 'null' : typeof objects;
460+
// biome-ignore lint/suspicious/noConsole: this package has no logger abstraction; a failing store must be visible
461+
console.error(`[LaunchDarkly] Skill store listed skills as ${typeName} rather than an object`);
462+
return { objects: {}, error: `the skill store listed skills as ${typeName} rather than an object` };
463+
}
464+
return { objects: objects as Record<string, RawSkillObject>, error: null };
426465
}
427466

428467
// ---------------------------------------------------------------------------
@@ -470,18 +509,19 @@ export type Resolution = {
470509
* versions of one key and only it can pick between them; `null` asks for the
471510
* newest. The equality check afterwards is kept as a **defense**, not as the
472511
* selection mechanism: the store is untrusted, so an answer that is not the
473-
* version that was asked for is withheld rather than returned.
512+
* version that was asked for is withheld rather than returned. The key is
513+
* checked the same way and for the same reason: identity is read off the object
514+
* itself, so an answer served under a different key would otherwise be returned
515+
* under the caller's key while carrying its own.
474516
*/
475517
export function resolveFromStore(store: SkillStore, key: string, wantedVersion: number | null): Resolution {
476518
let raw: RawSkillObject | null | undefined;
477519
try {
478520
raw = store.getObject(SKILL_OBJECT_KIND, key, wantedVersion);
479521
} catch (error) {
480-
const name = error instanceof Error ? error.constructor.name : 'unknown error';
481-
const message = error instanceof Error ? error.message : String(error);
482522
// biome-ignore lint/suspicious/noConsole: this package has no logger abstraction; a failing store must be visible
483-
console.error(`[LaunchDarkly] Skill store threw while retrieving '${key}': ${message}`);
484-
return { error: `the skill store threw ${name}: ${message}`, reason: 'store_unavailable', unavailable: true };
523+
console.error(`[LaunchDarkly] Skill store threw while retrieving '${key}': ${storeThrew(error)}`);
524+
return { error: storeThrew(error), reason: 'store_unavailable', unavailable: true };
485525
}
486526

487527
if (typeof raw !== 'object' || raw === null || Array.isArray(raw)) {
@@ -492,6 +532,12 @@ export function resolveFromStore(store: SkillStore, key: string, wantedVersion:
492532
if (skill === null) {
493533
return { error: `skill '${key}' failed integrity verification and was withheld`, reason: 'integrity_failure' };
494534
}
535+
if (skill.key !== key) {
536+
return {
537+
error: `skill '${key}' is not available: the store answered under key '${skill.key}'`,
538+
reason: 'wrong_version',
539+
};
540+
}
495541
if (wantedVersion !== null && skill.version !== wantedVersion) {
496542
return {
497543
error: `skill '${key}' version ${wantedVersion} is not available (the store holds version ${skill.version})`,

packages/client/src/skills-fs.ts

Lines changed: 2 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -504,13 +504,8 @@ function resolveAll(deadline: number, onUnavailable: OnUnavailable): { requests:
504504
// Deliberately not via allSkills(), which reports a throwing store as an empty
505505
// result — that would look like "every skill was revoked" and let prune delete
506506
// the lot.
507-
let objects: Record<string, unknown>;
508-
try {
509-
objects = allRawObjects(store);
510-
} catch (error) {
511-
const name = error instanceof Error ? error.constructor.name : 'unknown error';
512-
return unavailableRun(unavailable(`the skill store threw ${name}: ${messageOf(error)}`), onUnavailable);
513-
}
507+
const { objects, error } = allRawObjects(store);
508+
if (error !== null) return unavailableRun(unavailable(error), onUnavailable);
514509

515510
return {
516511
requests: Object.entries(objects).map(([key, raw]) => pendingForRaw(key, raw)),

packages/client/src/skills.ts

Lines changed: 2 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -253,18 +253,8 @@ export async function getSkills(refs: ReadonlyArray<SkillReference | string>): P
253253
* configured.
254254
*/
255255
export async function allSkills(): Promise<Skill[]> {
256-
const store = requireStore();
257-
258-
let objects: Record<string, RawSkillObject>;
259-
try {
260-
objects = allRawObjects(store);
261-
} catch (error) {
262-
// biome-ignore lint/suspicious/noConsole: this package has no logger abstraction; a failing store must be visible
263-
console.error(
264-
`[LaunchDarkly] Skill store threw while listing skills: ${error instanceof Error ? error.message : String(error)}`,
265-
);
266-
return [];
267-
}
256+
const { objects, error } = allRawObjects(requireStore());
257+
if (error !== null) return [];
268258

269259
const skills: Skill[] = [];
270260
for (const raw of Object.values(objects)) {

0 commit comments

Comments
 (0)