fix(llmrails): load library files deterministically (2/5)

[Pouyanpi]

Summary

Sorts library traversal during LLMRails initialization so recorded rail tests do not depend on filesystem walk order.

Why

Recorded tests expose that library bot message insertion order can vary across platforms when files are loaded in filesystem order.

What Changed

• Sorts directories during library os.walk traversal.
• Sorts .co files before loading them.

Review Notes

This is the only runtime change in the stack.

Stack Position

Part 2 of 5.

• Previous: test(recorded): add replay harness (1/5) #1974
• Next: test(recorded): add client cassette coverage (3/5) #1976

Stack Context

This stack decomposes recorded end-to-end replay coverage into reviewable slices. The PRs should be reviewed against their parent branch in the stack.

Please review each PR against its parent branch, not directly against the root base branch, except for part 1.

Order	PR	Branch	Base
1	#1974	stack/recorded-tests-01-harness	develop
2	#1975	stack/recorded-tests-02-deterministic-library-load	stack/recorded-tests-01-harness
3	#1976	stack/recorded-tests-03-clients	stack/recorded-tests-02-deterministic-library-load
4	#1977	stack/recorded-tests-04-public-api	stack/recorded-tests-03-clients
5	#1978	stack/recorded-tests-05-library-rails	stack/recorded-tests-04-public-api

Validation

poetry check --lock
poetry lock --no-update
poetry install --with dev
poetry run pytest tests/recorded --block-network -q
pre-commit hooks passed during commit creation

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

This PR makes library and prompt file loading order deterministic by sorting directory and file lists during os.walk traversal in both llmrails.py and prompts.py. A new test in test_prompt_override.py verifies that _load_prompts() returns prompts in sorted file order even when the filesystem yields them out of order.

• llmrails.py: adds dirs.sort() before recursion and wraps the files loop with sorted(files), ensuring .co library files are always loaded in the same order regardless of platform filesystem walk order.
• prompts.py: adds dirs.sort() and files.sort() before the filename loop, mirroring the same fix for YAML prompt files.
• tests/test_prompt_override.py: adds test_load_prompts_sorts_files_for_deterministic_overrides, which monkeypatches os.walk to return files out of order and asserts the resulting prompt list is sorted.

Confidence Score: 5/5

Safe to merge — both changes are narrow, additive sorts with no logic alterations beyond load order.

The only runtime changes are two pairs of sort calls that make directory and file traversal order stable. No data is dropped, no control flow is altered, and the fix is strictly additive. The new test directly validates the file-sort behavior in _load_prompts.

No files require special attention.

Important Files Changed

Filename	Overview
nemoguardrails/llm/prompts.py	Adds dirs.sort() and files.sort() inside the os.walk loop so prompt YAML files are loaded in a stable, platform-independent order.
nemoguardrails/rails/llm/llmrails.py	Adds dirs.sort() and switches to for file in sorted(files): during library .co file traversal, making bot_message insertion order deterministic across platforms.
tests/test_prompt_override.py	Adds a new test that verifies _load_prompts() returns files in sorted order by passing an out-of-order list through a monkeypatched os.walk stub.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[LLMRails.__init__] --> B[os.walk library_path]
    B --> C[dirs.sort in-place for os.walk recursion order]
    C --> D[for file in sorted files]
    D --> E{file.endswith .co?}
    E -- Yes --> F[parse_colang_file]
    F --> G[extend config.flows]
    F --> H[insert bot_messages if not already set]
    E -- No --> D

    I[_load_prompts] --> J[os.walk prompts_dir]
    J --> K[dirs.sort + files.sort]
    K --> L[for filename in files]
    L --> M{.yml or .yaml?}
    M -- Yes --> N[yaml.safe_load]
    N --> O[extend prompts list]
    M -- No --> L

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[LLMRails.__init__] --> B[os.walk library_path]
    B --> C[dirs.sort in-place for os.walk recursion order]
    C --> D[for file in sorted files]
    D --> E{file.endswith .co?}
    E -- Yes --> F[parse_colang_file]
    F --> G[extend config.flows]
    F --> H[insert bot_messages if not already set]
    E -- No --> D

    I[_load_prompts] --> J[os.walk prompts_dir]
    J --> K[dirs.sort + files.sort]
    K --> L[for filename in files]
    L --> M{.yml or .yaml?}
    M -- Yes --> N[yaml.safe_load]
    N --> O[extend prompts list]
    M -- No --> L

Loading

_{Reviews (10): Last reviewed commit: "test(prompts): isolate prompt loading or..." | Re-trigger Greptile}

[tgasser-nv]

Looks good, need to add unit-tests for the library traversal as well as prompts before merging.

[github-actions]

Staged Fern docs preview: https://nvidia-preview-pr-1975.docs.buildwithfern.com/nemo/guardrails