Skip to content

Fail rspec on unexpected DEPRECATION WARNING in CI#2692

Merged
yahonda merged 1 commit into
rsim:masterfrom
yahonda:raise-on-deprecation
May 4, 2026
Merged

Fail rspec on unexpected DEPRECATION WARNING in CI#2692
yahonda merged 1 commit into
rsim:masterfrom
yahonda:raise-on-deprecation

Conversation

@yahonda
Copy link
Copy Markdown
Collaborator

@yahonda yahonda commented May 4, 2026

Summary

Today, an unexpected ActiveSupport::Deprecation warning fired during the rspec run shows up only as a stderr line in the CI log. It's easy to miss until much later. This PR turns that into a hard test failure when CI=true (the default on every GitHub Actions runner).

Mechanism

spec/spec_helper.rb gains an around(:each) hook gated on ENV["CI"]. For each example it:

  1. Redirects $stderr to an outer StringIO.
  2. Runs the example.
  3. Restores $stderr.
  4. If the example passed but the captured buffer contains "DEPRECATION WARNING", raises with a message naming both supported "I expected this" mechanisms.

The hook stacks correctly with the two patterns specs already use to handle intentional deprecations:

  • expect { ... }.to output(/.../).to_stderr — the matcher itself redirects $stderr inside the block, so output it captures never reaches the outer hook.
  • OracleEnhanced.deprecator.silence { ... }Deprecation#warn short-circuits before any behavior runs, so nothing is written at all.

So no existing spec needs to change.

Negative-test verification

To confirm the hook actually catches a leaked deprecation, the should connect to database using service_name example in connection_spec.rb (whose OracleEnhanced.deprecator.silence wrap was added by PR #2689) was temporarily un-silenced locally and run with CI=true:

Failure/Error:
  raise <<~MSG
    Unexpected DEPRECATION WARNING leaked to stderr from this example. ...
  MSG

RuntimeError:
  Unexpected DEPRECATION WARNING leaked to stderr from this example. Either
  assert on it with `expect { ... }.to output(/.../).to_stderr` or wrap the
  call in `OracleEnhanced.deprecator.silence { ... }` if the deprecation
  is intentional and not under test. Captured:

  DEPRECATION WARNING: Setting `:database` to a value that starts with `/` ...

Restoring the silence wrapper makes the example pass again. The wrapper change is not part of the diff in this PR — it was a local validation only.

Workflow YAMLs

Every workflow that runs rspec gets CI: true set explicitly:

  • test.yml
  • test_11g.yml
  • test_11g_ojdbc11.yml
  • test_prepared_statements_false.yml
  • ruby_head.yml
  • jruby_head.yml

GitHub Actions already sets CI=true on every runner, so this is technically redundant — but the explicit declaration documents intent in the workflow file and protects against an upstream policy change.

Lint-only workflows (rubocop.yml, linting.yml, check_method_signatures.yml, check_method_visibility.yml) and non-rspec workflows (devcontainer.yml, release.yml) are left alone — they don't load spec_helper.rb so the env var would have no effect.

Test plan

  • bundle exec rspec — 748 examples, 0 failures, 12 pending (no behavior change outside CI)
  • CI=true bundle exec rspec — 748 examples, 0 failures, 12 pending (every existing intentional-deprecation site is correctly bracketed by to_stderr or silence)
  • Negative-test: temporarily un-silenced should connect to database using service_name, ran CI=true bundle exec rspec, observed the expected RuntimeError. Reverted before commit.
  • bundle exec rubocop — clean

🤖 Generated with Claude Code

Adds an `around(:each)` hook in spec_helper.rb that, when `ENV["CI"]`
is truthy, captures stderr for the duration of the example. If the
captured stderr contains "DEPRECATION WARNING" — i.e. a deprecation
fired but no `expect { ... }.to output(/.../).to_stderr` matcher and
no `OracleEnhanced.deprecator.silence { ... }` block consumed it —
the example is failed with a message naming both supported
mechanisms. Outside CI the suite is unchanged.

The hook stacks correctly with both intentional-deprecation
mechanisms:

  - `expect { ... }.to output(/.../).to_stderr` redirects `$stderr`
    inside the matcher's block, so output written there doesn't reach
    the outer capture.
  - `deprecator.silence` short-circuits inside `Deprecation#warn`
    before any behavior runs, so nothing is written at all.

Verified locally by temporarily removing the
`OracleEnhanced.deprecator.silence { ... }` wrapper that PR rsim#2689
added around the `should connect to database using service_name`
example: the example fails with the expected message naming the
leaked deprecation; restoring the silence makes it pass again.

Set `CI: true` explicitly in every workflow that runs rspec
(`test.yml`, `test_11g.yml`, `test_11g_ojdbc11.yml`,
`test_prepared_statements_false.yml`, `ruby_head.yml`,
`jruby_head.yml`). GitHub Actions already sets `CI=true` on every
runner; the explicit declaration documents intent in the workflow
file. Lint-only workflows (`rubocop.yml`, `linting.yml`,
`check_method_signatures.yml`, `check_method_visibility.yml`) and
non-rspec workflows (`devcontainer.yml`, `release.yml`) are left
alone — they don't load spec_helper.rb so the env var would have
no effect.

Test plan:

  - `bundle exec rspec`               — 748 examples, 0 failures
  - `CI=true bundle exec rspec`       — 748 examples, 0 failures
  - With the silence wrapper temporarily removed and `CI=true`, the
    targeted example fails with the expected RuntimeError naming
    the leaked deprecation.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@yahonda yahonda merged commit 958d1f2 into rsim:master May 4, 2026
12 checks passed
@yahonda yahonda deleted the raise-on-deprecation branch May 4, 2026 15:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant