Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kvs: remove dropcache #6622

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

chu11
Copy link
Member

@chu11 chu11 commented Feb 10, 2025

Problem: KVS dropcache code was added very early on in KVS prototyping for testing. It is basically not used anymore. If we want to test something that requires a "clean" slate for the KVS, we can just start a new instance and test with that.

Solution: Remove all KVS dropcache code. Remove all associated tests. Update one regression test to be accomplished through two instances rather than one to emulate an emptying of the cache.

Fixes #6608

Problem: KVS dropcache code was added very early on in KVS
prototyping for testing.  It is basically not used anymore.  If we
want to test something that requires a "clean" slate for the KVS,
we can just start a new instance and test with that.

Solution: Remove all KVS dropcache code.  Remove all associated
tests.  Update one regression test to be accomplished through two
instances rather than one to emulate an emptying of the cache.

Fixes flux-framework#6608
Copy link
Member

@garlick garlick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@chu11
Copy link
Member Author

chu11 commented Feb 11, 2025

one builder failed with not ok 8 - t4583-free-range-test, which I've never seen before. Just logging, will restart builder.

  expecting success: flux job attach f2bjg5Q4
  flux-job: task(s) Killed
  free-range-test: Re-launching test script under flux-start
  free-range-test: Starting a child instance with flat topology
  free-range-test: Started job f3Fcoj35
  free-range-test: Current overlay status of f3Fcoj35:
  0 fv-az1384-857: full
  ├─ 1 fv-az1384-857: full
  ├─ 2 fv-az1384-857: full
  └─ 3 fv-az1384-857: full
  free-range-test: Launch a sleep job within f3Fcoj35:
  f3tdXo6P
         JOBID USER     ST NTASKS NNODES  RUNTIME
             . runner    R      4      4   16.02s .
      f3Fcoj35 runner    R      4      4   9.111s └── flux
      f3tdXo6P runner    R      4      4   1.020s     └── sleep
  free-range-test: Killing rank 3 (pid 272449) and all children
  free-range-test: Wait for exception event in f3Fcoj35
  flux-start: 3 (pid 272449) Killed
  Feb 10 19:17:43.085878 UTC 2025 broker.err[0]: fv-az1384-857 (rank 3) failed
  1739215063.069683 exception type="node-failure" severity=2 userid=1001 note="shell rank 3 (on fv-az1384-857): Killed"
  Feb 10 19:17:43.186783 UTC 2025 broker.err[0]: dead to Flux: fv-az1384-857 (rank 3)
  free-range-test: But running a 3 node job in f3Fcoj35 still works:
  fv-az1384-857
  fv-az1384-857
  fv-az1384-857
  free-range-test: Overlay status of f3Fcoj35 should show rank lost:
  0 fv-az1384-857: degraded
  ├─ 1 fv-az1384-857: full
  ├─ 2 fv-az1384-857: full
  └─ 3 fv-az1384-857: lost lost connection
  free-range-test: Call flux shutdown on f3Fcoj35
  free-range-test: job f3Fcoj35 should exit cleanly (no hang) and a zero exit code:
  1739215072.700225 finish status=9
  free-range-test: dump output from job:
  
  9.648s: job.exception f3Fcoj35 type=node-failure severity=2 shell rank 3 (on fv-az1384-857): Killed
  flux-job: job shell Killed
  Feb 10 19:17:43.080180 UTC 2025 broker.err[0]: fv-az1384-857 (rank 3) failed
  
  Feb 10 19:17:43.182554 UTC 2025 broker.err[0]: dead to Flux: fv-az1384-857 (rank 3)
  
  Feb 10 19:17:52.841664 UTC 2025 broker.err[0]: rc2.0: /usr/src/flux-core-0.71.0-59-g860ec21a7/t/issues/t4583-free-range-test.sh Exited (rc=137) 21.8s
  flux-start: 0 (pid 272426) exited with rc=137
  Feb 10 19:17:57.829983 UTC 2025 broker.err[0]: rc2.0: /usr/src/flux-core-0.71.0-59-g860ec21a7/t/issues/t4583-free-range-test.sh Exited (rc=137) 31.5s
  not ok 8 - t4583-free-range-test

Problem: The kvs.dropcache RPC target no longer exists, yet
the "flux kvs dropcache" command still exists.

Remove "flux kvs dropcache" support.
Problem: The flux kvs dropcache command has been removed, but is
still documented.

Remove "dropcache" from flux-kvs(1).
Problem: The kvs.dropcache RPC target has been removed, but the
library function flux_kvs_dropcache() still exists.

Remove flux_kvs_dropcache() from libkvs.
Problem: The flux kvs dropcache command has been removed, but
bash completions still exist for it.

Remove bash completions for flux kvs dropcache.
Problem: The flux_kvs_dropcache() API function has been removed,
but Python still has a binding to it.

Remove the kvs dropcache() function.
@chu11
Copy link
Member Author

chu11 commented Feb 11, 2025

OMG, I'm so stupid. I forgot that there was a flux kvs dropcache command and associated library functions. @garlick please take a look at the extra commits I just pushed.

Copy link

codecov bot commented Feb 11, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 83.84%. Comparing base (27f4390) to head (8c105be).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6622      +/-   ##
==========================================
- Coverage   83.86%   83.84%   -0.02%     
==========================================
  Files         534      534              
  Lines       88358    88305      -53     
==========================================
- Hits        74100    74043      -57     
- Misses      14258    14262       +4     
Files with missing lines Coverage Δ
src/bindings/python/flux/kvs.py 95.19% <ø> (+0.25%) ⬆️
src/cmd/flux-kvs.c 84.53% <ø> (-0.01%) ⬇️
src/common/libkvs/kvs.c 93.44% <ø> (+0.48%) ⬆️
src/modules/kvs/kvs.c 74.31% <ø> (+0.34%) ⬆️

... and 7 files with indirect coverage changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

kvs: remove dropcache
2 participants