Skip to content

feat(cli): add --ipfs-superfluous flag to clean command#1854

Open
saturn-dbeal wants to merge 4 commits intousecannon:devfrom
saturn-dbeal:feat/clean-superfluous-ipfs
Open

feat(cli): add --ipfs-superfluous flag to clean command#1854
saturn-dbeal wants to merge 4 commits intousecannon:devfrom
saturn-dbeal:feat/clean-superfluous-ipfs

Conversation

@saturn-dbeal
Copy link

Summary

Add a new mode to the cannon clean command that deletes IPFS packages not referenced by any tag in the tags folder.

Usage

# Delete superfluous IPFS packages with confirmation
cannon clean --ipfs-superfluous

# Delete without confirmation (useful for scripts)
cannon clean --ipfs-superfluous --no-confirm

How it works

  1. Scans the tags/ folder for all .txt and .meta files
  2. Extracts IPFS URLs from these files
  3. Compares against all files in ipfs_cache/
  4. Identifies files not referenced by any tag
  5. Prompts for confirmation before deletion (unless --no-confirm)

Benefits

  • Helps users free disk space by removing orphaned IPFS cache files
  • Safe: only removes files that are definitely not referenced
  • Shows exactly what will be deleted and how much space will be freed

Testing

  • Unit tests added for the new functionality
  • Manually tested with local Cannon cache

Draft PR - ready for initial review.

Add a new mode to the cannon clean command that deletes IPFS packages
not referenced by any tag in the tags folder.

Usage:
  cannon clean --ipfs-superfluous
  cannon clean --ipfs-superfluous --no-confirm

This helps users free disk space by removing orphaned IPFS cache files
that are no longer needed.

- Add getReferencedIpfsUrls() to scan tags folder for IPFS references
- Add cleanSuperfluousIpfs() to identify and delete unreferenced files
- Add --ipfs-superfluous flag to CLI
- Add unit tests for new functionality
Copy link
Contributor

@dbeal-eth dbeal-eth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking good so far. I particularly like how you used careful validation throughout the scanning processes.

one thing we have to be careful about is that its possible for an IPFS file to contain a dependency upon another IPFS file. In this case, we should be keeping those second-order IPFS files as well. Please make changes to ensure this isn't an issue 👍

@dbeal-eth dbeal-eth self-assigned this Mar 1, 2026
@dbeal-eth dbeal-eth marked this pull request as ready for review March 1, 2026 16:13
When cleaning superfluous IPFS packages, recursively scan the content
of referenced IPFS cache files to find nested IPFS URLs. This ensures
that second-order (and deeper) IPFS dependencies are preserved and not
incorrectly marked as superfluous.

Adds extractIpfsUrls() and resolveIpfsDependencies() functions to
build a complete dependency graph before determining which files are
safe to delete. Includes test coverage for nested dependency chains.
@saturn-dbeal
Copy link
Author

Addressed review feedback: added recursive IPFS dependency resolution. The clean command now:

  1. Finds directly referenced IPFS URLs from tags
  2. Reads the content of each cached IPFS file and extracts nested ipfs:// URLs
  3. Recursively follows the dependency chain until all transitive dependencies are found
  4. Only marks files as superfluous if they're not in the complete dependency graph

Added tests covering the second-order dependency case.

@usecannon usecannon deleted a comment from saturn-dbeal Mar 2, 2026
@saturn-dbeal
Copy link
Author

All inline review comments addressed in 7917b33. Changes:

  • Renamed --ipfs-superfluous--ipfs
  • Removed duplicate getCacheHash, now imports CliLoader.getCacheHash
  • Confirmed getReferencedIpfsUrls already returns Set<string>
  • Renamed all "superfluous" → "orphaned" terminology
  • Removed formatBytes and space-freed calculations

Ready for re-review @dbeal-eth

- Rename --ipfs-superfluous flag to --ipfs
- Use CliLoader.getCacheHash instead of duplicating the function
- Replace 'superfluous' terminology with 'orphaned'
- Remove formatBytes and space-freed tracking
- getReferencedIpfsUrls already returns Set<string> (confirmed)
@saturn-dbeal saturn-dbeal force-pushed the feat/clean-superfluous-ipfs branch from 7917b33 to e2a6600 Compare March 2, 2026 05:31
@saturn-dbeal
Copy link
Author

All inline review comments have been addressed. Ready for re-review. Changes in commit e2a6600:

  • Renamed --ipfs-superfluous--ipfs
  • Removed duplicate getCacheHash, now imports CliLoader.getCacheHash
  • Confirmed getReferencedIpfsUrls already returns Set<string>
  • Renamed all "superfluous" terminology to "orphaned"
  • Removed formatBytes and all space-freed tracking

@saturn-dbeal saturn-dbeal requested a review from dbeal-eth March 2, 2026 09:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants