Skip to content

Add is_notice flag to the --classify option #3822 #4142

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from
Open

Add is_notice flag to the --classify option #3822 #4142

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/source/cli-reference/help-text-options.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -154,8 +154,8 @@ The Following Help Text is displayed, i.e. This is the help text for Scancode Ve
scans for key, top-level files. Key files are top-
level codebase files such as COPYING, README and
package manifests as reported by the --classify
option "is_legal", "is_readme", "is_manifest" and
"is_top_level" flags.
option "is_legal", "is_readme", "is_manifest",
"is_notice" and "is_top_level" flags.
--tallies-with-details Compute tallies of license, copyright and other scans
at the codebase level, keeping intermediate details
at the file and directory level.
Expand Down Expand Up @@ -639,7 +639,7 @@ The Following Text is displayed, i.e. This is the available plugins for Scancode
required_plugins:
options:
help_group: post-scan, name: tallies_key_files: --tallies-key-files
help: Compute tallies for license, copyright and other scans for key, top-level files. Key files are top-level codebase files such as COPYING, README and package manifests as reported by the --classify option "is_legal", "is_readme", "is_manifest" and "is_top_level" flags.
help: Compute tallies for license, copyright and other scans for key, top-level files. Key files are top-level codebase files such as COPYING, README and package manifests as reported by the --classify option "is_legal", "is_readme", "is_manifest", "is_notice" and "is_top_level" flags.
doc:
Compute tallies of a scan at the codebase level for only key files.

Expand Down Expand Up @@ -669,7 +669,7 @@ The Following Text is displayed, i.e. This is the available plugins for Scancode
--------------------------------------------
Plugin: scancode_pre_scan:classify class: summarycode.classify_plugin:FileClassifier
codebase_attributes:
resource_attributes: is_legal, is_manifest, is_readme, is_top_level, is_key_file
resource_attributes: is_legal, is_manifest, is_readme, is_top_level, is_key_file, is_notice
sort_order: 30
required_plugins:
options:
Expand Down
2 changes: 2 additions & 0 deletions docs/source/cli-reference/scan-options-post.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1018,6 +1018,7 @@ To see all plugins available via command line help, use ``--plugins``.
- "is_readme"
- "is_top_level"
- "is_key_file"
- "is_notice"

A key-file is a top-level file, that is either a legal (LICENSE/COPYING etc), manifest or a
readme file.
Expand Down Expand Up @@ -1087,6 +1088,7 @@ To see all plugins available via command line help, use ``--plugins``.
"is_readme": false,
"is_top_level": true,
"is_key_file": false,
"is_notice": false,
"tallies": {
"detected_license_expression": [
{
Expand Down
2 changes: 2 additions & 0 deletions docs/source/cli-reference/scan-options-pre.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ Pre-Scan Options
- readme
- top-level
- manifest
- notice

A manifest file in computing is a file containing metadata for a group of accompanying
files that are part of a set or coherent unit.
Expand All @@ -80,6 +81,7 @@ Pre-Scan Options
"is_readme": true,
"is_top_level": true,
"is_key_file": true
"is_notice": false,
}

----
Expand Down
6 changes: 3 additions & 3 deletions docs/source/rst_snippets/post_scan_options.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,9 @@ All "Post-Scan" Options
for key, top-level files, with occurrence counts.
Key files are top-level codebase files such as
COPYING, README and package manifests as reported
by the ``--classify`` option: "is_legal",
"is_readme", "is_manifest" and "is_top_level"
flags.
by the ``--classify`` option: "is_legal"
, "is_readme", "is_manifest", "is_notice" and
"is_top_level" flags.

Sub-Option of: ``--classify`` and ``--summary``.

Expand Down
7 changes: 6 additions & 1 deletion src/summarycode/classify.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,9 @@ def get_relative_path(root_path, path):
'readme',
)

NOTICE_STARTS_ENDS = (
"notice",
)

def check_resource_name_start_and_end(resource, STARTS_ENDS):
"""
Expand All @@ -111,6 +114,7 @@ def set_classification_flags(resource,
_LEGAL=LEGAL_STARTS_ENDS,
_MANIF=MANIFEST_ENDS,
_README=README_STARTS_ENDS,
_NOTICE=NOTICE_STARTS_ENDS,
):
"""
Set classification flags on the `resource` Resource
Expand All @@ -119,8 +123,9 @@ def set_classification_flags(resource,

resource.is_legal = is_legal = check_resource_name_start_and_end(resource, _LEGAL)
resource.is_readme = is_readme = check_resource_name_start_and_end(resource, _README)
resource.is_notice = is_notice = check_resource_name_start_and_end(resource, _NOTICE)
# FIXME: this will never be picked up as this is NOT available in a pre-scan plugin
has_package_data = bool(getattr(resource, 'package_data', False))
resource.is_manifest = is_manifest = path.endswith(_MANIF) or has_package_data
resource.is_key_file = (resource.is_top_level and (is_readme or is_legal or is_manifest))
resource.is_key_file = (resource.is_top_level and (is_readme or is_legal or is_manifest or is_notice))
return resource
5 changes: 4 additions & 1 deletion src/summarycode/classify_plugin.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,10 @@ class FileClassifier(PostScanPlugin):

('is_key_file',
Boolean(help='True if this file is "top-level" file and either a '
'legal, readme or manifest file.')),
'legal, notice, readme or manifest file.')),

('is_notice',
Boolean(help='True if this file is likely a notice file')),

# ('is_doc',
# Boolean(help='True if this file is likely a documentation file.')),
Expand Down
2 changes: 1 addition & 1 deletion src/summarycode/tallies.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -304,7 +304,7 @@ class KeyFilesTallies(PostScanPlugin):
help='Compute tallies for license, copyright and other scans for key, '
'top-level files. Key files are top-level codebase files such '
'as COPYING, README and package manifests as reported by the '
'--classify option "is_legal", "is_readme", "is_manifest" '
'--classify option "is_legal", "is_readme", "is_manifest", "is_notice" '
'and "is_top_level" flags.',
help_group=POST_SCAN_GROUP,
required_options=['classify', 'tallies']
Expand Down
9 changes: 9 additions & 0 deletions tests/formattedcode/data/common/manifests-expected.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1934,6 +1934,7 @@ files:
is_readme: no
is_top_level: yes
is_key_file: no
is_notice: no
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could not find any example of is_notice: True in any of the tests here. Could you add a NOTICE file somewhere in the full scancode scan tests so we can test for this?

Copy link
Contributor Author

@aayushkdev aayushkdev Apr 6, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might be mistaken, but from what I understand, the formattedcode tests are designed to scan only code files, not key files. and since there are no existing test cases for key files within formattedcode, I have not added a test case there.

but I have added a new test for notice in multiple_package_data in summarycode as there was no testcase there for notice

detected_license_expression:
detected_license_expression_spdx:
license_detections: []
Expand Down Expand Up @@ -1974,6 +1975,7 @@ files:
is_readme: no
is_top_level: yes
is_key_file: no
is_notice: no
detected_license_expression:
detected_license_expression_spdx:
license_detections: []
Expand Down Expand Up @@ -2092,6 +2094,7 @@ files:
is_readme: no
is_top_level: no
is_key_file: no
is_notice: no
detected_license_expression: cddl-1.0
detected_license_expression_spdx: CDDL-1.0
license_detections:
Expand Down Expand Up @@ -2173,6 +2176,7 @@ files:
is_readme: no
is_top_level: yes
is_key_file: no
is_notice: no
detected_license_expression:
detected_license_expression_spdx:
license_detections: []
Expand Down Expand Up @@ -2390,6 +2394,7 @@ files:
is_readme: no
is_top_level: no
is_key_file: no
is_notice: no
detected_license_expression: apache-2.0
detected_license_expression_spdx: Apache-2.0
license_detections:
Expand Down Expand Up @@ -2481,6 +2486,7 @@ files:
is_readme: no
is_top_level: yes
is_key_file: no
is_notice: no
detected_license_expression:
detected_license_expression_spdx:
license_detections: []
Expand Down Expand Up @@ -2631,6 +2637,7 @@ files:
is_readme: no
is_top_level: no
is_key_file: no
is_notice: no
detected_license_expression: mit
detected_license_expression_spdx: MIT
license_detections:
Expand Down Expand Up @@ -2702,6 +2709,7 @@ files:
is_readme: no
is_top_level: yes
is_key_file: no
is_notice: no
detected_license_expression:
detected_license_expression_spdx:
license_detections: []
Expand Down Expand Up @@ -2935,6 +2943,7 @@ files:
is_readme: no
is_top_level: no
is_key_file: no
is_notice: no
detected_license_expression: lgpl-3.0
detected_license_expression_spdx: LGPL-3.0-only
license_detections:
Expand Down
Loading
Loading