Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

License Export - Updated test scenarios #20

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

License Export - Updated test scenarios #20

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f663da0
Updated test scenarios
Feb 3, 2025
6f6c045
Removed test data reference + Updated steps
Feb 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
152 changes: 136 additions & 16 deletions tests/features/licenseexport_cdx.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,38 +36,158 @@ Scenario: User Downloads license information for CycloneDX SBOM from SBOM Explor
Scenario: Verify the files on downloaded CycloneDX SBOM license ZIP
Given User has Downloaded the License information for CycloneDX SBOM
When User extracts the Downloaded license ZIP file
Then Extracted files should contain two CSVs, one for Package License combination and another one for License reference
Then Extracted files should contain two CSVs, one for Package license information and another one for License reference

Scenario: Verify the headers on CycloneDX SBOM package License CSV file
Given User extracted the CycloneDX SBOM license compressed file
When User Opens the package license combination file
Then The file should have the following headers - name, namespace, group, version, package reference, license, license name and alternate package reference
When User Opens the package license information file
Then The file should have the following headers - name, namespace, group, version, package reference, license, license name, license expression and alternate package reference

Scenario: Verify the headers on CycloneDX SBOM License reference CSV file
Given User extracted the CycloneDX SBOM license compressed file
When User Opens the license reference file
Then The file should have the following headers - licenseId, name, extracted text and comment

Scenario: Verify the license information for a package on the CycloneDX SBOM with single license
Given User is on package license combination file
When User selects a package with Single license information
Then name column should contain the value of component.name field from SBOM json
Scenario: Verify the contents on CycloneDX SBOM license reference CSV file
Given User is on license reference file
When User selects a license from the list of licenses
Then The License reference CSV should be empty
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there may be mistake here:

Why to first select license from a list ... to then assert that reference file is empty?
I think either it should be just that file is empty (as that means selecting package/license there is impossible as there are none present).
Or maybe it is not empty and in that case the other assert was to be about some column content?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a mistake - updated the feature file

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@queria Thanks for the feedback - I have included most of your suggestions. Please find the details below:

  • Updated the steps with your suggestion like "version" column should match "metadata.component.version" from SBOM
  • Agree on the test data, I have included them as for our release testing. I have removed it.
  • We may need to make a decision on the generic part. I'm somewhat inclined to use generic steps and handle data validation through code. My idea is to make the test steps more generic and parameterize the SBOM file and package information. So based on the test scenario we could pass the information.


Scenario: Verify the license information for a package on the CycloneDX SBOM with single license id
# Test data https://drive.google.com/drive/folders/1Z6y6gMegutBeUuc_8LkpYxKeGz_KYG9H?usp=drive_link
# sbom - cdx_sbom.json package - pkg:maven/io.quarkus/[email protected]?type=jar
Given User is on SBOM license information file
When User selects a package with Single license id
Then name column should contain the value of SBOM name from metadata.component.name field from SBOM json
And namespace column should be empty
And group column should contain the value of component.group field from SBOM json
And version column should contain the value of component.version field from SBOM json
And group column should contain the value of metadata.component.group field from SBOM json
And version column should contain the value of metadata.component.version field from SBOM json
And package reference column should contain the value of components.purl from SBOM json
And license column should contain the value of components.license.id field from SBOM json
And license name column should contain the value of components.license.name field from SBOM json
And license name column should be empty
And license expression column should be empty
And alternate package reference column should be empty

Scenario: Verify the license information for a package on the CycloneDX SBOM with single license id with alternate package reference
# Test data https://drive.google.com/drive/folders/1Z6y6gMegutBeUuc_8LkpYxKeGz_KYG9H?usp=drive_link
# sbom - tc_1730_license_escape.json package - pkg:pkg:npm/%40gradio/[email protected]
Given User is on SBOM license information file
When User selects a package with Single license id with cpe information
Then name column should contain the value of SBOM name from metadata.component.name field from SBOM json
And namespace column should be empty
And group column should contain the value of metadata.component.group field from SBOM json
And version column should contain the value of metadata.component.version field from SBOM json
And package reference column should contain the value of components.purl from SBOM json
And license column should contain the value of components.license.id field from SBOM json
And license name column should be empty
And license expression column should be empty
And alternate package reference column should contain the value of CPE from components.cpe field from SBOM json

Scenario: Verify the license information for a package on the CycloneDX SBOM with single license name
# Test data https://drive.google.com/drive/folders/1Z6y6gMegutBeUuc_8LkpYxKeGz_KYG9H?usp=drive_link
# sbom - cdx_sbom.json package - pkg:maven/org.bouncycastle/[email protected]?type=jar
Given User is on SBOM license information file
When User selects a package with Single license name
Then name column should contain the value of SBOM name from metadata.component.name field from SBOM json
And namespace column should be empty
And group column should contain the value of metadata.component.group field from SBOM json
And version column should contain the value of metadata.component.version field from SBOM json
And package reference column should contain the value of components.purl from SBOM json
And license column should be empty
mrrajan marked this conversation as resolved.
Show resolved Hide resolved
And license name column should contain the value of license name from components.license.name field from SBOM json
And license expression column should be empty
And alternate package reference column should be empty

Scenario: Verify the license information for a package on the CycloneDX SBOM with single license name with alternate package reference
# Test data https://drive.google.com/drive/folders/1Z6y6gMegutBeUuc_8LkpYxKeGz_KYG9H?usp=drive_link
# sbom - tc_1730_license_escape.json package - pkg:pypi/[email protected]
Given User is on SBOM license information file
When User selects a package with Single license name with cpe information
Then name column should contain the value of SBOM name from metadata.component.name field from SBOM json
And namespace column should be empty
And group column should contain the value of metadata.component.group field from SBOM json
And version column should contain the value of metadata.component.version field from SBOM json
And package reference column should contain the value of components.purl from SBOM json
And license column should be empty
And license name column should contain the value of license name from components.license.name field from SBOM json
And license expression column should be empty
And alternate package reference column should contain the value of CPE from components.cpe field from SBOM json

Scenario: Verify the license information for a package on the CycloneDX SBOM with single license expression
# Test data https://drive.google.com/drive/folders/1Z6y6gMegutBeUuc_8LkpYxKeGz_KYG9H?usp=drive_link
# sbom - cdx_sbom.json package - pkg:maven/javax.activation/[email protected]?type=jar
Given User is on SBOM license information file
When User selects a package with Single license expression
Then name column should contain the value of SBOM name from metadata.component.name field from SBOM json
And namespace column should be empty
And group column should contain the value of metadata.component.group field from SBOM json
And version column should contain the value of metadata.component.version field from SBOM json
And package reference column should contain the value of components.purl from SBOM json
And license column should be empty
And license name column should be empty
And license expression column should contain the value of whole license expression in a single row from components.license.expression field from SBOM json
And alternate package reference column should be empty

Scenario: Verify the license information for a package on the CycloneDX SBOM with single license expression with alternate package reference
# Test data https://drive.google.com/drive/folders/1Z6y6gMegutBeUuc_8LkpYxKeGz_KYG9H?usp=drive_link
# sbom - tc_1730_license_escape.json package - pkg:rpm/rhel/[email protected]?arch=x86_64&upstream=annobin-12.31-2.el9.src.rpm&distro=rhel-9.4
Given User is on SBOM license information file
When User selects a package with Single license expression with cpe information
Then name column should contain the value of SBOM name from metadata.component.name field from SBOM json
And namespace column should be empty
And group column should contain the value of metadata.component.group field from SBOM json
And version column should contain the value of metadata.component.version field from SBOM json
And package reference column should contain the value of components.purl from SBOM json
And license column should be empty
And license name column should be empty
And license expression column should contain the value of whole license expression in a single row from components.license.expression field from SBOM json
And alternate package reference column should contain the value of CPE from components.cpe field from SBOM json

Scenario: Verify the license information for a package on the CycloneDX SBOM with multiple license ids
# Test data https://drive.google.com/drive/folders/1Z6y6gMegutBeUuc_8LkpYxKeGz_KYG9H?usp=drive_link
# sbom - cdx_sbom.json package - pkg:maven/jakarta.el/[email protected]?type=jar
Given User is on SBOM license information file
When User selects a package with multiple license sections
Then The report should have multiple rows for the same package corresponding to each license section
And Values on columns name, namespace, group, version, package reference, license name, license expression and alternate package reference should be same for all the rows
And Column license id for each row should contain the value from the components.license.id field of the corresponding license section

Scenario: Verify the license information for a package on the CycloneDX SBOM with multiple license names
# Test data https://drive.google.com/drive/folders/1Z6y6gMegutBeUuc_8LkpYxKeGz_KYG9H?usp=drive_link
# sbom - cdx_sbom.json package - pkg:maven/xpp3/[email protected]?type=jar
Given User is on SBOM license information file
When User selects a package with multiple license sections
Then The report should have multiple rows for the same package corresponding to each license section
And Values on columns name, namespace, group, version, package reference, license id, license expression and alternate package reference should be same for all the rows
And Column license name for each row should contain the value from the components.license.name field of the corresponding license section

Scenario: Verify the license information for a package on the CycloneDX SBOM with license id and license name
mrrajan marked this conversation as resolved.
Show resolved Hide resolved
# Test data https://drive.google.com/drive/folders/1Z6y6gMegutBeUuc_8LkpYxKeGz_KYG9H?usp=drive_link
# sbom - cdx_sbom.json package -pkg:maven/ch.qos.logback/[email protected]?type=jar
Given User is on SBOM license information file
When User selects a package with multiple license sections
Then The report should have multiple rows for the same package corresponding to each license section
And Values on columns name, namespace, group, version, package reference, license expression and alternate package reference should be same for all the rows
And Column license id should contain the value of components.license.id field on one row and on the same row license.name column should be empty
And Column license name should contain the value of the components.license.name on another row and on the same row license.id column should be empty

Scenario: Verify CycloneDX SBOM level license information on license export
# Test data https://drive.google.com/drive/folders/1Z6y6gMegutBeUuc_8LkpYxKeGz_KYG9H?usp=drive_link
# sbom - cdx_sbom.json
Given User is on SBOM license information file
Then name column should contain the value of SBOM name from metadata.component.name field from SBOM json
And namespace column should be empty
And group column should contain the value of metadata.component.group field from SBOM json
And version column should contain the value of metadata.component.version field from SBOM json
And package reference column should contain the value of metadata.component.purl from SBOM json
And license column should contain the value from metadata.component.licenses.license.id field of the SBOM json
And license name column should be empty
And license expression column should be empty
And alternate package reference column should be empty

Scenario: Verify the license information for a package on the CycloneDX SBOM with multiple licenses
Given User is on package license combination file
Given User is on SBOM license information file
When User selects a package with multiple license information
Then Package should have Rows equivalent to number of licenses
And All the package rows should be loaded with identical values for the columns name, namespace, group, version, package
And License column should be loaded with the unique licenses of the package from SBOM json

Scenario: Verify the contents on CycloneDX SBOM license reference CSV file
Given User is on license reference file
When User selects a license from the list of licenses
Then The License reference CSV should be empty
Loading