Skip to content

Commit bc6cdf7

Browse files
Add KonfluxSource backend [ROK-1347] (#692)
This commit adds a new 'KonfluxSource' backend that loads RPM advisory metadata from local JSON files rather than querying remote ET APIs. This is one component required for delivery of RPMs built in Konflux. Note: Future modifications will likely be required as additional functionality is implemented. For example, the 'src' attribute on RpmPushItem (currently set to None) may need to be modified to support "upload from URL" functionality when that is added in the future. Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
1 parent 0cf44d1 commit bc6cdf7

13 files changed

Lines changed: 1136 additions & 0 deletions

File tree

docs/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ A library for accessing push items from various sources.
1313
sources/koji
1414
sources/registry
1515
sources/errata
16+
sources/konflux
1617
sources/pub
1718
model/base
1819
model/files

docs/sources/konflux.rst

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
Source: konflux
2+
================
3+
4+
The ``konflux`` push source allows the loading of content from local JSON files
5+
organized by advisory. This source is designed for use with Konflux-generated
6+
advisory metadata and does not require network access or external API calls.
7+
8+
Supported content types:
9+
10+
* RPMs
11+
* Advisories
12+
13+
The source is designed to be extensible and can support additional content types
14+
(such as modules, container images, etc.) in the future as needed.
15+
16+
konflux source URLs
17+
-------------------
18+
19+
The base form of a konflux source URL is:
20+
21+
``konflux:base-directory?advisories=RHXA-XXXX:0001[,RHXA-XXXX:0002[,...]]``
22+
23+
For example, referencing a single advisory would look like:
24+
25+
``konflux:/path/to/konflux/data?advisories=RHSA-2020:0509``
26+
27+
Multiple advisories can be specified with a comma-separated list:
28+
29+
``konflux:/path/to/konflux/data?advisories=RHSA-2020:0509,RHSA-2020:0510``
30+
31+
The base directory should contain subdirectories named after each advisory ID.
32+
Each advisory subdirectory must contain:
33+
34+
* ``advisory_cdn_metadata.json`` - Advisory metadata (title, severity, references, packages, etc.)
35+
* ``advisory_cdn_filelist.json`` - RPM file list with checksums, signing keys, and repository destinations
36+
37+
Directory structure
38+
...................
39+
40+
Example directory structure::
41+
42+
/path/to/konflux/data/
43+
├── RHSA-2020:0509/
44+
│ ├── advisory_cdn_metadata.json
45+
│ └── advisory_cdn_filelist.json
46+
└── RHSA-2020:0510/
47+
├── advisory_cdn_metadata.json
48+
└── advisory_cdn_filelist.json
49+
50+
File format
51+
...........
52+
53+
**advisory_cdn_metadata.json**
54+
55+
This file contains advisory metadata in the standard Errata Tool format, including:
56+
57+
* Advisory ID, title, description, severity
58+
* Package list with checksums
59+
* References (CVEs, Bugzilla links, etc.)
60+
* Release information
61+
62+
**advisory_cdn_filelist.json**
63+
64+
This file contains build and RPM information::
65+
66+
{
67+
"build-nvr": {
68+
"rpms": {
69+
"rpm-filename.rpm": ["repo1", "repo2", ...]
70+
},
71+
"checksums": {
72+
"md5": {
73+
"rpm-filename.rpm": "checksum-value"
74+
},
75+
"sha256": {
76+
"rpm-filename.rpm": "checksum-value"
77+
}
78+
},
79+
"sig_key": "signing-key-id"
80+
}
81+
}
82+
83+
Differences from `ErrataSource`
84+
...............................
85+
86+
Unlike the `ErrataSource`, the `KonfluxSource`:
87+
88+
* Reads from local JSON files rather than querying the Errata API
89+
* Does not require Koji integration
90+
* Does not currently support filtering by architecture (this use case may be supported in the future)
91+
* Currently produces RPMs and advisories (additional content types such as modules and container images can be supported in the future)
92+
* RPM push items have ``src=None`` (no local RPM files, only metadata)
93+
94+
Python API reference
95+
--------------------
96+
97+
.. autoclass:: pushsource.KonfluxSource
98+
:members:
99+
:special-members: __init__

docs/userguide.rst

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -91,6 +91,9 @@ For detailed information, see the API reference of the associated class.
9191
| errata | ``errata:https://errata.example.com?errata=RHBA-2020:1234`` | :class:`~pushsource.ErrataSource` | Obtain RPMs, container images and advisory |
9292
| | | | metadata from Errata Tool |
9393
+--------------+-----------------------------------------------------------------------------+-------------------------------------+----------------------------------------------------+
94+
| konflux | ``konflux:/path/to/data?advisories=RHSA-2020:0509`` | :class:`~pushsource.KonfluxSource` | Obtain RPMs and advisory metadata from local |
95+
| | | | JSON files organized by advisory |
96+
+--------------+-----------------------------------------------------------------------------+-------------------------------------+----------------------------------------------------+
9497
| file | ``file:/tmp/file-to-push`` | n/a | Obtain a single file-backed item of various types. |
9598
+--------------+-----------------------------------------------------------------------------+ | |
9699
| cgw | ``cgw:/tmp/yaml-file-to-push`` | | |

src/pushsource/__init__.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,7 @@
3838
from pushsource._impl.backend import (
3939
ErrataSource,
4040
KojiSource,
41+
KonfluxSource,
4142
StagedSource,
4243
RegistrySource,
4344
PubSource,

src/pushsource/_impl/backend/__init__.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
from .errata_source import ErrataSource
22
from .koji_source import KojiSource
3+
from .konflux_source import KonfluxSource
34
from .staged import StagedSource
45
from .registry_source import RegistrySource
56
from .direct import DirectSource
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
from .konflux_source import KonfluxSource
2+
3+
__all__ = ["KonfluxSource"]
Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
import json
2+
import logging
3+
import os
4+
5+
from ...compat_attr import attr
6+
7+
LOG = logging.getLogger("pushsource.konflux")
8+
9+
10+
@attr.s()
11+
class KonfluxAdvisoryData(object):
12+
"""Stores advisory data loaded from JSON files."""
13+
14+
advisory_id = attr.ib(type=str)
15+
metadata = attr.ib(type=dict) # advisory_cdn_metadata content
16+
filelist = attr.ib(type=dict) # advisory_cdn_filelist content
17+
18+
19+
class KonfluxLoader(object):
20+
"""Loads advisory data from local JSON files."""
21+
22+
def __init__(self, base_dir):
23+
"""Initialize loader with base directory.
24+
25+
Parameters:
26+
base_dir (str):
27+
Base directory containing advisory subdirectories.
28+
"""
29+
self._base_dir = base_dir
30+
31+
def load_advisory_data(self, advisory_id):
32+
"""Load both advisory_cdn_metadata and advisory_cdn_filelist.
33+
34+
Parameters:
35+
advisory_id (str):
36+
Advisory ID (e.g., "RHSA-2020-0509")
37+
38+
Returns:
39+
KonfluxAdvisoryData: Named tuple with metadata and filelist
40+
41+
Raises:
42+
FileNotFoundError: If JSON files don't exist
43+
ValueError: If JSON is invalid or malformed
44+
"""
45+
advisory_dir = os.path.join(self._base_dir, advisory_id)
46+
47+
LOG.debug("Loading advisory data for %s from %s", advisory_id, advisory_dir)
48+
49+
metadata = self._load_json(
50+
os.path.join(advisory_dir, "advisory_cdn_metadata.json")
51+
)
52+
filelist = self._load_json(
53+
os.path.join(advisory_dir, "advisory_cdn_filelist.json")
54+
)
55+
56+
LOG.info("Successfully loaded advisory data for %s", advisory_id)
57+
58+
return KonfluxAdvisoryData(
59+
advisory_id=advisory_id, metadata=metadata, filelist=filelist
60+
)
61+
62+
def _load_json(self, filepath):
63+
"""Load and parse a JSON file with error handling.
64+
65+
Parameters:
66+
filepath (str):
67+
Path to JSON file to load
68+
69+
Returns:
70+
dict: Parsed JSON data
71+
72+
Raises:
73+
FileNotFoundError: If file doesn't exist
74+
ValueError: If JSON is invalid
75+
"""
76+
if not os.path.exists(filepath):
77+
raise FileNotFoundError("Required file not found: %s" % filepath)
78+
79+
LOG.debug("Loading JSON file: %s", filepath)
80+
81+
with open(filepath, "r") as f:
82+
try:
83+
return json.load(f)
84+
except json.JSONDecodeError as e:
85+
raise ValueError("Invalid JSON in %s: %s" % (filepath, str(e))) from e

0 commit comments

Comments
 (0)