Skip to content

Connectus split packs routing - #5483

Open
israelpoli wants to merge 28 commits into
masterfrom
connectus-split-packs-routing
Open

Connectus split packs routing#5483
israelpoli wants to merge 28 commits into
masterfrom
connectus-split-packs-routing

Conversation

@israelpoli

Copy link
Copy Markdown
Contributor

TEST

@israelpoli
israelpoli requested a review from a team as a code owner August 9, 2026 14:19
@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown

🚫 SDK Nightly required

This PR modifies files that require the SDK Nightly pipeline to be run before it can be merged:

  • demisto_sdk/commands/content_graph/interface/neo4j/queries/isolation.py
  • demisto_sdk/commands/content_graph/strict_objects/release_notes_config_test.py
  • demisto_sdk/commands/validate/validators/GR_validators/GR116_cross_destination_dependency.py
  • demisto_sdk/commands/validate/validators/GR_validators/GR116_cross_destination_dependency_all_files.py
  • demisto_sdk/commands/validate/validators/GR_validators/GR116_cross_destination_dependency_list_files.py
  • demisto_sdk/commands/content_graph/commands/common.py
  • demisto_sdk/commands/content_graph/commands/create.py
  • demisto_sdk/commands/content_graph/commands/update.py
  • demisto_sdk/commands/content_graph/common.py
  • demisto_sdk/commands/content_graph/interface/graph.py
  • demisto_sdk/commands/content_graph/interface/neo4j/neo4j_graph.py
  • demisto_sdk/commands/content_graph/interface/neo4j/queries/common.py
  • demisto_sdk/commands/content_graph/interface/neo4j/queries/dependencies.py
  • demisto_sdk/commands/content_graph/interface/neo4j/queries/nodes.py
  • demisto_sdk/commands/content_graph/interface/neo4j/queries/relationships.py
  • demisto_sdk/commands/content_graph/objects/pack.py
  • demisto_sdk/commands/content_graph/objects/pack_metadata.py
  • demisto_sdk/commands/content_graph/objects/repository.py
  • demisto_sdk/commands/content_graph/parsers/pack.py
  • demisto_sdk/commands/content_graph/parsers/repository.py
  • demisto_sdk/commands/content_graph/strict_objects/pack_meta_data.py
  • demisto_sdk/commands/content_graph/strict_objects/release_notes_config.py

Required action:

  1. Run the SDK Nightly pipeline against this branch.
  2. Add a link to the nightly run in this PR's description.
  3. Add the nightly-run-passed label once the run has passed (or nightly-run-skipped if you have a documented reason not to run it, with reviewer approval).
Alternative: run a Content build instead of the full SDK Nightly (click to expand)

If your change is scoped (e.g. a single new validator or a small bug fix), you can run a Content build against this SDK branch instead of the full SDK Nightly pipeline. When your change is a new validator, make sure it is registered in the Content repo's validation_config.toml so the Content build's run-validations job picks it up via demisto-sdk validate -a (the -a "all files" mode is what run-validations uses; a -g "git-diff" run will not exercise your new validator on unchanged files). Once the Content build is green, add the nightly-run-passed label to satisfy this gate.

This check will re-run automatically when a label is added or removed.

Comment thread demisto_sdk/commands/content_graph/objects/repository.py Fixed
Comment thread demisto_sdk/commands/content_graph/tests/split_packs_test.py Fixed
Comment thread demisto_sdk/commands/content_graph/strict_objects/pack_meta_data.py Fixed
Derived (split) packs were assigned source = <original pack name>, which is
the Managed Content feature directory:
<bucket>/<bucket_path>/<source>/<pack_id>/. A derived Gmail pack therefore
landed under the "Gmail" feature instead of "connectus".

Introduce resolve_derived_pack_source() as the single source of truth, with
precedence: per-pack derived_source in pack_metadata.json > DERIVED_PACK_SOURCE
env var > DEFAULT_DERIVED_PACK_SOURCE ("connectus"). The env var is read per
call rather than at import (unlike ENABLE_SPLIT_PACKS) so it stays testable and
settable by CI.

derived_source is declared in both metadata models because StrictPackMetadata
sets extra = Extra.forbid, which would otherwise reject the field outright.

The link back to the originating pack is unaffected; it is carried by
derived_from.
Comment thread demisto_sdk/commands/content_graph/tests/split_packs_test.py Fixed
@github-actions

Copy link
Copy Markdown

Changelog(s) in markdown:

  • Added a managed_pack_id field to every entry written by ContentDTO.write_pack_destinations. The value is taken from the pack in the content graph, falls back to an optional caller-supplied pack id to managed pack id mapping, and is null when the pack has no managed counterpart. #5483

@coveralls

coveralls commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Coverage Report for CI Build 32958811761

Warning

Build has drifted: This PR's base is out of sync with its target branch, so coverage data may include unrelated changes.
Quick fix: rebase this PR. Learn more →

Warning

No base build found for commit 8c9574e on master.
Coverage changes can't be calculated without a base build.
If a base build is processing, this comment will update automatically when it completes.

Coverage: 86.693%

Details

  • Patch coverage: 37 uncovered changes across 9 files (448 of 485 lines covered, 92.37%).

Uncovered Changes

File Changed Covered %
demisto_sdk/commands/content_graph/parsers/pack.py 129 118 91.47%
demisto_sdk/commands/validate/validators/GR_validators/GR116_cross_destination_dependency.py 25 15 60.0%
demisto_sdk/commands/prepare_content/prepare_content_setup.py 7 1 14.29%
demisto_sdk/commands/content_graph/objects/pack.py 43 38 88.37%
demisto_sdk/commands/content_graph/interface/neo4j/queries/dependencies.py 24 23 95.83%
demisto_sdk/commands/content_graph/objects/repository.py 42 41 97.62%
demisto_sdk/commands/content_graph/parsers/repository.py 5 4 80.0%
demisto_sdk/commands/validate/validators/GR_validators/GR116_cross_destination_dependency_all_files.py 11 10 90.91%
demisto_sdk/commands/validate/validators/GR_validators/GR116_cross_destination_dependency_list_files.py 11 10 90.91%
Total (25 files) 485 448 92.37%

Coverage Regressions

Requires a base build to compare against. How to fix this →


Coverage Stats

Coverage Status
Relevant Lines: 63160
Covered Lines: 54755
Line Coverage: 86.69%
Coverage Strength: 1.73 hits per line

💛 - Coveralls

…rectory

write_pack_destinations now accepts the artifacts and managed artifacts directories used by ContentDTO.dump(), so each recorded artifact_path points at the directory packs are actually dumped into instead of the directory holding the JSON file. Managed packs are routed to the managed artifacts directory, and the final path segment is delegated to the same _artifact_path() helper dump() uses.
@github-actions

Copy link
Copy Markdown

Changelog(s) in markdown:

  • Added a managed_pack_id field to every entry written by ContentDTO.write_pack_destinations. The value is taken from the pack in the content graph, falls back to an optional caller-supplied pack id to managed pack id mapping, and is null when the pack has no managed counterpart. #5483
  • Fixed an issue where pack_destinations.json recorded an incorrect artifact_path, derived from the location of the JSON file rather than from the directory the packs are actually dumped into. #5483

source_platform: Optional[str] = Field(None, alias="source:platform")

@root_validator
def validate_coupling_overrides(cls, values):
Comment thread demisto_sdk/commands/content_graph/objects/repository.py Fixed
The ``(source_pack_id, target_pack_id)`` pairs of the pack-level
dependencies that were deleted.
"""
...
def obtain_invalid_content_items_using_graph(
self, content_items: Iterable[ContentTypes], validate_all_files: bool
) -> List[ValidationResult]:
content_id_to_objects = {item.object_id: item for item in content_items}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants