Add RDAP analyzer#3760
Open
thunderstornX wants to merge 2 commits into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds a new RDAP observable analyzer (and its registration) to query rdap.org for IP/domain/URL registration data, plus unit tests.
Changes:
- Introduces
Rdapobservable analyzer that calls rdap.org and normalizes 404 into{"found": False}. - Adds Django migration to register the analyzer configuration (
AnalyzerConfig) for RDAP. - Adds mocked unit tests covering IP/domain/URL behavior and error conditions.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
api_app/analyzers_manager/observable_analyzers/rdap.py |
Implements the RDAP observable analyzer logic and error handling. |
api_app/analyzers_manager/migrations/0193_analyzer_config_rdap.py |
Registers the new analyzer/config via data migration. |
tests/api_app/analyzers_manager/observable_analyzers/test_rdap.py |
Adds unit tests that mock HTTP calls and validate endpoint selection/results. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+103
to
+104
| python_path = plugin.pop("model") | ||
| Model = apps.get_model(*python_path.split(".")) |
Comment on lines
+115
to
+116
| python_path = plugin.pop("model") | ||
| Model = apps.get_model(*python_path.split(".")) |
Comment on lines
+26
to
+27
| def update(self) -> bool: | ||
| pass |
Comment on lines
+56
to
+57
| except requests.RequestException as e: | ||
| raise AnalyzerRunException(e) |
Author
|
Proof of a working analysis (per the new-analyzer checklist). 
Ran the RDAP analyzer on I also added the analyzer to the Raw JSON of the finished analysis (observable
|
|
This pull request has been marked as stale because it has had no activity for 10 days. If you are still working on this, please provide some updates or it will be closed in 5 days. |
Member
|
hey, thanks for contributing. Please address copilot requests and pull again from develop ( and adjust migration numbers) |
thunderstornX
force-pushed
the
analyzer-rdap
branch
from
b35560a to
d58804a
Compare
Author
|
Thanks for the review! Addressed the Copilot points: the migrations now pop |
Comment on lines
+35
to
+39
| elif self.observable_classification == Classification.URL: | ||
| hostname = urlparse(self.observable_name).hostname | ||
| if not hostname: | ||
| raise AnalyzerRunException(f"unable to extract a hostname from URL {self.observable_name}") | ||
| path = f"domain/{hostname}" |
Comment on lines
+60
to
+62
| result = response.json() | ||
| result["found"] = True | ||
| return result |
Comment on lines
+43
to
+48
| @patch("api_app.analyzers_manager.observable_analyzers.rdap.requests.get") | ||
| def test_url_resolves_to_host_domain(self, mock_get): | ||
| mock_get.return_value = self._ok({"objectClassName": "domain"}) | ||
| self._analyzer("https://sub.example.com/path?q=1", Classification.URL).run() | ||
| self.assertIn("rdap.org/domain/sub.example.com", mock_get.call_args.args[0]) | ||
|
|
thunderstornX
force-pushed
the
analyzer-rdap
branch
from
d58804a to
6d726d8
Compare
Author
|
Good catches, thanks. Added IP-aware routing so URL observables whose host is an IP (e.g. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an observable analyzer that queries the public RDAP bootstrap (https://rdap.org) for registration data of IPs, domains, and URLs. Closes #3759.
RDAP (RFC 9082 / 9083) is the IETF-standard, free, unauthenticated successor to WHOIS. The existing WHOIS analyzers are either paid (
whoisxmlapi) or registry-specific (whoisripe), so a free, global lookup fills a gap.ip,domain, andurl(resolved to its host) observables;maximum_tlpAMBER.{"found": false}rather than an error.Proof it works, against live
rdap.org(the analyzer follows the bootstrap redirect viarequests):tests/api_app/analyzers_manager/observable_analyzers/test_rdap.pymocks the HTTP layer and covers IP/domain/URL routing, the 404 path, and the unsupported-type and no-hostname error paths.