Refactor/date limit soc feedback command #38769

tomaskukk · 2025-02-26T08:36:18Z

Contributing to Cortex XSOAR Content

Make sure to register your contribution by filling the contribution registration form

The Pull Request will be reviewed only after the contribution registration form is filled.

Status

In Progress
Ready
In Hold - (Reason for hold)

Related Issues

fixes: No issue

Description

Added date parsing for the threat_feedback_reported_at_limit field which is used as an argument in a command. This allows users to pass the value in user-friendly formats.

Must have

Tests
Documentation

content-bot · 2025-02-26T08:38:00Z

Thank you for your contribution. Your generosity and caring are unrivaled! Make sure to register your contribution by filling the Contribution Registration form, so our content wizard @sapirshuker will know the proposed changes are ready to be reviewed.
For your convenience, here is a link to the contributions SLAs document.

content-bot · 2025-02-26T08:38:02Z

Hi @tomaskukk, thanks for contributing to the XSOAR marketplace. To receive credit for your generous contribution please follow this link.

sapirshuker

Hi @tomaskukk ,
Thank you for your contribution!
Good work :)
Please see my suggestions.
In addition please fix the pre-commit issues.
Please feel free to reach out to me with any questions - I'm available here or on slack :)
Thanks again

sapirshuker · 2025-02-27T12:37:23Z

Packs/Hoxhunt/ReleaseNotes/2_0_1.md

+
+##### Hoxhunt v2
+
+- Added support for more flexible date formats in the `threat_feedback_reported_at_limit` argument.


Suggested change

- Added support for more flexible date formats in the `threat_feedback_reported_at_limit` argument.

- Added support for more flexible date formats in the *threat_feedback_reported_at_limit* argument in the ***hoxhunt-incident-soc-feedback-send*** command.

sapirshuker · 2025-02-27T13:23:04Z

Packs/Hoxhunt/Integrations/HoxhuntV2/HoxhuntV2.py

@@ -457,6 +457,10 @@ def camel_to_title(text: str) -> str:
        result.append(char)
    return ''.join(result).strip().title()

+# Hoxhunt GQL API expects dates in ISO format
+def format_date_to_iso_string(date: datetime | None) -> str:


You have the same functionality in the CommonServerPython timestamp_to_datestring and date_to_timestamp
These methods already exist, you can make use of them.

The date_to_timestamp returns the date as int (so probably epoch unix timestamp), and timestamp_to_datestring takes in the date as int or str, so I would need to parse the datetime to int / str first.

Would using the built in strftime be okay here so no need to do extra conversions?

According to the docs, timestamp_to_datestring parse timestamp (milliseconds) to a date string and takes str or int, str such as "1541494441222" and int such as 1541494441222
So you need to use date_to_timestamp in order to convert the date to milliseconds and timestamp_to_datestring in order to convert the milliseconds to iso format. Another way is to use date.isoformat()

tomaskukk · 2025-02-27T16:24:06Z

Hey @sapirshuker, I fixed the documentation suggestion and commented on the other comment. Pre-commit is passing now (the validation passed locally without issues but raised issues on CI, even with newest demisto-sdk version 🤔).

Should be good to be reviewed again!

content-bot · 2025-03-02T08:28:10Z

For the Reviewer: Trigger build request has been accepted for this contribution PR.

content-bot · 2025-03-02T08:28:16Z

For the Reviewer: Successfully created a pipeline in GitLab with url: https://gitlab.xdr.pan.local/xdr/cortex-content/content/-/pipelines/2604160

sapirshuker · 2025-03-02T15:30:49Z

Packs/Hoxhunt/Integrations/HoxhuntV2/HoxhuntV2.yml

@@ -232,3 +232,6 @@ tests:
 defaultclassifier: Hoxhunt-classifier
 defaultmapperin: Hoxhunt-mapper-incoming
 defaultmapperout: Hoxhunt-mapper-outgoing
+sectionOrder:
+- Connect


Please add the section header to the to the beginning of the file

Sure! (this was formatting work of demisto-sdk format)

CLAassistant · 2025-03-03T06:30:44Z

All committers have signed the CLA.

sapirshuker

Hi, thank you for your contribution! Did you notice the comment I wrote in response to your question? If not, I will write it again regarding the use of existing functions to convert the date to the requested format. Thank you! Waiting for your response.

sapirshuker · 2025-03-03T09:16:02Z

Packs/Hoxhunt/Integrations/HoxhuntV2/HoxhuntV2.py

+
+
+def format_date_to_iso_string(date: datetime | None) -> str:
+    return date.strftime('%Y-%m-%dT%H:%M:%S.%f')[:-3] + 'Z'  # type: ignore


According to the docs, timestamp_to_datestring parse timestamp (milliseconds) to a date string and takes str or int, str such as "1541494441222" and int such as 1541494441222
So you need to use date_to_timestamp in order to convert the date to milliseconds and timestamp_to_datestring in order to convert the milliseconds to iso format. Another way is to use date.isoformat()

Sorry I missed this comment completely earlier. I will fix it now!

Done, used the date.isoformat!

content-bot · 2025-03-03T15:40:11Z

For the Reviewer: Trigger build request has been accepted for this contribution PR.

content-bot · 2025-03-03T15:40:16Z

For the Reviewer: Successfully created a pipeline in GitLab with url: https://gitlab.xdr.pan.local/xdr/cortex-content/content/-/pipelines/2625584

content-bot · 2025-03-03T15:40:18Z

For the Reviewer: Trigger build request has been accepted for this contribution PR.

content-bot · 2025-03-03T15:40:27Z

For the Reviewer: Successfully created a pipeline in GitLab with url: https://gitlab.xdr.pan.local/xdr/cortex-content/content/-/pipelines/2625588

sapirshuker

Hi @tomaskukk , thank you very much for the change, the contribution, and your patience. Working with dates can be frustrating and there are edge cases that need to be considered. While the contribution is short, we need to ensure that we are not causing bugs. We can schedule a meeting together to think about the solution together.

sapirshuker · 2025-03-04T10:05:27Z

Packs/Hoxhunt/Integrations/HoxhuntV2/HoxhuntV2.yml

@@ -153,7 +156,7 @@ script:
      type: unknown
    - name: threat_feedback_reported_at_limit
      required: true
-      description: Datetime limit as a iso string (e.g. "2024-10-30T08:37:42.359Z").
+      description: Datetime limit. Accepts (<number> <time unit>), e.g "7 days", "one month" or a iso string (e.g. "2024-10-30T08:37:42.359Z").


We need to add this change to the readme file.

Good point!

sapirshuker · 2025-03-04T10:09:02Z

Packs/Hoxhunt/Integrations/HoxhuntV2/HoxhuntV2.py

@@ -639,7 +645,7 @@ def get_modified_remote_data_command(client: Client, args: dict, params: dict):
    remote_args = GetModifiedRemoteDataArgs(args)
    last_update = remote_args.last_update
    last_update_parsed = dateparser.parse(last_update, settings={'TIMEZONE': 'UTC'})
-    last_update_utc = last_update_parsed.strftime('%Y-%m-%dT%H:%M:%S.%f')[:-3] + 'Z'  # type: ignore


Let's only change what's necessary for the command you want to modify (hoxhunt-incident-soc-feedback-send) to avoid from unwanted impacting on users.

sapirshuker · 2025-03-04T10:09:07Z

Packs/Hoxhunt/Integrations/HoxhuntV2/HoxhuntV2.py

@@ -577,7 +583,7 @@ def get_remote_data_command(client: Client, args: dict, params: dict):
    incident_id = parsed_args.remote_incident_id
    last_update = parsed_args.last_update
    last_update_parsed = dateparser.parse(last_update, settings={'TIMEZONE': 'UTC'})
-    last_update_utc = last_update_parsed.strftime('%Y-%m-%dT%H:%M:%S.%f')[:-3] + 'Z'  # type: ignore


Let's only change what's necessary for the command you want to modify (hoxhunt-incident-soc-feedback-send) to avoid from unwanted impacting on users.

sapirshuker · 2025-03-04T10:09:16Z

Packs/Hoxhunt/Integrations/HoxhuntV2/HoxhuntV2.py

@@ -502,7 +508,7 @@ def fetch_incidents(

    max_fetch_parsed = min(int(max_fetch), 100)

-    timefrom = first_fetch_parsed.strftime('%Y-%m-%dT%H:%M:%S.%f')[:-3] + 'Z'


Let's only change what's necessary for the command you want to modify (hoxhunt-incident-soc-feedback-send) to avoid from unwanted impacting on users.

sapirshuker · 2025-03-04T10:29:46Z

Packs/Hoxhunt/Integrations/HoxhuntV2/HoxhuntV2.py

@@ -742,7 +748,10 @@ def hoxhunt_send_incident_soc_feedback_command(client: Client, args: dict, param
    incident_id = args.get('incident_id')
    custom_message = args.get('custom_message')
    threat_feedback_reported_at_limit = args.get('threat_feedback_reported_at_limit')
-    response = client.send_incident_soc_feedback(incident_id, custom_message, threat_feedback_reported_at_limit)
+    threat_feedback_reported_at_limit_parsed = dateparser.parse(threat_feedback_reported_at_limit)  # type: ignore
+    date_limit = format_date_to_iso_string(threat_feedback_reported_at_limit_parsed)


I tested your use case with the following code:

date_utc=dateparser.parse("2024-10-30T08:37:42.359Z") (as mentioned in the yml) print(date_utc) >> 2024-10-30 08:37:42.359000+00:00 print(date_utc.isoformat(timespec='milliseconds')) >> 2024-10-30T08:37:42.359+00:00 print(date_utc.isoformat(timespec='milliseconds')+'Z') >> 2024-10-30T08:37:42.359+00:00Z

I don't think this is the wanted result, Please let me know if I'm wrong.
In addition when executing:
date_utc=dateparser.parse("7 days")
I get the result according to my timezone and you add "Z" to the output from "format_date_to_iso_string"
method meaning UTC so the the conversion is incorrect (What was also present before the change you made in the function format_date_to_iso_string in order to use the isoformat function).
Maybe let's figure out together what the use case and what we should do.
We can try and see if using the function arg_to_datetime can solve the problem for us.

Good catch, it is not indeed the result we want. Seems like the unit tests didn't catch this, which I was relying in. I wonder if it would just be safest to use the custom formatting function that we already had in place, as in the original version of the pull request, as it's proven to work? Is there a specific reason we can't use that, which is proven at this point.

I did not go with the date_to_timestamp and timestamp_to_datestring functions because they lost milliseconds in the process.

@tomaskukk
I saw your original solution but as I wrote, when you are executing:
date_utc=dateparser.parse("7 days")
You get the time in the current time zone, for example my timezone is GMT+2 the time now is 2025-03-05 13:41:10.382902
When executing date_utc=dateparser.parse("7 days") we get 2025-02-25 13:41:10.382902
Then print(date_utc.strftime('%Y-%m-%dT%H:%M:%S.%f')[:-3] + 'Z') as your original version you get >> 2025-02-25T13:41:10.382Z
mean the same time in UTC, I don't think this is the wanted result, Please let me know if I'm wrong.

date_7_days=dateparser.parse("7 Days") print(date_7_days) >>2025-02-25 13:41:10.382902 print(date_7_days.strftime('%Y-%m-%dT%H:%M:%S.%f')[:-3] + 'Z') >> 2025-02-25T13:41:10.382Z

Hey @tomaskukk,
We have additional function from common server python named arg_to_datetime
Please check if this meets your need.

Yeah that makes sense, I didn't realize in the beginning that the motivation of the comment was the timezone related issue 👍

I'll see and fix it (alongside the other comments)!

tomaskukk added 3 commits February 26, 2025 10:12

refactor date limit to be more flexible

1670a43

update release notes

212de4d

add test cases for new function

98fa46d

content-bot added Contribution Thank you! Contributions are always welcome! External PR Partner Support Level Indicates that the contribution is for Partner supported pack labels Feb 26, 2025

content-bot changed the base branch from master to contrib/hoxhunt_refactor/date-limit-soc-feedback-command February 26, 2025 08:37

content-bot assigned sapirshuker Feb 26, 2025

content-bot requested a review from sapirshuker February 26, 2025 08:37

content-bot assigned edik24 Feb 26, 2025

content-bot added Partner-Approved Contribution Form Filled Whether contribution form filled or not. Partner labels Feb 26, 2025

sapirshuker reviewed Feb 27, 2025

View reviewed changes

sapirshuker added the pending-contributor The PR is pending the response of its creator label Feb 27, 2025

tomaskukk added 4 commits February 27, 2025 17:49

refactor release notes to be more specific

fdd9d6e

format code with pre-commit script

7e96468

update .yml configuration file according to CI pre-commit

1aa4bd0

update release notes with docker image update

a033eaa

tomaskukk requested a review from sapirshuker February 27, 2025 16:24

sapirshuker added the ready-for-instance-test In contribution PRs, this label will cause a trigger of a build with a modified pack from the PR. label Mar 2, 2025

content-bot removed the ready-for-instance-test In contribution PRs, this label will cause a trigger of a build with a modified pack from the PR. label Mar 2, 2025

sapirshuker reviewed Mar 2, 2025

View reviewed changes

move section order to top of yml

4dbd9a7

tomaskukk requested a review from sapirshuker March 3, 2025 06:33

sapirshuker requested changes Mar 3, 2025

View reviewed changes

use built in isoformat method of date to convert to iso string

7dc49a4

tomaskukk requested a review from sapirshuker March 3, 2025 13:30

sapirshuker added the ready-for-instance-test In contribution PRs, this label will cause a trigger of a build with a modified pack from the PR. label Mar 3, 2025

content-bot removed the ready-for-instance-test In contribution PRs, this label will cause a trigger of a build with a modified pack from the PR. label Mar 3, 2025

sapirshuker requested changes Mar 4, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor/date limit soc feedback command #38769

Refactor/date limit soc feedback command #38769

tomaskukk commented Feb 26, 2025 •

edited

Loading

content-bot commented Feb 26, 2025

content-bot commented Feb 26, 2025

sapirshuker left a comment •

edited

Loading

sapirshuker Feb 27, 2025

sapirshuker Feb 27, 2025

tomaskukk Feb 27, 2025 •

edited

Loading

sapirshuker Mar 2, 2025 •

edited

Loading

tomaskukk commented Feb 27, 2025

content-bot commented Mar 2, 2025

content-bot commented Mar 2, 2025

sapirshuker Mar 2, 2025

tomaskukk Mar 3, 2025

CLAassistant commented Mar 3, 2025 •

edited

Loading

sapirshuker left a comment

sapirshuker Mar 3, 2025

tomaskukk Mar 3, 2025

tomaskukk Mar 3, 2025

content-bot commented Mar 3, 2025

content-bot commented Mar 3, 2025

content-bot commented Mar 3, 2025

content-bot commented Mar 3, 2025

sapirshuker left a comment

sapirshuker Mar 4, 2025

tomaskukk Mar 4, 2025

sapirshuker Mar 4, 2025

sapirshuker Mar 4, 2025

sapirshuker Mar 4, 2025

sapirshuker Mar 4, 2025 •

edited

Loading

tomaskukk Mar 4, 2025

sapirshuker Mar 4, 2025 •

edited

Loading

sapirshuker Mar 4, 2025

tomaskukk Mar 4, 2025


		##### Hoxhunt v2

		- Added support for more flexible date formats in the `threat_feedback_reported_at_limit` argument.



		def format_date_to_iso_string(date: datetime \| None) -> str:
		return date.strftime('%Y-%m-%dT%H:%M:%S.%f')[:-3] + 'Z' # type: ignore

		@@ -502,7 +508,7 @@ def fetch_incidents(

		max_fetch_parsed = min(int(max_fetch), 100)

		timefrom = first_fetch_parsed.strftime('%Y-%m-%dT%H:%M:%S.%f')[:-3] + 'Z'

Refactor/date limit soc feedback command #38769

Are you sure you want to change the base?

Refactor/date limit soc feedback command #38769

Conversation

tomaskukk commented Feb 26, 2025 • edited Loading

Contributing to Cortex XSOAR Content

Status

Related Issues

Description

Must have

content-bot commented Feb 26, 2025

content-bot commented Feb 26, 2025

sapirshuker left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tomaskukk Feb 27, 2025 • edited Loading

Choose a reason for hiding this comment

sapirshuker Mar 2, 2025 • edited Loading

Choose a reason for hiding this comment

tomaskukk commented Feb 27, 2025

content-bot commented Mar 2, 2025

content-bot commented Mar 2, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

CLAassistant commented Mar 3, 2025 • edited Loading

sapirshuker left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

content-bot commented Mar 3, 2025

content-bot commented Mar 3, 2025

content-bot commented Mar 3, 2025

content-bot commented Mar 3, 2025

sapirshuker left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sapirshuker Mar 4, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sapirshuker Mar 4, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tomaskukk commented Feb 26, 2025 •

edited

Loading

sapirshuker left a comment •

edited

Loading

tomaskukk Feb 27, 2025 •

edited

Loading

sapirshuker Mar 2, 2025 •

edited

Loading

CLAassistant commented Mar 3, 2025 •

edited

Loading

sapirshuker Mar 4, 2025 •

edited

Loading

sapirshuker Mar 4, 2025 •

edited

Loading