Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

postpone timezone regex evaluation until first use - shaves off time from package import #1181

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

beda42
Copy link

@beda42 beda42 commented Sep 7, 2023

This MR is related to issue #533 which is caused mainly by a time intensive parsing of regular expressions for timezone matching. This MR introduces a global object TzRegexCache and moves preparation of the regexps into it. The regexps are no longer parsed on startup, but during first use instead. On my machine it shaves off about 200 ms from the import time of dateparser thus reducing it to less than 20 % of the original import time.

nijel added a commit to nijel/weblate that referenced this pull request Oct 2, 2024
It can be expensive to import due to building thousands of regexps at
import time (see scrapinghub/dateparser#1181).
nijel added a commit to WeblateOrg/weblate that referenced this pull request Oct 2, 2024
It can be expensive to import due to building thousands of regexps at
import time (see scrapinghub/dateparser#1181).
@tobymao
Copy link

tobymao commented Jan 30, 2025

is this library still actively maintained? i ran into this issue and would like to fix this. my idea is instead to cache the regexes so that overall time gets faster. this solution will still incur regex parsing time, although it does speed up parsing.

but i only want to do a PR if it actually has a shot at making it in.

@Gallaecio
Copy link
Member

We are reviewing PRs, and may even have a release “soon” (see the most recent PRs).

@tobymao
Copy link

tobymao commented Jan 30, 2025

We are reviewing PRs, and may even have a release “soon” (see the most recent PRs).

amazing! would you accept my idea? is there somewhere to discuss with maintainers this approach before i start to make iterations faster?

@Gallaecio
Copy link
Member

You can either start a draft PR with little code and start the discussion there, or discuss it here.

tobymao added a commit to tobymao/dateparser that referenced this pull request Jan 30, 2025
this is different from pr scrapinghub#1181. that pr only makes import faster but
still incurs cost on the first usage. this one leverages an optional
cache.

closes scrapinghub#533
tobymao added a commit to tobymao/dateparser that referenced this pull request Jan 30, 2025
this is different from pr scrapinghub#1181. that pr only makes import faster but
still incurs cost on the first usage. this one leverages an optional
cache.

closes scrapinghub#533
@tobymao
Copy link

tobymao commented Jan 30, 2025

i've taken another stab at this @beda42 #1250

instead of lazy evaluation, i do caching so that we only pay the regex compile time cost once. it will break the cache if there's a version bump.

tobymao added a commit to tobymao/dateparser that referenced this pull request Jan 31, 2025
this is different from pr scrapinghub#1181. that pr only makes import faster but
still incurs cost on the first usage. this one leverages an optional
cache.

closes scrapinghub#533
tobymao added a commit to tobymao/dateparser that referenced this pull request Feb 5, 2025
this is different from pr scrapinghub#1181. it builds a cache at install time which
can be distributed.

closes scrapinghub#533
tobymao added a commit to tobymao/dateparser that referenced this pull request Feb 5, 2025
this is different from pr scrapinghub#1181. it builds a cache at install time which
can be distributed.

closes scrapinghub#533
tobymao added a commit to tobymao/dateparser that referenced this pull request Feb 5, 2025
this is different from pr scrapinghub#1181. it builds a cache at install time which
can be distributed.

closes scrapinghub#533
tobymao added a commit to tobymao/dateparser that referenced this pull request Feb 5, 2025
this is different from pr scrapinghub#1181. it builds a cache at install time which
can be distributed.

closes scrapinghub#533
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants