Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Race condition in creating OpenSearch indices #1474

Open
tballison opened this issue Feb 21, 2025 · 1 comment
Open

Race condition in creating OpenSearch indices #1474

tballison opened this issue Feb 21, 2025 · 1 comment
Assignees
@tballison
Milestone
3.3.0

Comments

@tballison
Copy link
Contributor

We should catch "index already exists" exceptions and log them. At startup, if there's more than one worker, there can be a "check then write" race condition on creating a new index. This causes the exception just mentioned.
@tballison tballison self-assigned this Feb 21, 2025
@rzo1 rzo1 added this to the 3.3.0 milestone Feb 21, 2025
@tballison
Copy link
Contributor Author 

at org.opensearch.client.IndicesClient.create(IndicesClient.java:159) ~[stormjar.jar:?]
at org.apache.stormcrawler.opensearch.IndexCreation.createIndex(IndexCreation.java:101) ~[stormjar.jar:?]
at org.apache.stormcrawler.opensearch.IndexCreation.checkOrCreateIndex(IndexCreation.java:43) ~[stormjar.jar:?]
at org.apache.stormcrawler.opensearch.persistence.StatusUpdaterBolt.prepare(StatusUpdaterBolt.java:194) ~[stormjar.jar:?]
at org.apache.storm.executor.bolt.BoltExecutor.init(BoltExecutor.java:128) ~[storm-client-2.7.0.jar:2.7.0]
at org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:138) ~[storm-client-2.7.0.jar:2.7.0]
at org.apache.storm.executor.bolt.BoltExecutor.call(BoltExecutor.java:54) ~[storm-client-2.7.0.jar:2.7.0]

tballison added a commit that referenced this issue Feb 21, 2025
@tballison
#1474 -- catch OpenSearchExceptions to prevent percolation of Runtime…
dac03b9 
…Exceptions, including check-then-update race condition in creating an index.
tballison added a commit that referenced this issue Feb 21, 2025
@tballison
#1474 -- code style
b184f2d
tballison added a commit that referenced this issue Feb 24, 2025
@tballison
#1474 -- add template+index names to logging
2e6a394
tballison added a commit that referenced this issue Feb 24, 2025
@tballison
#1474 -- catch OpenSearchExceptions to prevent percolation of Runtime… (
4c1b630 
#1476)

* #1474 -- catch OpenSearchExceptions to prevent percolation of RuntimeExceptions, including check-then-update race condition in creating an index.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tballison tballison

Labels
None yet
Projects
None yet
Milestone
3.3.0
Development

No branches or pull requests
2 participants
@tballison @rzo1