Implemented computation of segment replication stats at shard level #17055
Conversation
The method implemented here computes the segment replication stats at the shard level, instead of relying on the primary shard to compute stats based on reports from its replicas. Signed-off-by: Vinay Krishna Pudyodu <[email protected]>
vinaykpud
requested review from
anasalkouz,
andrross,
ashking94,
Bukhtawar,
CEHENKLE,
dblock,
dbwiddis,
gbbafna,
jed326,
kotwanikunal,
mch2,
msfroh,
nknize,
owaiskazi19,
reta,
Rishikesh1159,
sachinpkale,
saratvemulapalli,
shwetathareja,
sohami,
VachaShah,
jainankitk and
linuxpi
as code owners
github-actions
bot
added
enhancement
Signed-off-by: Vinay Krishna Pudyodu <[email protected]>
Signed-off-by: Vinay Krishna Pudyodu <[email protected]>
Signed-off-by: Vinay Krishna Pudyodu <[email protected]>
Signed-off-by: Vinay Krishna Pudyodu <[email protected]>
server/src/main/java/org/opensearch/common/io/VersionedCodecStreamWrapper.java
Show resolved
Hide resolved
server/src/main/java/org/opensearch/common/io/VersionedCodecStreamWrapper.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/index/shard/IndexShard.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/index/store/remote/metadata/RemoteSegmentMetadata.java
Outdated
Show resolved
Hide resolved
...ain/java/org/opensearch/index/store/remote/metadata/RemoteSegmentMetadataHandlerFactory.java
Show resolved
Hide resolved
server/src/main/java/org/opensearch/indices/replication/SegmentReplicator.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/indices/replication/SegmentReplicator.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/indices/replication/SegmentReplicator.java
Show resolved
Hide resolved
server/src/main/java/org/opensearch/indices/replication/SegmentReplicator.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/indices/replication/SegmentReplicationTargetService.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Vinay Krishna Pudyodu <[email protected]>
Signed-off-by: Vinay Krishna Pudyodu <[email protected]>
|
❌ Gradle check result for 7252842: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
…tService Signed-off-by: Vinay Krishna Pudyodu <[email protected]>
|
❌ Gradle check result for c993370: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Vinay Krishna Pudyodu <[email protected]>
|
❌ Gradle check result for 6879f4c: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Vinay Krishna Pudyodu <[email protected]>
|
❌ Gradle check result for 6ee1808: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Vinay Krishna Pudyodu <[email protected]>
server/src/main/java/org/opensearch/indices/replication/SegmentReplicationTargetService.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Vinay Krishna Pudyodu <[email protected]>
Signed-off-by: Vinay Krishna Pudyodu <[email protected]>
|
❌ Gradle check result for 04ba008: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Vinay Krishna Pudyodu <[email protected]>
There was a problem hiding this comment.
@vinaykpud Thanks for pushing on this, I think this is really close.
server/src/main/java/org/opensearch/indices/replication/SegmentReplicator.java
Outdated
Show resolved
Hide resolved
server/src/test/java/org/opensearch/indices/replication/SegmentReplicatorTests.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/indices/replication/SegmentReplicator.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/indices/replication/SegmentReplicator.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Vinay Krishna Pudyodu <[email protected]>
vinaykpud
force-pushed
the
node-stats
branch
from
7ff35b8 to
d89fdce
Compare
vinaykpud
force-pushed
the
node-stats
branch
from
d89fdce to
fcc57bf
Compare
Signed-off-by: Vinay Krishna Pudyodu <[email protected]>
|
❌ Gradle check result for 25fd006: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Vinay Krishna Pudyodu <[email protected]>
|
❌ Gradle check result for d8585f7: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Description
The method implemented here computes the segment replication stats at the shard level, instead of relying on the primary shard to compute stats based on reports from its replicas.
Method implemented in this PR serves the segment replication stats for following core APIs:
/_nodes/stats)/_cluster/stats)/_statsor/{index}/_stats)Related Issues
Resolves #16801
Related to #15306
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.