Skip to content

Commit 4b305f7

Browse files
committed
Feat: retries additional ClientError types
- We first noticed semi-frequent (1 every ~100GB), seemingly random XAmzContentSHA256Mismatch errors when uploading data to S3 on 2/27/2024. They appear to be transient errors, only needing one retry before succeeding. - There is also a less frequent (once every ~50TB or so) "Bad Request" error, which is also transient - Retrying both errors allows the uploads to succeed
1 parent caf15c8 commit 4b305f7

File tree

1 file changed

+4
-0
lines changed

1 file changed

+4
-0
lines changed

s3fs/core.py

+4
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,10 @@ async def _error_wrapper(func, *, args=(), kwargs=None, retries):
120120
err = e
121121
if "SlowDown" in str(e):
122122
await asyncio.sleep(min(1.7**i * 0.1, 15))
123+
elif "XAmzContentSHA256Mismatch" in str(e):
124+
await asyncio.sleep(min(1.7**i * 0.1, 15))
125+
elif "Bad Request" in str(e):
126+
await asyncio.sleep(min(1.7**i * 0.1, 15))
123127
else:
124128
break
125129
except Exception as e:

0 commit comments

Comments
 (0)