Skip to content

Commit 182504c

Browse files
authored
concurrency limit 64 -> 10 (#3578)
* concurrency limit 64 -> 10 * update reference to concurrency limit in docs * update tests * remove release note
1 parent 4bc70bd commit 182504c

File tree

4 files changed

+12
-8
lines changed

4 files changed

+12
-8
lines changed

changes/3526.feature.md

Lines changed: 0 additions & 1 deletion
This file was deleted.

docs/user-guide/performance.md

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -175,13 +175,18 @@ Coming soon.
175175

176176
## Parallel computing and synchronization
177177

178-
Zarr is designed to support parallel computing and enables concurrent reads and writes to arrays. This section covers how to optimize Zarr's concurrency settings for different parallel computing scenarios.
178+
Zarr is designed to support parallel computing and enables concurrent reads and writes to arrays.
179+
This section covers how to optimize Zarr's concurrency settings for different parallel computing
180+
scenarios.
179181

180182
### Concurrent I/O operations
181183

182-
Zarr uses asynchronous I/O internally to enable concurrent reads and writes across multiple chunks. The level of concurrency is controlled by the `async.concurrency` configuration setting, which determines the maximum number of concurrent I/O operations.
184+
Zarr uses asynchronous I/O internally to enable concurrent reads and writes across multiple chunks.
185+
The level of concurrency is controlled by the `async.concurrency` configuration setting, which
186+
determines the maximum number of concurrent I/O operations.
183187

184-
The default value is 64, which provides good performance for most workloads. You can adjust this value based on your specific needs:
188+
The default value is 10, which is a conservative value. You may get improved performance by tuning
189+
the concurrency limit. You can adjust this value based on your specific needs:
185190

186191
```python
187192
import zarr

src/zarr/core/config.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -97,7 +97,7 @@ def enable_gpu(self) -> ConfigSet:
9797
"order": "C",
9898
"write_empty_chunks": False,
9999
},
100-
"async": {"concurrency": 64, "timeout": None},
100+
"async": {"concurrency": 10, "timeout": None},
101101
"threading": {"max_workers": None},
102102
"json_indent": 2,
103103
"codec_pipeline": {

tests/test_config.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@ def test_config_defaults_set() -> None:
5454
"order": "C",
5555
"write_empty_chunks": False,
5656
},
57-
"async": {"concurrency": 64, "timeout": None},
57+
"async": {"concurrency": 10, "timeout": None},
5858
"threading": {"max_workers": None},
5959
"json_indent": 2,
6060
"codec_pipeline": {
@@ -100,15 +100,15 @@ def test_config_defaults_set() -> None:
100100
]
101101
)
102102
assert config.get("array.order") == "C"
103-
assert config.get("async.concurrency") == 64
103+
assert config.get("async.concurrency") == 10
104104
assert config.get("async.timeout") is None
105105
assert config.get("codec_pipeline.batch_size") == 1
106106
assert config.get("json_indent") == 2
107107

108108

109109
@pytest.mark.parametrize(
110110
("key", "old_val", "new_val"),
111-
[("array.order", "C", "F"), ("async.concurrency", 64, 128), ("json_indent", 2, 0)],
111+
[("array.order", "C", "F"), ("async.concurrency", 10, 128), ("json_indent", 2, 0)],
112112
)
113113
def test_config_defaults_can_be_overridden(key: str, old_val: Any, new_val: Any) -> None:
114114
assert config.get(key) == old_val

0 commit comments

Comments
 (0)