Skip to content

Commit cd31285

Browse files
committed
update
1 parent 6ffb1fa commit cd31285

File tree

2 files changed

+3
-3
lines changed

2 files changed

+3
-3
lines changed

README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -260,7 +260,7 @@ hf_uri = "hf://datasets/leonardPKU/clevr_cogen_a_train/data"
260260
ds = ld.StreamingDataset(hf_uri)
261261

262262
for _ds in ds:
263-
print(f"{_ds[1]=}; {_ds[2]}")
263+
print(f"{_ds[1]}; {_ds[2]}")
264264
```
265265

266266
You don’t need to worry about indexing the dataset or any other setup. **LitData** will **handle all the necessary steps automatically** and `cache` the `index.json` file, so you won't have to index it again.
@@ -314,7 +314,7 @@ hf_uri = "hf://datasets/open-thoughts/OpenThoughts-114k/data"
314314
ds = ld.StreamingDataset(hf_uri, item_loader=ParquetLoader(), index_path="hf-index-dir")
315315

316316
for _ds in ds:
317-
print(f"{_ds[0]=}; {_ds[1]}\n")
317+
print(f"{_ds[0]}; {_ds[1]}\n")
318318
```
319319

320320
 

src/litdata/utilities/parquet.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ def delete_thread(rmq: Queue) -> None:
2626
file_path = rmq.get()
2727
if file_path is None: # Sentinel value to exit
2828
break
29-
if os.path.exists(file_path):
29+
with suppress(FileNotFoundError):
3030
os.remove(file_path)
3131

3232
# before exiting, just sleep for some time, to complete any pending deletions

0 commit comments

Comments
 (0)