feat: add support to stream multiple optimized datasets using StreamingDataset

[deependujha]

Before submitting

• Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure to update the docs?
• Did you write any new necessary tests?

What does this PR do?

Fixes # (issue).

This PR introduces a new feature to the StreamingDataset class, enabling it to accept multiple input_dir paths and combine their index.json files while preserving their respective remote path sources. This allows streaming from multiple buckets simultaneously without the need to copy or merge files, resulting in significant speed improvements.

testing code

import litdata as ld

optimized_datasets = ["s3://bucket1/ds1", "s3://bucket2/ds2", "s3://bucket3/ds8"]

ds = ld.StreamingDataset(optimized_datasets)

dataloader = ld.StreamingDataloader(ds, batch_size = 4)

for batch in dataloader:
    ...

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in GitHub issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃