[docs] Add a concept guide on sharding iterable datasets

[zokost]

What does this PR do?

Adds concept_guides/iterable_dataset_sharding.md, plus a toctree entry and a cross-link from
concept_guides/internal_mechanism.md. Documentation only — no code changes.

The docs explain that map-style datasets are sharded by swapping the batch_sampler, but the iterable-dataset path
is only mentioned in passing. Two consequences of that path keep coming up and are invisible at runtime:

1. Read amplification. IterableDatasetShard shards elements, so every process iterates the whole underlying
   dataset and keeps 1 / num_processes of it. Correct, but the data is read num_processes times (and
   num_processes × num_workers with dataloader workers). For streaming pipelines over remote storage that is the
   dominant cost, and nothing in the docs says it will happen.

2. Double sharding. A dataset that already shards by source (files, shards, byte ranges) gets wrapped anyway,
   so the stream is sharded a second time and each process silently keeps a fraction of the data it was given. No
   error, no hang — the loss curve looks normal.

Related: #3547 (where the suggested workaround is to not prepare the dataloader) and #3124; the equivalent issue on
the datasets side is huggingface/datasets#6594. This PR documents the current behaviour and the ways around it —
it does not change any of it.

The guide covers what prepare does to an iterable dataset, the measured cost, when to shard by source yourself,
the dispatch_batches=True alternative, and what you take responsibility for once you stop preparing the dataloader
(device placement, the end-of-epoch gradient sync via end_of_dataloader, RNG sync, and equal batch counts per
process).

Numbers

Every number in the guide was measured on main, not estimated — 8 shards of 10 records, num_processes=4,
batch_size=2:

	records per process	unique records seen	shard reads
dataset alone	—	80	8
naive dataset + prepare	20, 20, 20, 20	80	32 (4×)
self-sharding dataset, no prepare	20, 20, 20, 20	80	8
self-sharding dataset + prepare	6, 6, 6, 6	24	8

Reproduction script

from accelerate.data_loader import IterableDatasetShard
from torch.utils.data import DataLoader, IterableDataset

FILES = [[f"f{f}-r{r}" for r in range(10)] for f in range(8)]
reads = {"n": 0}


class Naive(IterableDataset):
    def __iter__(self):
        for f in FILES:
            reads["n"] += 1
            yield from f


class SelfSharded(IterableDataset):
    def __init__(self, ws, rank):
        self.ws, self.rank = ws, rank

    def __iter__(self):
        for f in FILES[self.rank :: self.ws]:
            reads["n"] += 1
            yield from f


def run(ds, ws, rank, bs=2, wrap=True):
    if wrap:
        ds = IterableDatasetShard(ds, batch_size=bs, num_processes=ws, process_index=rank)
        ds.epoch = 0
    return [x for b in DataLoader(ds, batch_size=bs) for x in b]


WS = 4
for name, factory, wrap in [
    ("naive + prepare", lambda r: Naive(), True),
    ("self-sharded, no prepare", lambda r: SelfSharded(WS, r), False),
    ("self-sharded + prepare", lambda r: SelfSharded(WS, r), True),
]:
    reads["n"] = 0
    out = [run(factory(r), WS, r, wrap=wrap) for r in range(WS)]
    print(f"{name:<26} per-process={[len(o) for o in out]} "
          f"unique={len(set().union(*map(set, out)))}/80 shard-reads={reads['n']}")

Built locally with doc-builder; the new page and the edited one convert to MDX and their internal links resolve.

Happy to cut the page down, fold it into internal_mechanism.md, or move it to usage_guides/ if you'd rather have
it there.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Documentation: @SunMarc