Make tensorflow optional in requirements

[aireenmei]

Description

1. Base Dependencies:
   • requirements.txt: Removed tensorflow, tensorflow-datasets, tensorflow-text, and seqio.
2. Overrides:
   • tpu_overrides.txt & cuda12_overrides.txt: Removed hardcoded TensorFlow pins.
3. Opt-in TF Layer:
   • tf_requirements.txt: New dedicated file for optional TF packages.
   • install_pre_train_extra_deps.py: Added --with-tf CLI flag and WITH_TF=true environment
   variable support.
4. Generated:
   • tpu-requirements.txt, cuda12-requirements.txt, decoupled-requirements.txt.

b/555209521

Tests

Generated environments following this instruction for cases with or without TF. Confirmed it works as expected.

Checklist

Before submitting this PR, please make sure (put X in square brackets):

• I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
• I have necessary comments in my code, particularly in hard-to-understand areas.
• I have run end-to-end tests tests and provided workload links above if applicable.
• I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.

[gemini-code-assist]

Code Review

This pull request deprecates the legacy TFDS input pipeline in favor of the Grain pipeline and makes TensorFlow-related packages optional dependencies. It introduces a new tf_requirements.txt file and updates the installation scripts and documentation to support an optional --with-tf flag. Additionally, various generated requirements files are updated with bumped dependency versions. The review feedback recommends removing capture_output=True from the subprocess installation command to prevent the CLI from appearing hung, and suggests adding seqio to the optional TensorFlow requirements since it is needed for the legacy pipeline.

[codecov]

Codecov Report

❌ Patch coverage is 80.00000% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/maxtext/common/gcloud_stub.py	80.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!