[filesystem] Evaluate DefaultCredentialsProvider for S3 authentication

[litiliu]

Search before asking

• I searched in the issues and found nothing similar.

Description

PR #3699 migrates the Fluss S3 filesystem plugin from AWS SDK v1 to AWS SDK v2. The current Glue guide documents separate fs.s3a.aws.credentials.provider lists for ECS Fargate, EKS with IRSA, and EC2.

As a follow-up to the review discussion in #3699, evaluate whether the guide can instead recommend the AWS SDK v2 software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider in server.yaml. This would let the AWS SDK own environment detection while still allowing users to override the provider for specialized setups.

Verification

• Confirm that Hadoop S3A 3.4.3 can load DefaultCredentialsProvider through fs.s3a.aws.credentials.provider.
• Manually verify the configuration with ECS task-role credentials.
• Manually verify the configuration with EKS IRSA credentials.
• Manually verify the configuration with an EC2 instance profile.
• Verify both server-side remote.data.dir access and the relevant Fluss filesystem security-token/client access path.
• Check whether provider fallback introduces noticeable cold-start delays.
• If the verification succeeds, simplify the Glue guide and document how users can configure a specific provider when needed.

Related discussion: #3699 (comment)
Related PR: #3699

Willingness to contribute

• I'm willing to submit a PR!