Add datafusion-cli features to allow cloud support to be omitted

[pepijnve]

Which issue does this PR close?

• Closes Support reduced dependency graph size for datafusion-cli #23412.

Rationale for this change

By adding opt-out features for s3, gcs and http, users of the datafusion-cli crate can choose to omit these dependencies. This reduces the dependency graph and final binary size.

What changes are included in this PR?

• Add features http, gcs, and s3 to datafusion-clis cargo file
• Add conditional compilation directives in the datafusion-cli code where necessary

Are these changes tested?

• Manually compiled various combinations of enabled features
• There are some clippy warnings related to unused imports and variables which proved hard to silence so I left this in place; not an issue for the default configuration

Are there any user-facing changes?

No, the new features are part of the default feature set so this should have little effect for existing users.

[ratmice]

No, the new features are part of the default feature set so this should have little effect for existing users.

There is one potential impact here which is worth noting, users that enable default-features = false, but rely upon the existing cloud support may need to explicitly enable the features now to continue using them.

[ratmice]

In case we do do want to try and avoid lint warnings, I think these should do it.

[ratmice]

If you do want to remove this unused import complaint when compiling without default features, it is possible to get rid of the clippy lint on this one with a line like:

Suggested change

	use datafusion_common::exec_err;
	#[cfg(any(feature = "s3", feature = "gcs"))]
	use datafusion_common::exec_err;

[ratmice]

Not sure how to add the suggestion in the github UI, it sadly ends up as a massive diff that doesn't look right. However, maybe you can fix these if you feel it's worthwhile with the following additions to the get_object_store.

    #[cfg_attr(not(any(feature = "s3", feature = "gcs")), expect(unused_variables))]
    table_options: &TableOptions,
    #[cfg_attr(not(any(feature = "s3", feature = "gcs")), expect(unused_variables))]
    resolve_region: bool,

[ratmice]

Only way I could think of to disable this one was,

        #[cfg_attr(not(any(feature = "s3", feature = "gcs")), expect(unused_mut))]
        let mut builder = SessionStateBuilder::from(state.clone());

[pepijnve]

In case we do do want to try and avoid lint warnings, I think these should do it.

I had done exactly this at first, but reverted it because I felt I was cluttering the code with a bunch of cruft only to satisfy the linter. Happy to put those back if that's preferred.

[pepijnve]

There is one potential impact here which is worth noting, users that enable default-features = false, but rely upon the existing cloud support may need to explicitly enable the features now to continue using them.

Indeed, perhaps this does warrant a short upgrade note. I had kind of assumed since there were no default features up until now that most users would not explicitly have opted out of default features.