Skip to content

chore: migrate gsutil usage to gcloud storage #7900

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
gurusai-voleti wants to merge 1 commit into
base: master
Choose a base branch
from
Open

chore: migrate gsutil usage to gcloud storage #7900

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/tutorials/tfx/gcp/vertex_pipelines_vertex_training.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -384,7 +384,7 @@
},
"outputs": [],
"source": [
"!gsutil cp gs://download.tensorflow.org/data/palmer_penguins/penguins_processed.csv {DATA_ROOT}/"
"!gcloud storage cp gs://download.tensorflow.org/data/palmer_penguins/penguins_processed.csv {DATA_ROOT}/"
]
},
{
Expand All @@ -404,7 +404,7 @@
},
"outputs": [],
"source": [
"!gsutil cat {DATA_ROOT}/penguins_processed.csv | head"
"!gcloud storage cat {DATA_ROOT}/penguins_processed.csv | head"
]
},
{
Expand Down Expand Up @@ -628,7 +628,7 @@
},
"outputs": [],
"source": [
"!gsutil cp {_trainer_module_file} {MODULE_ROOT}/"
"!gcloud storage cp {_trainer_module_file} {MODULE_ROOT}/"
]
},
{
Expand Down
4 changes: 2 additions & 2 deletions docs/tutorials/tfx/penguin_template.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -472,7 +472,7 @@
" - If the files are stored in GCS, you can use a path starting with\n",
" `gs://{bucket_name}/...`. Please make sure that you can access GCS from\n",
" your terminal, for example, using\n",
" [`gsutil`](https://cloud.google.com/storage/docs/gsutil).\n",
" [`gcloud`](https://cloud.google.com/sdk/gcloud/reference/storage).\n",
" Please follow\n",
" [authorization guide in Google Cloud](https://cloud.google.com/sdk/docs/authorizing)\n",
" if needed.\n",
Expand Down Expand Up @@ -1428,7 +1428,7 @@
},
"outputs": [],
"source": [
"!gsutil cp data/data.csv gs://{GOOGLE_CLOUD_PROJECT}-kubeflowpipelines-default/tfx-template/data/penguin/"
"!gcloud storage cp data/data.csv gs://{GOOGLE_CLOUD_PROJECT}-kubeflowpipelines-default/tfx-template/data/penguin/"
]
},
{
Expand Down
12 changes: 6 additions & 6 deletions tfx/examples/custom_components/container_components/download_grep_print_pipeline.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@
parameters={
'url': str,
},
# The component code uses gsutil to upload the data to GCS, so the
# container image needs to have gsutil installed and configured.
# The component code uses gcloud storage to upload the data to GCS, so the
# container image needs to have gcloud installed and configured.
# Fixing b/150670779 by merging cl/294536017 will lift this limitation.
image='google/cloud-sdk:278.0.0',
command=[
Expand Down Expand Up @@ -60,8 +60,8 @@
parameters={
'pattern': str,
},
# The component code uses gsutil to upload the data to GCS, so the
# container image needs to have gsutil installed and configured.
# The component code uses gcloud storage to upload the data to GCS, so the
# container image needs to have gcloud installed and configured.
# Fixing b/150670779 by merging cl/294536017 will lift this limitation.
image='google/cloud-sdk:278.0.0',
command=[
Expand Down Expand Up @@ -94,8 +94,8 @@
inputs={
'text': standard_artifacts.ExternalArtifact,
},
# The component code uses gsutil to upload the data to GCS, so the
# container image needs to have gsutil installed and configured.
# The component code uses gcloud storage to upload the data to GCS, so the
# container image needs to have gcloud installed and configured.
# Fixing b/150670779 by merging cl/294536017 will lift this limitation.
image='google/cloud-sdk:278.0.0',
command=[
Expand Down
12 changes: 6 additions & 6 deletions tfx/orchestration/test_pipelines/download_grep_print_pipeline.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@
parameters={
'url': str,
},
# The component code uses gsutil to upload the data to GCS, so the
# container image needs to have gsutil installed and configured.
# The component code uses gcloud storage to upload the data to GCS, so the
# container image needs to have gcloud installed and configured.
# Fixing b/150670779 by merging cl/294536017 will lift this limitation.
image='gcr.io/google.com/cloudsdktool/cloud-sdk:278.0.0',
command=[
Expand Down Expand Up @@ -60,8 +60,8 @@
parameters={
'pattern': str,
},
# The component code uses gsutil to upload the data to GCS, so the
# container image needs to have gsutil installed and configured.
# The component code uses gcloud storage to upload the data to GCS, so the
# container image needs to have gcloud installed and configured.
# Fixing b/150670779 by merging cl/294536017 will lift this limitation.
image='gcr.io/google.com/cloudsdktool/cloud-sdk:278.0.0',
command=[
Expand Down Expand Up @@ -94,8 +94,8 @@
inputs={
'text': standard_artifacts.ExternalArtifact,
},
# The component code uses gsutil to upload the data to GCS, so the
# container image needs to have gsutil installed and configured.
# The component code uses gcloud storage to upload the data to GCS, so the
# container image needs to have gcloud installed and configured.
# Fixing b/150670779 by merging cl/294536017 will lift this limitation.
image='gcr.io/google.com/cloudsdktool/cloud-sdk:278.0.0',
command=[
Expand Down
Loading