Cloud

You can remove all the files / objects in a Google Cloud Storage bucket using gsutil like this:

gsutil -m rm -r gs://my-bucket/\*

This will delete all the data in the bucket and there is usually no way to recover the data!

Problem:

While running a command like

gsutil rsync my-folder gs://my-bucket

you see an error message like

Building synchronization state...
Caught non-retryable exception while listing gs://mfwh-backups/: ServiceException: 401 Anonymous caller does not have storage.objects.list access to the Google Cloud Storage bucket.
CommandException: Caught non-retryable exception - aborting rsync

Solution:

This error is basically telling you that you are not logged in !

First, create a service account for the project on Google cloud: Direct link to service account page. You need to figure out depending on your setup what roles you want to assign to the service account. If you are lost and don’t know what to select, just assign it admin rights on the storage, but be aware that this might have security implications, as this account may also delete or create storage buckets etc.

Then open the page for that service account and create a new key!

This will give you a JSON file such as my-project-4d267a915c4e.json. Save it on the server or computer where you want to run gsutil. I recommend to save it in ~ (the user’s home folder) with the original filename, for example ~/my-project-4d267a915c4e.json.

Then you need to activate that service account using

gcloud auth activate-service-account --key-file [path to JSON file]

such as

gcloud auth activate-service-account --key-file [path to JSON file]

When I was trying to create a Oracle Cloud instance using Chrome, when I selected the Ubuntu image, the select image dialog closed and the image did not get selected.

However, the solution is simple: Use Firefox to create the instance. After that, you can continue to work with the instance using Chrome.

I recommend to always use either CentOS or Ubuntu. This way you will be compatible with other hosting providers and cloud vendors. Additionally, there is much more documentation regarding Ubuntu out there than for distributions like Oracle Linux.

Use this command to set the default zone for project myproject-123456 to europe-west4-a and the default region to europe-west4:

gcloud compute project-info add-metadata \
--metadata google-compute-default-region=europe-west4,google-compute-default-zone=europe-west4-a\
--project myproject-123456

Also see the official reference for more detailed information.

If you want to connect to a Google Cloud VM instance (my-instance in this example) from your command line using SSH, use this command:

rsync -Pavz [local file] $(gcloud compute instances list --filter="name=my-instance" --format "get(networkInterfaces[0].accessConfigs[0].natIP)"):

The subcommand (enclosed in $(...) ) finds the correct external IP address for your instance (see How to find IP address of Google Cloud VM instance on command line for more details), so this command boils down to for example

rsync -Pavz [local file] 35.207.77.101:

Using the -Pavz option is not specifically neccessary but these are the options I regularly use for rsync file transfers. You can use any rsync options, Google Cloud does not impose any specific restrictions here. For reference see the rsync manpage.

In case you want to use a different username for the SSH login, you can of course prefix the $(...) section like this:

rsync -Pavz [local file] [email protected]$(gcloud compute instances list --filter="name=my-instance" --format "get(networkInterfaces[0].accessConfigs[0].natIP)"):

If you want to connect to a Google Cloud VM instance (my-instance in this example) from your command line using SSH, you have two options:

Directly connect using gcloud

This will always work if your instance has SSH enabled, even if it does not have an external IP:

gcloud compute ssh my-instance --zone $(gcloud compute instances list --filter="name=my-instance" --format "get(zone)" | awk -F/ '{print $NF}')

Note that your have to replace my-instance by your actual instance name two times in the command above. The subcommand (enclosed in $(...) ) finds the correct zone for your instance since at the time of writing this article gcloud compute ssh will not work unless you set the correct zone for that instance. See How to find zone of Google Cloud VM instance on command line for more details.

Connect using external IP

You can also use gcloud to get the external IP and connect to it using your standard SSH client.

ssh $(gcloud compute instances list --filter="name=my-instance" --format "get(networkInterfaces[0].accessConfigs[0].natIP)")

This has the added advantage that your will be able to use this in other SSH-like command like rsync.

For reference also see the official manual on Securely Connecting to Instances