Cloud

Dec

2022

How to fix Oracle Cloud VNC connection: no matching host key type found. Their offer: ssh-rsa

Problem:

When trying to connect to your Oracle Cloud instance using VNC using Linux, you’ll get a command like

ssh -o ProxyCommand='ssh -W %h:%p -p 443 ocid1.instanceconsoleconnection.oc1.eu-frankfurt-1.antheljtwxs32nycl7rgwekcj4t2pecwwcsm7mgzy5c3tt3iiovq564wubta@instance-console.eu-frankfurt-1.oci.oraclecloud.com' -N -L localhost:5900:ocid1.instance.oc1.eu-frankfurt-1.antheljtwxs32nycblplzbuamqsqbi4ipz377f3qhs6a4tdh74j673jfsjtq:5900 ocid1.instance.oc1.eu-frankfurt-1.antheljtwxs32nycblplzbuamqsqbi4ipz377f3qhs6a4tdh74j673jfsjtq

but you see error messages like

Unable to negotiate with 130.61.0.255 port 443: no matching host key type found. Their offer: ssh-rsa

Solution:

Add the following text at the end of your ~/.ssh/config and retry:

Host *
    HostkeyAlgorithms +ssh-rsa
    PubkeyAcceptedAlgorithms +ssh-rsa

This will make SSH accept RSA host keys.

Apr

2022

How to remove ALL objects in Google Cloud Storage bucket using gsutil

You can remove all the files / objects in a Google Cloud Storage bucket using gsutil like this:

gsutil -m rm -r gs://my-bucket/\*

This will delete all the data in the bucket and there is usually no way to recover the data!

Apr

2022

How to install gcsfuse on Ubuntu in 15 seconds

export GCSFUSE_REPO=gcsfuse-`lsb_release -c -s`
echo "deb http://packages.cloud.google.com/apt $GCSFUSE_REPO main" | sudo tee /etc/apt/sources.list.d/gcsfuse.list
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
sudo apt-get update
sudo apt-get install gcsfuse

If that doesn’t work (such as for Ubuntu 21.10 impish at the time of writing this post), use the following method:

curl -L -O https://github.com/GoogleCloudPlatform/gcsfuse/releases/download/v0.39.2/gcsfuse_0.39.2_amd64.deb
sudo dpkg --install gcsfuse_0.39.2_amd64.deb
rm gcsfuse_0.39.2_amd64.deb

This is a summary from the official docs.

Apr

2022

How to fix gsutil 401 Anonymous caller does not have storage.objects.list access to the Google Cloud Storage bucket

Problem:

While running a command like

gsutil rsync my-folder gs://my-bucket

you see an error message like

Building synchronization state...
Caught non-retryable exception while listing gs://mfwh-backups/: ServiceException: 401 Anonymous caller does not have storage.objects.list access to the Google Cloud Storage bucket.
CommandException: Caught non-retryable exception - aborting rsync

Solution:

This error is basically telling you that you are not logged in !

First, create a service account for the project on Google cloud: Direct link to service account page. You need to figure out depending on your setup what roles you want to assign to the service account. If you are lost and don’t know what to select, just assign it admin rights on the storage, but be aware that this might have security implications, as this account may also delete or create storage buckets etc.

Then open the page for that service account and create a new key!

This will give you a JSON file such as my-project-4d267a915c4e.json. Save it on the server or computer where you want to run gsutil. I recommend to save it in ~ (the user’s home folder) with the original filename, for example ~/my-project-4d267a915c4e.json.

Then you need to activate that service account using

gcloud auth activate-service-account --key-file [path to JSON file]

such as

gcloud auth activate-service-account --key-file [path to JSON file]

Apr

2022

How to install gcloud on Ubuntu in 10 seconds

sudo snap install google-cloud-cli --classic

This is the summary from the official docs. I recommend to install it as snap package as opposed to a deb package since it will auto-update, it’s much easier to use and just works better out of the box in my experience.

Jan

2022

How to fix Oracle cloud Create Instance: Can’t select Ubuntu image

When I was trying to create a Oracle Cloud instance using Chrome, when I selected the Ubuntu image, the select image dialog closed and the image did not get selected.

However, the solution is simple: Use Firefox to create the instance. After that, you can continue to work with the instance using Chrome.

Aug

2021

What image / distribution should you use on Oracle Cloud?

I recommend to always use either CentOS or Ubuntu. This way you will be compatible with other hosting providers and cloud vendors. Additionally, there is much more documentation regarding Ubuntu out there than for distributions like Oracle Linux.

Aug

2021

Oracle Cloud free tier VM.Standard.E2.1.Micro /proc/cpuinfo

processor       : 0
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7551 32-Core Processor
stepping        : 2
microcode       : 0x1000065
cpu MHz         : 1996.246
cache size      : 512 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 1
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core ssbd ibpb vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr virt_ssbd arat arch_capabilities
bugs            : sysret_ss_attrs null_seg spectre_v1 spectre_v2 spec_store_bypass
bogomips        : 3992.49
TLB size        : 1024 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

processor       : 1
vendor_id       : AuthenticAMD
cpu family      : 23
model           : 1
model name      : AMD EPYC 7551 32-Core Processor
stepping        : 2
microcode       : 0x1000065
cpu MHz         : 1996.246
cache size      : 512 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 1
apicid          : 1
initial apicid  : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core ssbd ibpb vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr virt_ssbd arat arch_capabilities
bugs            : sysret_ss_attrs null_seg spectre_v1 spectre_v2 spec_store_bypass
bogomips        : 3992.49
TLB size        : 1024 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

Aug

2021

How to disable virtual cloud network firewall on Oracle Cloud

When running VM instances on Oracle Cloud, you might want to use all ports, not just the few ports that are open by default. This post shows how to disable the Virtual Cloud Network firewall altogether. Additionally, you need to configure the instance firewall e.g. via SSH. For Ubuntu, see How to disable instance firewall on Ubuntu on Oracle Cloud.

You need to use firefox to access the UI. It will not work in chrome!

First login to the cloud network dashboard at https://cloud.oracle.com/networking/vcns

Now click the virtual cloud network:

Now click Security lists on the bottom left:

Click the Default security list (which has been automatically created)

Click Add Ingress Rules

Enter source 0.0.0.0/0 (i.e. all IP addresses) and IP protocol All protocols:

Now click Save changes and don’t forget to configure your instance firewall

Aug

2021

How to disable instance firewall on Ubuntu on Oracle Cloud

Note: This describes how to disable the firewall on an Ubuntu instance. You additionally need to configure the cloud network security list! See How to disable virtual cloud network firewall on Oracle Cloud for details!

The Oracle firewall is iptables based. We can disable the Ubuntu instance firewall using

sudo iptables -F
sudo netfilter-persistent save

Explanation:

iptables -F: Flush (remove all) iptables rules
netfilter-persistent save Save empty ruleset to disk so it will be reloaded on reboot.

Aug

2021

How to fix Nextcloud 4047 InnoDB refuses to write tables with ROW_FORMAT=COMPRESSED or KEY_BLOCK_SIZE.

Problem:

When trying to maintenance:repair your Nextcloud instance, e.g. during a utf8mb4 upgrade, you see an error message like

ERROR: An exception occurred while executing a query: SQLSTATE[HY000]: General error: 4047 InnoDB refuses to write tables with ROW_FORMAT=COMPRESSED or KEY_BLOCK_SIZE.

Solution:

You need to turn off innodb-read-only-compressed. I do this by starting my MySQL docker with

--skip-innodb-read-only-compressed

Full command line which worked for me:

--transaction-isolation=READ-COMMITTED --binlog-format=ROW --innodb-file-per-table=1 --skip-innodb-read-only-compressed

Full docker-compose.yml nextcloud section:

nextcloud-db:
  image: mariadb
  command: --transaction-isolation=READ-COMMITTED --binlog-format=ROW --innodb-file-per-table=1 --skip-innodb-read-only-compressed
  restart: always
  volumes:
    - ./nextcloud-db:/var/lib/mysql
  environment:
    - MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}
    - MYSQL_PASSWORD=${MYSQL_PASSWORD}

Jul

2019

Installing apt packages using cloud-init

This cloud-init example installs nginx on Debian- or Ubuntu- based systems:

packages:
- nginx

If you want to enable upgrading packages, use:

package_upgrade: true
packages:
- nginx

Apr

2019

How to fix Google Cloud Build ignoring .dockerignore

Problem:

You want to run a docker image build on Google Cloud Build, but the client is trying to upload a huge context image to Google Cloud even though you have added all your large directories to your .dockerignore and the build works fine locally.

Solution:

Google Cloud Build ignores .dockerignore by design – the equivalent is called .gcloudignore.

You can copy the .dockerignore behaviour for gcloud by running

cp .dockerignore .gcloudignore

Apr

2019

How to set default zone for Google Cloud project using gcloud command-line tool

Use this command to set the default zone for project myproject-123456 to europe-west4-a and the default region to europe-west4:

gcloud compute project-info add-metadata \
--metadata google-compute-default-region=europe-west4,google-compute-default-zone=europe-west4-a\
--project myproject-123456

Also see the official reference for more detailed information.

Apr

2019

How to expand Kubernetes Physical Volume Claim (PVC)

Important note: By default, volumes will not be resized immediately but instead require a restart of the associated pod.

First, ensure that you have set allowVolumeExpansion: true for the storage class of your PVC. See our previous post on How to allow Physical Volume Claim (PVC) resize for Kubernetes storage class for more details.

We can expand the volume (named myapp-myapp-pvc-myapp-myapp-1 in this example) by running

kubectl patch pvc/"myapp-myapp-pvc-myapp-myapp-1" \
  --namespace "default" \
  --patch '{"spec": {"resources": {"requests": {"storage": "40Gi"}}}}'

Ensure that you have replaced the name of the PVC (myapp-myapp-pvc-myapp-myapp-1 in this example) and the storage size. It’s only possible to increase the size of the volume / expand it and not to downsize / shrink it. If your size is less than the previous value, you’ll see this error message:

The PersistentVolumeClaim "myapp-myapp-pvc-myapp-myapp-1" is invalid: spec.resources.requests.storage: Forbidden: field can not be less than previous value

After running this command, the PVC will be in the FileSystemResizePending state.

In order for the update to have effect, you’ll need to force Kubernetes to re-create all the pods for your deployment. To find out how to do this, read our post on How to force restarting all Pods in a Kubernetes Deployment.

For reference, see the official documentation on expanding persistent volumes

Apr

2019

How to force restarting all Pods in a Kubernetes Deployment

In contrast to classical deployment managers like systemd or pm2, Kubernetes does not provide a simple restart my application command.

However there’s an easy workaround: If you chance anything in your configuration, even innocuous things that don’t have any effect, Kubernetes will restart your pods.

Consider configuring a rolling update strategy before doing this if you are updating a production application that should have minimal downtime.

In this example we’ll assume you have a StatefulSet your want to update and it’s named elasticsearch-elasticsearch. Be sure to fill in the actual name of your deployment here.

kubectl patch statefulset/elasticsearch-elasticsearch -p \
  "{\"spec\":{\"template\":{\"metadata\":{\"annotations\":{\"dummy-date\":\"`date +'%s'`\"}}}}}"

This will just set a dummy-date annotation which does not have any effect.

You can monitor the update by

kubectl rollout status statefulset/elasticsearch-elasticsearch

Credits for the original solution idea to pstadler on GitHub.

Apr

2019

How to configure Google Cloud Kubernetes Elasticsearch Cluster with internal load balancer

Google Cloud offers a convenient way of installing an ElasticSearch cluster on top of a Google Cloud Kubernetes cluster. However, the documentation tells you to expose the ElasticSearch instance using

kubectl patch service/"elasticsearch-elasticsearch-svc" \
  --namespace "default" \
  --patch '{"spec": {"type": "LoadBalancer"}}'

However this command will expost ElasticSearch to an external IP which will make it publically accessible in the default configuration.

Here’s the equivalent command that will expose ElasticSearch to an internal load balancer with an internal IP address that will only be available from Google Cloud.

kubectl patch service/"elasticsearch-elasticsearch-svc" \
  --namespace "default" \
  --patch '{"spec": {"type": "LoadBalancer"}, "metadata": {"annotations": {"cloud.google.com/load-balancer-type": "Internal"}}}'

You might need to replace the name of your service (elasticsearch-elasticsearch-svc in this example) and possibly your namespace.

Apr

2019

How to install MicroK8S (MicroKubernetes) on Ubuntu in 30 seconds

This set of commands will install & start MikroK8S (MikroKubernetes) on Ubuntu and similar Linux distributions.

sudo snap install microk8s --classic
sudo snap install kubectl --classic
sudo microk8s.enable # Autostart on boot
sudo microk8s.start # Start right now
# Wait until microk8s has started
until microk8s.status ; do sleep 1 ; done
# Enable some standard modules
microk8s.enable dashboard registry istio

For reference see the official quickstart manual.

Apr

2019

How to fix kubectl unknown shorthand flag: ‘f’ in -f

Problem:

You want to run a Kubernetes kubectl command like

kubectl -f my-app-deployment.yaml

but you see this error message after kubectl prints its entire help page:

unknown shorthand flag: 'f' in -f

Solution:

You are missing an actual command to kubectl. Most likely you want create something on your Kubernetes instance, in which case you want to run this instead:

kubectl create -f my-app-deployment.yaml

You might also want to apply or replace your config instead. Note that apply does not automatically restart your Kubernetes Pods. Read How to fix Kubernetes kubectl apply not restarting pods for more information.

Apr

2019

How to rsync to Google Cloud VM instance on command line

If you want to connect to a Google Cloud VM instance (my-instance in this example) from your command line using SSH, use this command:

rsync -Pavz [local file] $(gcloud compute instances list --filter="name=my-instance" --format "get(networkInterfaces[0].accessConfigs[0].natIP)"):

The subcommand (enclosed in $(...) ) finds the correct external IP address for your instance (see How to find IP address of Google Cloud VM instance on command line for more details), so this command boils down to for example

rsync -Pavz [local file] 35.207.77.101:

Using the -Pavz option is not specifically neccessary but these are the options I regularly use for rsync file transfers. You can use any rsync options, Google Cloud does not impose any specific restrictions here. For reference see the rsync manpage.

In case you want to use a different username for the SSH login, you can of course prefix the $(...) section like this:

rsync -Pavz [local file] sshuser@$(gcloud compute instances list --filter="name=my-instance" --format "get(networkInterfaces[0].accessConfigs[0].natIP)"):

Problem:

Solution:

Problem:

Solution:

Problem:

Solution:

Problem:

Solution:

Problem:

Solution:

Search

Categories