Posted on Leave a comment

Azure Linux virtual machine not booting – disk issue

There could be cases that you have not configured the disk correctly inside a Linux azure virtual machine , or the disk mount point to change because of a disk increase. This circumstance will make your virtual machine fail and boot into emergency mode. In order to fix this issue you should first get a terminal windows on the Azure virtual machine as you cannot ssh. This can be done by selecting Help -> Serial Console

By checking the boot logs you can find the issue which in my case is the mounting of the data disk. The data disk is added as an entry on /etc/fstab but the mount point changed and Linux filesystem cannot locate it.

In order to resolve this issue you should mount the disk using the UUID to be 100% sure for the mount point. First locate the disk by using

fdisk -l

After locating the mount point run blkid command to find the UUID of the disk.

blkid

When you find the UUID use it in your /etc/fstab file instead of the mount point as shown below.

Posted on Leave a comment

Access Managed Identity from container inside VM – Azure

Managed identity is the best practice regarding security when accessing resources on Azure. There are many ways you can use it for service to service communication. Sometimes though you can use nested managed identity in more complex scenarios like the one demonstrated below. In this guide we will enable managed identity on a virtual machine and we will access this managed identity within a container that runs on that specific virtual machine. This case can be useful in complex deployment scenarios where you have multiple containers inside a virtual machine and you want to deploy using managed identity on azure.

The first thing you will need is the system assigned managed identity on the virtual machine.

Then you can run your containers inside the virtual machine. In my case the containers are windows based as a result I will use the route print command to show the routing table.

Run the following Commands to expose the managed identity endpoint

$gateway = (Get-NetRoute | Where { $_.DestinationPrefix -eq '0.0.0.0/0' } | Sort-Object RouteMetric | Select NextHop).NextHop
$ifIndex = (Get-NetAdapter -InterfaceDescription "Hyper-V Virtual Ethernet*" | Sort-Object | Select ifIndex).ifIndex
New-NetRoute -DestinationPrefix 169.254.169.254/32 -InterfaceIndex $ifIndex -NextHop $gateway -PolicyStore ActiveStore # metadata API

After the successful add of the route the managed identity endpoint should be redirected in the gateway and from there you will be able to authenticate.

We can verify the procedure by executing a key vault managed identity secret retrieval.

$token = Invoke-WebRequest -Uri 'http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https%3A%2F%2Fvault.azure.net' -Headers @{Metadata="true"} -UseBasicParsing
$tokenvalue = ($token.Content | ConvertFrom-Json).access_token

Retrieve secret:

Invoke-WebRequest -Uri "https://test.vault.azure.net/secrets/testsecret/d9ce520dfdfdf4bdc9a41f5572069708c?api-version=7.3" -Headers @{Authorization = "Bearer $tokenvalue"} -UseBasicParsing

At last you can login using Managed Identity from the container using the powershell module.

References:

Co authored with Giannis Anastasiou @ Vivawallet

Posted on 2 Comments

error: the namespace from the provided object does not match the namespace. You must pass –namespace to perform this operation

When you need to copy a secret from one namespace to another in a Kubernetes cluster you may face the below error.

error: the namespace from the provided object "" does not match the namespace "". You must pass --namespace to perform this operation

The issue can be found because of the origin namespace that is referenced inside the secret. In order to bypass you can export the secret and change the referenced namespace.

kubectl get secret mysecret --namespace=ns1 -o yaml > export.yaml

In order to get it work you will need to change the namespace according to your new namespace in the export.yaml and apply it.

namespace: ns2
Posted on Leave a comment

Failed to start up WiredTiger under any compatibility version. This may be due to an unsupported upgrade or downgrade

When you upgrade a standalone mongodb instance you can end up having the issue described below. When you have a version oldest than 4.4 in mongodb you should first upgrade to version 4.2 and then to version 4.4. During those steps however you must not forget to change the featureCompatibilityVersion. If you do not change this property mongodb will stop working.

"ctx":"initandlisten","msg":"Failed to start up WiredTiger under any compatibility version. This may be due to an unsupported upgrade or downgrade."}

Upgrade a Standalone to 4.4 — MongoDB Manual

You can get your current compatibility Version by using the mongo shell.

db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } )

In this end you should change your compatibility to 4.2 before upgrading to 4.4 in order for the upgrade to be performed successfully.

db.adminCommand( { setFeatureCompatibilityVersion: "4.2" } )