Month: April 2023

Posted on by — Leave a comment

Error from server (Forbidden): nodes is forbidden: User “” cannot list resource “nodes” in API group “” at the cluster scope

kubelogin is a client-go credential (exec) plugin implementing azure authentication. This plugin provides features that are not available in kubectl. It is supported on kubectl v1.11+ and you can bypass interactive authentication with it. This means that you do not have to enter a device code login when interacting with AKS.

I had to use the tool for managed identity authentication with Kubernetes service. In the documentation you can find instructions on how to use it for cases like user login, service principal, managed identity.

https://azure.github.io/kubelogin/concepts/login-modes/msi.html

Although I was following the correct instructions I was struggling with the error shown below when I was executing kubectl commands.

This error was due to the fact that I was not requesting the admin credentials on the kubectl command.

When I was asking for credentials with the below command I ended up with the error.

az aks get-credentials--resource-group rg --name clustername

As I had assigned the Kubernetes admin Cluster role on my managed identity I was able to execute kubectl commands when using.

az aks get-credentials --admin --resource-group rg --name clustername
Posted on by — Leave a comment

Get powershell command result as string

Sometimes you may end up with wrong results on powershell because of the return object. A detailed demonstration can be located below where the return object is not a string and the evaluation of equals is not correct.

For example lets assume that we need to check docker status from powershell and catch this result through the string that is returned. When docker is not running you can expect a similar message like the below.

By getting the result of the docker info command into a variable we can see that the return object is of type Object in powershell.

When you try to use the contains functions with this object in order to evaluate the docker status you will end up with a false result as is not evaluated correctly.

In order to resolve this issue you should specify that the result should be a string with Out-String function.

Then when you evaluate the expression with Contains function this is performed as expected and the correct result is returned.

Posted on by — Leave a comment

Find resource groups that contain tags on Azure using az cli

Sometimes you may need to massively search for resource groups or resources on Azure that have tags set. For example you may have some tags like temp resource that you want to delete with cron jobs. I myself wanted such a script and the first thing to do was to ask chatGPT about this. Although the provided answer is a good starting point I wanted a version that will bring all the resources that have a tag, and not a specific tag. I was searching for tags in general and not for a specific tag.

For such case I created my own script using az cli. In order to use it you will need to first login inside azure with your credentials.

az login

and then set your subscription

az account set --subscription "ID"

The script which brings resource-groups with tags can be found below.

$rgroups = az group list | ConvertFrom-Json
Write-Host Total Resource groups: $rgroups.Count  

$tags = @()
foreach ($item in $rgroups)
{
 if ( -not [string]::IsNullOrEmpty($item.tags)  ) { $tags+= $item } 
}
Write-Host Resource groups with Tags: $tags.Count  

echo $tags

When you run the script you can get the total number of resource groups and the ones that contain tags. You can then use the tags object to loop through the items with tags.

You can use the same logic to find also resources with tags inside an azure subscription

$resources = az resource list | ConvertFrom-Json
Write-Host Total Resources: $resources.Count  

$tags = @()
foreach ($item in $resources)
{
 if ( -not [string]::IsNullOrEmpty($item.tags)  ) { $tags+= $item } 
}
Write-Host Resources with Tags: $tags.Count  

echo $tags

Youtube video:

Posted on by — Leave a comment

Monitoring Windows service on Azure with Event Viewer and Log Analytics

A Log Analytics workspace is a unique environment for log data from Azure Monitor and other Azure services, such as Microsoft Sentinel and Microsoft Defender for Cloud. Expect from that it can be used also for monitoring combined with Azure alerts given that you create the appropriate query.

The below query can be used to monitor a windows server service by querying log analytics. In more detail it searches for eventID=7036 which indicates the service stopped status.

Query code in Kusto language will return the service name, state and time of the event.

Event
| where TimeGenerated >ago(1h)
| where EventLog  == "System" and EventID ==7036 and Source == "Service Control Manager" 
| parse kind=relaxed EventData with * '<Data Name="param1">' Windows_Service_Name '</Data><Data Name="param2">' Windows_Service_State '</Data>'*
| sort by TimeGenerated desc
| where Windows_Service_Name startswith "Docker Desktop" and Windows_Service_State contains "stopped"
| project Computer, Windows_Service_Name, Windows_Service_State, TimeGenerated

You can use the above query to create a azure alert when a service is found as stopped. As I want to monitor the Docker Desktop service, I will need to use that in the where clause of the query where Windows_Service_Name. The alert logic should indicate when a result is returned as a row in a given timeframe then an alert should be generated. This happens because a row is returned only when the event is captured on the event viewer. This means that the service stopped during the TimeGenerated interval of the query. The frequency of evaluation will be the time on which we want to repeat the log analytics query. For example if we need to search every 5 minutes for a stopped service then we should add 5 minutes there.

Finally the alert will be triggered and inform you about windows stopped services.