Tag: linux

Posted on by — 1 Comment

Deploy windows and linux virtual machines on Azure using terraform

Terraform is one of the best automation providers for DevOps purposes used by hundred of Engineers. It is an open source tool that can be used by anyone for free. In this article I will explain how to deploy windows and linux virtual machines on Azure using a Terraform template.

First things first you will need to have the az cli installed. Then you will have to set your subscription on your current powershell session. 

az account set --subscription "12abc123-4567-1234-12345-asdr4334fsd"

Then you will need to create an app role assignment for your subscription. This will be used from terraform for the provision of the resources.

az ad sp create-for-rbac --role="Contributor" --scopes="/subscriptions/12abc123-4567-1234-12345-asdr4334fsd"

That’s all. You can now deploy your resources through terraform. In the links below I have provided my Github repository along with instructions for the template use.

A tricky part of the deployment is the vm image selection. In order to locate the available azure images names you can use:

az vm image list

Enumeration of available images:

[
  {
    "offer": "CentOS",
    "publisher": "OpenLogic",
    "sku": "7.5",
    "urn": "OpenLogic:CentOS:7.5:latest",
    "urnAlias": "CentOS",
    "version": "latest"
  },
  {
    "offer": "debian-10",
    "publisher": "Debian",
    "sku": "10",
    "urn": "Debian:debian-10:10:latest",
    "urnAlias": "Debian",
    "version": "latest"
  },
  {
    "offer": "flatcar-container-linux-free",
    "publisher": "kinvolk",
    "sku": "stable",
    "urn": "kinvolk:flatcar-container-linux-free:stable:latest",
    "urnAlias": "Flatcar",
    "version": "latest"
  },
  {
    "offer": "openSUSE-Leap",
    "publisher": "SUSE",
    "sku": "42.3",
    "urn": "SUSE:openSUSE-Leap:42.3:latest",
    "urnAlias": "openSUSE-Leap",
    "version": "latest"
  },
  {
    "offer": "RHEL",
    "publisher": "RedHat",
    "sku": "7-LVM",
    "urn": "RedHat:RHEL:7-LVM:latest",
    "urnAlias": "RHEL",
    "version": "latest"
  },
  {
    "offer": "SLES",
    "publisher": "SUSE",
    "sku": "15",
    "urn": "SUSE:SLES:15:latest",
    "urnAlias": "SLES",
    "version": "latest"
  },
  {
    "offer": "UbuntuServer",
    "publisher": "Canonical",
    "sku": "18.04-LTS",
    "urn": "Canonical:UbuntuServer:18.04-LTS:latest",
    "urnAlias": "UbuntuLTS",
    "version": "latest"
  },
  {
    "offer": "WindowsServer",
    "publisher": "MicrosoftWindowsServer",
    "sku": "2019-Datacenter",
    "urn": "MicrosoftWindowsServer:WindowsServer:2019-Datacenter:latest",
    "urnAlias": "Win2019Datacenter",
    "version": "latest"
  },
  {
    "offer": "WindowsServer",
    "publisher": "MicrosoftWindowsServer",
    "sku": "2016-Datacenter",
    "urn": "MicrosoftWindowsServer:WindowsServer:2016-Datacenter:latest",
    "urnAlias": "Win2016Datacenter",
    "version": "latest"
  },
  {
    "offer": "WindowsServer",
    "publisher": "MicrosoftWindowsServer",
    "sku": "2012-R2-Datacenter",
    "urn": "MicrosoftWindowsServer:WindowsServer:2012-R2-Datacenter:latest",
    "urnAlias": "Win2012R2Datacenter",
    "version": "latest"
  },
  {
    "offer": "WindowsServer",
    "publisher": "MicrosoftWindowsServer",
    "sku": "2012-Datacenter",
    "urn": "MicrosoftWindowsServer:WindowsServer:2012-Datacenter:latest",
    "urnAlias": "Win2012Datacenter",
    "version": "latest"
  },
  {
    "offer": "WindowsServer",
    "publisher": "MicrosoftWindowsServer",
    "sku": "2008-R2-SP1",
    "urn": "MicrosoftWindowsServer:WindowsServer:2008-R2-SP1:latest",
    "urnAlias": "Win2008R2SP1",
    "version": "latest"
  }
]

In order to narrow down and find Ubuntu available images (use grep instead of Select-string for Unix environments)

az vm image list-offers -p canonical -l eastus | Select-String name

Ubuntu images names for east us region:

 "name": "0001-com-ubuntu-confidential-vm-experimental",
    "name": "0001-com-ubuntu-confidential-vm-focal",
    "name": "0001-com-ubuntu-confidential-vm-test-focal",
    "name": "0001-com-ubuntu-minimal-focal-daily",
    "name": "0001-com-ubuntu-minimal-groovy-daily",
    "name": "0001-com-ubuntu-minimal-hirsute-daily",
    "name": "0001-com-ubuntu-minimal-impish-daily",
    "name": "0001-com-ubuntu-minimal-jammy-daily",
    "name": "0001-com-ubuntu-private-fips-motorola",
    "name": "0001-com-ubuntu-pro-advanced-sla",
    "name": "0001-com-ubuntu-pro-advanced-sla-att",
    "name": "0001-com-ubuntu-pro-advanced-sla-csw",
    "name": "0001-com-ubuntu-pro-advanced-sla-dd",
    "name": "0001-com-ubuntu-pro-advanced-sla-nestle",
    "name": "0001-com-ubuntu-pro-advanced-sla-servicenow",
    "name": "0001-com-ubuntu-pro-advanced-sla-shell",
    "name": "0001-com-ubuntu-pro-advanced-sla-ub01",
    "name": "0001-com-ubuntu-pro-advanced-sla-unp",
    "name": "0001-com-ubuntu-pro-bionic",
    "name": "0001-com-ubuntu-pro-bionic-fips",
    "name": "0001-com-ubuntu-pro-focal",
    "name": "0001-com-ubuntu-pro-focal-fips",
    "name": "0001-com-ubuntu-pro-hidden-msft-fips",
    "name": "0001-com-ubuntu-pro-microsoft",
    "name": "0001-com-ubuntu-pro-trusty",
    "name": "0001-com-ubuntu-pro-xenial",
    "name": "0001-com-ubuntu-pro-xenial-fips",
    "name": "0001-com-ubuntu-server-eoan",
    "name": "0001-com-ubuntu-server-focal",
    "name": "0001-com-ubuntu-server-focal-daily",
    "name": "0001-com-ubuntu-server-groovy",
    "name": "0001-com-ubuntu-server-groovy-daily",
    "name": "0001-com-ubuntu-server-hirsute",
    "name": "0001-com-ubuntu-server-hirsute-daily",
    "name": "0001-com-ubuntu-server-impish",
    "name": "0001-com-ubuntu-server-impish-daily",
    "name": "0001-com-ubuntu-server-jammy-daily",
    "name": "0002-com-ubuntu-minimal-bionic-daily",
    "name": "0002-com-ubuntu-minimal-disco-daily",
    "name": "0002-com-ubuntu-minimal-focal-daily",
    "name": "0002-com-ubuntu-minimal-xenial-daily",
    "name": "0003-com-ubuntu-minimal-eoan-daily",
    "name": "0003-com-ubuntu-server-trusted-vm",
    "name": "test-ubuntu-premium-offer-0002",
    "name": "Ubuntu15.04Snappy",
    "name": "Ubuntu15.04SnappyDocker",
    "name": "UbuntuServer",

Specific information about an image:

az vm image list -p canonical -l eastus --offer 0001-com-ubuntu-pro-bionic --all --sku pro-18_04-lts

Inside linux or windows folder depending on the resource you want to deploy apply your terraform configuration

terraform init
terraform apply

After the successful run of the terraform script.

In order to delete the environment you can run

terraform destroy

By committing destroy your eight resources that deal with your virtual machine will disappear.

Repository for the code:

https://github.com/geralexgr/terraform-az-lin-win

Video tutorial on YouTube:

Posted on by — Leave a comment

Extend swap size on Redhat – Installer up to 128GB

If you try to allocate more than 128GB on swap partition for a Redhat installation you will notice that is not possible through installer. This is a known bug on Redhat bugzilla that is mentioned as resolved. However I tried to allocate 256GB swap with a RedHat 8.2 installer and I got the maximum size which is 128GB. In this article you will learn how to increase swap size manually.

First validate that there is available space on the volume group. (140g available on my case)

Then extend the swap logical volume

Deactivate swap file

format swap

Reactivate swap partition.

You can verify swap space with 

free -g
Posted on by — Leave a comment

Configuring firewalld on Linux systems – zone creation

Firewalld is the default firewall module on newer Linux distributions that replaced its ancestor iptables.

One of its biggest advantage is firewall-cmd tool that makes easy to configure your own policies/zones through command line.

When installed, firewalld should be enabled and started.

systemctl enable firewalld; systemctl start firewalld

The default zone that is configured after installation is public on which the default network interface is added.

You can get a list of services allowed with the command:

firewall-cmd --zone=public --list-all

For the sake of the article we will create a new zone and allow some services on it.

Create a new zone with:

firewall-cmd --permanent --new-zone=custom

Make custom zone your default:

firewall-cmd --set-default-zone=custom

Reload firewalld module so that changes take place. Everytime you need to change a firewall setting a reload must take place.

systemctl reload firewalld

Add a custom ssh or application port on your created zone

firewall-cmd --permanent --add-port=11233/tcp --zone=custom

Add a build in service with a known port:

firewall-cmd --permanent --add-service=https --zone=custom

Add an IP address that could access your zone:

firewall-cmd --permanent --zone=custom --add-source=192.168.1.254

Posted on by — Leave a comment

Mastering NTP configuration on Linux systems

The Network Time Protocol (NTP) is a networking protocol for clock synchronization between computer systems over packet-switched, variable-latency data networks and it is very important for various components and applications.

Many problems occur when time is not synchronized between systems even for time difference of milliseconds.

The latest case that I faced with a customer is composed of some power Linux systems that host SAP applications. Due to only some seconds time difference some flows could not be completed and messages could not be sent successfully between servers. This situation caused problems on client’s production and business departments and had to be resolved.

Although NTP was configured on the systems, the basic configuration was not enough to address correct time differences between systems and we had to alter the configuration.

Two of the most common used NTP daemons that one can use for Linux systems are ntpd and chronyd. In my case I had to alter configuration for ntpd, but same apply for chrony and the only change is where the configuration file is stored.

In order to modify ntpd settings one should edit /etc/ntp.conf file and for chronyd the conf file is located on /etc/chrony.conf

You can browse ntpd and chronyd manual pages with below commands:

man chrony.conf 
man ntpd.conf

Understanding NTPD and options:

The basic configuration of ntpd is the server keyword on the configuration file. However this could not be enough and you may encounter a big offset if your NTP provider is physically located on a long distance. (this was our case actually). 

server IP on ntpd.conf

The premises on which NTP provider was located had a distance of some km and the connection was established through a dedicated network line. This however was not enough to keep time accurate.

As you can see below, the offset of the ntp with the one random system on client infrastructure was approximately 1,8 seconds (output is milliseconds) . 

ntpq -pn
ntpd command will show you among others delay and offset statistics

The wikipedia article that is attached below, describes how NTP protocol works. In general a typical NTP client regularly polls one or more NTP servers. The client must compute its time offset and round-trip delay. Time offset θ, the difference in absolute time between the two clocks, is defined by below equation:

The important thing to understand is that NTP polling does not directly synchronize the local system  clock to the server clock; rather, a complex algorithm calculates an  adjustment value for each tick of the local system clock.

As a result, shorter  polling intervals cause NTP to make large but less accurate calculations that never stabilize, causing the local system clock to wander.

Longer  polling intervals allow NTP to calculate smaller tick adjustments that  stabilizes to a more accurate value, reducing wander in the local system  clock.

In many systems, administrators setup NTP only with the iburst option. This option only works for initial synchronization and will not be helpful on the normal system operation.

On the other hand, the burst option would be better, as on every synchronization attempt you will get more accurate calculations.

These options specify the minimum and maximum poll intervals for NTP
messages, in seconds as a power of two.

With burst option, chronyd/ntpd will shorten the interval between up to four requests to 2 seconds or less when it cannot get a good measurement from the server. The number of requests in the burst is limited by the current polling interval to keep the average interval at or above the minimum interval, i.e. the current interval needs to be at least two times longer than the minimum interval in order to allow a burst with two requests.

Adjust minpoll and maxpoll values:

These options specify the minimum and maximum poll intervals for NTP
messages, in seconds as a power of two.

The maximum poll interval defaults to 10 (1024 s), but can be increased by the maxpoll option to an upper limit of 17 (36 h).

The minimum poll interval defaults to 6 (64 s), but can be decreased by the minpoll option to a lower limit of 3 (8 s).

As described it would be better to increase maxpoll and minpoll so that you decrease traffic and improve accuracy.

In my case I altered the configuration to the below one and it seems that time differences between systems got better.

server IP burst minpoll 7 maxpoll 11

Check leap status:

ntpq -c rv
Leap_none = successful synchronization. Also offset is 76ms

Sources:

https://access.redhat.com/solutions/39194

https://en.wikipedia.org/wiki/Network_Time_Protocol