Mastering NTP configuration on Linux systems

The Network Time Protocol (NTP) is a networking protocol for clock synchronization between computer systems over packet-switched, variable-latency data networks and it is very important for various components and applications.

Many problems occur when time is not synchronized between systems even for time difference of milliseconds.

The latest case that I faced with a customer is composed of some power Linux systems that host SAP applications. Due to only some seconds time difference some flows could not be completed and messages could not be sent successfully between servers. This situation caused problems on client’s production and business departments and had to be resolved.

Although NTP was configured on the systems, the basic configuration was not enough to address correct time differences between systems and we had to alter the configuration.

Two of the most common used NTP daemons that one can use for Linux systems are ntpd and chronyd. In my case I had to alter configuration for ntpd, but same apply for chrony and the only change is where the configuration file is stored.

In order to modify ntpd settings one should edit /etc/ntp.conf file and for chronyd the conf file is located on /etc/chrony.conf

You can browse ntpd and chronyd manual pages with below commands:

Understanding NTPD and options:

The basic configuration of ntpd is the server keyword on the configuration file. However this could not be enough and you may encounter a big offset if your NTP provider is physically located on a long distance. (this was our case actually).

The premises on which NTP provider was located had a distance of some km and the connection was established through a dedicated network line. This however was not enough to keep time accurate.

As you can see below, the offset of the ntp with the one random system on client infrastructure was approximately 1,8 seconds (output is milliseconds) .

ntpd command will show you among others delay and offset statistics

The wikipedia article that is attached below, describes how NTP protocol works. In general a typical NTP client regularly polls one or more NTP servers. The client must compute its time offset and round-trip delay. Time offset θ, the difference in absolute time between the two clocks, is defined by below equation:

The important thing to understand is that NTP polling does not directly synchronize the local system  clock to the server clock; rather, a complex algorithm calculates an  adjustment value for each tick of the local system clock.

As a result, shorter  polling intervals cause NTP to make large but less accurate calculations that never stabilize, causing the local system clock to wander.

Longer  polling intervals allow NTP to calculate smaller tick adjustments that  stabilizes to a more accurate value, reducing wander in the local system  clock.

In many systems, administrators setup NTP only with the iburst option. This option only works for initial synchronization and will not be helpful on the normal system operation.

On the other hand, the burst option would be better, as on every synchronization attempt you will get more accurate calculations.

These options specify the minimum and maximum poll intervals for NTP
messages, in seconds as a power of two.

With burst option, chronyd/ntpd will shorten the interval between up to four requests to 2 seconds or less when it cannot get a good measurement from the server. The number of requests in the burst is limited by the current polling interval to keep the average interval at or above the minimum interval, i.e. the current interval needs to be at least two times longer than the minimum interval in order to allow a burst with two requests.

Adjust minpoll and maxpoll values:

These options specify the minimum and maximum poll intervals for NTP
messages, in seconds as a power of two.

The maximum poll interval defaults to 10 (1024 s), but can be increased by the maxpoll option to an upper limit of 17 (36 h).

The minimum poll interval defaults to 6 (64 s), but can be decreased by the minpoll option to a lower limit of 3 (8 s).

As described it would be better to increase maxpoll and minpoll so that you decrease traffic and improve accuracy.

In my case I altered the configuration to the below one and it seems that time differences between systems got better.

Check leap status:

Leap_none = successful synchronization. Also offset is 76ms

Sources:

https://access.redhat.com/solutions/39194

https://en.wikipedia.org/wiki/Network_Time_Protocol

Rescan storage disk capacity physical RHEL – multipath storage device

Sometimes infrastructure servers are physical and directly connected to storage and not virtual ones. In virtualization cases, one has to assign a new virtual disk on the virtual server and proceed with the expand. In case of a physical server one should do the following in order to assign more space on a partition or a lvm.

Find storage adapters and rescan them:

where X is 0 to 10 in my system (some online and some offline)

After re scanning storage adapters a new multipath device will be visible and ready to be used.

4 paths are available for this storage device ( disk 50G)

Initialize your disk and use it appropriately.

Configure Postfix to Send Mail Using an External SMTP Server

Sometimes due to network configuration you cannot use your SMTP server and you want to send emails through a proxy/relay. Below you can find how to configure a linux machine so that to act as an SMTP relay and forward emails to one SMTP server.

First if not installed you should install postfix package for your linux server depending on the distribution. For example on a RedHat server you should do:

Then you should edit the below postfix configuration lines. Postfix configuration is handled by /etc/postfix/main.cf

  • Change inet_interfaces from localhsot to all

Configure the source from which the emails will be sent. For example you can use a specific host or a subnet of hosts. In my case I chose a subnet so the second line should be uncommented.

  • Configure your subnet source Address and mask.
  • Add your SMTP server address in the latest line

Finally test your configuration by a powershell command to verify the functionality.

Manage Windows machines with Ansible (winrm)

Ansible is a very powerful automation tool that is developed from RedHat. Many large organizations rely on Ansible to automate tasks and procedures. In this article I will explain how one can use ansible to manage windows clients and servers.

In order to manage windows hosts ansible winrm plugin should be used to communicate with the client/server machine.

The first step is to verify that ansible is installed.

Alongside with ansible, on the control node the pywinrm module should be also installed. By default this one is not installed and one should do it manually.

The managed Windows client/server machines should be configured to allow remote connections. A very useful power shell script is already developed from other users and it needs only to be executed on the managed host.

If your execution policy is prohibiting this script to be executed, you should set-execution policy to RemoteSigned as shown below and then execute the powershell.

Control node should have network connectivity with the managed hosts.

Some environmental variables should be used, so that ansible knows how the connection will be performed (winrm). I included those variables in my inventory file as I created this lab just for demonstration. My inventory file looks like below:

Lastly make sure that the user that is used for the connection has administrative rights on the managed windows hosts. Otherwise some error codes will be returned.

Lastly confirm ansible on managed host is working by using win_ping module.

Documentation:

https://www.ansible.com/blog/connecting-to-a-windows-host