Build a Highly Available cluster for Centos/Redhat with pacemaker

With the latest version of pcs package there are some changes regarding the implementation of a highly available cluster which is based on centos/redhat operating system. Currently the latest package available for pcs is the 0.10.4 as shown below on which cluster with pacemaker 2.x and corosync 3.x are supported.

pcs package version

Firstly authenticate the nodes of the cluster (changed from previous versions)

Create and enable cluster on both nodes

Check cluster status after the creation and the online nodes.

Enable cluster on all nodes

Reboot one node to verify cluster functionality and check active nodes. When a node of cluster goes down, cluster is still online because of the second node.

Create a cluster resource of type IPaddr2 in order to support an application/service.

Ping the IP to verify that it is online

Shutdown the node on which cluster resource IPaddr2 is running and verify that IP is still accessible. You can verify during the shutdown and the resource migration from one node to another that icmp package respond is a bit slower. (3ms instead of <1ms)

Verify that cluster resource cluster-ip is online on the second node.

Centos cluster IP resource in stopped state – pacemaker

When a two nodes pcs cluster is created without a stonith device the below error may appear when creating an IPaddr2 cluster resource. After the creation the resource cannot be brought online.

In order to resolve the issue stonith-enabled setting should be disabled. Afterwards the resource would become online as shown on the below screenshot.

Corosync Cluster Engine exiting with status 8 – Failed to start

The below error can occur when one tries to create a new HA cluster with pcs package. Corosync fails to start with the below description.

Corosync error

The error can be resolved by editing /etc/hosts file and by commenting the first two lines for localhost. At the end I appended a new line that maps localhost with a specific IPv4 IP but this could be unnecessary.

After that when one tries to create and enable the cluster the result would be successful.

Rescan storage disk capacity physical RHEL – multipath storage device

Sometimes infrastructure servers are physical and directly connected to storage and not virtual ones. In virtualization cases, one has to assign a new virtual disk on the virtual server and proceed with the expand. In case of a physical server one should do the following in order to assign more space on a partition or a lvm.

Find storage adapters and rescan them:

where X is 0 to 10 in my system (some online and some offline)

After re scanning storage adapters a new multipath device will be visible and ready to be used.

4 paths are available for this storage device ( disk 50G)

Initialize your disk and use it appropriately.