Automating HAProxy using Ansible

Using ansible for configuring HAProxy with dynamic inventory over bare metal and AWS

Task Description:

12.1 Use Ansible playbook to Configure Reverse Proxy, i.e., HAproxy and update its configuration file automatically each time a new Managed node (Configured with Apache Webserver) joins the inventory.

12.2 Configure the same setup as 12.1 over AWS using instance over there.

To know about ansible visit: Industry Use Case on Automation Using Ansible-Demo | by Gursimar Singh | Medium

What is a Load Balancer?

Load balancing is defined as the methodical and efficient distribution of network or application traffic across multiple servers in a server farm. Each load balancer sits between client devices and backend servers, receiving and then distributing incoming requests to any available server capable of fulfilling them.

What is HAProxy?

HAProxy is a free, amazingly fast, and reliable solution offering high availability, load balancing, and proxying for TCP and HTTP-based applications. It is particularly suited for extremely high-traffic websites and powers quite a number of the world’s most visited ones. Over the years, it has become the de-facto standard open-source load balancer, is now shipped with most mainstream Linux distributions, and is often deployed by default on cloud platforms. Its mode of operation makes its integration into existing architectures very easy and riskless while still offering the possibility of not exposing fragile web servers to the internet.

TASK 12.1

In the inventory file, we create two groups, one for the load balancer and the other for the backend servers.

Now, the playbook for configuring:

To update the HAProxy Configuration file automatically each time a new Managed Node(Configured with Apache Webserver) joins the inventory.

{% for i in groups['backend_server'] %}
server app{{ loop.index }}  {{ i }}:80 check
{% endfor %}

Now, we run the playbook.

In the managed node, the configuration file is updated dynamically.

GitHub URL for the task: ARTH-Program/Task 12/12.1 at main · gursimarsm/ARTH-Program (github.com)

TASK 12.2

First, configure the Ansible dynamic inventory so that we can fetch IPs dynamically and launch all the operating systems over the cloud. Third, set up a load balancer through HAProxy, and all the files and folder related to this task is in /ansible/arthtask12.2/ folder.

DYNAMIC INVENTORY SETUP

Here we will set up a dynamic inventory on AWS using boto3, ec2.yml, and ec2.ini files.

Boto3 : is the Amazon Web Services (AWS) Software Development Kit (SDK) for Python, allowing Python developers to write software that uses services like Amazon S3 and Amazon EC2. You can find the latest, most up-to-date documentation at our doc site, including a supported services list.

Step 1:

• Installing the required software and libraries

$ yum install python3 -y
$ pip install boto3
$ pip install boto

Step 2:

• Create a directory

$ mkdir /ansible/arthtask12/mydb

• Download ec2.yml and ec2.ini from the ansible official dynamic inventory GitHub link in the /ansible/arthtask12.2/mydb folder.
• Note: Both files should be in the same folder.

$ wget https://raw.githubusercontent.com/ansible/ansible/stable-2.9/contrib/inventory/ec2.py
$  wget https://raw.githubusercontent.com/ansible/ansible/stable-2.9/contrib/inventory/ec2.ini

• Make the ec2.py file executable:

$ chmod +x ec2.py

• Open the ec2.py file and change env python to python3 in the first line as the python code is written in python2, but we’ll be using python3. So, we need to change it.

Step 3:

• Set environment variable for authentication

$ export AWS_REGION: <YOUR-AWS-REGION-NAME-HERE>
$ export AWS_ACCESS_KEY_ID: <YOUR-AWS-ACCESS-KEY-HERE>
$ export AWS_SECRET_ACCESS_KEY: <YOUR-AWS-SECRET-KEY-HERE>

• Update the /ansible/arthtask12.2/mydb directory in the ansible configuration file and set AWS private_key and user through which we want to launch AWS OS on AWS.

• Note: Go to that folder where the private key is present and run:

$ chmod 400 <private_key_name>

Dynamic inventory configuration is done. Now we can check by using:

$ ansible all — list-hosts

LAUNCHING AWS INSTANCES USING ANSIBLE-PLAYBOOK

We have three operating systems, one for HAProxy configuration and another two for web server configuration.

Below is the ansible playbook code for EC2 instances named play.yml.

vim  /ansible/arthtask12.2/play.yml

We are launching an OS, so we don’t have any IP address now, and to use the ansible-playbook or ad-hock command, we need the IP address as the host. So, we will use localhost.

While launching EC2 instances, we need to give the aws-access-key and aws-secret-key, which are confidential. So, we create a file and set all the variable values in this file.

$ vim  /ansible/arthtask12.2/var.yml
      aws_access:  xxxxxxxx
      aws_secret:  xxxxxxxxxxxxxxxxxxxxxx
      region: ap-south-1

Here, for instance, providing instance — tags is essential. We will use the tags in the configuration of the HAProxy file dynamically.

Using the Playbook

The playbook file contains the tasks to be executed on the remote server.

Run command:

$ ansible-playbook play.yml

Now let's try to ping the nodes:

$ ansible all -m ping

As we have configured dynamic inventory, we don’t need to write the IPs of all the systems in the inventory file. Dynamic inventory will fetch all the details dynamically.

We can list all the details about all the systems using the following:

$ ./ec2.py — list

We will use these two tags: tag_name_loadbalancer and tag_name_web_server

The system setup is done.

CONFIGURATION REVERSE PROXY (WITH APACHE WEBSERVER)

First, install HAProxy software in your ansible managed node

$ yum install haproxy -y

Now, go to the /etc/haproxy and copy haproxy.cfg in your main task folder i.e. /ansible/arthtask12.2

$ cd /etc/haproxy
$ cp haproxy.cfg /ansible/arthtask12.2
$ ls
$ vim haproxy.cfg

Now, edit this haproxy.cfg file so that it can automatically update each time a new managed node is created.

Now, create one more ansible playbook name as main.yml

Here I am using the tags name, which was fetched using dynamic inventory

tag_name_loadbalancer   <This for haproxy configuration>
tag_name_web_server      <This for apache webserver>

We can also check the syntax error using

$ ansible-playbook main.yml — check

Now, let's run the playbook: main.yml

Everything is working good. Now, we can check load balancer is working or not.

We have configured HAProxy in <tag_name_loadbalancer>, and this contains the load balancer IP.

Now open <Load_balancer_IP> multiple times to verify that the load balancer is working correctly.

The PHP code added to the page is :

<pre>
<?php
print`/usr/sbin/ifconfig eth0`;
?>
</pre>/

This prints the system's IP address so we can refresh the page multiple times and notice the load is being distributed.

GitHub URL for the task: ARTH-Program/Task 12/12.2 at main · gursimarsm/ARTH-Program (github.com)

We have successfully completed the tasks!