Apache CloudStack is a turnkey orchestration platform to deliver IaaS clouds. As a cloud infrastructure scales to hundreds or thousands of servers, with production environments supporting multiple applications and services, high availability becomes a key requirement. Just as for OpenStack, it is possible to deploy the CloudStack management server in a multi-node installation. Since the management server uses a MySQL database to store the state of all its objects, the database could become a single point of failure. The CloudStack manual recommends MySQL replication with manual failover in the event of database loss. We believe there is a better way.
In this blog post, we’ll show you how to deploy redundant CloudStack management servers with MariaDB Galera Cluster on CentOS 6.5 64bit. We will have two load balancer nodes fronting the management servers and the database servers. Since CloudStack relies on MySQL’s GET_LOCK and RELEASE LOCK, which are not supported by Galera, we will redirect all database requests to only one MariaDB node and automatically failover to another node in case the former goes down. So, we’re effectively getting the HA benefits of Galera clustering (auto-failover, full consistency between DB nodes, no slave lag), while avoiding the Galera limitations as we’re not concurrently accessing all the nodes. We will deploy a two-node Galera Cluster (plus an arbitrator on a separate ClusterControl node).
Our setup will look like this:
Note that this blog post does not cover the installation of hypervisor and storage hosts. Our setup consists of 4 servers:
- lb1: HAproxy + keepalived (master)
- lb2: HAproxy + keepalived (backup) + ClusterControl + garbd
- mgm1: CloudStack Management + database server
- mgm2: CloudStack Management + database server