Cloud Bursting | A Complete Explanation

Curious about cloud bursting? We explain what cloud bursting is and how it allows you to use a hybrid cloud to maximize your computing resources.

What is cloud bursting?

Cloud bursting is a hybrid cloud deployment technique that combines private cloud and public cloud resources to deal with peak demand on IT resources. During peak periods, demand for an application can exceed the available resources running in an on-premises private cloud. When this happens the demand-on scalability from public cloud can be used to quickly support the excess demand in a “cloud burst”.

What Is Hybrid Cloud Architecture and How Does It Empower Cloud Bursting?

To better understand cloud bursting, it’s important to understand cloud architecture.

Cloud environments are built on networks of hardware and software. Unlike traditional servers or even server data centers, cloud environments bind together servers, databases, file systems, hardware, and software into a virtual environment that enables features like accessible storage, scalable computing, and high-performance file transfers and data processing. These resources are typically offered to users through vendors that build, maintain, and upgrade these systems, but systems can be off-site or on-premise.

With that in mind, there are three primary cloud environment configurations:

• Public Cloud: This is by far the most common cloud configuration, where multiple users will have private instances stationed on shared servers. While your cloud storage or computing instance is secured by software and separated from other cases, you will share server hardware and resources with multiple users, known as “tenants.” In fact, you can think of this as an actual apartment or hotel: each tenant has their room, but they are all in the same building. Public cloud provides secure, scalable, and cost-effective resources to all users.
• Private Cloud: The opposite of public cloud, private environments are single-tenant, meaning that that user only accesses the resources used by the user. Private cloud environments can be stationed both as third-party services or as dedicated on-premise architectures, but in both cases will offer additional security and customization options to the tenant—for a price.
• Hybrid Cloud: As the name suggests, hybrid environments combine private and public architecture to leverage the benefits of both. You’ll often find hybrid cloud users storing critical files in private cloud instances and then putting other information or resources into the public part of the system where scalability is more paramount.

Even though most of our general understanding of cloud environments comes from a public cloud, many enterprise organizations focus on hybrid clouds that give them some of the benefits of both while mitigating the limitations. One of these limitations involves the ability to scale with demand.

How Does Cloud Bursting Work?

One of the most significant selling points of cloud infrastructure is its high levels of flexibility and scalability. This is generally true for the public cloud, where on-demand resources are readily available. When it comes to private cloud environments, however, scalability is a bit limited due to restrictions on private resources and cost.

Enterprise companies using hybrid cloud infrastructure environments, or even private cloud environments with public resource access, can minimize waste through the process of hybrid cloud bursting.

Essentially, a cloud bursting deployment model works as such:

• An organization has private or hybrid cloud infrastructure in place for its daily use.
  This infrastructure serves everyday purposes. Periodically, demands on this on-premises infrastructure exceed the available resources on-premises. Scaling resources in a private cloud to support the excess demand would be a costly and time-consuming effort to procure and deploy new server, network, and storage resources in the data center.
• Using public cloud resources the organization can quickly scale on-demand for a short period of time to support the excess demand. This operation is the “burst” to the cloud.
• Once peak demand passes, the need for those extra cloud-based resources passes, the entire environment in the public cloud be scaled down and even scaled away.

What Are the Different Types of Cloud Bursting?

Not all cloud bursting is created equal, and different configurations can serve other use cases.
Broadly speaking, there are three types of cloud bursts:

• Manual Bursting. Involves your organization manually provisioning resources during a burst, giving you more control and allowing for more strategic bursts, especially when you need to maintain the freedom of local system resources.
• Automated Bursting: As the name suggests, automated bursting relies on predefined rules and policies that define how resources will be allocated and under which conditions those resources will be used for bursting. This usually allows fast and responsive bursting without direct admin intervention.
• Cloud Bursting for Distributed Load Balancing. This type revolves around the concept of “load balancing,” or the process of provisioning cloud resources, monitoring, and storage, and then deploying those resources. With distributed load bursting, applications are distributed to provisioned public clouds, and once private cloud thresholds are met, workloads are automatically offloaded to the shared instance of the application.

What Are Some Challenges and Cloud Bursting Use Cases?

Cloud bursting, while a relatively simple concept, is still a complex and involved process. One of the best practices and best first steps that you can take to approaching cloud bursting as a part of your infrastructure is to plan.

Think of it this way: moving between private (off-site or on-premise) and the public cloud takes a lot of work. From provisioning resources to transferring data and applications to those resources, you need to understand precisely how you want to accomplish that task rapidly, in real time, and without it being a drain to the resources you are trying to allocate for performance.

This is where the concept of cloud elasticity comes in. Elasticity is how a system can bend and mold itself to the needs of the user. That’s not to say that cloud environments change entirely to meet the needs of users. However, it is to say that an elastic infrastructure that can quickly provision resources will find more success during a cloud burst.

Achieving this means a few things:

• Understand how your cloud environment works. Are you using third-party hybrid systems? Is your private cloud on-premise? Do you have on-premise servers that supplement resources? Any of these resources can serve as a part of an effective burst.
• Define and organize your underlying software systems. If you’re running high-performance workloads that you want to stretch out onto public instances to handle demand, you might face a bottleneck in terms of system performance. Cloud file systems and optimized data throughout can strangle performance no matter how you’re expanding your system.
• Map out what workloads should expand into the public cloud and which shouldn’t. Not everything needs to push out to public space during a burst. You might find that only specific aspects of an analytics or public-facing interface need to scale rapidly, while back-end workloads are acceptable on private servers. By mapping out needs beforehand, you won’t waste resources during a burst.
• Layout compliance and regulatory obligations. Public cloud bursting seems like an excellent idea for nearly any and every organization. Remember, however, that public environments might not meet the compliance requirements. Take a cloud bursting example with retail operations to handle customer data, which may leave that data vulnerable and you in noncompliance with PCI DSS regulations, which in turn can damage your reputation with major payment processors.

Cloud bursting best supports organizations that have temporary and often predictable upswings in computational demand. Retailers with online stores can often use carefully planned cloud bursts to support increased pressure on online sales portals. Additionally, IT management or information officers can scale data processing to the cloud to avoid expanding or reducing physical data centers in ways that disrupt employment or customer experience.

WEKA, the Data Platform for Cloud

The WEKA Data Platform provides the performance and scalability you need to affordably run performance-intensive applications like AI or HPC in any cloud. WEKA eliminates the need to over-provision resources in the cloud to meet demanding performance requirements and removes the need to manage multiple copies of data, simplifying and accelerating data pipelines while dramatically reducing costs. Customers who use WEKA for AI, HPC and other performance-intensive workloads in the cloud reduce infrastructure costs by 50%, accelerate data-intensive workloads by 65%, and accelerate data pipelines by 90%, leading to increased revenue by as much as 20%.

Additional Resources

WEKA on AWS
Hybrid Cloud Storage
Cloud Storage