AWS instance planning strategy is crucial to your long-term cloud gains. Taking the time to understand and pick the right instances for your apps is well worth the time and effort as it will directly impact your AWS pricing and bill.
Do you know your application well enough to be able to zero in on a specific instance? Even if you do, can you do it at scale across hundreds or thousands of instances? Which instance type is the best to use when the only differences between them are things you can’t measure—HVM vs. Paravirtualization; or EBS vs. instance storage? Which one provides the best financial value? Should you just replicate your existing allocation to minimize risk? How can you balance the two?
Read on to see how you can surgically tackle these questions to make the right choices for your organization.
AWS pricing strategy follows well-established norms of economics; you get significant price breaks for:
- 1. Buying at scale—via a volume discount program
- 2. Long-term commitments—in 1- and 3-year lengths
- 3. Prepayment—advance payments result in bigger discounts
An early commitment of resources and cash allows AWS to better plan their resource allotments; in return, they pass on their savings from that benefit to you.
But this implies that you are able to pin down your resource requirements in advance. Wait—that’s early binding! It doesn’t sound very cloud-like, does it?
AWS realized this early on; in 2009 they introduced EC2 Reserved Instances (RI). The specific definition and capability of an RI has evolved over time, but the idea has stayed the same: an RI is a like a prepaid voucher for an instance. You buy your instances, and then the AWS cashier applies that sum to whichever instance meets the Terms and Conditions of the voucher.
Over time, AWS has greatly expanded those voucher Terms and Conditions to give you greater flexibility. But of course, this flexibility comes at a cost: the greater the flexibility, the lower the discount.
RIs are a middle ground. You get a pricing break by making a commitment to a specific set of resources, without having to tie down those resources to a specific implementation/instance. This idea has worked so well in EC2 that AWS has rolled out the concept of reservations for other services as well, including RDS, DynamoDB, CloudFront, and RedShift. These all support capacity reservation in return for discounts.
But we’re getting ahead of ourselves. We’ll do a deep dive into RIs and best practices in a future article. For now, let’s focus on the basics of instance planning and discuss how to get your EC2 planning right—even before you start the optimization process.
EC2 Instance Proliferation
What started off as a small, well-differentiated menu of AWS EC2 instance families has now become a laundry list of choices. In 2013, there were 10 EC2 instance types; as of October 2018, nearly 150 instance types were available in any given AWS region, differentiated across dozens of parameters.
The net result is that you have a lot of choice when it comes to picking the right instance. Choice is good, right?
Yes—if you know how to leverage it. It is good to be able to tailor resources to a specific need, but capability assumes your specific need is very well understood. Many times it is—if you have a GPU-driven application, for example, or an in-memory database. But for the vast majority of “middle ground” applications, so much choice makes it easy either to end up in “analysis paralysis,” or simply to make the wrong choice.
For example, of the 150-odd instance types, there are 21 instance choices for four vCPUs. If you wanted a 4 vCPU server with (roughly) 32GB of memory, there are 10 choices. Each has approximately the same number of EC2 compute units, but with a price differential of 3x among them (ranging from r4.xlarge at $.266/hr. to i2.xlarge at $.853/hr.).
Choosing wrong carries a penalty. EC2 instance pricing follows an exponential curve (see Figure 1 for a high-level breakdown and Figure 2 for a detailed look at just the C instance family). This means that oversizing instances—which may be a good way to reduce risk—has a significant financial impact. It’s important to get your instance sizing right at the beginning, or be able to course-correct as soon as possible.
Data-Driven Instance Choice
The only practical way to solve a data-intensive problem is with Data-Driven Decision-Making. With HyperCloud™ Analytics, we regularly benchmark over 400 million pricing combinations of instances and services across different clouds. This data is collated against inventory, performance, and utilization data collected from either your existing IT systems or your detailed AWS bill.
This information allows us to give you an accurate instance recommendation that is based on actual allocation, performance, and utilization history. Analytics gives you not only the low-risk “as-is” translation into an EC2 instance, but also calculates and prices a “right-sized” recommendation (see screenshot for details).
By eliminating guesswork and using actual data to drive decision-making, HyperCloud Analytics can ensure that you’re taking the best tradeoff between risk and cost-benefit.
Iterative Cost Optimization
“Dear Cloud, What Have You Done For Me Lately?“
Getting it right the first time around is important; getting it right every time is priceless. The cloud is hardly static. Economies of scale drive cloud economics: CSPs regularly drop prices, and new services come online. In addition, business requirements change, and application profiles are constantly evolving.
It is important to stay ahead of the pricing curve by continuously re-analyzing your pricing and looking for opportunities to optimize. By regularly monitoring both your cloud spend and your performance data, HyperCloud Analytics can find opportunities for optimization
Continuous change also creates interesting arbitrage opportunities that can result in unexpected savings. The application planning function in HyperCloud Analytics can use cost-cloud arbitrage across different clouds to determine the best placement for your application.
Once you’ve migrated to the cloud, HyperCloud Analytics can project cost savings from both in-cloud arbitrage (moving to the same instance in a different region or moving to a different instance type), and cross-cloud arbitrage (moving to a comparable instance in another cloud). Figure 4 shows the different options for arbitrage available for a running EC2 instance. Options shown include comparable EC2 instances, as well as instances in other clouds (in this case, GCP), along with projected costs.
This iterative cost-optimization ensures that you are continuously leveraging the best the cloud has to offer in terms of cost/benefit.
The HyperGrid™ Difference
HyperCloud Analytics from HyperGrid enables key functionality in the cloud:
- Data-Driven Instance Choices. HyperCloud Analytics allows you to find the right EC2 instance, based on price and performance, by using data-driven decision-making. This process compares over 400 million price points across public clouds and benchmarks them against performance and utilization data gathered from your on-prem IT systems or your AWS bill.
- Continuous Instance Optimization. By regularly evaluating your utilization and billing, HyperCloud Analytics can make intelligent recommendations on instance, region, price, and performance that are the best fit for your needs. In addition, Analytics will benchmark AWS pricing against other comparable clouds, so you can leverage cross-cloud arbitrage, if feasible.
The combination of these makes it easier for you to identify and eliminate cloud overspend and maximize the benefit of your cloud investment. Read more on AWS instance planning.
All this goodness—and we haven’t even gotten to RIs yet! Stay tuned to our blogs, we will discuss how to use RIs to further streamline your AWS EC2 costs. We will examine RI best practices.
Manoj Nair joins HyperGrid from HPE where he was GM and VP of Product Management for Converged Infrastructure. His team was responsible for driving the Product Strategy and Roadmap across all elements of the Converged Portfolio & Infrastructure Management. Prior to HPE, Manoj was SVP leading strategy and R&D for the Public Cloud solutions at EMC. This was an incubation team working across the EMC federation of companies. Previously, Manoj was SVP & GM at RSA – responsible for IAM & Authentication product lines. Previously he led R&D and Product Management for RSA Security Management portfolio. Manoj also led R&D for EMC's internal incubation project, EMC Infoscape, as well as the architecture of the EMC PowerPath product family. Manoj has also held development and research positions at Data General, Novell and US NSF funded Research Labs. He is also the holder of over a dozen patents granted by USPTO in Systems Software, File systems, Information Management and Security. Manoj holds a M.S. in Computer Science from Clemson University.
Forbes Technology Council, Official Member 2018
- AWS Instance Planning – Getting It Right the First Time Around - November 9, 2018
- Cloud Management Platforms Aren’t Dead - October 31, 2018
- HyperCloud™ Analytics Is Now Available on AWS Marketplace - September 18, 2018