DaaS (DevOps as a Service) – Part 1

  • Cloud

Joshua Seidel
August 11, 2021 142 views

Two individuals from a DevOps team

What is IaC? When did it start? What kinds of tools are there? Which one should I choose? Let’s tackle each of these one by one throughout this blog.

What is IaC?

IaC is Infrastructure as Code, this means writing code, scripts, automation to bring servers, load balancers, databases etc. UP and online. Rather than logging into your favorite flavor of Cloud computing provider, or your local VM on-premise system and configuring and launching your resource, you’re writing code that you execute like a set of directions to bring everything up in one simple command. This way of automating infrastructure allows you and your company to scale, expand, shrink, modify your infrastructure easily, maintaining sanity, replicating it to other datacenters, and arguably most importantly keeping track of changes to your infrastructure in the same source control systems developers have been using to track changings in software.

 

When did it start?

That’s pretty tough to say… do we count bash scripts engineers have written to invoke CLI commands or APIs? If so, no one knows… Infrastructure as Code is a concept. It’s not a specific tool, but how you are converting the provisioning of resources via Code rather than a set of instructions on how to build a server and setup a database.

 

What kinds of tools are out there?

I’m going to cover what I would consider the Top 3 used tools that fall under IaC (for AWS), some may add other tools, but I’m talking about the toolsets that can and are designed to manage infrastructure.

Terraform: My personal favorite, and the go to for Softrams. Terraform was initially released on July 28th, 2014 from HashiCorp. I want to come out now and say, I’m biased… I’ve used every toolset I’m about to cover, and this is by far my favorite… we will get into why later on. Terraform is written in Go, a pretty easy to understand language you can look at the code with minimal experience in coding and understand what its doing and why. It also helps that the community is great and huge. The code is well commented, and modules are well documented. It supports all the major players. AWS, Azure, Google Cloud, Oracle Clout, VMware and OpenStack. Terraforms large community helps it stay up to date pretty well with changes that the providers it supports make. While Terraform doesn’t get updated as fast as let’s say CloudFormation, but it is fast enough for all our use cases. The one great thing is its open source. If the module you’re using doesn’t support something, take a quick Udemy lesson on go, and write the upgrade. For example, a few years ago, I worked for a company that used Chef, we needed terraform to be able to call chef and utilize an encrypted databag. Terraform didn’t support it. So I learned Go and summited a change. Terraform also allows you to organize your code as you see fit, within custom build modules, in 1 HUGE file or thousands of tiny files, its really up to you. Terraform knows based on *correct references* “hey don’t create the Route53 record that requires the IP address, until the server is created. It also can “self heal”. Kill of a resource manually, it will rebuild it the next time you run it, exactly how your built it the time before.

 

CloudFormation: Do we call CloudFormation a IaC tool. Yes… Kind of… CloudFormation is a PaaS from Amazon that allows you to script out YAML or JSON based instructions(code) for how you want to launch and configure resources in AWS by the Console or via a CLI command. You can actually launch a cloud formation stack via Terraform too ;-). What’s great about CloudFormation is it is updated… RIGHT AWAY. When Amazon releases a new product in AWS, the CloudFormation resources to manage it come with it. It’s also pretty well documented of course and has really good support behind it (AWS Support). The Bad… If statements are a bit of a pain, as its need to be written in its own way, it need someone to build it that has experience. You can’t do If, Else, you need to do If, If, If, If. And at the end of the day it gets ugly to look at, and not very straight forward for those who don’t know CloudFormation to be able to look at it and understand right away what it’s doing. It’s also Linear and does not make relationships between resources obvious. Thus, in a lot of cases if you delete a resource created by a cloud formation stack, the cloud formation stack dies. It doesn’t always know or say “oh, that resource in my instructions isn’t there anymore?!!? Let me launch it again”. Something that does stand out in CloudFormation from the others is the Amazon made CloudFormation Designer. This tool gives you a visual representation of your cloud formation stack, and actually allows you to create stacks and launch them using this visual tool. This is a great feature for those engineers out there that want or need visual representation and like the GUI feel vs a IDE or terminal editor.

 

Ansible: I would call Ansible the stateless engine that could. Like the saying “the little engine that could”. The reason I use that statement, is ansible is pretty light weight. To use Ansible you need: a terminal, install ansible, write playbooks. But it doesn’t just function as a IaC tool, it works as a CM (Configuration Management) tool. This would allow a team or company to focus on one toolset to handle provisioning their resources and configuring them down to the tiniest detail. Ansible is written in Python, a popular codebase which is known for its efficiency and performance. Ansible was released to the public on February 20th, 2012 and has a very strong following. So much so, RedHat acquired it in 2015. Although the tool is written in Python; all the average engineer or developer needs to know is how to google and write in YAML. Many say you can master ansible in as little as 2 hours. Ansible has a HUGE following, and HUGE community backing it up. It’s actually Softrams GO TO configuration management toolset, but we will cover CM Tools in another series. Ansible again is stateless, unlike CloudFormation and Terraform, Yes, I know of Tower and AWX but we’re not talking about them yet J . Every time you run a playbook against a server, it’s like you never ran it before. Ansible will talk to the server, usually over SSH. Essentially ask it questions or check for “things” and then make changes as needed or let you know that it’s OK. Ansibles provisioners or what they call “cloud_modules” support a good number of providers like AWS, Azure, Cloud Stack, Google Cloud, OpenStack and VMware, as well as various other smaller set of modules for Docker or various DNS, firewall etc.

 

Which one should I choose?! And WHY

For IaC we choose Terraform. Terraform is a great tool, and easy to learn.

 

So how do we utilize it… Softrams builds out Infrastructure functions as modules. For example, creating an ECS cluster using a custom AMI. There are a few parts here, for one of our main customer CMS, they have what is known as a Gold Image, this Gold Image is a pre-hardened version of Windows or RedHat, in this specific example RedHat, that has all the modifications and “pre-reqs” installed. With our ECS module we have to build it in a way that it creates all the resources that is utilized by a cluster, ECS Hosts (ec2 instances with ECS agent and docker installed), load balancer (in our case a ALB), target groups, security groups and IAM roles. By making a “module”. We can build out a “set” of resources that are needed to work together to build the ECS cluster as its own IaC codebase. We then just need to reference that module in our main terraform and pass along variables to instruct Terraform to build a unique cluster. This setup goes great with a Terraform feature called Workspaces.

 

Module Layout Example:

VPC:

-rw-r–r–  dhcp_options.tf

-rw-r–r–  flow_logs.tf

-rw-r–r–  internet_gateway.tf

-rw-r–r–  nat_gateway.tf

-rw-r–r–  route53.tf

-rw-r–r–  routes.tf

-rw-r–r–  subnets.tf

-rw-r–r–  variables.tf

-rw-r–r–  vpc.tf

 

The above fileset is broken out into functions of a VPC in AWS. Each file organized how we felt was best, working together by references.

 

All you then need to do is go into your main.tf and call the module

 

module “vpc” {

source           = “./vpc”

product         = “${var.product}”

env                 = “${var.env}”

cidr                 = “${var.cidr}”

data1_cidr    = “${var.data1_cidr}”

data2_cidr    = “${var.data2_cidr}”

app1_cidr     = “${var.app1_cidr}”

app2_cidr     = “${var.app2_cidr}”

dmz1_cidr     = “${var.dmz1_cidr}”

dmz2_cidr     = “${var.dmz2_cidr}”

dns_suffix     = “${var.dns_suffix}”

build               = “${var.build_vpc}”

build_route53 = “${var.build_route53}”

}

 

On to Workspaces, they allow you to create separate states, in our case, separate environments that could use difference instance sizes, different IP ranges or even not build some modules depending on the need. All while still using the same code base, same modules, just with a separate set of variables. Ref: https://www.terraform.io/docs/state/workspaces.html.

Lastly Terraform is versatile, works on multiple cloud providers, once you learn HCL or read through the extensive documentation which always include an example, it’s very easy to use and very feature rich. You can call your CM toolset like, Ansible, Chef or Puppet from Terraform, so once a resource gets created you can seamlessly have Terraform kick off a CM script against it to configure it.



More Stories

  • Kubernetes GitOps with ArgoCD

    Oreon Lothamer
    July 1, 2021

    While Kubernetes provides a robust platform for managing container workflows it can become cumbersome to manage, especially at scale. GitOps is a methodology for managing Kubernetes where Git is the single source of truth.

  • Staying Safe Online

    Ryan Mathus
    July 7, 2021

    In a world where everything is connected to the internet and all of the data you could possibly want is at your fingertips, it is even more important to stay safe online and use some of the basic techniques mentioned.

  • The How-To of Automation

    Ryan Mathus
    April 6, 2021

    Automation has improved security protocol, reduced remediation, and sped up breach detections. Our platform code called Eagle helps manage software supply chain and enables continuous compliance and security monitoring relying on open-source tool eco-system.