There’s an approach that we seek at Panintelligence known informally as “Deployment at the click of a button”. At times, it can be an elusive goal but it’s something that we often aim to achieve on behalf of our customers. The idea is that we want to be able to create the complete, functioning software system by clicking a single button.
The button is a nice metaphor while also being a literal thing in AWS CloudFormation. It’s also emblematic of the principles of simplicity, comprehensiveness, and consistency.
For example, while you might be able to do the same with a single command, you would need to consider the setup and configuration required to run that single command which might reduce simplicity and consistency.
In this post, I describe the principles and motivation, user base, scope of a complete software system, assumptions and prerequisites, alternative scenarios, and documentation required for one click deployment of our BI analytics tools.
Principles
These are the three principles that the “one-button everything” mantra is based upon:
Comprehensive – The goal is to orchestrate the full solution stack, not partial implementations of the solution stack .
Consistent – Works the same way every time. Documentation is similar across solution stacks. Once you require “one-off” implementations, it makes it susceptible to errors.
Simple – Few steps and dependencies. Make it difficult to make mistakes.
These three principles guide the design of these one-button systems.
The users are us
The users of your one-button systems are often other engineers within your organization. A tired argument you might hear is that you don’t need to create simple systems for other engineers since they’re technical too.
I could not disagree more with this reasoning. As engineers, we should not be spending time on repetitive, failure-prone activities and put the burden on others – at scale. This belief doesn’t best serve the needs of the organization as most engineers should be spending time on providing features to users who receive value for their work.
What is the complete software system?
A common question we get is “what makes up a complete software system?” To us, the complete software system refers to all of the infrastructure and software that composes the system. For example, this includes:
- Networks (e.g., VPC)
- Compute (EC2, Containers, Serverless, etc.)
- Storage (e.g., S3, EBS, etc.)
- Database and Data (RDS, DynamoDB, etc.)
- Version control repositories (e.g., CodeCommit)
- Deployment Pipelines
- Orchestration of software delivery workflows
- Execution of these workflows
- Building and deploying application/Service code
- Test execution
- Static Analysis
- Security hardening, tests and analysis
- Notification systems
- Monitoring systems
Frequent problem scenarios when building a complete software system
Engineers will encounter problems when building infrastructure and applications on AWS. It is important to understand what type of problems may interfere with your work. Here are some examples that engineers may face:
“I am not getting an HTTP status code of 200 server response and unable to access my web application”
Engineers would have to configure the AWS Virtual Private Cloud (VPC) networking settings to enable users to communicate with the application. This can be achieved by configuring AWS Route 53, route tables, security groups, Network Access control list (NACL), Internet gateway, Load balancers and more.
“My application does not have enough space to run”
You can configure storage to EC2 instances with Elastic Block Store (EBS), Elastic File system (EFS), S3 and more. You have different methods of storage based on requirements and usage. As a result, it is important to know what type of storage and how much you need, or your systems may fail.
“I can’t SSH into my EC2 instance”
You can access your EC2 instances in multiple ways such as session manager, SSH on your local machine, create a Bastion host and more. However, you would need to make sure your networking and security configurations are correct in order to access your EC2 instance. Also, you may need to install a session manager agent on it. You can read the AWS documentation on SSH here.
As shown above, there are many elements building and configurations may cause problems. If you don’t have a set of instructions or good documentation, you will not have consistent successful builds.
What is AWS Cloudformation and how can Infrastructure as code can help?
Infrastructure as Code (IaC) means to manage your IT infrastructure using configuration files. AWS CloudFormation is a service that helps you model and set up your Amazon Web Services resources so that you can spend less time managing those resources and more time focusing on your applications that run in AWS.
You create a template that describes all the AWS resources that you want (like Amazon EC2 instances or Amazon RDS DB instances), and AWS CloudFormation takes care of provisioning and configuring those resources for you.
You don’t need to individually create and configure AWS resources and figure out what’s dependent on what; AWS CloudFormation handles all of that. You can read documentation on AWS Cloudformation here.
Prerequisites
In order for you to use AWS Cloudformation, you will need an AWS account and knowledge about AWS services so you are aware of the configurations you can set.
AWS Cloudformation examples
Instead of using AWS Management console on the web browser, I will show you some code snippets on how to configure your services by text, which can be JSON or YAML.
- Parameters - you can set values for your parameters into your Cloudformation stack and can always update your Cloudformation stack if values change over time. For example, providing the subnet groups for services or changing the tags so you can quickly identify your resources. As you can see below, I have stated “Parameters” and within parameters, I've provided “VPCCidrBlock”. I’ve also stated what “Type” it is and if an engineer does not want to change any values, the default value is set.
- Resources create and build the services. Here is an example of creating a VPC and attaching it to an internet gateway below. You can see the “Type” which is the AWS resource you wish to create and within that resource you can set the properties and configurations.
Panintelligence_VPC.YAML:
YAML
Parameters:
VPCCidrBlock:
Type: String
Default: 10.0.0.0/16
Description: The cidrblock for the VPC
Resources:
VPC:
Type: 'AWS::EC2::VPC'
Properties:
CidrBlock:
Ref: VPCCidrBlock
EnableDnsSupport: true
EnableDnsHostnames: true
Tags:
- Key: Name
Value: !Sub '${AWS::StackName}'
InternetGateway:
Type: 'AWS::EC2::InternetGateway'
DependsOn: VPC
AttachGateway:
Type: 'AWS::EC2::VPCGatewayAttachment'
Properties:
VpcId: !Ref VPC
InternetGatewayId: !Ref InternetGateway
You can import your parameters in a json format so don’t have to do it manually on AWS management console Cloudformation page.
Parameters.JSON:
JSON
{
{
"ParameterKey": "VPCCidrBlock",
"ParameterValue": "10.0.0.0/16"
}
}
How can you deploy?
AWS CLI:
In AWS CLI, you can run this line of code below and it will output the stack ID:
Bash
aws cloudformation create-stack --stack-name panintelligence /
--template-body file://Panintelligence_VPC.yaml /
--parameters file://parameters.json /
--capabilities CAPABILITY_NAMED_IAM
AWS Management console:
You could also use AWS management console by uploading the file. Go into AWS CloudFormation and create a new resource:
There are multiple ways to upload the code within CloudFormation, upload the YAML file from local machine:
After clicking next, I would need to go through the steps. The next step is parameters where I specify the stack name and the VPC cidr block:
Continue following the steps until you can “Create stack” and wait for the magic green words of “CREATE COMPLETE”:
Now go into your VPC console and you will see the resources have been created based on your specification.
AWS CloudFormation can provide clarity of how to build a complete software system on AWS. Engineers will be able to describe each resource of how they are configured and connected to other resources. Engineers can use AWS CloudFormation as a tool to debug the system due to scanning how it is built without spending some time on AWS Management console.
Within minutes you can deploy an AWS CloudFormation stack with your environment configured. With consistency and simple deployment, it is almost deployment of a button.
How does Panintelligence use AWS CloudFormation?
At Panintelligence, the cloud team has built two CloudFormation stacks to quickly support the customers for a Proof of Concept. That is an EC2 and ECS deployment stacks. This allows the customer to understand what resources they need and examples of configurations.
Also, while Panintelligence has manual deployment documentation to support customers it can take a considerable amount of time compared to CloudFormation stacks. By using our stack templates, we can get Panintelligence deployed within minutes and allow the customer to focus on their own data on the dashboard.
In addition, there are currently 200 AWS services that you can use so there is a variety of options on
how to build your infrastructure. While Panintelligence currently has two CloudFormation stacks, Panintelligence is also improving and creating more templates to meet different requirements using different AWS services. At Panintelligence we value customer feedback to help us to use the full potential of AWS.
Please take a look at the Panintelligence GitHub CloudFormation scripts here.
Myles Pucknell is a Cloud Engineer at Panintelligence who is fascinated with anything to do with cloud technology. He is always tinkering with new technology and looking for ways to automate and make processes more efficient. Read more blogs from the Panintelligence Engineering team here.
