Security best practices recommend the latest base images (e.g. AWS AMI ) for spinning VM on the cloud. Up to date software patches reduce the risk of any security breach. This emphasizes a strong need to set up an Image Factory to automate the image creation process with the latest software versions.
Also, every organization has a standard list of software to be installed for a given image. Only the essential software must be part of the list to reduce the blast surface from a security standpoint. This list of software can either be installed once the VM is created or when the VM is in the process of creation. However, both these techniques are time-consuming and hence not recommended.
It would be preferred to bake this software list in the base image itself. HashiCorp Packer is an open-source tool that specializes in building automated machine images for multiple platforms from a single source configuration.
HashiCorp Packer Installation
Refer to HashiCorp documentation for Packer installation based on your hardware OS.
GitHub Actions
For this demo, we will use GitHub Actions to create CI/CD pipeline to automate this workflow and eventually push the baked image (AMI) in AWS. It is a platform to automate tasks within the software development lifecycle. It's an event-driven framework, which means we can carry series of commands for a given event or can be scheduled for one-off or repetitive tasks. (e.g. Execute a Test Suite on Pull Request creation, Adding labels to issues, Lint checks, etc.)
Actions are defined in YAML files, which allows pipeline workflow to be triggered using any GitHub events like on creation of Pull Requests, on code commits, and much more.
Prerequisites
- AWS User with Programmatic access
- AWS Access Key ID
- AWS Secret Access Key
- AWS IAM Privileges to create EC2 Instance (create, modify and delete EC2 instances). Refer documentation for the full list of IAM permissions required to run the amazon-ebs builder.
In this post, we will bake Open JDK (Java 8) in our Ubuntu base Image and push it into AWS. Packer configurations can be written in HCL (.pkr.hcl file extension) and JSON (.pkr.json) formats. We will use the HCL language for this demo.
Reference GitHub repository - pkr-aws-ubuntu-java
Code Time
Let us start writing Packer configuration. (I am using a Linux machine for this demo)
Packer Configuration
Create Project folder pkr-aws-ubuntu-java
mkdir pkr-aws-ubuntu-java && cd $_
Create a file named aws-demo.pkr.hcl
touch aws-demo.pkr.hcl
Open your favorite IDE (e.g. VSCode). Copy the below code in aws-demo.pkr.hcl file.
packer {
required_plugins {
amazon = {
version = ">= 0.0.2"
source = "github.com/hashicorp/amazon"
}
}
}
The packer {} block contains Packer settings, including a required Packer version. The required_plugins block in the Packer block, specifies the plugin required by the template to build your image. The plugin block contains a version and source attribute.
Source block
The source block configures a specific builder plugin, which is then invoked by the build block. Source blocks use builders and communicators to define virtualization type, image launch type, etc.
Copy the following code to aws-demo.pkr.hcl file.
variable "ami_prefix" {
type = string
default = "packer-aws-ubuntu-java"
}
locals {
timestamp = regex_replace(timestamp(), "[- TZ:]", "")
}
source "amazon-ebs" "ubuntu_java" {
ami_name = "${var.ami_prefix}-${local.timestamp}"
instance_type = "t2.micro"
region = "us-east-1"
source_ami_filter {
filters = {
name = "ubuntu/images/*ubuntu-xenial-16.04-amd64-server-*"
root-device-type = "ebs"
virtualization-type = "hvm"
}
most_recent = true
owners = ["099720109477"]
}
ssh_username = "ubuntu"
}
Variable ami_prefix is used to define the AMI image. Local variable timestamp helps ensure uniqueness to AMI name.
The amazon-ebs builder launches the source AMI, runs provisioners within this instance, then repackages it into an EBS-backed AMI. This builder configuration launches a t2.micro AMI in the us-east-1 region using an ubuntu:xenial AMI as the base image.
It creates the AMI named packer-aws-ubuntu-java+timestamp. AMI names must be unique else it will throw an error.
It also uses the SSH communicator - by specifying the ssh_username attribute. Packer is then able to SSH into EC2 instance using a temporary keypair and security group to provision your instances.
Build Block
The build block defines what Packer should do with the EC2 instance after it launches.
build {
name = "packer-ubuntu"
sources = [
"source.amazon-ebs.ubuntu_java"
]
provisioner "shell" {
inline = [
"echo Install Open JDK 8 - START",
"sleep 10",
"sudo apt-get update",
"sudo apt-get install -y openjdk-8-jdk",
"echo Install Open JDK 8 - SUCCESS",
]
}
}
provisioner block helps automate modifications to your base image. It leverages shell scripts, file uploads, and integrations with modern configuration management tools such as Ansible, Chef, etc.
The above provisioner defines a shell provisioner and installs Open JDK 8 in the base image.
The final file aws-demo.pkr.hcl should look as below.
packer {
required_plugins {
amazon = {
version = ">= 0.0.2"
source = "github.com/hashicorp/amazon"
}
}
}
variable "ami_prefix" {
type = string
default = "packer-aws-ubuntu-java"
}
locals {
timestamp = regex_replace(timestamp(), "[- TZ:]", "")
}
source "amazon-ebs" "ubuntu_java" {
ami_name = "${var.ami_prefix}-${local.timestamp}"
instance_type = "t2.micro"
region = "us-east-1"
source_ami_filter {
filters = {
name = "ubuntu/images/*ubuntu-xenial-16.04-amd64-server-*"
root-device-type = "ebs"
virtualization-type = "hvm"
}
most_recent = true
owners = ["099720109477"]
}
ssh_username = "ubuntu"
}
build {
name = "packer-ubuntu"
sources = [
"source.amazon-ebs.ubuntu_java"
]
provisioner "shell" {
inline = [
"echo Install Open JDK 8 - START",
"sleep 10",
"sudo apt-get update",
"sudo apt-get install -y openjdk-8-jdk",
"echo Install Open JDK 8 - SUCCESS",
]
}
}
GitHub Actions
Create a new file in the .github/workflows directory named github-actions-packer.yml
We will schedule this workflow to run in the wee hours - let's say 04:00 Hrs in the morning.
name - The name of your workflow. GitHub displays the names of your workflows on your repository's actions page - "AWS AMI using Packer Config"
name: AWS AMI using Packer Config
on - (Required) The name of the GitHub event that triggers the workflow. We have configured to trigger the workflow on schedule.
on:
schedule:
# * is a special character in YAML so you have to quote this string
- cron: '0 4 * * *'
jobs - A workflow run is made up of one or more jobs. These jobs can run in parallel or sequentially. Each job executes in a runner environment specified by runs-on.
job name - The name of the job displayed on GitHub.
runs-on - (Required) Determines the type of machine to run the job on. The machine can be either a GitHub-hosted runner or a self-hosted runner. Available GitHub-hosted runner types are: windows-latest / windows-2019 / windows-2016 / ubuntu-latest / ubuntu-20.04 etc.
jobs:
packer:
runs-on: ubuntu-latest
name: packer
steps - Sequence of tasks called steps within a Job. They can execute commands, set up tasks, or run actions in your repository, a public repository, or action published in a Docker registry.
The first step is to check out the source code in the runner environment.
Checkout V2- This action checks out your repository under $GITHUB_WORKSPACE, so your workflow can access it.
steps:
- name: Checkout Repository
uses: actions/checkout@v2
To ensure access to the AWS Cloud environment we need to configure AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in the runner environment. The values for these variables will be configured as GitHub Secrets in the below section.
Configure AWS Credentials - This action configures AWS credential and region environment variables for use in other GitHub Actions.
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
# aws-session-token: ${{ secrets.AWS_SESSION_TOKEN }}
# if you have/need it
aws-region: us-east-1
Init initializes the Packer configuration used in the GitHub action workflow.
# Initialize Packer templates
- name: Initialize Packer Template
uses: hashicorp/packer-github-actions@master
with:
command: init
Validate checks whether the configuration has been properly written. It will throw an error otherwise.
# validate templates
- name: Validate Template
uses: hashicorp/packer-github-actions@master
with:
command: validate
arguments: -syntax-only
target: aws-demo.pkr.hcl
Build executes the Packer configuration.
# build artifact
- name: Build Artifact
uses: hashicorp/packer-github-actions@master
with:
command: build
arguments: "-color=false -on-error=abort"
target: aws-demo.pkr.hcl
env:
PACKER_LOG: 1
The complete file github-actions-packer.yml will look as below.
---
name: AWS AMI using Packer Config
on:
schedule:
# * is a special character in YAML so you have to quote this string
- cron: '0 4 * * *'
jobs:
packer:
runs-on: ubuntu-latest
name: packer
steps:
- name: Checkout Repository
uses: actions/checkout@v2
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
# aws-session-token: ${{ secrets.AWS_SESSION_TOKEN }}
# if you have/need it
aws-region: us-east-1
# Initialize Packer templates
- name: Initialize Packer Template
uses: hashicorp/packer-github-actions@master
with:
command: init
# validate templates
- name: Validate Template
uses: hashicorp/packer-github-actions@master
with:
command: validate
arguments: -syntax-only
target: aws-demo.pkr.hcl
# build artifact
- name: Build Artifact
uses: hashicorp/packer-github-actions@master
with:
command: build
arguments: "-color=false -on-error=abort"
target: aws-demo.pkr.hcl
env:
PACKER_LOG: 1
The source code is ready and can be pushed to the GitHub repository. As configured, the workflow will be triggered at 04:00 hrs in the morning.
