Enterprise-ready golden images with one engineer

David Webster
4 min readSep 15, 2020
Batu Caves Kuala Lumpur by Bernard Spragg. NZ

Simplicity is critical when trying to be day one, especially when you need to deliver the images, thousands of accounts will use when creating their VMs. No longer do you have to get tied in the nitty-gritty of the AWS API by copying images to regions before sharing them or even needing complex pipelines consisting of many different AWS services, tools, and code to glue everything together. Like all things that have been custom created and are required regularly, they quickly get replaced by managed services. Last year AWS released EC2 Image Builder, a service that aims to solve all this complexity by allowing you to centralize image creation and sharing so that your organization can meet its compliance needs, with great ease, no packer, and no code.

A thing of the past

How EC2 Image Builder works

You can imagine EC2 Image Builder as something that takes in a custom/AWS/Marketplace AMI as input, installs some software, tests the image, and then outputs the AMI to one or many AWS accounts in as many different regions that you specify.

I do not want to bore you with the workings of Cloudformation, but it is nice to see how it correlates to the different parts that make up an EC2 Image Builder pipeline and for if you want to check out the code on Github.

First of all, you start by defining a recipe that describes which components you want to run in the build and test phase. These can be custom created components or the AWS pre-built components. Here I chose a Windows AMI and included the AWS pre-built component for STIG hardened images. I have also selected the AWS pre-built inspector component for testing the AMI against CIS standards, which will send the findings to AWS Inspector. AWS has done a great example of this recently with a blog about building STIG images.

Next, you need to create a distribution config that can share the AMI with multiple accounts and regions and the infrastructure config for defining the size of the instance building your AMI. You can also set TerminateInstanceOnFailure to false to investigate the AMI if you encounter any errors.

Finally, you define the pipeline, which combines the configs and recipes and where you can set the frequency of your pipeline execution.

In summary, when the pipeline runs, it goes through the three steps of building, testing, and then finally distributing the image. The build phase will install the STIG component, which will be executed and start hardening the running EC2. In reality, the component is simply a YAML file of either BASH or Powershell commands to run. Once the build phase completes, an AMI is created, and the EC2 is terminated, the test phase starts, and the Inspector agent is installed on the running EC2. After the AMI passes the test phase, the EC2 is terminated, and the AMI is then distributed to all the accounts across regions specified in the recipe’s distribution config.

Sharing the responsibility and ensuring compliance

You may wish to enforce the use and only the use of the images created by your EC2 Image Builder pipelines. AWS recommends resource-based tags, but if you are looking for something a little more preventative, you can achieve this using the following creative SCP, which denies the launching of an EC2 instance unless it is from the EC2 Image Builder account (imagebuilder_account_id), which is running the pipeline. However, this by itself limits the users to only those images. You may want these accounts to be able to create AMIs based on these base AMIs so that they can still add custom software; in this case, it is as straightforward as adding the account that the images are then shared with to the ec2:owner as we did earlier with the image_builder_account_id.

And there you have it end-to-end; the entire process took just over an hour to deploy the Cloudformation, run inspector tests, and output an enterprise-ready hardened AMI with CIS findings that you can take directly to your risk or compliance person on day one and as promised with only one engineer.

--

--