Automating Static Website Deployment, Part 1

科技动态 2017-06-26 阅读原文

This article was originally posted here .

Now that I have 8 static websites deployed into AWS using S3 and CloudFront I need to automate the deployment process so that I can make changes quickly and get them to production with less effort. To that end I have developed some scripts and configurations that automate the deployment of this site to a QA environment and production when changes are pushed to either branch. I will be describing this work in probably 3 parts. This is the first.


I describe creating a Hugo Website but you can probably drop in any static site generator.

Building a Hugo Website

A Hugo web site is extremely easy to start. First create a new hugo site:

$ hugo new site

One thing you will almost certainly need is a theme . I use a git submodule link to import the theme. First you'll need to initialize the git repo:

$ cd
$ git init .

Next, you'll need to add the submodule you want. I will use the hugo-bootstrap-premium theme:

$ mkdir -p themes
$ cd themes
$ git submodule add ""
$ cd ..

Now you need to edit your config.toml:

baseURL = ""
languageCode = "en-us"
title = "CFM Test Site"
theme = "hugo-bootstrap-premium"

After that you need to start creating content. For instance:

$ hugo new post/

This will create a new file in the content directory named "post/" with some boilerplate front matter.

Please see Hugo's Site for details on building and maintaining a website using hugo. At this point we have a small but functional website using a theme. Next we will look at how to build and deploy this website to AWS automatically after each push.

Continuous Delivery

We are going to develop a set of scripts and configuration files that will allow the site to be continously deployed to an appropriate environment based on the branch changes were made to. In this example we will use GitFlow to manage our code.

$ git flow init

NOTE: Be sure to have installed git-flow first.

If you took the defaults when prompted, you will be on a branch named "develop". This will be our "QA" branch. When we push changes to this branch the system will automatically deliver our generated site to our QA environment.

Creating the Envronments

We have to have someplace to push this site. We are going to use S3 and Cloudfront to deploy our static website. Please see Hosting a Static Website for Pennies a Month for instructions on how to set this up. For now I am going to assume that we have already done this. That would mean we have a buckets named for QA and for Production created and CloudFront distributions created that serves up content for and .

Creating the Code Repository

Now we need to create the code repo in AWS CodeCommit.

$ aws codecommit create-repository 
    --repository-description "A test website"

The output will look like this:

    "repositoryMetadata": {
        "repositoryName": "",
        "cloneUrlSsh": "ssh://",
        "lastModifiedDate": 1497199943.645,
        "repositoryDescription": "A test website",
        "cloneUrlHttp": "",
        "creationDate": 1497199943.645,
        "repositoryId": "9663fef8-f28f-4806-aa09-e8218a774f7c",
        "Arn": "",
        "accountId": "############"

Now set the remote and push the current code to it:

$ git remote add origin ""
$ git add .
$ git commit -m "initial checking of"
$ git push --set-upstream origin develop

Now we can set the default branch so that new clones will use develop automatically:

$ aws codecommit update-default-branch --repository-name 
    --default-branch-name develop

Building the Website

To build the website, we need to run hugo and package the public site into a tarball.

Here is a script that will do that. We will put it in a subdirectory named 'website-tools' and call it ''.


usage() { 
  echo "usage: $0 -b  [-v ] [-d ] [-D]" 1>&2
  echo "  -b Specifies the base directory to build in." 1>&2
  echo "  -v The version tag to put on the tarball. Defaults to 'local'" 1>&2
  echo "  -d The base domain for the site. Default uses baseURL from config." 1>&2
  echo "  -D Turn on drafts." 1>&2
  exit -1

while getopts ":b:v:d:D" o; do
  case "${o}" in
      HUGO_OPT="--baseURL http://${OPTARG}/ ${HUGO_OPT}"
      HUGO_OPT="--buildDrafts ${HUGO_OPT}"
shift $((OPTIND-1))

cd "${DIR}" || exit -2

hugoOutput=$(mktemp /tmp/hugoOutput-XXXXXX)
hugo $HUGO_OPT 2>&1 | tee -a "${hugoOutput}"
ERRCOUNT=$(grep -c "^ERROR:" "${hugoOutput}")
if (( ERRCOUNT == 0 )); then
  cd "${DIR}/public" || exit -3
  tar czf "${DIR}/public-${VERSION}.tar.gz" -- *
  exit $?
  echo "$ERRCOUNT errors in hugo output" 1>&2
  exit "${ERRCOUNT}"

This script does 3 things:

  1. It builds the website using hugo
  2. It looks for lines beginning with "ERROR:" in the output and fails if one is encountered.
  3. It tars the "public/" folder into a tarball with the name public-(VERSION).tar.gz where the (VERSION) is either 'local' or a value passed into the script.

If we want drafts we can pass '-D' on the command line for this script. The '-d' option lets us override the domain name of the site so that links go to the correct domain name. Hugo will generate full URLs based on the "baseURL" setting in the config file. So overriding this for QA will prevent QA from having links into Production.

Presumably we want drafts in QA and not in Production so the command lines for our build in QA would be:

$ ./website-tools/ . -D -d -v develop-1234567

And Prduction will be:

$ ./website-tools/ . -v master-1234567

Deploying the Website

To deploy the website we need to copy the contents of the "public/" directory generated in the build step to the origin bucket in S3. After that is complete, we need to issue an invalidation request for the cloudfront distribution.

Here is a script that does that. We will put it in ./website-tools as well and call it


usage() {
  echo "usage: -h  [-b ] [-d ]"
  exit -1

while getopts ":b:h:d:" o; do
  case "${o}" in 

if [ -z "${HOST}" ]; then

if ! aws s3 sync "${DIR}" "s3://${HOST}/" --acl 'public-read'; then
  echo "ERROR attempting to push content to origin"
  exit -2

# if the distribution id is provided, then it is fronted by a CloudFront distribution.
# Issue the invalidation.
if [[ "${DISTRIBUTION_ID}" != "" ]]; then
  read -r -d '' JSON <<JSON
  "Paths": {
    "Quantity": 3,
    "Items": ["/*","/index.html","/"]
  "CallerReference": "invalidate-all-$(TZ=GMT date "+%Y%m%dT%H%M%S")"  
  INVALIDATION_ID=$(aws cloudfront create-invalidation 
      --distribution-id "${DISTRIBUTION_ID}" 
      --invalidation-batch "${JSON}" 
      | jq -r .Invalidation.Id)
  while : ; do
    sleep 10
    LOOPS=$((LOOPS + 1))
    INVALIDATION_STATUS=$(aws cloudfront get-invalidation 
        --distribution-id "${DISTRIBUTION_ID}" 
        --id "${INVALIDATION_ID}" 
        | jq -r .Invalidation.Status)
    echo -n "."
    if [ "$INVALIDATION_STATUS" == "Completed" ] || ((LOOPS == 30)); then
  if [ "$INVALIDATION_STATUS" != "Completed" ]; then
    echo "ERROR: Invalidation status check timed out. (300 seconds)"
    exit -3
    echo "Invalidation Complete."

This script does 3 things:

  1. Uses aws s3 sync to transfer the content of public/ to the origin bucket.
  2. Issues an invalidation to the passed distribution id.
  3. Waits for that invalidation to move to a "Completed" state or upto 5 minutes which ever is shorter.

So now, to deploy our current code to QA we issue the following command:

$ ./website-tools/ -h -d E3ODFZSADK1LN8

NOTE:Your distribution id will, of course, be different.

andProduction will look like this:

$ ./website-tools/ -h -d E3EHZFMLC8IG8T

Coming Up

In Part 2, I will discuss the process of creating the AWS CodeBuild buildspec.yml including how to detect what branch you are building so that you can deploy to the correct environment.


I have been writing software since 1979 when my father bought the family an Apple ][+. Since then I have spent the last 39 years learning how to program. I am still learning and I am teaching. I love to mentor and guide programmers not only because I can teach them but also because they teach me.

The Practical Developer

责编内容by:The Practical Developer阅读原文】。感谢您的支持!


AWS Cost and Usage Report Analysis with an... Home Blog ELK Stack AWS Cost and Usage Report Analysis wi...
Linux安装git和maven的详细过程 一、使用yum安装git 当前安装环境是centos6.5 由于在CentOS6的yum源中已经有git的版本了,可以直接使用yum源进行安装...
Fuzzing arbitrary functions in ELF binaries I decided to give a descent test to theLIEF project. Executable parsers are not...
git quick tip – branching and merging Sometimes you want to do experiment work or wants to patch the git master...
Terraform in an AWS Multi Account Environment Terraform is a great tool to spin up environments on AWS—or in other clouds. ...