AWS and other interesting stuff

A CloudFormation Custom Resource For CloudFront Origin Access Identities (OAI)

· by Steve Hogg · Read in about 5 min · (861 Words)
AWS AWS CloudFormation AWS CloudFront AWS Lambda Serverless Framework

A CloudFormation Custom Resource For CloudFront Origin Access Identities (OAI)

CloudFormation does not currently support OriginAccessIdentity (OAI) resources. Here are a couple of ways to deal with this:

1) Create the OriginAccessIdentity via CLI and pass it to CloudFormation using a parameter

Create the OAI via the CLI …

$ aws cloudfront create-cloud-front-origin-access-identity --cloud-front-origin-access-identity-config CallerReference=h4-q-and-a-oai,Comment=h4-q-and-a-oai

… use a parameter in CloudFormation template …

...
"Parameters": {
  "OriginAccessIdentity": {
    "Type": "String",
    "Description": "OAI (create not supported within CF directly)",
    "Default": ""
  }
},
...

… input the parameter when updating the stack:

$ OAI=$(aws cloudfront list-cloud-front-origin-access-identities \
  | jq '.CloudFrontOriginAccessIdentityList.Items[] | select(.Comment == "h4-q-and-a-oai") | .Id' \
  | sed 's/"//g')
$ aws cloudformation update-stack --stack-name H4QAndA \
  --template-body file://h4-q-and-a.json \
  --parameters ParameterKey=OriginAccessIdentity,ParameterValue=$OAI

2) Use a CloudFormation CustomResource to create/delete the OriginAccessIdentity

You can create a Custom Resource backed by an AWS Lambda function. I like using the Serverless Framework for deploying Lambda functions as it is a tidy way to manage/deploy code and configuration in one place.

mkdir cloudformation-cloudfront-oai
cd $_

Create a serverless.yml file with this content:

service: cloudformation-cloudfront-oai

provider:
  name: aws
  runtime: python2.7
  stage: prod
  region: ap-southeast-2

  iamRoleStatements:
   - Effect: "Allow"
     Action:
       - "cloudfront:ListCloudFrontOriginAccessIdentities"
       - "cloudfront:GetCloudFrontOriginAccessIdentity"
       - "cloudfront:GetCloudFrontOriginAccessIdentityConfig"
       - "cloudfront:CreateCloudFrontOriginAccessIdentity"
       - "cloudfront:DeleteCloudFrontOriginAccessIdentity"
     Resource: "*"

functions:
  get_oai:
    handler: oai.handler

resources:
  Description: "ServerLess: CloudFormation CustomResource for CloudFront OAI"

Custom Resources need to PUT a standard set of variables to a callback URL “ResponseURL”. Ryan Brown wrote a useful library to help make this easy, so install this in a vendor directory:

$ mkdir vendor
$ wget https://raw.githubusercontent.com/ryansb/cfn-wrapper-python/master/cfn_resource.py -O vendor/cfn_resource.py

Create the oai.py file with handlers for creating and deleting Origin Access Identities:

import sys
sys.path.append('./vendor')
import boto3
import logging
import cfn_resource


logger = logging.getLogger()
logger.setLevel(logging.INFO)

cfclient = boto3.client('cloudfront')

# set `handler` as the entry point for Lambda
handler = cfn_resource.Resource()


def get_oai(oai_name):
    result = cfclient.list_cloud_front_origin_access_identities()
    items = result['CloudFrontOriginAccessIdentityList']['Items']
    logger.exception('Items: %s' % items)
    id = filter(lambda n: n.get('Comment') == oai_name, items)[0]['Id']

    result = cfclient.get_cloud_front_origin_access_identity_config(Id=id)
    etag = result['ETag']

    return {'id': id, 'ETag': etag}


def create_oai(oai_name):
    id = 'UNKNOWN'
    try:
        oai_config = get_oai(oai_name)
        id = oai_config['id']
        etag = oai_config['ETag']
    except IndexError:
        config = {
            'CallerReference': oai_name,
            'Comment': oai_name
        }
        result = cfclient.create_cloud_front_origin_access_identity(
            CloudFrontOriginAccessIdentityConfig=config)
        id = result['CloudFrontOriginAccessIdentity']['Id']
        pass

    return id


def delete_oai(oai_name):
    try:
        oai_config = get_oai(oai_name)
        id = oai_config['id']
        etag = oai_config['ETag']
        result = cfclient.delete_cloud_front_origin_access_identity(
            Id=id, IfMatch=etag)
    except IndexError:
        raise

    return id


@handler.create
def create(event, context):
    props = event['ResourceProperties']
    oai_name = props['OAIName']
    oai_id = create_oai(oai_name)

    return {
        "PhysicalResourceId": oai_id,
        "Data": {
            "OriginAccessIdentity": oai_id
        }
    }


@handler.delete
def delete(event, context):
    props = event['ResourceProperties']
    oai_name = props['OAIName']
    oai_id = delete_oai(oai_name)

    return {"PhysicalResourceId": oai_id}

Deploy the function:

$ serverless deploy

Get the ARN for the function:

$ serverless info

Then use the function as a Custom Resource in CloudFormation:

{
  ...
  "Resources": {
      "MySiteOAI": {
        "Type": "Custom::OAI",
        "Properties": {
         "ServiceToken": "arn:aws:lambda:ap-southeast-2:<ACCOUNT_ID>:function:cloudformation-cloudfront-oai-prod-get_oai",
         "Region": "ap-southeast-2",
         "OAIName": "my-site-oai"
        }
      }
      ...
      "MySiteS3Bucket": {
        "Type": "AWS::S3::Bucket",
        "Properties": {
          "BucketName": "mysite.nz",
          "WebsiteConfiguration": {
             "IndexDocument": "index.html",
             "ErrorDocument": "404.html"
          }
        }
      },
      "MySiteS3BucketPolicy": {
        "Type": "AWS::S3::BucketPolicy",
        "Properties": {
          "Bucket": { "Ref": "MySiteS3Bucket" },
          "PolicyDocument" : {
             "Version":"2012-10-17",
             "Statement":[{
                "Sid": "PolicyForCloudFrontPrivateContent",
                "Effect": "Allow",
                "Principal": {"AWS": {"Fn::Join": [ "", [
                  "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity ",
                  { "Fn::GetAtt": [ "MySiteOAI", "OriginAccessIdentity" ] } ]]} },
                "Action": ["s3:GetObject"],
                "Resource": "arn:aws:s3:::mysite.nz/*"
             }]
          }
        }
    },
    "MySiteCloudFront": {
      "Type": "AWS::CloudFront::Distribution",
      "Properties": {
        ...
        "Origins": [ {
          "DomainName": "mysite.nz.s3.amazonaws.com",
          "Id": "MySiteOrigin",
          "S3OriginConfig": {
            "OriginAccessIdentity": {"Fn::Join": [ "", [
              "origin-access-identity/cloudfront/",
              { "Fn::GetAtt": [ "MySiteOAI", "OriginAccessIdentity" ] } ]]}
          }
        } ],
        ...
      }
    }
  }
}

Configure CloudFront for a Single-Page Web App

Single page web apps typically need all requests redirected to index.html; the apps set different URLs in browsers’ location bar for convenience e.g. so that back/forward buttons still work, but the page in the location doesn’t actually exist. If the user refreshes the page they’ll get a 404 error.

Instead, you can configure CloudFront to redirect all requests to index.html

{
  ...
    "MySiteCloudFront": {
     "Type": "AWS::CloudFront::Distribution",
     "Properties": {
        "DistributionConfig": {
          ...
          "CustomErrorResponses": [
            {
               "ErrorCode": 403,
               "ResponsePagePath": "/index.html",
               "ResponseCode": "200",
               "ErrorCachingMinTTL": 300
            },
            {
               "ErrorCode": 404,
               "ResponsePagePath": "/index.html",
               "ResponseCode": "200",
               "ErrorCachingMinTTL": 300
            }
          ]
          ...
        }
     }
    }
  ...
}

Getting Hugo To Work With S3 and CloudFront

I use Hugo for this site. Hugo compiles Mark Down into HTML into various sub directories. e.g.

study/DynamoDB/index.html

We want study/DynamoDB/ to be directed to index.html automatically by S3 rather than giving a 404 error. To do this you use the following XML in: Bucket > Static Web Hosting > Enable Website Hosting > Edit Redirection Rules:

<RoutingRules>
    <RoutingRule>
        <Condition>
            <KeyPrefixEquals>/</KeyPrefixEquals>
        </Condition>
        <Redirect>
            <ReplaceKeyWith>index.html</ReplaceKeyWith>
        </Redirect>
    </RoutingRule>
</RoutingRules>

The CloudFormation JSON to do this is:

"MyS3Bucket": {
   "Type": "AWS::S3::Bucket",
   "Properties": {
      "BucketName": "your-website-name.nz",
      "WebsiteConfiguration": {
         "IndexDocument": "index.html",
         "ErrorDocument": "404.html",
         "RoutingRules": [{
            "RedirectRule": {
               "ReplaceKeyWith": "index.html"
            },
            "RoutingRuleCondition": {
               "KeyPrefixEquals": "/"
            }
         }]
      }
   }
}

This configuration works when using the S3 website URL, but to get it working when CloudFront is in-front, you need to use a Custom Origin rather than an S3 one. i.e. :

use your-website-name.nz.s3-website-ap-southeast-2.amazonaws.com

instead of your-website-name.nz.s3.amazonaws.com

This is because CloudFront uses the S3 REST API for S3 origins, but it uses standard HTTP and honours redirects with the Custom Origin.

For this reason, we can’t use an OriginAccessIdentity with Hugo hosted on S3.

Comments