Using Terraform to Create an Image Resizer with AWS Lambda and S3

DevOps Chapter

EngineeringJul 11, 2024.12 min read

Terraform is an open-source Infrastructure as Code (IaC) tool that allows you to define and provision cloud resources using a declarative language. Learn how Terraform, in collaboration with AWS, revolutionizes infrastructure management. This tutorial will guide you through creating an image resizer using AWS Lambda and S3, demonstrating how Terraform can efficiently manage cloud resources for sustainable and scalable solutions. By defining infrastructure as code, Terraform allows for the automation of infrastructure deployment, scaling, and management, making it an essential tool for modern cloud operations.

In today's cloud era, serverless architectures are increasingly favored for building scalable and cost-efficient applications. A typical use case is creating an image resizer service that processes images on-the-fly, which is especially beneficial for web applications that need to serve images in various sizes to enhance performance and user experience. This article will guide you through setting up an image resizer using AWS Lambda and S3, with Terraform managing the infrastructure.

Prerequisites

Before we start, ensure you have the following:

AWS Account: An active AWS account.
Terraform: Installed on your local machine. You can download it from here.
AWS CLI: Installed and configured with your AWS credentials.
Basic understanding of AWS services and Terraform concepts.
Basic knowledge of Python programming language

Architecture overview

This is the overall architecture that we are going to deploy through Terraform, which involves both S3 and AWS Lambda. This serverless architecture allows for scalable, cost-efficient image processing without the need to manage underlying infrastructure. By leveraging Terraform to manage this infrastructure, we ensure reproducibility and version control of our cloud resources, aligning with best practices in Infrastructure as Code (IaC). Terraform enables us to define infrastructure configurations in human-readable files, which can be versioned and shared like any other code. This approach ensures that our infrastructure is consistently reproducible across different environments and that changes are tracked through version control. Additionally, Terraform's state management capabilities, such as storing state in S3 or DynamoDB, prevent configuration drift and ensure infrastructure consistency. Our architecture encompasses the following components:

AWS S3 Buckets: Two S3 buckets will be created—one for storing the original images and another for the resized images. These buckets serve as the primary storage for our image processing pipeline. Two buckets are used in order to avoid a infinite loop of resealing.
AWS Lambda Function: A Lambda function will be implemented to handle the resizing of images. This function will be triggered automatically whenever a new image is uploaded to the original image bucket.
IAM Roles and Policies: Appropriate IAM roles and policies will be configured to grant the necessary permissions for the Lambda function to interact with the S3 buckets.
S3 Bucket Notification: The original image S3 bucket will be set up to trigger the Lambda function upon new uploads, ensuring seamless and automated image processing.

Step 1: Set Up the S3 Buckets

We’ll create two S3 buckets: one for storing the original images and another for the resized images. Terraform makes this process straightforward.

Create a new directory for your Terraform project and a file named main.tf:

provider "aws" {
  region = "REGION"
}

resource "aws_s3_bucket" "original_images" {
  bucket = "your-original-images-bucket"
}

resource "aws_s3_bucket" "resized_images" {
  bucket = "your-resized-images-bucket"
}
# Replace "your-original-images-bucket" and "your-resized-images-bucket" and "REGION" with unique names for your buckets.

Step 2: Create the Lambda Function

Next, we’ll create a Lambda function that will resize the images. This function will be triggered whenever a new image is uploaded to the original images bucket. First, let’s write the Lambda function code in Python:

Create a file named lambda_function.py:

import boto3
from PIL import Image
import io

s3 = boto3.client('s3')

def lambda_handler(event, context):
    bucket = event['Records'][0]['s3']['bucket']['name']
    key = event['Records'][0]['s3']['object']['key']
    
    response = s3.get_object(Bucket=bucket, Key=key)
    image = Image.open(response['Body'])
    
    resized_image = image.resize((100, 100))
    
    buffer = io.BytesIO()
    resized_image.save(buffer, 'JPEG')
    buffer.seek(0)
    # Replace 'your-resized-images-bucket' with the name of your resized images bucket.
    s3.put_object(
        Bucket='your-resized-images-bucket',
        Key=key,
        Body=buffer,
        ContentType='image/jpeg'
    )
    
    return {
        'statusCode': 200,
        'body': 'Image resized successfully'
    }

Since we are using external packages, we should create a requirements.txt file containing this entry below:

Pillow

Now, we should first build our Lambda layer package since we are using a third-party library. The build can be achieved using the official AWS Lambda Docker image.

docker run -v "$(PWD)":/var/task "public.ecr.aws/sam/build-python3.9" /bin/sh -c '
  mkdir -p packaged layer/python && \
  pip install -r requirements.txt -t layer/python && \
  cd layer && zip -q -r9 ../packaged/pil.zip python; \
  cd .. && rm -rf layer
'

Add the following code to your main.tf to package this code into a ZIP file using the data source:

#This data source aims to generate an archive from content, a file, or a directory of files.
data "archive_file" "function_package" {
  type        = "zip"
  source_file = "${path.root}/lambda_function.py"
  output_path = "${path.root}/packaged/lambda_function.zip"
}

Step 3: Define the Lambda Function in Terraform

Add the following to your main.tf file to create the Lambda function and the necessary IAM role:

# Lambda layer containing Python package
resource "aws_lambda_layer_version" "pillow_layer" {
  filename            = "${path.root}/packaged/pil.zip"
  layer_name          = "image-resizer-layer"
  source_code_hash    = filebase64sha256("${path.root}/packaged/pil.zip")
  compatible_runtimes = ["python3.9"]
}

# Lambda function Role definition
resource "aws_iam_role" "lambda_role" {
  name = "lambda_s3_exec_role"
  assume_role_policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Action = "sts:AssumeRole",
        Effect = "Allow",
        Sid    = "",
        Principal = {
          Service = "lambda.amazonaws.com",
        },
      },
    ],
  })
}
resource "aws_iam_role_policy_attachment" "lambda_s3_policy" {
  role       = aws_iam_role.lambda_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}

#Created Policy for IAM Role
resource "aws_iam_policy" "policy" {
  name        = "lambda_s3_policy"
  description = "A test policy"

  policy = <<EOF
   {
"Version": "2012-10-17",
"Statement":[
    {
        "Effect": "Allow",
        "Action": [
            "s3:GetObject"
        ],
        "Resource": "arn:aws:s3:::${aws_s3_bucket.original_images.bucket}/*"
    },
    {
        "Effect": "Allow",
        "Action": [
            "s3:PutObject"
        ],
        "Resource": "arn:aws:s3:::${aws_s3_bucket.resized_images.bucket}/*"
    }
]
} 
    EOF
}
resource "aws_iam_role_policy_attachment" "lambda_s3_policy" {
  role       = aws_iam_role.lambda_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}

resource "aws_lambda_function" "image_resizer" {
  filename         = "packaged/lambda_function.zip"
  function_name    = "image-resizer"
  role             = aws_iam_role.lambda_role.arn
  handler          = "lambda_function.lambda_handler"
  runtime          = "python3.9"
  layers           = [aws_lambda_layer_version.pillow_layer.arn]
  source_code_hash = data.archive_file.function_package.output_base64sha256
}
#This allows S3 to invoke the lambda.
resource "aws_lambda_permission" "allow_s3" {
  statement_id  = "AllowExecutionFromS3"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.image_resizer.function_name
  principal     = "s3.amazonaws.com"
  source_arn    = aws_s3_bucket.original_images.arn
}

Step 4: Configure S3 to Trigger Lambda

Finally, configure the original images in the S3 bucket to trigger the Lambda function on new uploads. Add the following to your main.tf file:

resource "aws_s3_bucket_notification" "bucket_notification" {
  bucket = aws_s3_bucket.original_images.id

  lambda_function {
    lambda_function_arn = aws_lambda_function.image_resizer.arn
    events              = ["s3:ObjectCreated:*"]
  }

  depends_on = [aws_lambda_permission.allow_s3]
}

Step 5: Deploy the Infrastructure

Now, let’s deploy the infrastructure. Run the following commands:

terraform init
terraform apply
Note: Terraform will create a state file to track the resources it manages. Ensure this file is stored securely and not committed to version control.
# Confirm terraform apply with yes.

Note: Terraform will create a state file to track the resources it manages. Ensure this file is stored securely and not committed to version control. Confirm terraform apply with yes.

Step 6: Test the Image Resized

Simply upload an image to the original images S3 bucket. You should see a resized image appear in the resized images S3 bucket shortly after.

As you can see here, the initial size of the image is 79.5 KB.

And now, the reduced size of the image is 2.8 KB.

For more information on Terraform and AWS, check out the official Terraform documentation and AWS Terraform provider documentation.

Share this story