Customizable monitoring script on AWS Lambda

Background

I have a few applications that I operate and monitoring them is a basic requirement. I utilized a script running on an EC2 instance in a Docker container that pinged the services and pushed some values to CloudWatch. Then I added some alarms to let me know if there is a problem. Even though it was also monitoring itself, it wasn't a problem as the alarm went off even for missing data.

It was working well, but I felt I shouldn't need a server to run a rather simple script every few minutes. This use-case is the perfect candidate for a serverless application: runs occasionally, and it runs fast.

Let's see how to migrate it to a serverless deployment!

What to monitor

For a website, common candidates to are the response status and the response length. I chose the latter and since an error response is usually much smaller than a normal one, I can set up an alarm based on that.

But there are more complicated scenarios:

• Calling a monitoring endpoint, and parse the returned information
• Connecting to some sort of database
• Accessing protected services

I wanted a script that can be deployed serverless with minimal effort, and can adapt to support a wide range of use-cases.

How to use the script

The monitoring script is based on NodeJS and as such, it can use all the rich ecosystem of npm.

First, clone the repository from GitHub.

The main functionality is in the handler.js. You'll find everything there to make it work.

As an example, the provided handler gets the response length of google.com and puts it to CloudWatch:

• The putMonitoring call interfaces with CloudWatch and puts the provided value to the provided metric
• The getResponseLength returns the length of an URL
• The performMonitoring glues the previous two together
• And finally, the index calls and wait for all monitoring to complete

By adding more functions that retrieve values from various sources, you can implement any monitoring you'd like.

For local development, call the functions that produce the values, without calling the CloudWatch code. This way by the time you finally deploy you'll have the critical parts tested.

CloudFormation

The script uses CloudFormation to deploy everything in one step. This is particularly useful, as even this simple Lambda function requires multiple resources. By packaging everything inside a template file, all the boilerplate is handled alongside the main functionality.

The provided template calls the function every minute, but can be configured.

The last important part is the policy of the Lambda execution role. In the sample template, it has access to the CloudWatch Logs in order to enable logging, as well as to put data into CloudWatch. Don't forget to add more permissions if any is needed.

How to deploy

As with most CloudFormation templates, you need some setting up:

• An Access and a Secret key with permissions to create all the resources
• The AWS CLI installed and configured
• An S3 bucket that you can upload the code to

Package the code

Since the code must be uploaded to an S3 bucket, the first step is to install all the npm dependencies and package the template.

Use the script below, just fill in the bucket name where the AWS CLI will upload the lambda code. Make sure that the bucket is in the same region where you'll deploy the template.

cd src && npm i && cd .. && mkdir -p .tmp && aws cloudformation package --template-file cloudformation.yml --s3-bucket <bucket> --output-template-file .tmp/output.yml

Deploy

To deploy the template, use this script, and fill in the stack name:

aws cloudformation deploy --template-file .tmp/output.yml --stack-name <stack name> --capabilities CAPABILITY_IAM

After the deployment is successful, you'll see the values appearing on the CloudWatch console. You can then go ahead and define alarms that will trigger when there is a problem.

Conclusion

This script is a simple solution to monitor your resources in a highly versatile way. You can define any custom code that produces a value to put into CloudWatch, so that it can easily support most use-cases. Deployed with CloudFormation also ensures that all the resources required to perform its function are handled together.