Stopping an EC2 instance that you don't need can save you money and improve security. In this post, I demonstrate how to use the AWS CDK to provision an EC2 instance that is automatically stopped when there are no open SSH or Session Manager connections to it.

These days I most often use an EC2 instance as a Bastion server. I use SSH or Session Manager to connect to it and access a protected database from there. This is usually just for occasional maintenance or debugging. An "always on" instance for this purpose feels wasteful since it would just sit idle most of the time. Because I only use it with an SSH (or Session Manager) connection we can automate the process of stopping it when there are no remaining connections. The primary downside of this is that I must manually start the instance (and wait for it to boot) when I need to use it. In my case, it is a tradeoff I'm happy to make.

Below I demonstrate how to use the AWS CDK to provision an EC2 instance that is automatically stopped when there are no open SSH or Session Manager connections to it. The full source code is available on GitHub at https://github.com/mpvosseller/cdk-ec2-autostop.

The approach we take is as follows:

  1. Install a bash script report-metrics.sh on the instance to determine whether it is "active" or not and publish the result as a CloudWatch metric.
  2. Configure a cronjob on the instance to run report-metrics.sh once a minute.
  3. Configure a CloudWatch Alarm to monitor the metric and stop the instance when it is reported inactive for more than 15 minutes.

Report Metrics Script

Below is the report-metrics.sh script and it does most of the heavy lifting. It runs on the instance and determines whether it should be considered "active" or not. It then publishes a CloudWatch metric with the result. We consider the instance active if it has any open SSH or Session Manager connections to it or if the instance was recently booted.

We use various tools and services to accomplish this job:

#!/bin/bash

# Publish a CloudWatch metric to report whether this instance should be considered active or not.
# We consider it active when there are any open SSH or Session Manager connections to it or if it
# was recently booted.

availabilityZone=$(curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone)
region=$(echo "${availabilityZone}" | sed 's/[a-z]$//')
instanceId=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)
stackName=$(aws --region "${region}" ec2 describe-instances --instance-id "${instanceId}" --query 'Reservations[*].Instances[*].Tags[?Key==`aws:cloudformation:stack-name`].Value' | jq -r '.[0][0][0]')
uptimeSeconds=$(cat /proc/uptime | awk -F '.' '{print $1}')
uptimeMinutes=$((uptimeSeconds / 60))
ssmConnectionCount=$(aws ssm describe-sessions --filters "key=Target,value=${instanceId}" --state Active --region "${region}" | jq '.Sessions | length')
sshConnectionCount=$(/usr/sbin/ss -o state established '( sport = :ssh )' | grep -i ssh | wc -l)
((totalConnectionCount = ssmConnectionCount + sshConnectionCount))

# note that "ssh over ssm" connections are double counted

isActive=0
if [ "${totalConnectionCount}" -gt 0 ] || [ "${uptimeMinutes}" -lt 15 ]; then
    isActive=1
fi

metricNameSpace="${stackName}"
aws --region "${region}" cloudwatch put-metric-data --metric-name "ConnectionCount" --dimensions InstanceId="${instanceId}" --namespace "${metricNameSpace}" --value "${totalConnectionCount}"
aws --region "${region}" cloudwatch put-metric-data --metric-name "UptimeMinutes" --dimensions InstanceId="${instanceId}" --namespace "${metricNameSpace}" --value "${uptimeMinutes}"
aws --region "${region}" cloudwatch put-metric-data --metric-name "Active" --dimensions InstanceId="${instanceId}" --namespace "${metricNameSpace}" --value "${isActive}"

Crontab File

Below is the crontab file. Nothing exciting here. It is a standard crontab file that requests the report-metrics.sh script to be run every minute.

* * * * * /home/ec2-user/report-metrics.sh

CDK Application

Below is the main CDK application code. If you are familiar with the AWS CDK it should be mostly straightforward. The most important aspects are:

import * as cloudwatch from '@aws-cdk/aws-cloudwatch'
import * as actions from '@aws-cdk/aws-cloudwatch-actions'
import * as ec2 from '@aws-cdk/aws-ec2'
import * as iam from '@aws-cdk/aws-iam'
import { CfnOutput, Construct, Duration, Stack, StackProps } from '@aws-cdk/core'

export class CdkEc2AutostopStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props)

    const vpc = ec2.Vpc.fromLookup(this, 'Vpc', {
      isDefault: true,
    })

    const keyName = '' // for ssh you must enter the name of your ec2 key pair

    const instance = new ec2.Instance(this, 'Instance', {
      init: ec2.CloudFormationInit.fromConfig(
        new ec2.InitConfig([
          ec2.InitPackage.yum('jq'),
          ec2.InitFile.fromFileInline(
            '/home/ec2-user/report-metrics.sh',
            './assets/report-metrics.sh',
            {
              owner: 'ec2-user',
              group: 'ec2-user',
              mode: '000744',
            }
          ),
          ec2.InitFile.fromFileInline('/home/ec2-user/crontab', './assets/crontab', {
            owner: 'ec2-user',
            group: 'ec2-user',
            mode: '000444',
          }),
          ec2.InitCommand.shellCommand('sudo -u ec2-user crontab /home/ec2-user/crontab'),
        ])
      ),
      instanceName: 'AutoStopInstance',
      vpc,
      keyName: keyName || undefined,
      instanceType: ec2.InstanceType.of(ec2.InstanceClass.T3, ec2.InstanceSize.NANO),
      machineImage: ec2.MachineImage.latestAmazonLinux({
        generation: ec2.AmazonLinuxGeneration.AMAZON_LINUX_2,
        cpuType: ec2.AmazonLinuxCpuType.X86_64,
      }),
    })

    // WARNING: this opens port 22 (ssh) publicly to any IPv4 address    
    // instance.connections.allowFrom(ec2.Peer.anyIpv4(), ec2.Port.tcp(22), 'SSH access');

    // permissions for session manager
    instance.addToRolePolicy(
      new iam.PolicyStatement({
        actions: ['ssmmessages:*', 'ssm:UpdateInstanceInformation', 'ec2messages:*'],
        resources: ['*'],
      })
    )

    // permissions for report-metrics.sh script
    instance.addToRolePolicy(
      new iam.PolicyStatement({
        actions: ['ec2:DescribeInstances', 'ssm:DescribeSessions', 'cloudwatch:PutMetricData'],
        resources: ['*'],
      })
    )

    const alarm = new cloudwatch.Alarm(this, 'Alarm', {
      alarmName: `Idle Instance - ${this.stackName}`,
      metric: new cloudwatch.Metric({
        // this metric is generated by report-metrics.sh
        namespace: this.stackName,
        metricName: 'Active',
        dimensions: {
          InstanceId: instance.instanceId,
        },
        statistic: 'maximum',
        period: Duration.minutes(15),
      }),
      comparisonOperator: cloudwatch.ComparisonOperator.LESS_THAN_THRESHOLD,
      threshold: 1,
      evaluationPeriods: 1,
      treatMissingData: cloudwatch.TreatMissingData.NOT_BREACHING,
    })
    alarm.addAlarmAction(new actions.Ec2Action(actions.Ec2InstanceAction.STOP))

    new CfnOutput(this, 'InstanceId', {
      description: 'Instance ID of the host. Use this to connect via SSM Session Manager',
      value: instance.instanceId,
    })
  }
}

Once deployed, I get an EC2 instance that can be manually started when needed and it will be automatically stopped when there are no remaining SSH or Session Manager connections to it. This lets me sleep just a tiny bit better at night.

The full code can be found on GitHub at https://github.com/mpvosseller/cdk-ec2-autostop.

If you found this helpful or have some feedback please let me know.