Autorecovery restores a failed EC2 instance without waiting for someone to intervene. AWS may recover instances after a single physical host fails, but it may not do so after a larger outage, such as a rack failure. A CloudWatch alarm can initiate recovery when the instance’s system status check fails. section 13.0 section 13.1
A Jenkins continuous integration server is a good fit when a brief interruption is acceptable but prolonged downtime would prevent colleagues from testing and deploying software. The same approach can suit other applications with that tolerance, including FTP and VPN servers. section 13.1
Set up recovery alongside the instance:
- Create a VPC, launch an EC2 instance in it, and install the application during bootstrap. A CloudFormation template can define these resources and the recovery alarm together. section 13.1
- Configure an AWS/EC2 CloudWatch alarm for the instance’s StatusCheckFailed_System metric. The example uses the Maximum statistic, a 60-second period, five evaluation periods, and a threshold greater than zero. section 13.1
- Set the alarm action to the EC2 recovery action, arn:aws:automate:${AWS::Region}:ec2:recover, and identify the instance with the InstanceId dimension. section 13.1
Listing 13.2 Creating a CloudWatch alarm to recover a failed EC2 instance
RecoveryAlarm: #1
Type: 'AWS::CloudWatch::Alarm'
Properties:
AlarmDescription: 'Recover EC2 instance when underlying hardware fails.'
Namespace: 'AWS/EC2' #2
MetricName: 'StatusCheckFailed_System' #3
Statistic: Maximum #4
Period: 60 #5
EvaluationPeriods: 5 #6
ComparisonOperator: GreaterThanThreshold #7
Threshold: 0 #8
AlarmActions: #9
- !Sub 'arn:aws:automate:${AWS::Region}:ec2:recover'
Dimensions:
- Name: InstanceId #10
Value: !Ref Instance
The CloudFormation alarm listing shows the metric, evaluation settings, recovery action, and instance dimension together. section 13.1
Autorecovery supports high availability, not fault tolerance: recovery can involve a short interruption. If even that interruption is unacceptable, recovering one instance is not enough; distributing the workload across multiple virtual machines is another availability approach. section 13.0
