(CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

2. About me • Guy Davies, Senior Systems Engineer, Sophos Ltd Sophos use AWS extensively in our operations to support our anti- malware, anti-spam, and threat-detection software and hardware devices. Also provide security products for AWS customers to complement their cloud security profiles: • UTM [ free trial available! ] • Secure Server for Linux [ AMI available in Marketplace ] www.sophos.com/aws

3. What to expect from the session • How to schedule tasks in AWS Lambda • Overview of the various options available • Building up a pure Lambda scheduling infrastructure • Resources and templates to implement this yourself

4. Events? Scheduling?

5. Why would I want to schedule Lambda? • Lambda designed as event-driven computing • Event-driven computing is awesome • Most of the time you want to trigger because something happens… HOWEVER

6. Why would I want to schedule Lambda? Sometimes you just plain need to do something on a schedule. Examples: • Log cleanup • Batching up statistics • Alarm clock • Infrastructure automation

7. Scheduling options on traditional infrastructure Unix • cron: recurring tasks */2 * * * * do_something • at: run a task at a specific time at 1615 oct 7 <<< “mail –s’AWS talk!’ guy.davies@sophos.com” Windows • Scheduled tasks • AT

8. Options for scheduling Lambda functions (1) 1. Spin up an Amazon EC2 instance, and use crontab to invoke Lambda • Why bother running Lambda at all? • Many folks want a pure-Lambda deployment • Running an instance means more to manage • Not hugely financially efficient

9. Options for scheduling Lambda functions (2) 2. Unreliable Town Clock (townclock.io) • Awesome public Amazon SNS topic • Chimes every fifteen minutes • Community supported • Has a 15 minute granularity

10. Options for scheduling Lambda functions (3) 3. Others… • Trigger from Amazon SWF • Trigger from an instance in AWS Data Pipeline • Trigger from an AWS CloudTrail upload into an Amazon S3 bucket All of these could be your solution! But what if we want a “pure” Lambda implementation that’s managed by AWS?

11. A pure Lambda scheduler

12. How do we generate a timing signal in AWS? 0 0.2 0.4 0.6 0.8 1 Photo: ‘Signetics NE5555N, …’ by Stefan506 is licenced by CC BY-SA 3.0. Source: https://en.wikipedia.org/wiki/555_timer_IC Amazon CloudWatch

13. CloudWatch as a time signal 1. Set an alarm on a CloudWatch metric 2. Alarm goes into ALARM state: • Triggers Amazon SNS which triggers Lambda • Lambda inverts the state of the metric 3. Alarm goes into OK state: • Triggers SNS which triggers Lambda • Lambda inverts the state of the metric

14. Configuring the CloudWatch alarm

15. Configuring the CloudWatch alarm Once the metric is inverted, the alarm will trigger at the top of the next minute: 1-minute resolution

16. Configuring the CloudWatch alarm All three states trigger the same SNS notification. The Lambda function figures which was the trigger and sets the CloudWatch metric to the opposite state

17. Putting it together Lambda cron function CloudWatch metric CloudWatch alarm triggers SNS topic10 Invoke further Lambda functions (if scheduled)

18. The Lambda function

19. Lambda function 1. The “event” is the SNS notification that is sent by CloudWatch. 2. Invert the value of the CloudWatch metric. 3. Jobs and schedule are managed as a separate JSON file in the bundle. 4. Invoke any Lambda functions in the schedule that are scheduled to be run this minute. 5. Done.

20. Main Lambda function exports.handler = function(event, context) { async.waterfall([ function (callback) { flip_cloudwatch(event,callback); }, read_crontab, execute_lambdas ], function (err) { if (err) context.fail(err); else context.succeed(); }); };

21. Flipping CloudWatch The event is an SNS message from CloudWatch: var snsmessage = JSON.parse(event.Records[0].Sns.Message); If it’s just gone into alarm, we want to reset to zero. Likewise if it’s just gone into OK, reset to 1. if (snsmessage.NewStateValue == 'ALARM') { value = 0.0 } else if (snsmessage.NewStateValue == 'OK' || snsmessage.NewStateValue == 'INSUFFICIENT_DATA') { value = 1.0 }; Push the new value to CloudWatch: var params = { MetricData: [ { MetricName: 'LambdaCron', Timestamp: new Date, Unit: 'None', Value: value } ], Namespace: 'LambdaCron' }; cloudwatch.putMetricData(params, function(err, data) { . . . });

22. Crontab-like Lambda configuration { "jobs": [ { "schedule": "*/3 * * * *", "function": "testfunction", "args": { "key1": "test1", "key3": "test3", "key2": "test2" } } ] }

23. Checking the schedule (1) Use the fantastic cron-parser library (https://github.com/harrisiirak/cron-parser/ MIT licenced) var parser = require('cron-parser'); Create a Date object which refers to the top of the current minute to compare the schedule to. var d = new Date(); d.setSeconds(0); d.setMilliseconds(0); Parse the crontab to find the ‘next’ runtime for the job (= now if it needs to run) var interval = parser.parseExpression(job["schedule"],{currentDate: d}); var runtime = interval.next();

24. Run each job that needs running if (datestring == runtimestring) { var lambda = new AWS.Lambda(); var params = { FunctionName: job["function"], InvocationType: "Event", Payload: JSON.stringify(job["args"]) }; lambda.invoke(params, function(err,data) { if (err) iteratorcallback(err); else iteratorcallback(null); }); } }

25. Demo

26. Use cases Trialling this for intelligent scaling on a schedule • AWS scheduled scaling has limitations: • Absolute number of desired instances • Limit of 120 tasks per month • Lambda function triggered by lambda-cron could do for example: • Every morning at 08:00 UTC, add 20% of capacity unless we have > 30 instances already running

27. Reliability and monitoring Empirically pretty reliable. Running since April without (much!) intervention. Even comes back after outages!

28. Reliability and monitoring Monitoring: CloudWatch! • Lambda invocation metrics will tell you that it’s running • Application-level monitoring on the jobs you are triggering 60 invocations / hr

29. Summary 1. Use CloudWatch alarms as two states to provide a timing signal to Lambda. 2. Trigger off all three states to enhance reliability. 3. Allows us to schedule tasks purely within Lambda. 4. The cron function invokes once a minute.

30. Resources Github: github.com/g-a-d/lambda-cron Email: guy.davies@sophos.com AWS CloudFormation stack Lambda function

31. Resources Libraries used: • async: https://github.com/caolan/async • avoid nested-callback-hell (great for folks more used to procedural programming) • MIT license • cron-parser: https://github.com/harrisiirak/cron-parser • parse crontabs • MIT license

32. Resources Remember to check www.sophos.com/aws • UTM for AWS (free trial available) • Secure server for AWS • Free trials, free home use AV

33. Remember to complete your evaluations!

34. Thank you!

(CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

Amazon Web Services

(CMP407) Lambda as Cron: Scheduling Invocations in AWS Lambda

A particular slide catching your eye?