AWS October Webinar Series - AWS Lambda Best Practices: Python, Scheduled Jobs, and More

AWS October Webinar Series - AWS Lambda Best Practices: Python, Scheduled Jobs, and More

1. 1. © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Presenter Vyom Nagrani, Sr. Product Manager, AWS Lambda Q&A Moderator Ajay Nair, Sr. Product Manager, AWS Lambda October 29th, 2015 AWS Lambda Best Practices: Python, Scheduled Jobs, and More
2. 2. AWS Lambda: A compute service that runs your code in response to events Lambda functions: Serverless, trigger-based code execution Triggered by events: • Direct Sync and Async invocations • Put to an Amazon S3 bucket • Call to an API Gateway endpoint • And many more … Makes it easy to • Perform data-driven auditing, analysis, and notification • Build back-end services that perform at scale
3. 3. Continuous ScalingNo Servers to Manage AWS Lambda automatically scales your application by running code in response to each trigger. Your code runs in parallel and processes each trigger individually, scaling precisely with the size of the workload. Subsecond Metering With AWS Lambda, you are charged for every 100ms your code executes and the number of times your code is triggered. You don't pay anything when your code isn't running. AWS Lambda automatically runs your code without requiring you to provision or manage servers. Just write the code and upload it to Lambda. Benefits of AWS Lambda for building a server-less data processing engine 1 2 3
4. 4. AWS Lambda – how it works Bring your own code • Node.JS, Java, Python • Java = Any JVM based language such as Scala, Clojure, etc. • Bring your own libraries Flexible invocation paths • Event or RequestResponse invoke options • Existing integrations with various AWS services Simple resource model • Select memory from 128MB to 1.5GB in 64MB steps • CPU & Network allocated proportionately to RAM • Reports actual usage Fine grained permissions • Uses IAM role for Lambda execution permissions • Uses Resource policy for AWS event sources
5. 5. AWS Lambda – how it works Authoring functions • AWS SDK built in • Handle inbound traffic • Use processes, threads, /tmp, sockets, … Deployment options • Author directly using the console WYSIWYG editor • Package code as a ZIP and upload to Lambda or to S3 Stateless functions • Persist data using S3 / DynamoDB / Elasticache • No affinity to infrastructure (can’t “log in to the box”) Monitoring and Logging • Metrics in Amazon CloudWatch – Requests, Errors, Latency, Throttles • Logs in CloudWatch Logs
6. 6. AWS Lambda – event sources for creating real time data processing applications Amazon CloudWatch Amazon SNS … and the list will continue to grow! Amazon DynamoDBAmazon KinesisAmazon S3
7. 7. AWS Lambda – event sources for creating web, mobile & IoT application backends Amazon Cognito Sync Twilio Integration … and the list will continue to grow! Amazon IoT ActionsAmazon Echo SkillsAmazon API Gateway
8. 8. AWS Lambda – event sources for creating service extensions and serverless workers Amazon SES Actions Zapier Integration … and the list will continue to grow! Amazon SWF TasksAWS CloudFormation Custom Resources AWS Lambda Scheduled Events
9. 9. Lambda update: 5 new features announced earlier this month at re:Invent 2015 1. Python functions 2. Increased function duration 3. Function versioning & aliasing 4. Scheduled functions (Cron) 5. Accessing Resources in a VPC From a Lambda Function [coming soon] Details at: https://aws.amazon.com/blogs/aws/aws-lambda-update-python-vpc- increased-function-duration-scheduling-and-more/
10. 10. New feature: Python 2.7 support in AWS Lambda Description: Native support for writing Lambda functions in Python Benefit: Create Lambda functions in Python, complete with built-in access to the AWS SDK for Python, no need for learning new programming language for teams used to building applications in Python How it works: In the runtime selection for a Lambda function, you will now see Python in addition to Node and Java (Blueprints also available for Python)
11. 11. Sample Lambda function in Python - An Amazon S3 trigger that retrieves metadata for the object
12. 12. New feature: Increased function duration Description: Increase 60s max time duration of Lambda functions to 300s Benefit: processes larger objects and run more complex code in Lambda – e.g. video transcoding, Extract-Transform-Load (ETL) applications How it works: In the runtime selection for a Lambda function, you will now see Python in addition to Node and Java (Blueprints also available for Python)
13. 13. New feature: Function versioning & aliasing Description: Upload and manage multiple versions of same Lambda function Benefit: Each time you upload a fresh copy of the code for a particular function, Lambda will automatically create a new version and assign it a number (1, 2,3, and so forth). You can also create named aliases and assign them to specific versions of the function code. How it works: The Lambda function ARN format is expanded to allow an optional qualifier specification, which represents a function version or a function alias
14. 14. How Lambda function versioning & aliasing works – development of function code Developing in AWS Lambda stays simple: • Upload code • Make changes at any time • Last update wins handler (event, context): return(“version1”) handler (event, context): return(“version2”) lambda.CreateFunction lambda.UpdateFunctionCode lambda.Invoke(myFunction) lambda.Invoke(myFunction)
15. 15. How Lambda function versioning & aliasing works – publishing new versions Publish a new version from development at any time: • “Copied” dev version to a numbered version • Published versions are read-only (including configuration) • Simple, integer counter per function handler (event, context): return(“version1”) handler (event, context): return(“version2”) lambda.PublishVersion lambda.PublishVersion lambda.Invoke(myFunction:1) lambda.Invoke(myFunction:2)
16. 16. How Lambda function versioning & aliasing works – creating aliases for function versions Create a named alias to any version: • Allows function owner to map ARNs to code • Can be updated without changing clients • Development version gets default alias of $LATEST handler (event, context): return(“version1”) handler (event, context): return(“version2”) lambda.CreateAlias(“prod”) lambda.CreateAlias(“beta”) lambda.Invoke(myFunction:prod) lambda.Invoke(myFunction:beta)
17. 17. New feature: Scheduled functions (Cron) Description: Invoke and run Lambda functions on a schedule Benefit: You can now run batch jobs or periodic cleanup, aggregation, or monitoring jobs by creating a recurring schedule event that can trigger a Lambda function How it works: When selecting the event source for a Lambda function, you can select “scheduled event” from the dropdown list and create a rate() or cron() schedule
18. 18. Lambda function code for building a Canary
19. 19. Sneak Peek: Accessing Resources in a VPC From a Lambda Function [Coming soon] Description: Access resources behind a VPC from inside Lambda functions Benefit: You will soon be able to access resources in a private IP range in your VPC, including EC2, ELB, RDS, ElastiCache and Redshift How it works: When creating Lambda functions, you will be able to select the VPC subnets and security groups, and the Lambda function executions can then access private endpoints in that VPC IP range
20. 20. In case you missed these updates: New AWS Lambda partner blueprints
21. 21. In case you missed these updates: Amazon Simple Email Service Inbound Rules // Spam check if (sesNotification.receipt.spamVerdict.status === 'FAIL‘ || sesNotification.receipt.virusVerdict.status === 'FAIL') console.log('Dropping spam');
22. 22. In case you missed these updates: Amazon CloudWatch Logs Processing AWS LambdaAmazon CloudWatch Logs Amazon DynamoDB Amazon S3 Amazon Redshift Scan, audit, or index log entries in near real time
23. 23. In case you missed these updates: AWS IoT integration and blueprint for AWS Lambda
24. 24. Three Next Steps 1. Create and test your first Python Lambda function. You can use any third party library, even native ones. With a larger function timeout, you can now run more complex applications or data processing algorithms as Lambda functions. 2. Create multiple versions of your Lambda functions, add aliases to different versions, and try invoking different versions using the same client APIs by moving about the aliases on the backend. 3. Hook up your Lambda function to Scheduled Events to build your own health- monitor for any website or custom public endpoint – the first million invokes are on us each month, so your Canary is absolutely FREE!
25. 25. Thank you! Visit http://aws.amazon.com/lambda, the AWS Compute blog, and the Lambda forum to learn more and get started using Lambda.