(SPOT301) AWS Innovation at Scale | AWS re:Invent 2014
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

(SPOT301) AWS Innovation at Scale | AWS re:Invent 2014

  • 181 views
Uploaded on

This session, led by James Hamilton, VP and Distinguished Engineer, gives an insider view of some the innovations that help make the AWS cloud unique. He will show examples of AWS networking ...

This session, led by James Hamilton, VP and Distinguished Engineer, gives an insider view of some the innovations that help make the AWS cloud unique. He will show examples of AWS networking innovations from the interregional network backbone, through custom routers and networking protocol stack, all the way down to individual servers. He will show examples from AWS server hardware, storage, and power distribution and then, up the stack, in high scale streaming data processing. James will also dive into fundamental database work AWS is delivering to open up scaling and performance limits, reduce costs, and eliminate much of the administrative burden of managing databases. Join this session and walk away with a deeper understanding of the underlying innovations powering the cloud.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment

Views

Total Views
181
On Slideshare
179
From Embeds
2
Number of Embeds
1

Actions

Shares
Downloads
15
Comments
0
Likes
11

Embeds 2

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. November 12, 2014 | Las Vegas, NV James Hamilton, AWS VP & Distinguished Engineer
  • 2. “5X the cloud capacity in use than the aggregate total of the other 14 providers”
  • 3. Every day, AWS adds enough new server capacity to support all of Amazon’s global infrastructure when it was a $7B annual revenue enterprise
  • 4. 57% 8% 18% 13% 4% Servers Networking Equipment Power Distribution & Cooling Power Other Infrastructure Monthly Costs 3yr server & 10 yr infrastructure amortization
  • 5. AZ AZ AZ AZ AZ Transit Transit
  • 6. • Asynchronous replication between distant data centers • Committing to an SSD order 1 to 2ms • But, LA to New York is 74ms round trip • You can’t wait 74ms to commit a transaction • On failure, difficult & high skill decision: • Fail-over & lose transactions, or • Or don’t fail-over & lose availability • Difficult choice • Use AZs for no-admin failover • Sync works when <2ms • Can be combined with regional replication for very high availability 74ms
  • 7. AZ AZ AZ AZ AZ Transit Transit
  • 8. 26% 40% 25% 30% 35% 40% Multi-AZ RDS % of all RDS Synchronous Replication AZ1 AZ2
  • 9. SQL Txns Caching StoreStore Transform: Write to Read Format Optimized StoreStore AZ 1 AZ 2 StoreStore AZ 3 Transform: Write to Read Format Optimized Transform: Write to Read Format Optimized
  • 10. AZ 1 AZ 2 AZ 3
  • 11. Full 10Gbps Links Ingestion, Backup, & Restore Customer VPC Internal VPC JDBC/ODBC
  • 12. EBS
  • 13. Capture Submissions Process in Realtime Store in Redshift Clients Submitting Data AWS Metering: • Tens of millions records/sec • Multiple TB per hour • 100,000s of internal sources • Scales, low-cost, auditable, with real time alerting Amazon Kinesis: • Producers call put • Sequence # returned • Distributed over shards • Scales per shard at 1 MB/s & 1000 TPS Amazon Web Services AZ AZ AZ Durable, highly consistent storage replicates data across three data centers (availability zones) Aggregate and archive to S3 Millions of sources producing 100s of terabytes per hour Front End Authentication Authorization Ordered stream of events supports multiple readers Real-time dashboards and alarms Machine learning algorithms or sliding window analytics Aggregate analysis in Hadoop or a data warehouse Inexpensive: $0.028 per million puts
  • 14. 2008 2009 2010 2011 2012 2013 2014 On pace for over 449 new services or major features in 2014 280 159 82 61 48 24 449
  • 15. http://bit.ly/awsevals Thank You!