データセンターは世界にいくつ必要か

データセンターは世界にいくつ必要か

1. 1. { “名前” : “真壁 徹(まかべ とおる)”, “所属” : “日本マイクロソフト株式会社”, “役割” : “クラウド ソリューションアーキテクト”, “経歴” : “大和総研  HP Enterprise”, “特技” : “クラウド & オープンソース” }
2. 2. http://perspectives.mvdirona.com/2017/04/how-many-data-centers-needed- world-wide/
3. 3. http://fortune.com/2017/04/12/mark-hurd-oracle-data-centers/ "If I have two-times faster computers, I don't need as many data centers. If I can speed up the database, maybe I need one fourth as may data centers. I can go on and on about how tech drives this.“
4. 4. [Microsoft Global Datacenters and Network Infrastructure] https://www.youtube.com/watch?v=bqZrejosqWU
5. 5. 38announced regions 34available regions
6. 6. リージョンペア リージョン リージョン リージョン AWS re:Invent 2014 “AWS Innovation at Scale with James Hamilton” より
7. 7. 12.88ミリ秒18.92ミリ秒 9.00ミリ秒 測ってみました
8. 8. "There is some absolute data center size where the facility becomes “too big to fail.” This line is gray and open to debate but the limiting factor is how big of a facility can an operator lose before the lost resources and the massive network access pattern changes on failure can’t be hidden from customers.”
9. 9. 1.Microsoft: 35 MW with CloudHQ in Manassas, Virginia 2.Microsoft: 30 MW with EdgeConneX in Elk Grove Village, Illinois 3.Microsoft: 22 MW with CyrusOne in Ashburn, Virginia 4.Microsoft: 16 MW with DuPont Fabros Technology in Santa Clara, California 5.Microsoft: 13.5 MW with CyrusOne in Phoenix, Arizona 6.Microsoft: 9 MW with CyrusOne in San Antonio, Texas 7.Oracle: 6 MW with CyrusOne in Sterling, Virginia 8.Salesforce: 6 MW with QTS in Dallas, Texas 9.Oracle: 5 MW with CyrusOne in Ashburn, Virginia 10.Oracle: 5 MW with Digital Realty Trust in Ashburn, Virginia http://www.datacenterknowledge.com/archives/2017/01/18/who-leased-the-most-data-center-space-in-2016/ Microsoft、AWSともに自社建設、保有分は入っていないので雰囲気だけ
10. 10. Colocation Density 2.0+ Power Usage Effectiveness (PUE) 1.4 – 1.6 PUE Discrete servers Capacity 20 year technology Rack Density & deployment Minimized resource impact Generation 1 Generation 2 Containment Modular Hyper-scale 1.2 – 1.5 PUE 1.12 – 1.20 PUE 1.07 – 1.19 PUE Containers, PODs Scalability & sustainability Air & water Economization Differentiated SLAs Deployment Areas & ITPACs No more traditional IT Right-sized Faster time-to-market Outside air cooled Fully integrated Resilient software Common infrastructure Operational simplicity Flexible & scalable Generation 3 Generation 4 Generation 5
11. 11. http://natick.research.microsoft.com/
12. 12. 電気代
13. 13. Standard_D1_v2 10.404 East US 2 / -44.1% Standard_D2_v2 20.91 East US 2 / -44.4% Standard_D3_v2 41.718 East US 2 / -44% Standard_D4_v2 83.436 East US 2 / -44% Standard_D5_v2 166.872 East US 2 / -44% Standard_D11_v2 23.46 East US 2 / -35.2% Standard_D12_v2 46.818 East US 2 / -34.9% Standard_D13_v2 93.636 East US 2 / -34.9% Standard_D14_v2 187.17 East US 2 / -34.8% Standard_D15_v2 233.988 East US 2 / -34.7% Standard_D1_v2 6.936 East US 2 / -16.2% Standard_D2_v2 13.872 East US 2 / -16.2% Standard_D3_v2 27.744 East US 2 / -15.8% Standard_D4_v2 55.488 East US 2 / -15.8% Standard_D5_v2 110.874 East US 2 / -15.7% Standard_D11_v2 19.38 East US 2 / -21.6% Standard_D12_v2 38.658 East US 2 / -21.1% Standard_D13_v2 77.418 East US 2 / -21.2% Standard_D14_v2 154.836 East US 2 / -21.1% Standard_D15_v2 193.494 East US 2 / -21.1% Standard_D1_v2 8.976 East US 2 / -35.2% Standard_D2_v2 17.952 East US 2 / -35.2% Standard_D3_v2 35.802 East US 2 / -34.8% Standard_D4_v2 71.604 East US 2 / -34.8% Standard_D5_v2 143.31 East US 2 / -34.8% Standard_D11_v2 23.868 East US 2 / -36.3% Standard_D12_v2 47.94 East US 2 / -36.4% Standard_D13_v2 95.574 East US 2 / -36.2% Standard_D14_v2 191.148 East US 2 / -36.1% Standard_D15_v2 238.986 East US 2 / -36.1% Standard_D1_v2 10.914 East US 2 / -46.7% Standard_D2_v2 21.828 East US 2 / -46.7% Standard_D3_v2 43.656 East US 2 / -46.5% Standard_D4_v2 87.414 East US 2 / -46.6% Standard_D5_v2 174.828 East US 2 / -46.6% Standard_D11_v2 23.46 East US 2 / -35.2% Standard_D12_v2 46.818 East US 2 / -34.9% Standard_D13_v2 93.636 East US 2 / -34.9% Standard_D14_v2 187.17 East US 2 / -34.8% Standard_D15_v2 233.988 East US 2 / -34.7% Azure 仮想マシン(Linux)の価格(円)を、最安値リージョンと比較すると 東日本 西ヨーロッパ 西イギリス 東アジア(香港) m4.xlarge $0.258 -29% m4.2xlarge $0.516 -29% m4.4xlarge $1.032 -29%
14. 14. "Eventually it just make better sense to offer customers alternative regions rather than attempting to scale a single region and the argument in favor of multiple regions become even stronger when other factors are considered.”
15. 15. Azure Regionオンプレミス(ユーザー占有型) User User User User Storage Cluster Storage Cluster Server Cluster Server Cluster 共用部分が大きい
16. 16. Azure Regionオンプレミス(ユーザー占有型) User User User User Storage Cluster Storage Cluster Server Cluster Server Cluster 影響範囲 ストレージクラスターが故障した場合 影響範囲が大きい
17. 17. Azure Regionオンプレミス(ユーザー占有型) User User User User Storage Cluster Storage Cluster Server Cluster Server Cluster データセンター全体で共有している設備の障害影響は、 オンプレもAzureも同様
18. 18. どれも冗長化、自己回復機能を有するが、不具合や作業ミスの可能性はゼロではない ユーザーがそれに対処するにはマルチリージョン構成が有効
19. 19. (規模の違いはありますが)
20. 20. Azure Storage Cluster Storage Cluster Server Cluster Server Cluster
21. 21. Traditional on-premises Modern cloud Monolithic, centralized Design for predictable scalability Relational database Strong consistency Serial and synchronized processing Design to avoid failures (MTBF) Occasional big updates Manual management Snowflake servers Decomposed, de-centralized Design for elastic scale Polyglot persistence (mix of storage technologies) Eventual consistency Parallel and asynchronous processing Design for failure (MTTR) Frequent small updates Automated self-management Immutable infrastructure Azure Application Architecture Guide https://docs.microsoft.com/ja-jp/azure/architecture/guide/
22. 22. "But, without a more fundamental solution to the speed of light problem, many regions are the only practical way to effectively serve the entire planet for many workloads.”
23. 23. 車線数(帯域)は 増えている すでに僕のかんがえる 最速の車(光)
24. 24. "Imagine a $20,000 car in one market costing far more than $200,000 in another market.”
25. 25. 海底ケーブルへの投資 https://blogs.technet.microsoft.com/hybridcloud/2016/05/26/microsoft- and-facebook-to-build-subsea-cable-across-atlantic/ https://azure.microsoft.com/ja-jp/blog/microsoft-invests-in-subsea- cables-to-connect-datacenters-globally/ 160Tbps 80Tbps
26. 26. $azure location list --details info: Executing command location list + Getting ARM registered providers info: Getting locations... data: data: Location : japaneast data: DisplayName : East Japan data: data: […]
27. 27. 12.88ミリ秒18.92ミリ秒 9.00ミリ秒 測ってみました
28. 28. " In addition, some national jurisdictions will put in place legal restrictions than make it difficult to fully serve the market without a local region. ”
29. 29. Standard_D1_v2 10.404 East US 2 / -44.1% Standard_D2_v2 20.91 East US 2 / -44.4% Standard_D3_v2 41.718 East US 2 / -44% Standard_D4_v2 83.436 East US 2 / -44% Standard_D5_v2 166.872 East US 2 / -44% Standard_D11_v2 23.46 East US 2 / -35.2% Standard_D12_v2 46.818 East US 2 / -34.9% Standard_D13_v2 93.636 East US 2 / -34.9% Standard_D14_v2 187.17 East US 2 / -34.8% Standard_D15_v2 233.988 East US 2 / -34.7% Standard_D1_v2 6.936 East US 2 / -16.2% Standard_D2_v2 13.872 East US 2 / -16.2% Standard_D3_v2 27.744 East US 2 / -15.8% Standard_D4_v2 55.488 East US 2 / -15.8% Standard_D5_v2 110.874 East US 2 / -15.7% Standard_D11_v2 19.38 East US 2 / -21.6% Standard_D12_v2 38.658 East US 2 / -21.1% Standard_D13_v2 77.418 East US 2 / -21.2% Standard_D14_v2 154.836 East US 2 / -21.1% Standard_D15_v2 193.494 East US 2 / -21.1% Standard_D1_v2 8.976 East US 2 / -35.2% Standard_D2_v2 17.952 East US 2 / -35.2% Standard_D3_v2 35.802 East US 2 / -34.8% Standard_D4_v2 71.604 East US 2 / -34.8% Standard_D5_v2 143.31 East US 2 / -34.8% Standard_D11_v2 23.868 East US 2 / -36.3% Standard_D12_v2 47.94 East US 2 / -36.4% Standard_D13_v2 95.574 East US 2 / -36.2% Standard_D14_v2 191.148 East US 2 / -36.1% Standard_D15_v2 238.986 East US 2 / -36.1% Standard_D1_v2 10.914 East US 2 / -46.7% Standard_D2_v2 21.828 East US 2 / -46.7% Standard_D3_v2 43.656 East US 2 / -46.5% Standard_D4_v2 87.414 East US 2 / -46.6% Standard_D5_v2 174.828 East US 2 / -46.6% Standard_D11_v2 23.46 East US 2 / -35.2% Standard_D12_v2 46.818 East US 2 / -34.9% Standard_D13_v2 93.636 East US 2 / -34.9% Standard_D14_v2 187.17 East US 2 / -34.8% Standard_D15_v2 233.988 East US 2 / -34.7% 発想の逆転: 日本に置く必要のないシステムをUSに置けたら? 東日本 西ヨーロッパ 西イギリス 東アジア(香港) m4.xlarge $0.258 -29% m4.2xlarge $0.516 -29% m4.4xlarge $1.032 -29%
30. 30. " Oracle is hardly unique in having their own semiconductor team. Amazon does custom ASICs, Google acquired an ARM team and has done custom ASIC for machine learning. Microsoft has done significant work with FPGAs and is also an ARM licensee. ”
31. 31. 汎用・柔軟 効率・性能
32. 32. ( https://docs.microsoft.com/ja-jp/azure/virtual-machines/virtual-machines-linux-sizes )
33. 33. Host SmartNIC FPGA ToR NIC ASIC SmartNIC CPU
34. 34. Web search ranking Traditional software (CPU) server plane QPICPU QSFP 40Gb/s ToR FPGA CPU 40Gb/s QSFP QSFP Hardware acceleration plane Web search ranking Deep neural networks SDN offload SQL
35. 35. • Cavium ThunderX2 • Qualcomm Centriq 2400 • モバイルデバイス向けに数が伸 び続けるARMへの期待 • 電力あたり性能 • エコシステム • Windows ServerをARM対応中 • まずはAzure内、ユーザーが意識 しないところで • ソフトウェアとしての販売は未定 https://schd.ws/hosted_files/ocpussummit2017/f2/OCP17%20Workshop_Microsoft%20Project%20Olympus%20Servers_3_8_2017.pdf http://files.opencompute.org/oc/public.php?service=files&t=5e232f151532e2af3b9754af4ca79f0b
36. 36. http://files.opencompute.org/oc/public.php?service=files&t=14ab3cf25170b7a0a439e11a3d818c96 https://azure.microsoft.com/en-us/blog/ecosystem-momentum-positions-microsoft-s-project-olympus-as-de-facto-open-compute-standard/ • “Hyperscale GPU Accelerator” • NVIDIAとの共同開発 • Tesla P100 SMX2 x8 & NVLink • 4シャーシまで拡張可 • Internal PCIe Fabric Interconnect • 16サーバーまで共用可
37. 37. “Price reductions on L Series and announcing next generation Hyper- threaded virtual machines” (2017/4/3)