The OMR GC talk - Ruby Kaigi 2015

The OMR GC talk - Ruby Kaigi 2015

1. 1. It’s Dangerous to GC alone. Take this! The OMR GC talk
2. 2. Who are those handsome guys? Robert Young rwyoung@ca.ibm.com @rwy0717 Craig Lehmann craigl@ca.ibm.com @CraigLehmann 2
3. 3. IBM Runtime Technologies 3
4. 4. 4 VisualAge Smalltalk J9 JVM 1988 ? 1993 2016
5. 5. 5 VisualAge Smalltalk J9 JVM 1988 OMR 1993 2016
6. 6. An open source toolkit for language runtime technologies. OMR
7. 7. Execution Environment 7 Language-Agnostic Components Platform Abstraction Layer Garbage Collector Diagnostic Services Source Code Bytecode/AST Compiler Just-In-Time Compiler InterpreterSource Bytecode Compiler Interpreter
8. 8. Execution Environment 8 Language-Agnostic Components Platform Abstraction Layer Garbage Collector Diagnostic Services Source Code Bytecode/AST Compiler Just-In-Time Compiler InterpreterSource Bytecode Compiler Interpreter
9. 9. 9 A Runtime Toolkit for Many Languages JVM OMR Java App Python OMR Python App MRI OMR Ruby App
10. 10. In Ruby OMR
11. 11. 13 Getting started
12. 12. 14 How do we start? http://thefeedingdoctor.com/wp-content/uploads/2013/10/Kid-Confused.jpg
13. 13. 15 How do we start?
14. 14. 16
15. 15. Read the source 17
16. 16. Read the source 18
17. 17. 19 Craig! Robert! Tue Oct 5th - 1993
18. 18. • Fantastic worldwide community • Ruby has changed a lot over 20 years • MRI is a complex virtual machine with a lot of history behind it’s design decisions. 20
19. 19. What we noticed
20. 20. Objects 22 RVALUE • Direct object pointers • Conservative GC Conservative collection
21. 21. 23 Dynamic memory string arrayhash data … Managed Memory • Fixed sized objects • What about variable sized objects
22. 22. Dynamic memory string arrayhash data … Managed Memory malloc malloc malloc malloc Malloc Memory 24
23. 23. Dynamic memory string arrayhash data … malloc malloc malloc malloc 25 • No heap fragmentation • Easy conservative marking
24. 24. Dynamic memory string arrayhash data … malloc malloc malloc malloc 26 • Heavyweight allocation • Malloc fragmentation • Poor cache locality • Expensive sweeping
25. 25. Ruby has an incremental GC Mark Sweep Ruby 27 • GC work is interleaved with execution • Minimizes mutator pause times • Single threaded • Serialized with the mutator
26. 26. What we did
27. 27. Goals  Make allocations faster  Improve GC Time  Improve mutator performance  100% compatibility 29
28. 28. 100% compatibility  No C-extensions API changes  Pass all extended tests  Run Rails - unmodified 30
29. 29. Let’s make it work 31
30. 30. 32 Starting out with Mark Sweep static int gc_start(rb_objspace_t *objspace …) { … OMR_GC_SystemCollect(); ... } static int newobj_of(rb_objspace_t *objspace …) { ... obj = rb_omr_get_freeobj(th, sizeof(RVALUE)); ... }
31. 31. 33 Starting out with Mark Sweep Need to support conservative collection Conservative collection Object Map (bit map) • Object map allows us to support conservative collection • Bit Map used to keep track of objects • Results in ~ 1.6% memory overhead
32. 32. Let’s make it parallel 34
33. 33. Multithreaded GC (Stop the world) 35 mark sweep • Introducing Parallel Global GC • Aggressively parallelized • New APIs for parallelism • Support for thread pooling and task synchronization
34. 34. Marking in parallel 36 mark sweep Marking: • Scan roots in parallel • Break VM roots into subsets • Complete marking in parallel
35. 35. Sweeping in parallel 37 mark sweep Sweeping: • Free malloc space in parallel • Clean up objects in parallel • Move work out of finalization
36. 36. Hey… It works! 38
37. 37. New tools! 39
38. 38. Garbage Collection Memory Visualizer for Ruby MRI with zero changes to the tool 40
39. 39. Health Center for Ruby MRI: Live GC visualization – No code changes required 41
40. 40. Health Center for Ruby MRI: Method profiling 42
41. 41. How can we make CRuby faster? 43
42. 42. Allocations! 44
43. 43. Making allocation fast Thread-local heaps: • Cache regions of the heap per-thread • Allocate without locking • Better locality 45
44. 44. Thread local heap Object TLH Heap Next Alloc 47 TLH
45. 45. Thread local heap TLH Object TLH Heap Next Alloc 48
46. 46. Thread local heap TLH Object TLH Heap Next Alloc 49
47. 47. How can we make it faster? 50
48. 48. Lets look at off heap memory! 51
49. 49. How do we replace malloc/free? • malloc/free callouts are expensive • Rely on system for memory management concerns • Still susceptible to fragmentation and concurrency • Black box implementation • Replace malloc and free with a new allocator 52
50. 50. New built-in type! 55
51. 51. New built-in type OMRBuffers • Create a new, variable sized object type • Allocate all buffers on the heap as objects typedef struct OMRBuffer { VALUE flags; long size; } OMRBuffer; 56 buffer
52. 52. 58 OMRBuffers on the heap! string arrayhash data … Managed Memory buffer buffer buffer buffer Malloc Memory
53. 53. 59 Shorter sweeping with OMRBuffers mark sweep GC Before GC After mark sweep
54. 54. Getting user defined types on heap • RDatas are used to create C-extension types. • Typed RDatas have a new flag: RDATA_HEAP_ALLOCATED • Automatically heap allocates the data buffer at allocation • No free method results in no object finalization • Allows for heap allocation in extensions • Extremely easy to use 60
55. 55. OMRBuffer wins Improvements: • Eliminated obj_free() • Fast allocations • Better cache locality Challenges: • Heap fragmentation • Conservative collection • Concurrency 61
56. 56. Multithreaded allocations • MRI does some work in backgrounded threads • Use OMRBuffers in background threads • Introduce finer grained locking than the GVL • Allows for: • multithreaded allocations • GCing from a background thread 62
57. 57. Is Ruby Fast Yet? 63 8% better throughput 122.57 112.84 0 20 40 60 80 100 120 140 2.2-omr 2.2.3
58. 58. Experiments 64
59. 59. Generational GC in MRI • Experimenting with non-copying generational GC • Marking is already fast • Heap fragmentations issues • High memory overhead 65
60. 60. Segregated heap in MRI • Introduced a segregated heap into ruby • Heap divided into regions of fixed sized objects • Bounds maximum heap fragmentation 66
61. 61. 67 Concurrent GC in MRI • Adding Concurrent GC to Ruby • Ruby threads incrementally mark • Background thread scans • Parallelized sweep (STW) sweepmark
62. 62. What’s next 70
63. 63. OMR GC in Ruby – what’s next • Ongoing experimentation • Balanced • Semispace copying generational collection • compaction • Real time garbage collection 71
64. 64. What’s next for OMR? • Open source OMR • Make our Ruby experiments available • We want to hear from the experts (you) • Lets make OMR and Ruby the best they can be 72
65. 65. Ruby+OMR Tech Preview Download @ goo.gl/P3yXuy • Releasing a preview so you can start test- driving • Send us feedback! 73
66. 66. 74
67. 67. 75 We want to help
68. 68. Thank You! 76
69. 69. John Duimovich CTO IBM Runtimes duimovic@ca.ibm.com @jduimovich Ask us Anything! Mark Stoodley OMR Project Lead mstoodle@ca.ibm.com @mstoodle Robert Young OMR Developer rwyoung@ca.ibm.com @rwy0717 77 Charlie Gracie OMR GC Architect crgracie@ca.ibm.com @crgracie Craig Lehmann OMR Developer craigl@ca.ibm.com @craiglehmann
70. 70. The end!
71. 71. Who were those handsome guys again? Robert Young rwyoung@ca.ibm.com @rwy0717 Craig Lehmann craigl@ca.ibm.com @CraigLehmann 82
72. 72. Trademarks, Copyrights, Disclaimers 88 IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of other IBM trademarks is available on the web at "Copyright and trademark information" at http://www.ibm.com/legal/copytrade.shtml Other company, product, or service names may be trademarks or service marks of others. THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE. IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, NOR SHALL HAVE THE EFFECT OF, CREATING ANY WARRANTIES OR REPRESENTATIONS FROM IBM (OR ITS SUPPLIERS OR LICENSORS), OR ALTERING THE TERMS AND CONDITIONS OF ANY AGREEMENT OR LICENSE GOVERNING THE USE OF IBM PRODUCTS OR SOFTWARE. © Copyright International Business Machines Corporation 2015. All rights reserved.
73. 73. Additional Important Disclaimers • THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. • WHILST EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. • ALL PERFORMANCE DATA INCLUDED IN THIS PRESENTATION HAVE BEEN GATHERED IN A CONTROLLED ENVIRONMENT. YOUR OWN TEST RESULTS MAY VARY BASED ON HARDWARE, SOFTWARE OR INFRASTRUCTURE DIFFERENCES. • ALL DATA INCLUDED IN THIS PRESENTATION ARE MEANT TO BE USED ONLY AS A GUIDE. • IN ADDITION, THE INFORMATION CONTAINED IN THIS PRESENTATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM, WITHOUT NOTICE. • IBM AND ITS AFFILIATED COMPANIES SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. • NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF: • - CREATING ANY WARRANT OR REPRESENTATION FROM IBM, ITS AFFILIATED COMPANIES OR ITS OR THEIR SUPPLIERS AND/OR LICENSORS 89