Memory management in Linux Raghu Udiyar Technical Support Engineer Red Hat
What is memory? <ul><li>It is just a form of storage
Can a disk be considered memory? Yes, in a way
Only difference is with : </li><ul><li>Capacity – cost
Latency
Volatile </li></ul></ul>
Hardware <ul><li>CPU registers - ~1 cycle
CPU cache (L1, L2..) - 3 – 14 cycles
RAM (main memory) - ~250 cycles
Disk - ~40 million cycles </li></ul>
Physical Memory <ul><li>The map provides the physical addresses
These are then taken and divided into page frames of 4Kb
E.g. 4Gb = ~1 million pages </li></ul>
Virtual Memory <ul><li>The system never allocates physical memory directly
Every process has a virtual view of the memory
A process will think it is the only task running on the system
This virtual memory is then mapped to physical memory using page tables </li></ul>
Virtual Memory vi bash 2GB Physical Memory Virtual memory
Virtual Memory vi bash 2GB Physical Memory Virtual memory Page Table
Address Space <ul><li>On a 32bit system : </li><ul><li>4Gb : 3 Gb userspace / 1 Gb kernel space </li></ul><li>On a 64bi...
User and Kernel address space Kernel User Kernel User 32bit 64bit 3Gb 1GB 256TB 256TB
Memory Overcommit <ul><li>Allocate more then what is available
malloc(1G) = Ok
memset(1G) = Ok
malloc(4GB) = Ok
memset(4GB) = -ENOMEM
Kernel almost always grants Virtual memory
Upcoming SlideShare
Loading in...5
×

Memory management in Linux

14,315

Published on

A brief overview of the memory management subsystem in the Linux kernel.

Published in: Technology
0 Comments
31 Likes
Statistics
Notes
no profile picture user

  • Be the first to comment

Show More
No Downloads
Views
Total Views
14,315
On Slideshare
0
From Embeds
0
Number of Embeds
16
Actions
Shares
0
Downloads
0
Comments
0
Likes
31
Embeds 0
No embeds

No notes for slide

Memory management in Linux

  1. 1. Memory management in Linux Raghu Udiyar Technical Support Engineer Red Hat
  2. 2. What is memory? <ul><li>It is just a form of storage
  3. 3. Can a disk be considered memory? Yes, in a way
  4. 4. Only difference is with : </li><ul><li>Capacity – cost
  5. 5. Latency
  6. 6. Volatile </li></ul></ul>
  7. 7. Hardware <ul><li>CPU registers - ~1 cycle
  8. 8. CPU cache (L1, L2..) - 3 – 14 cycles
  9. 9. RAM (main memory) - ~250 cycles
  10. 10. Disk - ~40 million cycles </li></ul>
  11. 11. Physical Memory <ul><li>The map provides the physical addresses
  12. 12. These are then taken and divided into page frames of 4Kb
  13. 13. E.g. 4Gb = ~1 million pages </li></ul>
  14. 14. Virtual Memory <ul><li>The system never allocates physical memory directly
  15. 15. Every process has a virtual view of the memory
  16. 16. A process will think it is the only task running on the system
  17. 17. This virtual memory is then mapped to physical memory using page tables </li></ul>
  18. 18. Virtual Memory vi bash 2GB Physical Memory Virtual memory
  19. 19. Virtual Memory vi bash 2GB Physical Memory Virtual memory Page Table
  20. 20. Address Space <ul><li>On a 32bit system : </li><ul><li>4Gb : 3 Gb userspace / 1 Gb kernel space </li></ul><li>On a 64bit system : </li><ul><li>256 TB userspace / 256 TB kernel space </li></ul></ul>
  21. 21. User and Kernel address space Kernel User Kernel User 32bit 64bit 3Gb 1GB 256TB 256TB
  22. 22. Memory Overcommit <ul><li>Allocate more then what is available
  23. 23. malloc(1G) = Ok
  24. 24. memset(1G) = Ok
  25. 25. malloc(4GB) = Ok
  26. 26. memset(4GB) = -ENOMEM
  27. 27. Kernel almost always grants Virtual memory
  28. 28. Page fault occurs only when that memory is used </li></ul>
  29. 29. Out of Memory <ul><li>Side effect of overcommit is that system can run out of memory
  30. 30. In such cases the “OOM killer” is invoked
  31. 31. It will try to free memory by killing a task based on heuristics </li></ul>
  32. 32. OOM killer message
  33. 33. Page Cache <ul><li>Area in memory that caches files to improve disk i/o performance
  34. 34. All i/o in Linux goes through the page cache </li></ul>Write-back (delayed) Clean/Dirty pages Page cache Process Disk
  35. 35. Demonstration : Page Cache <ul><li># echo 1 > /proc/sys/vm/drop_caches
  36. 36. # dd if=/dev/zero of=512M-file count=512000 bs=1k
  37. 37. # time cat 512M-file &> /dev/null
  38. 38. # echo 1 > /proc/sys/vm/drop_caches
  39. 39. # time cat 512M-file &> /dev/null </li></ul>
  40. 40. Types of memory <ul><li>Kernel pages - fixed in memory
  41. 41. Program text - read only
  42. 42. File backed - clean or dirty pages
  43. 43. Anonymous pages – Not file-backed, e.g stack and heap
  44. 44. Only anonymous memory is paged out to swap area </li></ul>
  45. 45. Swap <ul><li>Area on disk that is used as if it's main memory
  46. 46. Considerably slow - ~40 million cycles
  47. 47. Swapping is a misnomer, pages paged out/in is more appropriate
  48. 48. Only anonymous pages are paged out </li></ul>
  49. 49. /proc/meminfo <ul>See : http://www.redhat.com/advice/tips/meminfo.html </ul>
  50. 50. Tools - vmstat
  51. 51. Tools - ps
  52. 52. Tools - sar <ul><li>Gives complete system stats over time
  53. 53. (10mins default)
  54. 54. Check “man sar” for details on all switches
  55. 55. E.g. : </li><ul><ul><li>#sar -f sa05 -t
  56. 56. #sar -f sa05 -tq
  57. 57. #sar -f sa05 -tr
  58. 58. #sar -f sa05 -tW </li></ul></ul></ul>
  59. 59. Common Queries <ul><li>Linux ate my ram! ( http://www.linuxatemyram.com/ )
  60. 60. https://access.redhat.com/kb/docs/DOC-1139 </li></ul>
  61. 61. Common Queries <ul><li>Totaling rss in ps does not add up to what is shown in free
  62. 62. Check : https://access.redhat.com/kb/docs/DOC-52712 </li></ul>
  63. 63. Common Queries <ul><li>Why is system using swap?
  64. 64. https://access.redhat.com/kb/docs/DOC-3368
  65. 65. Swap usage doesn't necessarily indicate a problem
  66. 66. Instead, check #sar -tW for problems </li></ul>
  67. 67. Common Queries <ul><li>I want to stop system from using the page cache or, how to reduce the page cache size.
  68. 68. In most case, this is not advised, and arises from misunderstanding of how the page cache works
  69. 69. Controlling page cache size : https://access.redhat.com/kb/docs/DOC-41749 </li></ul>
  70. 70. Common Queries <ul><li>Why is pgpgin/pgpgout so high?
  71. 71. Some confuse this with swap in/out
  72. 72. These are just the number of sectors read in or written to disk.
  73. 73. See : https://access.redhat.com/kb/docs/DOC-61288 </li></ul>
  74. 74. Common Queries <ul><li>System OOM killed, what to do (32bit system)
  75. 75. Check : https://access.redhat.com/kb/docs/DOC-52474 </li></ul>
  76. 76. Thank you Questions?

×