ablog

HugePages は free コマンドで見ると used に計上される（2）

Linux

HugePages は free コマンドで見ると used に計上される - ablog で Huge Page は確保されるとOSカーネル内に確保され、free コマンドの used に計上されることを確認した。

今回はユーザプロセスが Huge Page を使用している時も used に計上されることを確認した。ちょっと検証の精度が怪しいので、後日ちゃんと調べる予定。

まず、Huge page を使用していない状態

# sysctl -w vm.nr_hugepages=0
vm.nr_hugepages = 0
# free -m
             total       used       free     shared    buffers     cached
Mem:         15779       1741      14038          0         42        804★
-/+ buffers/cache:        895★      14884
Swap:         7895          0       7895
# cat /proc/meminfo 
MemTotal:       16158544 kB
MemFree:        14373768 kB
Buffers:           43660 kB
Cached:           823360 kB
SwapCached:            0 kB
Active:           708308 kB
Inactive:         769356 kB
Active(anon):     610924 kB
Inactive(anon):   117960 kB
Active(file):      97384 kB
Inactive(file):   651396 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:       8085500 kB
SwapFree:        8085500 kB
Dirty:               784 kB
Writeback:             0 kB
AnonPages:        610960 kB
Mapped:           300480 kB
Shmem:            118236 kB
Slab:              73432 kB
SReclaimable:      32620 kB
SUnreclaim:        40812 kB
KernelStack:        3128 kB
PageTables:        34104 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    16164772 kB
Committed_AS:    2302772 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      382868 kB
VmallocChunk:   34359348048 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:       0 ★huge page は使われていない
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        8388 kB
DirectMap2M:    16459776 kB

vm.nr_hugepages を 200 に設定すると、used が増える

# sysctl -w vm.nr_hugepages=200
vm.nr_hugepages = 200
# free -m
             total       used       free     shared    buffers     cached
Mem:         15779       2144      13635          0         42        803★ほぼ変化なし（+1MB）
/+ buffers/cache:       1297★400MB増加      14482
Swap:         7895          0       7895
# cat /proc/meminfo 
MemTotal:       16158544 kB
MemFree:        13962992 kB
Buffers:           43828 kB
Cached:           823328 kB
SwapCached:            0 kB
Active:           710236 kB
Inactive:         769324 kB
Active(anon):     612672 kB
Inactive(anon):   117904 kB
Active(file):      97564 kB
Inactive(file):   651420 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:       8085500 kB
SwapFree:        8085500 kB
Dirty:               168 kB
Writeback:             0 kB
AnonPages:        612408 kB
Mapped:           301776 kB
Shmem:            118176 kB
Slab:              73324 kB
SReclaimable:      32864 kB
SUnreclaim:        40460 kB
KernelStack:        3120 kB
PageTables:        34120 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    15959972 kB
Committed_AS:    2306196 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      382868 kB
VmallocChunk:   34359348048 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:     200 ★huge page が 200　個確保されている
HugePages_Free:      200 ★200個全てまだ使われていない
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB ★huge page 1つのサイズは 2MB
DirectMap4k:        8388 kB
DirectMap2M:    16459776 kB

共有メモリを Huge page で使用するプログラムを実行する

# gdb ./hugepage-shm
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-60.el6)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /home/yazekats/Documents/github/yoheia/misc/hugepage_test/hugepage-shm...done.
(gdb) b 89
Breakpoint 1 at 0x40086f: file hugepage-shm.c, line 89.
(gdb) run
Starting program: /home/yazekats/Documents/github/yoheia/misc/hugepage_test/hugepage-shm 
shmid: 0x120023
shmaddr: 0x2aaaaac00000
Starting the writes:
................................................................................................................................................................................................................................................................

Breakpoint 1, main () at hugepage-shm.c:89
89		dprintf("Done.\n");
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.132.el6.x86_64
(gdb) 

used に計上されたままで、cached は増えていない

# free -m
             total       used       free     shared    buffers     cached
Mem:         15779       2174      13605          0         43        818★ほぼ変化なし（+15MB）
-/+ buffers/cache:       1312★+15MB      14467
Swap:         7895          0       7895
# cat /proc/meminfo 
MemTotal:       16158544 kB
MemFree:        13932904 kB
Buffers:           44756 kB
Cached:           837832 kB
SwapCached:            0 kB
Active:           726956 kB
Inactive:         781296 kB
Active(anon):     626508 kB
Inactive(anon):   124484 kB
Active(file):     100448 kB
Inactive(file):   656812 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:       8085500 kB
SwapFree:        8085500 kB
Dirty:              1400 kB
Writeback:             0 kB
AnonPages:        626264 kB
Mapped:           306720 kB
Shmem:            124824 kB
Slab:              73644 kB
SReclaimable:      33084 kB
SUnreclaim:        40560 kB
KernelStack:        3136 kB
PageTables:        35064 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    15959972 kB
Committed_AS:    2339268 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      382880 kB
VmallocChunk:   34359348048 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:     200
HugePages_Free:       72 ★128個の huge page が使われている
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        8388 kB
DirectMap2M:    16459776 kB

プログラムを終了すると

(gdb) quit
A debugging session is active.

	Inferior 1 [process 3981] will be killed.

Quit anyway? (y or n) y

# free -m
             total       used       free     shared    buffers     cached
Mem:         15779       2186      13593          0         47        830★ほぼ変化なし（+12MB）
-/+ buffers/cache:       1308★ほぼ変化なし      14471
Swap:         7895          0       7895
# cat /proc/meminfo 
MemTotal:       16158544 kB
MemFree:        13915492 kB
Buffers:           48328 kB
Cached:           854604 kB
SwapCached:            0 kB
Active:           725872 kB
Inactive:         799912 kB
Active(anon):     623680 kB
Inactive(anon):   122400 kB
Active(file):     102192 kB
Inactive(file):   677512 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:       8085500 kB
SwapFree:        8085500 kB
Dirty:              2696 kB
Writeback:             0 kB
AnonPages:        623452 kB
Mapped:           304164 kB
Shmem:            122724 kB
Slab:              74388 kB
SReclaimable:      33664 kB
SUnreclaim:        40724 kB
KernelStack:        3096 kB
PageTables:        34804 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    15959972 kB
Committed_AS:    2345268 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      382880 kB
VmallocChunk:   34359348048 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:     200
HugePages_Free:       72 ★128MBは使われたまま？
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        8388 kB
DirectMap2M:    16459776 kB

ツイートする

Permalink | コメント(0) | トラックバック(0) | 17:59