regression in memory fragmentation in tcmalloc with appends ops(2.5.0v2.2.0)

these data were obtained during the investigation of http://www.couchbase.com/issues/browse/MB-7887

test with 1MB thru 10MB

./testrunner -i 3nodes.ini -t memorysanitytests.MemorySanity.test_items_append,items=1000000,desired_item_size=10000,append_ratio=0.15,sasl_buckets=1,kv_verify=False,value_size=1000

Thus the appending is as follows

2014-01-10 07:08:42 | Item-appending of 150000 items starting ..
2014-01-10 07:18:12 | for 150000 items size was increased to 1000
2014-01-10 07:27:29 | for 150000 items size was increased to 2024
2014-01-10 07:36:41 | for 150000 items size was increased to 3048
2014-01-10 07:45:51 | for 150000 items size was increased to 4072
2014-01-10 07:57:11 | for 150000 items size was increased to 5096
2014-01-10 08:24:53 | for 150000 items size was increased to 6120
2014-01-10 08:54:01 | for 150000 items size was increased to 7144
2014-01-10 10:04:38 | for 150000 items size was increased to 8168
2014-01-10 10:59:41 | for 150000 items size was increased to 9192
2014-01-10 10:59:41 | The appending of 150000 items ended

build 2.2.0-821:

In 10.3.4.144 bucket default, total_fragmentation_bytes + the total_allocated_bytes = 1540784160
In 10.3.4.144 bucket default, mem_used = 604608360
In 10.3.4.144 bucket default, the difference between actual memory used by memcached and mem_used is 2.54840035622 times
In 10.3.4.146 bucket default, total_fragmentation_bytes + the total_allocated_bytes = 1437491168
In 10.3.4.146 bucket default, mem_used = 554534408
In 10.3.4.146 bucket default, the difference between actual memory used by memcached and mem_used is 2.59224882579 times
In 10.3.4.145 bucket default, total_fragmentation_bytes + the total_allocated_bytes = 1524727840
In 10.3.4.145 bucket default, mem_used = 608097400
In 10.3.4.145 bucket default, the difference between actual memory used by memcached and mem_used is 2.50737437785 times

In 10.3.4.144 bucket bucket0, total_fragmentation_bytes + the total_allocated_bytes = 1546715392
In 10.3.4.144 bucket bucket0, mem_used = 573434776
In 10.3.4.144 bucket bucket0, the difference between actual memory used by memcached and mem_used is 2.69728216135 times
In 10.3.4.146 bucket bucket0, total_fragmentation_bytes + the total_allocated_bytes = 1481072992
In 10.3.4.146 bucket bucket0, mem_used = 565021864
In 10.3.4.146 bucket bucket0, the difference between actual memory used by memcached and mem_used is 2.62126669137 times
In 10.3.4.145 bucket bucket0, total_fragmentation_bytes + the total_allocated_bytes = 1532223104
In 10.3.4.145 bucket bucket0, mem_used = 560881912
In 10.3.4.145 bucket bucket0, the difference between actual memory used by memcached and mem_used is 2.73181051344 times

build 2.5.0-1036:

In 10.3.4.148 bucket default, total_fragmentation_bytes + the total_allocated_bytes = 1593826880
In 10.3.4.148 bucket default, mem_used = 539971928
In 10.3.4.148 bucket default, the difference between actual memory used by memcached and mem_used is 2.95168470314 times
In 10.3.4.147 bucket default, total_fragmentation_bytes + the total_allocated_bytes = 1677467072
In 10.3.4.147 bucket default, mem_used = 584436664
In 10.3.4.147 bucket default, the difference between actual memory used by memcached and mem_used is 2.87022901766 times
In 10.3.4.149 bucket default, total_fragmentation_bytes + the total_allocated_bytes = 1532477440
In 10.3.4.149 bucket default, mem_used = 575876168
In 10.3.4.149 bucket default, the difference between actual memory used by memcached and mem_used is 2.66112321564 times

In 10.3.4.148 bucket bucket0, total_fragmentation_bytes + the total_allocated_bytes = 1604477248
In 10.3.4.148 bucket bucket0, mem_used = 558971400
In 10.3.4.148 bucket bucket0, the difference between actual memory used by memcached and mem_used is 2.87041027144 times
In 10.3.4.147 bucket bucket0, total_fragmentation_bytes + the total_allocated_bytes = 1685250592
In 10.3.4.147 bucket bucket0, mem_used = 568804984
In 10.3.4.147 bucket bucket0, the difference between actual memory used by memcached and mem_used is 2.9627915356 times
In 10.3.4.149 bucket bucket0, total_fragmentation_bytes + the total_allocated_bytes = 1571995840
In 10.3.4.149 bucket bucket0, mem_used = 560109160
In 10.3.4.149 bucket bucket0, the difference between actual memory used by memcached and mem_used is 2.80658834431 times

here I see the results on 2.5.0-1036 became worse by about 10 percent than on 2.2:

for example, bucket default
AVG(2.5)/ AVG(2.2) = (2.95 + 2.87 + 2.66)/(2.54 + 2.59 + 2.50) =~1.111

ry to clarify the size of the items.

./testrunner -i 3nodes.ini -t memorysanitytests.MemorySanity.test_items_append,items=1000000,desired_item_size=10000,append_ratio=0.15,sasl_buckets=1,kv_verify=False,value_size=1000

in pre-setup we load 1M items(items=1000000) with 1000Byte size(value_size=1000)=~1KB

then we append 15%(append_ratio=0.15) of them until we reach item size a little less 10KB(desired_item_size=10000Byte)

2014-01-10 07:08:42 | Item-appending of 150000 items starting ..
2014-01-10 07:18:12 | for 150000 items size was increased to 1000 Byte
2014-01-10 07:27:29 | for 150000 items size was increased to 2024 Byte
2014-01-10 07:36:41 | for 150000 items size was increased to 3048 Byte
2014-01-10 07:45:51 | for 150000 items size was increased to 4072 Byte
2014-01-10 07:57:11 | for 150000 items size was increased to 5096 Byte
2014-01-10 08:24:53 | for 150000 items size was increased to 6120 Byte
2014-01-10 08:54:01 | for 150000 items size was increased to 7144 Byte
2014-01-10 10:04:38 | for 150000 items size was increased to 8168 Byte
2014-01-10 10:59:41 | for 150000 items size was increased to 9192 Byte
2014-01-10 10:59:41 | The appending of 150000 items ended

and another comparison:
test
./testrunner -i andrei2.ini -t memorysanitytests.MemorySanity.test_items_append,items=2000,desired_item_size=3000000,append_ratio=0.15,sasl_buckets=1,kv_verify=False,value_size=1000000,append_size=40000

steps: 2 buckets
2K items with size 1M(value_size=1000000 Byte)

append 300 items (append_ratio=0.15) with delta 40K (append_size=40000 Byte) until we reach item size a little less 3MB(desired_item_size=3000000 Byte)

build 2.2.0-821:

In 10.3.4.144 bucket default, total_fragmentation_bytes + the total_allocated_bytes = 1335845776
In 10.3.4.144 bucket default, mem_used = 608118296
In 10.3.4.144 bucket default, the difference between actual memory used by memcached and mem_used is 2.19668736295 times
In 10.3.4.146 bucket default, total_fragmentation_bytes + the total_allocated_bytes = 1366215208
In 10.3.4.146 bucket default, mem_used = 614061672
In 10.3.4.146 bucket default, the difference between actual memory used by memcached and mem_used is 2.22488272807 times
In 10.3.4.145 bucket default, total_fragmentation_bytes + the total_allocated_bytes = 1336260640
In 10.3.4.145 bucket default, mem_used = 592927640
In 10.3.4.145 bucket default, the difference between actual memory used by memcached and mem_used is 2.25366562436 times

In 10.3.4.144 bucket bucket0, total_fragmentation_bytes + the total_allocated_bytes = 1336082080
In 10.3.4.144 bucket bucket0, mem_used = 544514488
In 10.3.4.144 bucket bucket0, the difference between actual memory used by memcached and mem_used is 2.45371263657 times
In 10.3.4.146 bucket bucket0, total_fragmentation_bytes + the total_allocated_bytes = 1361140336
In 10.3.4.146 bucket bucket0, mem_used = 576695448
In 10.3.4.146 bucket bucket0, the difference between actual memory used by memcached and mem_used is 2.36024116493 times
In 10.3.4.145 bucket bucket0, total_fragmentation_bytes + the total_allocated_bytes = 1336205024
In 10.3.4.145 bucket bucket0, mem_used = 570649352
In 10.3.4.145 bucket bucket0, the difference between actual memory used by memcached and mem_used is 2.34155181166 times

build 2.5.0-1046:

In 10.3.4.148 bucket default, total_fragmentation_bytes + the total_allocated_bytes = 1700445864
In 10.3.4.148 bucket default, mem_used = 620101656
In 10.3.4.148 bucket default, the difference between actual memory used by memcached and mem_used is 2.74220500388 times
In 10.3.4.147 bucket default, total_fragmentation_bytes + the total_allocated_bytes = 1694731184
In 10.3.4.147 bucket default, mem_used = 608031784
In 10.3.4.147 bucket default, the difference between actual memory used by memcached and mem_used is 2.78724110909 times
In 10.3.4.149 bucket default, total_fragmentation_bytes + the total_allocated_bytes = 1786141968
In 10.3.4.149 bucket default, mem_used = 620130840
In 10.3.4.149 bucket default, the difference between actual memory used by memcached and mem_used is 2.88026631283 times

In 10.3.4.148 bucket bucket0, total_fragmentation_bytes + the total_allocated_bytes = 1700446208
In 10.3.4.148 bucket bucket0, mem_used = 580733208
In 10.3.4.148 bucket bucket0, the difference between actual memory used by memcached and mem_used is 2.9281022414 times
In 10.3.4.147 bucket bucket0, total_fragmentation_bytes + the total_allocated_bytes = 1691246496
In 10.3.4.147 bucket bucket0, mem_used = 535454056
In 10.3.4.147 bucket bucket0, the difference between actual memory used by memcached and mem_used is 3.15852775238 times
In 10.3.4.149 bucket bucket0, total_fragmentation_bytes + the total_allocated_bytes = 1735417856
In 10.3.4.149 bucket bucket0, mem_used = 563604536
In 10.3.4.149 bucket bucket0, the difference between actual memory used by memcached and mem_used is 3.079141038 times

and now I see regression in 25%

AVG(2.5)/ AVG(2.2) = (2.74 + 2.78 + 2.88)/(2.19 + 2.2 + 2.25)=~1.25