Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-4461

replication cursor stuck from slave 1 to slave 2 , hence high number of checkpoint items in slave1

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 1.7.2
    • Fix Version/s: 2.0-beta
    • Component/s: couchbase-bucket
    • Security Level: Public
    • Labels:
    • Environment:
      Ubuntu 10.04.3 LTS x86_64, Membase 1.7.2, Amazon m1.large instances, 16-node cluster.

      Description

      Two out of sixteen nodes are ejecting active items because their mem_used is above the high water mark. The other nodes are well below. Customer says that keys are of various sizes, but the larger ones should be spread out randomly across the different nodes. Number of keys on all nodes is roughly equal.

      The two problem nodes show ep_value_size much larger than a healthy node. However, looking at the sqlite data files, there's no significant difference in size of the files on disk (as seen, for example, in */membase.log).

      FYI, the rise in data size seems to have started on these two nodes after a different node, 10.254.7.150, stopped responding to REST and membase was restarted (with 'service membase-server restart').

      The mbcollect_info data for these servers are in the S3 . The logs are named:

      membase 16: a good node, for comparison
      membase 07 and membase 14: the trouble nodes that are ejecting items due to large memory usage
      membase 11: the node that was restarted on Saturday

      Can someone please take a look at this, and help me understand why the ep_value_size might be bloating up for these two nodes?

      Thanks,

      Tim

      # Subject Project Status CR V
      For Gerrit Dashboard: &For+MB-4461=message:MB-4461

        Activity

        TimSmith Tim Smith (Inactive) created issue -
        farshid Farshid Ghods (Inactive) made changes -
        Field Original Value New Value
        Issue Type Story [ 6 ] Bug [ 1 ]
        farshid Farshid Ghods (Inactive) made changes -
        Description Two out of sixteen nodes are ejecting active items because their mem_used is above the high water mark. The other nodes are well below. Customer says that keys are of various sizes, but the larger ones should be spread out randomly across the different nodes. Number of keys on all nodes is roughly equal.

        The two problem nodes show ep_value_size much larger than a healthy node. However, looking at the sqlite data files, there's no significant difference in size of the files on disk (as seen, for example, in */membase.log).

        FYI, the rise in data size seems to have started on these two nodes after a different node, 10.254.7.150, stopped responding to REST and membase was restarted (with 'service membase-server restart').

        The mbcollect_info data for these servers are in the S3 at customers.couchbase.com/umami/ . The logs are named:

        membase 16: a good node, for comparison
        membase 07 and membase 14: the trouble nodes that are ejecting items due to large memory usage
        membase 11: the node that was restarted on Saturday


        Can someone please take a look at this, and help me understand why the ep_value_size might be bloating up for these two nodes?

        Thanks,

        Tim

        Two out of sixteen nodes are ejecting active items because their mem_used is above the high water mark. The other nodes are well below. Customer says that keys are of various sizes, but the larger ones should be spread out randomly across the different nodes. Number of keys on all nodes is roughly equal.

        The two problem nodes show ep_value_size much larger than a healthy node. However, looking at the sqlite data files, there's no significant difference in size of the files on disk (as seen, for example, in */membase.log).

        FYI, the rise in data size seems to have started on these two nodes after a different node, 10.254.7.150, stopped responding to REST and membase was restarted (with 'service membase-server restart').

        The mbcollect_info data for these servers are in the S3 . The logs are named:

        membase 16: a good node, for comparison
        membase 07 and membase 14: the trouble nodes that are ejecting items due to large memory usage
        membase 11: the node that was restarted on Saturday


        Can someone please take a look at this, and help me understand why the ep_value_size might be bloating up for these two nodes?

        Thanks,

        Tim

        farshid Farshid Ghods (Inactive) made changes -
        Assignee Mike Wiederhold [ mikew ]
        farshid Farshid Ghods (Inactive) made changes -
        Summary ep_value_size large on two nodes, but sqlite files not bloated replica node is not persisting replication items to disk from memory and hence checkpoint items are kept in memory
        farshid Farshid Ghods (Inactive) made changes -
        Comment [ grepping for mem_used shows that it increased from 2..... to 4... and it is bouncing back and forth between 4.1 and 4.2

        ./ns_server.log:mem_used 3781928470
        ./ns_server.log:mem_used 3785524778
        ./ns_server.log:mem_used 3793641927
        ./ns_server.log:mem_used 3796850266
        ./ns_server.log:mem_used 3802705555
        ./ns_server.log:mem_used 3816069541
        ./ns_server.log:mem_used 3807969675
        ./ns_server.log:mem_used 3811358027
        ./ns_server.log:mem_used 3776635340
        ./ns_server.log:mem_used 3750806756
        ./ns_server.log:mem_used 3765881048
        ./ns_server.log:mem_used 3782645146
        ./ns_server.log:mem_used 3792286113
        ./ns_server.log:mem_used 3777521605
        ./ns_server.log:mem_used 3772382610
        ./ns_server.log:mem_used 3775429318
        ./ns_server.log:mem_used 3787245808
        ./ns_server.log:mem_used 3802814288
        ./ns_server.log:mem_used 3812506176
        ./ns_server.log:mem_used 3811904819
        ./ns_server.log:mem_used 3819497312
        ./ns_server.log:mem_used 3820309558
        ./ns_server.log:mem_used 3827466798
        ./ns_server.log:mem_used 3837120979
        ./ns_server.log:mem_used 3820473335
        ./ns_server.log:mem_used 3824534778
        ./ns_server.log:mem_used 3833182726
        ./ns_server.log:mem_used 3830486917
        ./ns_server.log:mem_used 3837908130
        ./ns_server.log:mem_used 3808073337
        ./ns_server.log:mem_used 3795467526
        ./ns_server.log:mem_used 3797466021
        ./ns_server.log:mem_used 3793598684
        ./ns_server.log:mem_used 3787538301
        ./ns_server.log:mem_used 3794329831
        ./ns_server.log:mem_used 3765985607
        ./ns_server.log:mem_used 3764534838
        ./ns_server.log:mem_used 3770317130
        ./ns_server.log:mem_used 3778489283
        ./ns_server.log:mem_used 3777330502
        ./ns_server.log:mem_used 3787285798
        ./ns_server.log:mem_used 3803153218
        ./ns_server.log:mem_used 3814246273
        ./ns_server.log:mem_used 3819251237
        ./ns_server.log:mem_used 3823159457
        ./ns_server.log:mem_used 3810006985
        ./ns_server.log:mem_used 3817249902
        ./ns_server.log:mem_used 3827893236
        ./ns_server.log:mem_used 3839810332
        ./ns_server.log:mem_used 3827161408
        ./ns_server.log:mem_used 3827909896
        ./ns_server.log:mem_used 3806745551
        ./ns_server.log:mem_used 3808227941
        ./ns_server.log:mem_used 3819558325
        ./ns_server.log:mem_used 3831404828
        ./ns_server.log:mem_used 3831878446
        ./ns_server.log:mem_used 3823619348
        ./ns_server.log:mem_used 3813853883
        ./ns_server.log:mem_used 3822595717
        ./ns_server.log:mem_used 3838127490
        ./ns_server.log:mem_used 3845584371
        ./ns_server.log:mem_used 3849808071
        ./ns_server.log:mem_used 3852541457
        ./ns_server.log:mem_used 3817086945
        ./ns_server.log:mem_used 3822498815
        ./ns_server.log:mem_used 3824093731
        ./ns_server.log:mem_used 3844026408
        ./ns_server.log:mem_used 3840031885
        ./ns_server.log:mem_used 3849206569
        ./ns_server.log:mem_used 3841524378
        ./ns_server.log:mem_used 3849333690
        ./ns_server.log:mem_used 3864814123
        ./ns_server.log:mem_used 3864922855
        ./ns_server.log:mem_used 3868662716
        ./ns_server.log:mem_used 3842522395
        ./ns_server.log:mem_used 3792940540
        ./ns_server.log:mem_used 3801193243
        ./ns_server.log:mem_used 3811891538
        ./ns_server.log:mem_used 3816011827
        ./ns_server.log:mem_used 3825488543
        ./ns_server.log:mem_used 3742566028
        ./ns_server.log:mem_used 3734005754
        ./ns_server.log:mem_used 3743716648
        ./ns_server.log:mem_used 3758480075
        ./ns_server.log:mem_used 3770798324
        ./ns_server.log:mem_used 3777108218
        ./ns_server.log:mem_used 3790071526
        ./ns_server.log:mem_used 3785033407
        ./ns_server.log:mem_used 3794187685
        ./ns_server.log:mem_used 3808951608
        ./ns_server.log:mem_used 3818602934
        ./ns_server.log:mem_used 3802362458
        ./ns_server.log:mem_used 3808832182
        ./ns_server.log:mem_used 3802630497
        ./ns_server.log:mem_used 3811389996
        ./ns_server.log:mem_used 3827635633
        ./ns_server.log:mem_used 3838666578
        ./ns_server.log:mem_used 3829743386
        ./ns_server.log:mem_used 3838610466
        ./ns_server.log:mem_used 3825475132
        ./ns_server.log:mem_used 3833643047
        ./ns_server.log:mem_used 3849500686
        ./ns_server.log:mem_used 3856282454
        ./ns_server.log:mem_used 3857763683
        ./ns_server.log:mem_used 3864384021
        ./ns_server.log:mem_used 3846232517
        ./ns_server.log:mem_used 3852155251
        ./ns_server.log:mem_used 3866795416
        ./ns_server.log:mem_used 3868725228
        ./ns_server.log:mem_used 3872368561
        ./ns_server.log:mem_used 3879907182
        ./ns_server.log:mem_used 3872681140
        ./ns_server.log:mem_used 3850146786
        ./ns_server.log:mem_used 3866517972
        ./ns_server.log:mem_used 3827125804
        ./ns_server.log:mem_used 3836608957
        ./ns_server.log:mem_used 3849016709
        ./ns_server.log:mem_used 3824436926
        ./ns_server.log:mem_used 3788653416
        ./ns_server.log:mem_used 3807179590
        ./ns_server.log:mem_used 3821233629
        ./ns_server.log:mem_used 3812035071
        ./ns_server.log:mem_used 3825152764
        ./ns_server.log:mem_used 3822139564
        ./ns_server.log:mem_used 3831961912
        ./ns_server.log:mem_used 3848647083
        ./ns_server.log:mem_used 3858538619
        ./ns_server.log:mem_used 3852395332
        ./ns_server.log:mem_used 3859785299
        ./ns_server.log:mem_used 3813223264
        ./ns_server.log:mem_used 3825128542
        ./ns_server.log:mem_used 3837225558
        ./ns_server.log:mem_used 3848998379
        ./ns_server.log:mem_used 3853789403
        ./ns_server.log:mem_used 3863620700
        ./ns_server.log:mem_used 3856104166
        ./ns_server.log:mem_used 3864249079
        ./ns_server.log:mem_used 3871133127
        ./ns_server.log:mem_used 3871411550
        ./ns_server.log:mem_used 3875563626
        ./ns_server.log:mem_used 3883315457
        ./ns_server.log:mem_used 3869593246
        ./ns_server.log:mem_used 3877340752
        ./ns_server.log:mem_used 3889444987
        ./ns_server.log:mem_used 3899312594
        ./ns_server.log:mem_used 3900492277
        ./ns_server.log:mem_used 3908137702
        ./ns_server.log:mem_used 3890232603
        ./ns_server.log:mem_used 3895340098
        ./ns_server.log:mem_used 3904126805
        ./ns_server.log:mem_used 3895072871
        ./ns_server.log:mem_used 3900803055
        ./ns_server.log:mem_used 3877309858
        ./ns_server.log:mem_used 3819430628
        ./ns_server.log:mem_used 3828433076
        ./ns_server.log:mem_used 3840347207
        ./ns_server.log:mem_used 3854145892
        ./ns_server.log:mem_used 3858539288
        ./ns_server.log:mem_used 3867028466
        ./ns_server.log:mem_used 3861938268
        ./ns_server.log:mem_used 3870817272
        ./ns_server.log:mem_used 3882031316
        ./ns_server.log:mem_used 3893222567
        ./ns_server.log:mem_used 3898396575
        ./ns_server.log:mem_used 3905071872
        ./ns_server.log:mem_used 3887571476
        ./ns_server.log:mem_used 3896652392
        ./ns_server.log:mem_used 3881923161
        ./ns_server.log:mem_used 3891416091
        ./ns_server.log:mem_used 3897006586
        ./ns_server.log:mem_used 3907629701
        ./ns_server.log:mem_used 3895877496
        ./ns_server.log:mem_used 3901332286
        ./ns_server.log:mem_used 3914837049
        ./ns_server.log:mem_used 3926376053
        ./ns_server.log:mem_used 3927220479
        ./ns_server.log:mem_used 3903227269
        ./ns_server.log:mem_used 3874865348
        ./ns_server.log:mem_used 3885288985
        ./ns_server.log:mem_used 3897940993
        ./ns_server.log:mem_used 3909246963
        ./ns_server.log:mem_used 3914037120
        ./ns_server.log:mem_used 3924508154
        ./ns_server.log:mem_used 3918218078
        ./ns_server.log:mem_used 3926003471
        ./ns_server.log:mem_used 3938870506
        ./ns_server.log:mem_used 3949258683
        ./ns_server.log:mem_used 3952965371
        ./ns_server.log:mem_used 3960392049
        ./ns_server.log:mem_used 3941579339
        ./ns_server.log:mem_used 3948059836
        ./ns_server.log:mem_used 3956813089
        ./ns_server.log:mem_used 3960257746
        ./ns_server.log:mem_used 3960309698
        ./ns_server.log:mem_used 3968268249
        ./ns_server.log:mem_used 3932320536
        ./ns_server.log:mem_used 3941763924
        ./ns_server.log:mem_used 3956720882
        ./ns_server.log:mem_used 3967772183
        ./ns_server.log:mem_used 3961654872
        ./ns_server.log:mem_used 3930353777
        ./ns_server.log:mem_used 3926899578
        ./ns_server.log:mem_used 3938449607
        ./ns_server.log:mem_used 3952194089
        ./ns_server.log:mem_used 3966049512
        ./ns_server.log:mem_used 3969065251
        ./ns_server.log:mem_used 3978802875
        ./ns_server.log:mem_used 3967268252
        ./ns_server.log:mem_used 3969887553
        ./ns_server.log:mem_used 3982567213
        ./ns_server.log:mem_used 3990930694
        ./ns_server.log:mem_used 3975884842
        ./ns_server.log:mem_used 3984889940
        ./ns_server.log:mem_used 3974199979
        ./ns_server.log:mem_used 3982692169
        ./ns_server.log:mem_used 3995872679
        ./ns_server.log:mem_used 4006790407
        ./ns_server.log:mem_used 4010549443
        ./ns_server.log:mem_used 4018121078
        ./ns_server.log:mem_used 4001555190
        ./ns_server.log:mem_used 4008740661
        ./ns_server.log:mem_used 4019681936
        ./ns_server.log:mem_used 4026920537
        ./ns_server.log:mem_used 4015626768
        ./ns_server.log:mem_used 4024809240
        ./ns_server.log:mem_used 4007356084
        ./ns_server.log:mem_used 3996550136
        ./ns_server.log:mem_used 4036620555
        ./ns_server.log:mem_used 4055972625
        ./ns_server.log:mem_used 4073635084
        ./ns_server.log:mem_used 4074634587
        ./ns_server.log:mem_used 4045129541
        ./ns_server.log:mem_used 4037838349
        ./ns_server.log:mem_used 4045293450
        ./ns_server.log:mem_used 4050590664
        ./ns_server.log:mem_used 4051035176
        ./ns_server.log:mem_used 4052180899
        ./ns_server.log:mem_used 4047684139
        ./ns_server.log:mem_used 4046205113
        ./ns_server.log:mem_used 4031406894
        ./ns_server.log:mem_used 4037345300
        ./ns_server.log:mem_used 4034772644
        ./ns_server.log:mem_used 4006516597
        ./ns_server.log:mem_used 4008986982
        ./ns_server.log:mem_used 4020785077
        ./ns_server.log:mem_used 4041685431
        ./ns_server.log:mem_used 4055851877
        ./ns_server.log:mem_used 4041502332
        ./ns_server.log:mem_used 4056900491
        ./ns_server.log:mem_used 4057885146
        ./ns_server.log:mem_used 4071705310
        ./ns_server.log:mem_used 4073910137
        ./ns_server.log:mem_used 4086672848
        ./ns_server.log:mem_used 4093120966
        ./ns_server.log:mem_used 4105804188
        ./ns_server.log:mem_used 4104409066
        ./ns_server.log:mem_used 4085887572
        ./ns_server.log:mem_used 4080160525
        ./ns_server.log:mem_used 4074654500
        ./ns_server.log:mem_used 4077425129
        ./ns_server.log:mem_used 4093958245
        ./ns_server.log:mem_used 4088869002
        ./ns_server.log:mem_used 4098396253
        ./ns_server.log:mem_used 4107520080
        ./ns_server.log:mem_used 4115239508
        ./ns_server.log:mem_used 4100605465
        ./ns_server.log:mem_used 4112209184
        ./ns_server.log:mem_used 4101149284
        ./ns_server.log:mem_used 4110617152
        ./ns_server.log:mem_used 4109973954
        ./ns_server.log:mem_used 4113656627
        ./ns_server.log:mem_used 4089277229
        ./ns_server.log:mem_used 4110053626
        ./ns_server.log:mem_used 4109857021
        ./ns_server.log:mem_used 4122118072
        ./ns_server.log:mem_used 4143807466
        ./ns_server.log:mem_used 4109457456
        ./ns_server.log:mem_used 4113922330
        ./ns_server.log:mem_used 4134252075
        ./ns_server.log:mem_used 4127474707
        ./ns_server.log:mem_used 4138727835
        ./ns_server.log:mem_used 4140320635
        ./ns_server.log:mem_used 4143547716
        ./ns_server.log:mem_used 4145126063
        ./ns_server.log:mem_used 4100871998
        ./ns_server.log:mem_used 4103943202
        ./ns_server.log:mem_used 4117923506
        ./ns_server.log:mem_used 4131918013
        ./ns_server.log:mem_used 4132219136
        ./ns_server.log:mem_used 4142145097
        ./ns_server.log:mem_used 4142077711
        ./ns_server.log:mem_used 4128357373
        ./ns_server.log:mem_used 4104674249
        ./ns_server.log:mem_used 4130072389
        ./ns_server.log:mem_used 4126325067
        ./ns_server.log:mem_used 4140488270
        ./ns_server.log:mem_used 4154667077
        ./ns_server.log:mem_used 4149064131
        ./ns_server.log:mem_used 4149639427
        ./ns_server.log:mem_used 4155359099
        ./ns_server.log:mem_used 4159675717
        ./ns_server.log:mem_used 4144350226
        ./ns_server.log:mem_used 4149704382
        ./ns_server.log:mem_used 4126454179
        ./ns_server.log:mem_used 4140654625
        ./ns_server.log:mem_used 4159999148
        ./ns_server.log:mem_used 4159721814
        ./ns_server.log:mem_used 4164217282
        ./ns_server.log:mem_used 4161199231
        ./ns_server.log:mem_used 4155177904
        ./ns_server.log:mem_used 4164156179
        ./ns_server.log:mem_used 4185057829
        ./ns_server.log:mem_used 4185376397
        ./ns_server.log:mem_used 4178393198
        ./ns_server.log:mem_used 4197651256
        ./ns_server.log:mem_used 4194299281
        ./ns_server.log:mem_used 4202035420
        ./ns_server.log:mem_used 4203749835
        ./ns_server.log:mem_used 4212040909
        ./ns_server.log:mem_used 4210332233
        ./ns_server.log:mem_used 4212355099
        ./ns_server.log:mem_used 4197928690
        ./ns_server.log:mem_used 4209977417
        ./ns_server.log:mem_used 4217353058
        ./ns_server.log:mem_used 4223654140
        ./ns_server.log:mem_used 4229371289
        ./ns_server.log:mem_used 4239573307
        ./ns_server.log:mem_used 4243551258
        ./ns_server.log:mem_used 4177188654
        ./ns_server.log:mem_used 4197500679
        ./ns_server.log:mem_used 4199995358
        ./ns_server.log:mem_used 4212809893
        ./ns_server.log:mem_used 4177021118
        ./ns_server.log:mem_used 4161525879
        ./ns_server.log:mem_used 4175396412
        ./ns_server.log:mem_used 4194018519
        ./ns_server.log:mem_used 4197096535
        ./ns_server.log:mem_used 4191206950
        ./ns_server.log:mem_used 4215127604
        ./ns_server.log:mem_used 4217761334
        ./ns_server.log:mem_used 4234327319
        ./ns_server.log:mem_used 4256914803
        ./ns_server.log:mem_used 4258073133
        ./ns_server.log:mem_used 4254539046
        ./ns_server.log:mem_used 4273359194
        ./ns_server.log:mem_used 4274345968
        ./ns_server.log:mem_used 4275419542
        ./ns_server.log:mem_used 4280031098
        ./ns_server.log:mem_used 4268186248
        ./ns_server.log:mem_used 4256380403
        ./ns_server.log:mem_used 4274713646
        ./ns_server.log:mem_used 4273044496
        ./ns_server.log:mem_used 4270576564
        ./ns_server.log:mem_used 4282239770
        ./ns_server.log:mem_used 4235804317
        ./ns_server.log:mem_used 4240891333
        ./ns_server.log:mem_used 4255082981
        ./ns_server.log:mem_used 4225182928
        ./ns_server.log:mem_used 4224584091
        ./ns_server.log:mem_used 4249306487
        ./ns_server.log:mem_used 4243687866
        ./ns_server.log:mem_used 4254933590
        ./ns_server.log:mem_used 4282500731
        ./ns_server.log:mem_used 4249176265
        ./ns_server.log:mem_used 4251594588
        ./ns_server.log:mem_used 4269250073
        ./ns_server.log:mem_used 4247721327
        ./ns_server.log:mem_used 4239099938
        ./ns_server.log:mem_used 4261297293
        ./ns_server.log:mem_used 4256173587
        ./ns_server.log:mem_used 4237662194
        ./ns_server.log:mem_used 4260473429
        ./ns_server.log:mem_used 4261512529
        ./ns_server.log:mem_used 4261202196
        ./ns_server.log:mem_used 4282547901
        ./ns_server.log:mem_used 4248167430
        ./ns_server.log:mem_used 4247745837
        ./ns_server.log:mem_used 4267876114
        ./ns_server.log:mem_used 4267973795
        ./ns_server.log:mem_used 4217812270
        ./ns_server.log:mem_used 4235710670
        ./ns_server.log:mem_used 4234514245
        ./ns_server.log:mem_used 4249474470
        ./ns_server.log:mem_used 4259421082
        ./ns_server.log:mem_used 4240483420
        ./ns_server.log:mem_used 4252261309
        ./ns_server.log:mem_used 4275364438
        ./ns_server.log:mem_used 4276798243
        ./ns_server.log:mem_used 4288226683
        ./ns_server.log:mem_used 4321749928
        ./ns_server.log:mem_used 4151659289
        ./ns_server.log:mem_used 4158962985
        ./ns_server.log:mem_used 4180110686
        ./ns_server.log:mem_used 4179122151
        ./ns_server.log:mem_used 4189028896
        ./ns_server.log:mem_used 4198904358
        ./ns_server.log:mem_used 4175011405
        ./ns_server.log:mem_used 4126262719
        ./ns_server.log:mem_used 4149558863
        ./ns_server.log:mem_used 4150921766
        ./ns_server.log:mem_used 4148832953
        ./ns_server.log:mem_used 4161880176
        ./ns_server.log:mem_used 4161051335
        ./ns_server.log:mem_used 4166169673
        ./ns_server.log:mem_used 4184971845
        ./ns_server.log:mem_used 4180668361
        ./ns_server.log:mem_used 4191305056
        ./ns_server.log:mem_used 4206151904
        ./ns_server.log:mem_used 4193433783
        ./ns_server.log:mem_used 4188908704
        ./ns_server.log:mem_used 4190718623
        ./ns_server.log:mem_used 4190566647
        ./ns_server.log:mem_used 4184051366
        ./ns_server.log:mem_used 4194057720
        ./ns_server.log:mem_used 4195846930
        ./ns_server.log:mem_used 4206830757
        ./ns_server.log:mem_used 4214797308
        ./ns_server.log:mem_used 4191447240
        ./ns_server.log:mem_used 4205310509
        ./ns_server.log:mem_used 4226147514
        ./ns_server.log:mem_used 4232467065
        ./ns_server.log:mem_used 4238550254
        ./ns_server.log:mem_used 4260462260
        ./ns_server.log:mem_used 4237288717
        ./ns_server.log:mem_used 4247206268
        ./ns_server.log:mem_used 4230719524
        ./ns_server.log:mem_used 4269633285
        ./ns_server.log:mem_used 4232455177
        ./ns_server.log:mem_used 4211959854
        ./ns_server.log:mem_used 4211046396
        ./ns_server.log:mem_used 4203578920
        ./ns_server.log:mem_used 4223995988
        ./ns_server.log:mem_used 4202482875
        ./ns_server.log:mem_used 4212662948
        ./ns_server.log:mem_used 4234256710
        ./ns_server.log:mem_used 4191496022
        ./ns_server.log:mem_used 4207557961
        ./ns_server.log:mem_used 4221373550
        ./ns_server.log:mem_used 4224042557
        ./ns_server.log:mem_used 4216063982
        ./ns_server.log:mem_used 4238504425
        ./ns_server.log:mem_used 4234634830
        ./ns_server.log:mem_used 4224451284
        ./ns_server.log:mem_used 4236147698
        ./ns_server.log:mem_used 4228655590
        ./ns_server.log:mem_used 4232227255
        ./ns_server.log:mem_used 4245811576
        ./ns_server.log:mem_used 4242447924
        ./ns_server.log:mem_used 4252551497
        ./ns_server.log:mem_used 4256837248
        ./ns_server.log:mem_used 4241715685
        ./ns_server.log:mem_used 4248348160
        ./ns_server.log:mem_used 4267606356
        ./ns_server.log:mem_used 4268531492
        ./ns_server.log:mem_used 4279213204
        ./ns_server.log:mem_used 4298196122
        ./ns_server.log:mem_used 4272093484
        ./ns_server.log:mem_used 4280796410
        ./ns_server.log:mem_used 4294337746
        ./ns_server.log:mem_used 4294902278
        ./ns_server.log:mem_used 4305138777
        ./ns_server.log:mem_used 4310350093
        ./ns_server.log:mem_used 4289337086
        ./ns_server.log:mem_used 4286146427
        ./ns_server.log:mem_used 4299297631
        ./ns_server.log:mem_used 4300769431
        ./ns_server.log:mem_used 4311619255
        ./ns_server.log:mem_used 4323856143
        ./ns_server.log:mem_used 4323911657
        ./ns_server.log:mem_used 4197580729
        ./ns_server.log:mem_used 4219663563
        ./ns_server.log:mem_used 4217499093
        ./ns_server.log:mem_used 4212877664
        ./ns_server.log:mem_used 4162234380
        ./ns_server.log:mem_used 4137830797
        ./ns_server.log:mem_used 4056611211
        ./ns_server.log:mem_used 4078366679
        ./ns_server.log:mem_used 4080166822
        ./ns_server.log:mem_used 4091444270
        ./ns_server.log:mem_used 4100768993
        ./ns_server.log:mem_used 4105240530
        ./stats.log: mem_used: 4100160283 ]
        sharon Sharon Barr (Inactive) made changes -
        Fix Version/s 1.8.1 [ 10249 ]
        farshid Farshid Ghods (Inactive) made changes -
        Summary replica node is not persisting replication items to disk from memory and hence checkpoint items are kept in memory replication cursor stuck from slave 1 to slave 2 , hence high number of checkpoint items in slave1
        farshid Farshid Ghods (Inactive) made changes -
        Affects Version/s 1.7.2 [ 10203 ]
        Affects Version/s 1.7.1.1 [ 10210 ]
        farshid Farshid Ghods (Inactive) made changes -
        Environment Ubuntu 10.04.3 LTS x86_64, Membase 1.7.1.1, Amazon m1.large instances, 16-node cluster. Ubuntu 10.04.3 LTS x86_64, Membase 1.7.2, Amazon m1.large instances, 16-node cluster.
        sharon Sharon Barr (Inactive) made changes -
        Fix Version/s 1.8.0 [ 10248 ]
        dipti Dipti Borkar made changes -
        Labels customer
        Fix Version/s 1.8.0 [ 10248 ]
        farshid Farshid Ghods (Inactive) made changes -
        Comment [ Integrated in github-ep-engine-2-0 #131 (See [http://qa.hq.northscale.net/job/github-ep-engine-2-0/131/])
             ]
        chiyoung Chiyoung Seo made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        farshid Farshid Ghods (Inactive) made changes -
        Component/s couchbase-bucket [ 10173 ]
        Component/s ep_engine [ 10013 ]
        maria Maria McDuff (Inactive) made changes -
        Fix Version/s 2.0-beta [ 10113 ]
        Fix Version/s 1.8.2 [ 10249 ]
        mikew Mike Wiederhold made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            mikew Mike Wiederhold
            Reporter:
            TimSmith Tim Smith (Inactive)
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews