Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-10121

cbrecovery hangs if it terminated in console

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Won't Fix
    • Major
    • feature-backlog
    • 2.5.0
    • ns_server, tools, UI
    • Security Level: Public
    • None
    • 2.5.0-1059

    Description

      steps:

      run cbrecovery and then terminate it in console

      andrey@baranouski:~/atg-store$ /opt/couchbase/bin/cbrecovery http://10.3.3.233:8091 http://10.3.3.141:8091 -b default -B default -u Administrator -p password -U Administrator -P password
      Missing vbuckets to be recovered:[

      {"node": "ns_1@10.3.4.144", "vbuckets": [15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 342, 343, 344, 345, 346, 347, 348, 349, 350, 351, 352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363, 364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 375, 376, 377, 378, 379, 380, 381, 382, 383, 384, 385, 386, 387, 388, 389, 390, 391, 392, 393, 394, 395, 396, 397, 398, 399, 400, 401, 402, 403, 404, 405, 406, 407, 408, 409, 410, 411, 412, 413, 414, 415, 416, 417, 418, 419, 420, 421, 422, 423, 424, 425, 426, 427, 428, 429, 430, 431, 432, 433, 434, 435, 436, 437, 438, 439, 440, 441, 442, 443, 444, 445, 446, 447, 448, 449, 450, 451, 452, 453, 454, 455, 456, 457, 458, 459, 460, 461, 462, 463, 464, 465, 466, 467, 468, 469, 470, 471, 472, 473, 474, 475, 476, 477, 478, 479, 480, 481, 482, 483, 484, 485, 486, 487, 488, 489, 490, 491, 492, 493, 494, 495, 496, 497, 498, 499, 500, 501, 502, 503, 504, 505, 506, 507, 508, 509, 510, 511, 512]}

      ]
      [ ] 0.0% (0/estimated 10 msgs)
      bucket: default, msgs transferred...
      : total | last | per sec
      byte : 0 | 0 | 0.0
      transfer data only. bucket design docs will be skipped.
      done
      ^Cinterrupted.
      andrey@baranouski:~/atg-store$ curl http://Administrator:password@10.3.3.141:8091/pools/default/tasks
      [

      {"type":"recovery","bucket":"default","uuid":"7fd15e10ca2c8d06f0bc11545372c8ce","recommendedRefreshPeriod":10,"stopURI":"/pools/default/buckets/default/controller/stopRecovery?recovery_uuid=7fd15e10ca2c8d06f0bc11545372c8ce","commitVBucketURI":"/pools/default/buckets/default/controller/commitVBucket?recovery_uuid=7fd15e10ca2c8d06f0bc11545372c8ce","recoveryStatusURI":"/pools/default/buckets/default/recoveryStatus?recovery_uuid=7fd15e10ca2c8d06f0bc11545372c8ce"}

      ,

      {"type":"rebalance","status":"notRunning"}

      ,

      {"cancelURI":"/controller/cancelXDCR/b2585fdf4118cf9b90e4b9be4b021525%2Fdefault%2Fdefault","settingsURI":"/settings/replications/b2585fdf4118cf9b90e4b9be4b021525%2Fdefault%2Fdefault","status":"running","replicationType":"capi","id":"b2585fdf4118cf9b90e4b9be4b021525/default/default","source":"default","target":"/remoteClusters/b2585fdf4118cf9b90e4b9be4b021525/buckets/default","continuous":true,"type":"xdcr","recommendedRefreshPeriod":10,"changesLeft":0,"docsChecked":7,"docsWritten":7,"errors":["2014-02-04 05:33:54 [Vb Rep] Error replicating vbucket 60. Please see logs for details.","2014-02-04 05:33:54 [Vb Rep] Error replicating vbucket 36. Please see logs for details.","2014-02-04 05:33:54 [Vb Rep] Error replicating vbucket 30. Please see logs for details.","2014-02-04 05:33:54 [Vb Rep] Error replicating vbucket 26. Please see logs for details.","2014-02-04 05:33:54 [Vb Rep] Error replicating vbucket 26. Please see logs for details.","2014-02-04 05:33:54 [Vb Rep] Error replicating vbucket 18. Please see logs for details.","2014-02-04 05:33:54 [Vb Rep] Error replicating vbucket 14. Please see logs for details.","2014-02-04 05:33:54 [Vb Rep] Error replicating vbucket 14. Please see logs for details.","2014-02-04 05:33:54 [Vb Rep] Error replicating vbucket 12. Please see logs for details.","2014-02-04 05:33:53 [Vb Rep] Error replicating vbucket 4. Please see logs for details."]}

      ]andrey@baranouski:~/atg-store$
      andrey@baranouski:~/atg-store$
      andrey@baranouski:~/atg-store$
      andrey@baranouski:~/atg-store$ curl http://Administrator:password@10.3.3.141:8091/pools/default/buckets/default/recoveryStatus?recovery_uuid=7fd15e10ca2c8d06f0bc11545372c8ce
      {"uuid":"7fd15e10ca2c8d06f0bc11545372c8ce","code":"ok","recoveryMap":[

      {"node":"ns_1@10.3.4.144","vbuckets":[16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,342,343,344,345,346,347,348,349,350,351,352,353,354,355,356,357,358,359,360,361,362,363,364,365,366,367,368,369,370,371,372,373,374,375,376,377,378,379,380,381,382,383,384,385,386,387,388,389,390,391,392,393,394,395,396,397,398,399,400,401,402,403,404,405,406,407,408,409,410,411,412,413,414,415,416,417,418,419,420,421,422,423,424,425,426,427,428,429,430,431,432,433,434,435,436,437,438,439,440,441,442,443,444,445,446,447,448,449,450,451,452,453,454,455,456,457,458,459,460,461,462,463,464,465,466,467,468,469,470,471,472,473,474,475,476,477,478,479,480,481,482,483,484,485,486,487,488,489,490,491,492,493,494,495,496,497,498,499,500,501,502,503,504,505,506,507,508,509,510,511,512]}

      ]}andrey@baranouski:~/atg-store$
      andrey@baranouski:~/atg-store$ date
      Tue Feb 4 16:58:00 FET 2014
      andrey@baranouski:~/atg-store$ curl http://Administrator:password@10.3.3.141:8091/pools/default/buckets/default/recoveryStatus?recovery_uuid=7fd15e10ca2c8d06f0bc11545372c8ce
      {"uuid":"7fd15e10ca2c8d06f0bc11545372c8ce","code":"ok","recoveryMap":[

      {"node":"ns_1@10.3.4.144","vbuckets":[16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,342,343,344,345,346,347,348,349,350,351,352,353,354,355,356,357,358,359,360,361,362,363,364,365,366,367,368,369,370,371,372,373,374,375,376,377,378,379,380,381,382,383,384,385,386,387,388,389,390,391,392,393,394,395,396,397,398,399,400,401,402,403,404,405,406,407,408,409,410,411,412,413,414,415,416,417,418,419,420,421,422,423,424,425,426,427,428,429,430,431,432,433,434,435,436,437,438,439,440,441,442,443,444,445,446,447,448,449,450,451,452,453,454,455,456,457,458,459,460,461,462,463,464,465,466,467,468,469,470,471,472,473,474,475,476,477,478,479,480,481,482,483,484,485,486,487,488,489,490,491,492,493,494,495,496,497,498,499,500,501,502,503,504,505,506,507,508,509,510,511,512]}

      ]}andrey@baranouski:~/atg-store$ date
      Tue Feb 4 17:00:47 FET 2014

      result: cbrecovery is "in progress" but in fact it hangs

      see also screenshot on UI where we can not determine the progress or status

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            bcui Bin Cui (Inactive)
            andreibaranouski Andrei Baranouski
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty