Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-54266

[Upgrade] - Online upgrade using swap rebalance from 6.6.5-MP8 -> 6.6.5-MP9 with n2n enabled + encryption level strict fails while rebalancing cbas



    • Bug
    • Resolution: Duplicate
    • Major
    • 6.6.6
    • 6.6.5
    • analytics
    • 6.6.5-10080 ---> 6.6.5-10117
    • Untriaged
    • Centos 64-bit
    • 1
    • Unknown
    • Analytics Sprint 6


      Steps to Repro
      1. Create a 3 node cluster on 6.6.5-10080 .
      2. Enable n2n encryption and set encryption level to strict using following commands.

      curl -v -u Administrator:password -X POST http://localhost:8091/internalSettings -d "canEnableStrictEncryption=true" 
      /opt/couchbase/bin/couchbase-cli node-to-node-encryption -c http://localhost:8091 -u Administrator -p password --enable
      curl -v -u Administrator:password http://localhost:8091/settings/security -d "clusterEncryptionLevel=strict"

      3. Create a few buckets and load data.
      4. Start upgrade using swap rebalance by adding a 6.6.5-10117 node(after enabling n2n + strict level ) and removing a 6.6.5-10080 node.

      I noticed following exits and rebalance failures on the incoming 6.6.5-10117 node 11:51:46 PM 25 Oct, 2022

      Service 'cbas' exited with status 1. Restarting. Messages:
      2022-10-25T23:51:46.243-07:00 INFO CBAS.cbas Using plain authentication for user <ud>@cbas</ud>
      2022-10-25T23:51:46.244-07:00 INFO CBAS.cbas Using plain authentication for user <ud>@cbas</ud>
      2022-10-25T23:51:46.244-07:00 INFO CBAS.cbas audit: created new audit service
      2022-10-25T23:51:46.252-07:00 INFO CBAS.cbas clusterCompatibility init'd to 393222
      2022-10-25T23:51:46.289-07:00 FATA CBAS.cbas my node not found in nodes ([{a10b1913268c11aed9d52e1e0d7f5766 1688875630395392 map[cbas-version:6.6.5-10080 cc-http-port:9111 controller-id:1 host: ns-server-port:8091 num-iodevices:2 starting-partition-id:2 svc-http-port:8095]} {cf365d0b127c37e0a502e7e2a1d80b2f 1688875630395648 map[cbas-version:6.6.5-10080 cc-http-port:9111 controller-id:0 host: ns-server-port:8091 num-iodevices:2 starting-partition-id:0 svc-http-port:8095]} {7d30bf2aea28a7cba3a9d56cfda68558 1688875630395392 map[cbas-version:6.6.5-10080 cc-http-port:9111 controller-id:2 host: ns-server-port:8091 num-iodevices:2 starting-partition-id:4 svc-http-port:8095]}])
     11:51:46 PM 25 Oct, 2022

      Rebalance exited with reason {service_rebalance_failed,cbas,
      Rebalance Operation Id = 7fa166c0eb20c796893dcc53a97b6163

      cbcollect_info attached.


        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.



              Balakumaran.Gopal Balakumaran Gopal
              Balakumaran.Gopal Balakumaran Gopal
              0 Vote for this issue
              6 Start watching this issue



                Gerrit Reviews

                  There are no open Gerrit changes