Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-63179

[System Test] Replica violation seen during system test run

    XMLWordPrintable

Details

    • Untriaged
    • Linux x86_64
    • 0
    • Unknown

    Description

      Script to Repro

      ./sequoia -client 172.23.97.180:2375 -provider file:centos_second_cluster.yml -test tests/integration/7.2/test_7.2.yml -scope tests/integration/7.2/scope_7.2_magma.yml -scale 1 -repeat 0 -log_level 0 -version 7.2.6-8109 -skip_setup=false -skip_test=false -skip_teardown=true -skip_cleanup=false -continue=false -collect_on_error=false -stop_on_error=false -duration=604800 -show_topology=true
      

      After almost 2 days of run we saw ReplicaViolation in query logs.

      172.23.218.182

      2024-08-15T20:54:34.414-07:00 [Error] 
      MemoryQuota: 20142096384
      CpuQuota: 8
      --- Violations for index <idx3_DPYmy1bY 1 (replica 1), bucket7, scope_1, coll_1> (mem 0, cpu 0) at node 172.23.218.186:8091
              Cannot move to 172.23.218.164:8091: ReplicaViolation (free mem 1.67772e+07T, free cpu 8)
              Can move to 172.23.218.157:8091: NoViolation (free mem 2.41148G, free cpu 8)
      --- Violations for index <idx3_DPYmy1bY 1 (replica 2), bucket7, scope_1, coll_1> (mem 0, cpu 0) at node 172.23.218.186:8091
              Cannot move to 172.23.218.164:8091: ReplicaViolation (free mem 1.67772e+07T, free cpu 8)
              Can move to 172.23.218.157:8091: NoViolation (free mem 2.41148G, free cpu 8)
      2024-08-15T20:54:34.414-07:00 [Error] Encounter planner error for index.  Use round robin strategy for planning. Error:
      MemoryQuota: 20142096384
      CpuQuota: 8
      --- Violations for index <idx3_DPYmy1bY 1 (replica 1), bucket7, scope_1, coll_1> (mem 0, cpu 0) at node 172.23.218.186:8091
              Cannot move to 172.23.218.164:8091: ReplicaViolation (free mem 1.67772e+07T, free cpu 8)
              Can move to 172.23.218.157:8091: NoViolation (free mem 2.41148G, free cpu 8)
      --- Violations for index <idx3_DPYmy1bY 1 (replica 2), bucket7, scope_1, coll_1> (mem 0, cpu 0) at node 172.23.218.186:8091
              Cannot move to 172.23.218.164:8091: ReplicaViolation (free mem 1.67772e+07T, free cpu 8)
              Can move to 172.23.218.157:8091: NoViolation (free mem 2.41148G, free cpu 8)
      : Index:bucket7, scope_1, coll_1, idx3_DPYmy1bY
      

      Attachments

        Activity

          People

            dhananjay.kshirsagar Dhananjay Kshirsagar
            Balakumaran.Gopal Balakumaran Gopal
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:

              PagerDuty