Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-44637

[BP MB-44327 6.6.2] - Duplicate mutation due to a race during rebalance

    XMLWordPrintable

Details

    • Untriaged
    • 1
    • Yes

    Description

      Build: 7.0.0-4454

      Test:

      • Deploy 5 handler bucket op,timers,n1ql,sbm,curl
      • load documents to source bucket of handlers
      • Rebalance in / out the eventing node 
      • Delete collections which has bindings for bucket op and timers
      • wait for mutation to complete
      • Seeing duplicate mutations in sbm where we haven't deleted the bindings

      ./testrunner -i /tmp/testexec.120861.ini -p get-cbcollect-info=True,GROUP=bucket_op,skip_log_scan=False,bucket_storage=couchstore -t eventing.eventing_rebalance_collection.EventingRebalanceCollection.test_eventing_rebalance_in_delete_recreate_collections,nodes_init=4,services_init=kv-eventing-index-n1ql,dataset=default,groups=simple,reset_services=True,doc-per-day=20,GROUP=bucket_op,java_sdk_client=True
       
      Exception: duplicated data in destination bucket which is not expected. Current : 40571 Expected : 40320  dcp_backlog : 0  TIMERS_IN_PAST : 0 lcb_exceptions : {'172.23.122.82': {}, '172.23.122.97': {}, '172.23.123.62': {'12009': 466, '209': 466}}

       ./testrunner -i /tmp/testexec.120861.ini -p get-cbcollect-info=True,GROUP=bucket_op,skip_log_scan=False,bucket_storage=couchstore -t eventing.eventing_rebalance_collection.EventingRebalanceCollection.test_eventing_rebalance_out_delete_recreate_collections,nodes_init=6,services_init=kv-eventing-eventing-eventing-index-n1ql,dataset=default,groups=simple,reset_services=True,doc-per-day=20,GROUP=bucket_op,java_sdk_client=True
       
       
      Exception: duplicated data in destination bucket which is not expected. Current : 40392 Expected : 40320  dcp_backlog : 0  TIMERS_IN_PAST : 0 lcb_exceptions : {'172.23.123.62': {'211': 37}}

      Attachments

        1. 172.23.122.116-20210212-2353-diag.zip
          7.34 MB
        2. 172.23.122.117-20210212-2353-diag.zip
          24.79 MB
        3. 172.23.122.118-20210212-2353-diag.zip
          21.65 MB
        4. 172.23.122.138-20210222-2309-diag.zip
          26.97 MB
        5. 172.23.122.139-20210222-2309-diag.zip
          38.31 MB
        6. 172.23.122.140-20210222-2309-diag.zip
          43.29 MB
        7. 172.23.122.141-20210222-2309-diag.zip
          12.08 MB
        8. 172.23.122.153-20210222-2309-diag.zip
          38.03 MB
        9. 172.23.122.179-20210222-2309-diag.zip
          31.46 MB
        10. 172.23.122.182-20210222-2309-diag.zip
          43.42 MB
        11. 172.23.122.82-20210212-2353-diag.zip
          29.91 MB
        12. 172.23.122.90-20210212-2353-diag.zip
          41.99 MB
        13. 172.23.122.97-20210212-2353-diag.zip
          46.98 MB
        14. 172.23.123.61-20210212-2353-diag.zip
          40.41 MB
        15. 172.23.123.62-20210212-2353-diag.zip
          40.23 MB
        16. 172.23.123.65-20210222-2309-diag.zip
          51.43 MB
        17. test_9.log
          415 kB
        18. test.log
          390 kB
        19. test-1.log
          252 kB

        Issue Links

          Activity

            People

              vinayaka.kamath Vinayaka Kamath (Inactive)
              jeelan.poola Jeelan Poola
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                PagerDuty