Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-15249

[system tests] goxdcr exits many times on src cluster during rebalance

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Critical
    • 4.0.0
    • 4.0.0
    • XDCR
    • Security Level: Public
    •  4.0.0-3082
    • Untriaged
    • Centos 64-bit
    • Unknown

    Description

      steps:
      1. 3 nodes in cluster, 4 buckets. run data loader more then a day
      2. setup replication from SRC to cluster DEST for all buckets.
      3. rebalance in at SRC cluster
      rebalance in at DEST cluster

      on src cluster:

      Port server goxdcr on node 'babysitter_of_ns_1@127.0.0.1' exited with status 1. Restarting. Messages: github.com/couchbase/goxdcr/factory.*XDCRFactory.SetStartSeqno·fm(0x7fba15a09e90, 0xc2119fc280, 0x0, 0x0)
      /home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/factory/xdcr_factory.go:139 +0x4d
      created by github.com/couchbase/goxdcr/pipeline.(*GenericPipeline).Start
      /home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline/generic_pipeline.go:178 +0x4dc
      [goport] 2015/06/09 02:22:11 /opt/couchbase/bin/goxdcr terminated: exit status 2 ns_log000 ns_1@172.23.105.22 02:22:11 - Tue Jun 9, 2015
      Port server goxdcr on node 'babysitter_of_ns_1@127.0.0.1' exited with status 1. Restarting. Messages: github.com/couchbase/goxdcr/gen_server.(*GenServer).run(0xc20b2586c0)
      /home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:78 +0x998
      created by github.com/couchbase/goxdcr/gen_server.(*GenServer).Start_server
      /home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/gen_server/gen_server.go:68 +0x75
      [goport] 2015/06/09 02:19:55 /opt/couchbase/bin/goxdcr terminated: exit status 2 ns_log000 ns_1@172.23.105.22 02:19:55 - Tue Jun 9, 2015
      Bucket "AbRegNums" loaded on node 'ns_1@172.23.105.22' in 0 seconds. ns_memcached000 ns_1@172.23.105.22 02:18:53 - Tue Jun 9, 2015
      Reset auto-failover count auto_failover000 ns_1@172.23.105.156 02:18:51 - Tue Jun 9, 2015
      Rebalance completed successfully.
      ns_orchestrator001 ns_1@172.23.105.156 02:18:51 - Tue Jun 9, 2015
      Port server goxdcr on node 'babysitter_of_ns_1@127.0.0.1' exited with status 1. Restarting. Messages: github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).supervising(0xc209c861c0, 0x0, 0x0)
      /home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:185 +0x46f
      created by github.com/couchbase/goxdcr/supervisor.(*GenericSupervisor).Start
      /home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/supervisor/generic_supervisor.go:142 +0x230
      [goport] 2015/06/09 02:17:15 /opt/couchbase/bin/goxdcr terminated: exit status 2 ns_log000 ns_1@172.23.105.22 02:17:16 - Tue Jun 9, 2015
      Port server goxdcr on node 'babysitter_of_ns_1@127.0.0.1' exited with status 1. Restarting. Messages: net/http.func·019()
      /usr/local/go/src/net/http/transport.go:520 +0x42
      created by net/http.(*Transport).getConn
      /usr/local/go/src/net/http/transport.go:522 +0x335
      [goport] 2015/06/09 02:16:57 /opt/couchbase/bin/goxdcr terminated: exit status 2 ns_log000 ns_1@172.23.105.158 02:16:57 - Tue Jun 9, 2015
      Port server goxdcr on node 'babysitter_of_ns_1@127.0.0.1' exited with status 1. Restarting. Messages: github.com/couchbase/goxdcr/parts.(*XmemNozzle).receiveResponse(0xc20c814000, 0xc20a3761c0, 0xc20c8141c8)
      /home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/xmem_nozzle.go:1423 +0x2342
      created by github.com/couchbase/goxdcr/parts.(*XmemNozzle).Start
      /home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/parts/xmem_nozzle.go:751 +0x4b4
      [goport] 2015/06/09 02:16:30 /opt/couchbase/bin/goxdcr terminated: exit status 2 ns_log000 ns_1@172.23.105.156 02:16:30 - Tue Jun 9, 2015
      Port server goxdcr on node 'babysitter_of_ns_1@127.0.0.1' exited with status 1. Restarting. Messages: github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).checkpointing(0xc20a201590)
      /home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:635 +0x419
      created by github.com/couchbase/goxdcr/pipeline_svc.(*CheckpointManager).Start
      /home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/checkpoint_manager.go:195 +0x4c5
      [goport] 2015/06/09 02:16:06 /opt/couchbase/bin/goxdcr terminated: exit status 2 ns_log000 ns_1@172.23.105.157 02:16:06 - Tue Jun 9, 2015

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              xiaomei Xiaomei Zhang (Inactive)
              andreibaranouski Andrei Baranouski
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty