14:16:04 [cen-s723.perf.couchbase.com] run: /opt/couchbase/bin/cbindex -auth=Administrator:password -server cen-s705.perf.couchbase.com:8091 -type create -bucket bucket-1 -scope scope-1 -collection collection-389 -fields \\\`coins\\\`  -using memdb -index i389-6 -with {\\\"defer_build\\\":true} -refresh_settings=true
14:16:06 [cen-s723.perf.couchbase.com] out: fatal error: concurrent map iteration and map write
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 1 [running]:
14:16:06 [cen-s723.perf.couchbase.com] out: runtime.throw(0xd6d9ed, 0x26)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/runtime/panic.go:608 +0x72 fp=0xc0007b4c38 sp=0xc0007b4c08 pc=0x42fdb2
14:16:06 [cen-s723.perf.couchbase.com] out: runtime.mapiternext(0xc0007b4d88)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/runtime/map.go:790 +0x525 fp=0xc0007b4cc0 sp=0xc0007b4c38 pc=0x412865
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/manager/client.(*metadataRepo).listDefnWithValidInstNoLock(0xc00034c730, 0xd90b08, 0xc000190720)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/manager/client/metadata_provider.go:3954 +0x45f fp=0xc0007b4df8 sp=0xc0007b4cc0 pc=0xb0a04f
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/manager/client.(*MetadataProvider).findIndexByName(0xc0001906e0, 0x7ffee377cd4e, 0x6, 0x7ffee377ccf7, 0x8, 0x7ffee377cd07, 0x7, 0x7ffee377cd1b, 0xe, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/manager/client/metadata_provider.go:2799 +0x96 fp=0xc0007b4e98 sp=0xc0007b4df8 pc=0xb03a16
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/manager/client.(*MetadataProvider).CreateIndexWithPlan(0xc0001906e0, 0x7ffee377cd4e, 0x6, 0x7ffee377ccf7, 0x8, 0x7ffee377cd07, 0x7, 0x7ffee377cd1b, 0xe, 0x7ffee377cd41, ...)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/manager/client/metadata_provider.go:385 +0xab fp=0xc0007b4fe0 sp=0xc0007b4e98 pc=0xaf178b
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/queryport/client.(*metadataClient).CreateIndex(0xc000190630, 0x7ffee377cd4e, 0x6, 0x7ffee377ccf7, 0x8, 0x7ffee377cd07, 0x7, 0x7ffee377cd1b, 0xe, 0x7ffee377cd41, ...)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/meta_client.go:327 +0x242 fp=0xc0007b5128 sp=0xc0007b4fe0 pc=0xb39052
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/queryport/client.(*GsiClient).CreateIndex4(0xc0001e6100, 0x7ffee377cd4e, 0x6, 0x7ffee377ccf7, 0x8, 0x7ffee377cd07, 0x7, 0x7ffee377cd1b, 0xe, 0x7ffee377cd41, ...)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/client.go:566 +0x413 fp=0xc0007b5508 sp=0xc0007b5128 pc=0xb2b063
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/querycmd.HandleCommand(0xc0001e6100, 0xc000486000, 0xc0005ebc00, 0xe3c4c0, 0xc00000e018, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/querycmd/docmd.go:276 +0x35a7 fp=0xc0007b5e60 sp=0xc0007b5508 pc=0xb6ab07
14:16:06 [cen-s723.perf.couchbase.com] out: main.main()
14:16:06 [cen-s723.perf.couchbase.com] out: 	goproj/src/github.com/couchbase/indexing/secondary/cmd/cbindex/main.go:92 +0x262 fp=0xc0007b5f98 sp=0xc0007b5e60 pc=0xb6e1e2
14:16:06 [cen-s723.perf.couchbase.com] out: runtime.main()
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/runtime/proc.go:201 +0x207 fp=0xc0007b5fe0 sp=0xc0007b5f98 pc=0x431727
14:16:06 [cen-s723.perf.couchbase.com] out: runtime.goexit()
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/runtime/asm_amd64.s:1333 +0x1 fp=0xc0007b5fe8 sp=0xc0007b5fe0 pc=0x45fd81
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 6 [syscall]:
14:16:06 [cen-s723.perf.couchbase.com] out: os/signal.signal_recv(0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/runtime/sigqueue.go:139 +0x9c
14:16:06 [cen-s723.perf.couchbase.com] out: os/signal.loop()
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/os/signal/signal_unix.go:23 +0x22
14:16:06 [cen-s723.perf.couchbase.com] out: created by os/signal.init.0
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/os/signal/signal_unix.go:29 +0x41
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 9 [select]:
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/cbauth/cbauthimpl.(*tlsNotifier).loop(0xc00000cc00)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/godeps/src/github.com/couchbase/cbauth/cbauthimpl/impl.go:352 +0xa3
14:16:06 [cen-s723.perf.couchbase.com] out: created by github.com/couchbase/cbauth/cbauthimpl.NewSVCForTest
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/godeps/src/github.com/couchbase/cbauth/cbauthimpl/impl.go:505 +0x26b
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 10 [select]:
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/cbauth/cbauthimpl.(*cfgChangeNotifier).loop(0xc00000cc20)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/godeps/src/github.com/couchbase/cbauth/cbauthimpl/impl.go:272 +0xb7
14:16:06 [cen-s723.perf.couchbase.com] out: created by github.com/couchbase/cbauth/cbauthimpl.NewSVCForTest
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/godeps/src/github.com/couchbase/cbauth/cbauthimpl/impl.go:506 +0x294
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 11 [IO wait]:
14:16:06 [cen-s723.perf.couchbase.com] out: internal/poll.runtime_pollWait(0x7efddced9f00, 0x72, 0xc0001977c8)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/runtime/netpoll.go:173 +0x66
14:16:06 [cen-s723.perf.couchbase.com] out: internal/poll.(*pollDesc).wait(0xc000538118, 0x72, 0xffffffffffffff00, 0xe3d120, 0x1827148)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/internal/poll/fd_poll_runtime.go:85 +0x9a
14:16:06 [cen-s723.perf.couchbase.com] out: internal/poll.(*pollDesc).waitRead(0xc000538118, 0xc0003a5000, 0x1000, 0x1000)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d
14:16:06 [cen-s723.perf.couchbase.com] out: internal/poll.(*FD).Read(0xc000538100, 0xc0003a5000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/internal/poll/fd_unix.go:169 +0x179
14:16:06 [cen-s723.perf.couchbase.com] out: net.(*netFD).Read(0xc000538100, 0xc0003a5000, 0x1000, 0x1000, 0x6f4a1b, 0xc000160de8, 0x5d)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/fd_unix.go:202 +0x4f
14:16:06 [cen-s723.perf.couchbase.com] out: net.(*conn).Read(0xc0001e2040, 0xc0003a5000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/net.go:177 +0x68
14:16:06 [cen-s723.perf.couchbase.com] out: bufio.(*Reader).Read(0xc0001dc480, 0xc00050e001, 0xdff, 0xdff, 0x191, 0x1, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/bufio/bufio.go:216 +0x22f
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/cbauth/revrpc.(*minirwc).Read(0xc0005143c0, 0xc00050e001, 0xdff, 0xdff, 0x416cc5, 0x203000, 0x203000)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:101 +0x4d
14:16:06 [cen-s723.perf.couchbase.com] out: encoding/json.(*Decoder).refill(0xc000370420, 0xc0001e510a, 0x9)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/encoding/json/stream.go:159 +0x102
14:16:06 [cen-s723.perf.couchbase.com] out: encoding/json.(*Decoder).readValue(0xc000370420, 0x0, 0x0, 0xbe50a0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/encoding/json/stream.go:134 +0x222
14:16:06 [cen-s723.perf.couchbase.com] out: encoding/json.(*Decoder).Decode(0xc000370420, 0xc27e60, 0xc000316bc0, 0x7d766f, 0xc000322ed8)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/encoding/json/stream.go:63 +0x78
14:16:06 [cen-s723.perf.couchbase.com] out: net/rpc/jsonrpc.(*serverCodec).ReadRequestHeader(0xc000316ba0, 0xc0001e51a0, 0xbc2480, 0xc0004c78d0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/rpc/jsonrpc/server.go:66 +0x76
14:16:06 [cen-s723.perf.couchbase.com] out: net/rpc.(*Server).readRequestHeader(0xc000322eb0, 0xe46480, 0xc000316ba0, 0xc0004c78d0, 0x16, 0xc0001e4cc0, 0x10000c000116c01, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/rpc/server.go:590 +0x67
14:16:06 [cen-s723.perf.couchbase.com] out: net/rpc.(*Server).readRequest(0xc000322eb0, 0xe46480, 0xc000316ba0, 0xc000322eb0, 0xc0004c7440, 0xc0004c7450, 0xc000466f80, 0xc0001e4cc0, 0xbc51c0, 0xc000168fc0, ...)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/rpc/server.go:550 +0x5a
14:16:06 [cen-s723.perf.couchbase.com] out: net/rpc.(*Server).ServeCodec(0xc000322eb0, 0xe46480, 0xc000316ba0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/rpc/server.go:465 +0x97
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/cbauth/revrpc.(*Service).Run(0xc0002fc7e0, 0xc000116f80, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:147 +0x509
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/cbauth/revrpc.BabysitService(0xc000197f80, 0xc0002fc7e0, 0xe3cac0, 0xc0001e40e0, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:243 +0x5c
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/cbauth.runRPCForSvc(0xc0002fc7e0, 0xc0001615e0, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/godeps/src/github.com/couchbase/cbauth/default.go:57 +0xbf
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/cbauth.startDefault.func1(0xc0002fc7e0, 0xc0001615e0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/godeps/src/github.com/couchbase/cbauth/default.go:66 +0x35
14:16:06 [cen-s723.perf.couchbase.com] out: created by github.com/couchbase/cbauth.startDefault
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/godeps/src/github.com/couchbase/cbauth/default.go:65 +0xcf
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 131 [sleep]:
14:16:06 [cen-s723.perf.couchbase.com] out: time.Sleep(0x2540be400)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/runtime/time.go:105 +0x14f
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/common.pollForDeletedBuckets()
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/common/dcp_seqno.go:949 +0x41
14:16:06 [cen-s723.perf.couchbase.com] out: created by github.com/couchbase/indexing/secondary/common.init.0
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/common/dcp_seqno.go:53 +0x84
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 126 [select]:
14:16:06 [cen-s723.perf.couchbase.com] out: net/http.(*persistConn).writeLoop(0xc000792120)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/transport.go:1885 +0x113
14:16:06 [cen-s723.perf.couchbase.com] out: created by net/http.(*Transport).dialConn
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/transport.go:1339 +0x966
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 129 [runnable]:
14:16:06 [cen-s723.perf.couchbase.com] out: net/http.(*persistConn).roundTrip(0xc000792120, 0xc000c44090, 0x0, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/transport.go:2101 +0x56a
14:16:06 [cen-s723.perf.couchbase.com] out: net/http.(*Transport).roundTrip(0x1817960, 0xc0007d0100, 0xc000c44060, 0xc000d54450, 0xc000d54444)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/transport.go:465 +0x9b1
14:16:06 [cen-s723.perf.couchbase.com] out: net/http.(*Transport).RoundTrip(0x1817960, 0xc0007d0100, 0x1817960, 0xbfc9d243ac7a3509, 0x1c3b5966b4)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/roundtrip.go:17 +0x35
14:16:06 [cen-s723.perf.couchbase.com] out: net/http.send(0xc0007d0000, 0xe3c1a0, 0x1817960, 0xbfc9d243ac7a3509, 0x1c3b5966b4, 0x18740e0, 0xc000320558, 0xbfc9d243ac7a3509, 0xc000195540, 0x1)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/client.go:250 +0x14b
14:16:06 [cen-s723.perf.couchbase.com] out: net/http.(*Client).send(0xc000155ec0, 0xc0007d0000, 0xbfc9d243ac7a3509, 0x1c3b5966b4, 0x18740e0, 0xc000320558, 0x0, 0x1, 0x1b)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/client.go:174 +0xfa
14:16:06 [cen-s723.perf.couchbase.com] out: net/http.(*Client).do(0xc000155ec0, 0xc0007d0000, 0x0, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/client.go:641 +0x2a8
14:16:06 [cen-s723.perf.couchbase.com] out: net/http.(*Client).Do(0xc000155ec0, 0xc0007d0000, 0xc000155ec0, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/client.go:509 +0x35
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/security.GetWithAuth(0xc0000362d0, 0x2d, 0xc0001959b8, 0x0, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/security/tls.go:539 +0x1c1
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/dcp.queryRestAPI(0xc0001e6400, 0xd53079, 0x6, 0xe3cae0, 0xc0010a3480, 0xbc6cc0, 0xc0001e6318, 0xc0007a09b8, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/dcp/pools.go:330 +0x17c
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/dcp.(*Client).parseURLResponse(0xc0001e6300, 0xd53079, 0x6, 0xbc6cc0, 0xc0001e6318, 0xe, 0xc000102098)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/dcp/pools.go:478 +0xb1
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/dcp.ConnectWithAuth(0xc000102080, 0x59, 0xe3cae0, 0xc0010a3480, 0x7efddcf356c0, 0x0, 0x40f86f, 0xc0007a0a98, 0x44aedf, 0x40, ...)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/dcp/pools.go:540 +0xd2
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/dcp.Connect(0xc000102080, 0x59, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/dcp/pools.go:546 +0xde
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/common.NewServicesChangeNotifier(0xc000102080, 0x59, 0xd54a2a, 0x7, 0x0, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/common/services_notifier.go:244 +0x376
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/queryport/client.(*metadataClient).watchClusterChanges(0xc000190630)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/meta_client.go:1993 +0x132
14:16:06 [cen-s723.perf.couchbase.com] out: created by github.com/couchbase/indexing/secondary/queryport/client.newMetaBridgeClient
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/meta_client.go:116 +0x4e1
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 123 [select]:
14:16:06 [cen-s723.perf.couchbase.com] out: net/http.(*persistConn).readLoop(0xc00041a120)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/transport.go:1761 +0x6b9
14:16:06 [cen-s723.perf.couchbase.com] out: created by net/http.(*Transport).dialConn
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/transport.go:1338 +0x941
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 124 [select]:
14:16:06 [cen-s723.perf.couchbase.com] out: net/http.(*persistConn).writeLoop(0xc00041a120)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/transport.go:1885 +0x113
14:16:06 [cen-s723.perf.couchbase.com] out: created by net/http.(*Transport).dialConn
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/transport.go:1339 +0x966
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 49 [select]:
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/queryport/client.(*schedTokenMonitor).updater(0xc00034c6e0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/meta_client.go:2264 +0xed
14:16:06 [cen-s723.perf.couchbase.com] out: created by github.com/couchbase/indexing/secondary/queryport/client.newSchedTokenMonitor
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/meta_client.go:2102 +0x260
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 146 [select]:
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/cbauth/metakv.doRunObserveChildren(0xc0005ec130, 0xd5db0e, 0x13, 0xc00011f760, 0xc0003d6180, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:305 +0x30c
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(0xc0005ec130, 0xd5db0e, 0x13, 0xc00011f760, 0xc0003d6180, 0x115, 0x116)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:263 +0x58
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/cbauth/metakv.RunObserveChildren(0xd5db0e, 0x13, 0xc00011f150, 0xc0003d6180, 0xc00011f100, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:392 +0x85
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/queryport/client.NewClientSettings.func1.1(0x0, 0x0, 0x0, 0x10, 0xc57540)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/settings.go:73 +0x118
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/common.(*RetryHelper).Run(0xc000193fa0, 0xc00011f100, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/common/retry_helper.go:36 +0x8f
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/queryport/client.NewClientSettings.func1(0xc000330240)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/settings.go:77 +0xad
14:16:06 [cen-s723.perf.couchbase.com] out: created by github.com/couchbase/indexing/secondary/queryport/client.NewClientSettings
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/settings.go:68 +0xd9
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 210 [chan receive]:
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/queryport/client.(*metadataClient).logstats(0xc000190630)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/meta_client.go:1363 +0x78
14:16:06 [cen-s723.perf.couchbase.com] out: created by github.com/couchbase/indexing/secondary/queryport/client.newMetaBridgeClient
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/meta_client.go:117 +0x503
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 211 [select]:
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/queryport/client.(*GsiClient).listenMetaChange(0xc0001e6100, 0xc0007ac000)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/client.go:1786 +0xb7
14:16:06 [cen-s723.perf.couchbase.com] out: created by github.com/couchbase/indexing/secondary/queryport/client.makeWithMetaProvider
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/client.go:1779 +0x21e
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 125 [select]:
14:16:06 [cen-s723.perf.couchbase.com] out: net/http.(*persistConn).readLoop(0xc000792120)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/transport.go:1761 +0x6b9
14:16:06 [cen-s723.perf.couchbase.com] out: created by net/http.(*Transport).dialConn
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/transport.go:1338 +0x941
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 46 [IO wait]:
14:16:06 [cen-s723.perf.couchbase.com] out: internal/poll.runtime_pollWait(0x7efddced9d60, 0x72, 0xc00011b9a0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/runtime/netpoll.go:173 +0x66
14:16:06 [cen-s723.perf.couchbase.com] out: internal/poll.(*pollDesc).wait(0xc000467198, 0x72, 0xffffffffffffff00, 0xe3d120, 0x1827148)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/internal/poll/fd_poll_runtime.go:85 +0x9a
14:16:06 [cen-s723.perf.couchbase.com] out: internal/poll.(*pollDesc).waitRead(0xc000467198, 0xc00073a000, 0x1000, 0x1000)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d
14:16:06 [cen-s723.perf.couchbase.com] out: internal/poll.(*FD).Read(0xc000467180, 0xc00073a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/internal/poll/fd_unix.go:169 +0x179
14:16:06 [cen-s723.perf.couchbase.com] out: net.(*netFD).Read(0xc000467180, 0xc00073a000, 0x1000, 0x1000, 0x5, 0x1071, 0x1072)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/fd_unix.go:202 +0x4f
14:16:06 [cen-s723.perf.couchbase.com] out: net.(*conn).Read(0xc0001e2060, 0xc00073a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/net.go:177 +0x68
14:16:06 [cen-s723.perf.couchbase.com] out: net/http.(*persistConn).Read(0xc00041a120, 0xc00073a000, 0x1000, 0x1000, 0x181, 0x0, 0xc965e0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/transport.go:1497 +0x75
14:16:06 [cen-s723.perf.couchbase.com] out: bufio.(*Reader).fill(0xc0001dca20)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/bufio/bufio.go:100 +0x10f
14:16:06 [cen-s723.perf.couchbase.com] out: bufio.(*Reader).ReadSlice(0xc0001dca20, 0xa, 0x2, 0x9, 0xd9f, 0xb, 0xda0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/bufio/bufio.go:341 +0x36
14:16:06 [cen-s723.perf.couchbase.com] out: net/http/internal.readChunkLine(0xc0001dca20, 0x1, 0x1061, 0x0, 0x0, 0x19)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/internal/chunked.go:122 +0x34
14:16:06 [cen-s723.perf.couchbase.com] out: net/http/internal.(*chunkedReader).beginChunk(0xc00030e990)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/internal/chunked.go:48 +0x32
14:16:06 [cen-s723.perf.couchbase.com] out: net/http/internal.(*chunkedReader).Read(0xc00030e990, 0xc00050c000, 0x1e00, 0x1e00, 0xc0005c80f0, 0xc00011bd30, 0x42eb12)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/internal/chunked.go:93 +0x125
14:16:06 [cen-s723.perf.couchbase.com] out: net/http.(*body).readLocked(0xc0007aa740, 0xc00050c000, 0x1e00, 0x1e00, 0x0, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/transfer.go:803 +0x6c
14:16:06 [cen-s723.perf.couchbase.com] out: net/http.(*body).Read(0xc0007aa740, 0xc00050c000, 0x1e00, 0x1e00, 0x0, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/transfer.go:795 +0xde
14:16:06 [cen-s723.perf.couchbase.com] out: net/http.(*bodyEOFSignal).Read(0xc0007aa780, 0xc00050c000, 0x1e00, 0x1e00, 0x0, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/transport.go:2255 +0xd6
14:16:06 [cen-s723.perf.couchbase.com] out: encoding/json.(*Decoder).refill(0xc0003709a0, 0xc000134300, 0xc00011bf70)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/encoding/json/stream.go:159 +0x102
14:16:06 [cen-s723.perf.couchbase.com] out: encoding/json.(*Decoder).readValue(0xc0003709a0, 0x0, 0x0, 0x1)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/encoding/json/stream.go:134 +0x222
14:16:06 [cen-s723.perf.couchbase.com] out: encoding/json.(*Decoder).Decode(0xc0003709a0, 0xbc55c0, 0xc00034c550, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/encoding/json/stream.go:63 +0x78
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/cbauth/metakv.doRunObserveChildren.func1(0xc00040a090, 0xc0001dce40, 0xc0004686c0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:288 +0x137
14:16:06 [cen-s723.perf.couchbase.com] out: created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:284 +0x1e8
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 170 [chan receive]:
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/gometa/common.(*PeerPipe).doSend(0xc0007aa700)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/gometa/common/peerPipe.go:165 +0x8c
14:16:06 [cen-s723.perf.couchbase.com] out: created by github.com/couchbase/gometa/common.NewPeerPipe
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/gometa/common/peerPipe.go:63 +0x118
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 76 [select]:
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/cbauth/metakv.doRunObserveChildren(0xc0005ec130, 0xd63f39, 0x1b, 0xc000332d80, 0xc000469080, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:305 +0x30c
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(0xc0005ec130, 0xd63f39, 0x1b, 0xc000332d80, 0xc000469080, 0x40f86f, 0xc0001e56c0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:263 +0x58
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/cbauth/metakv.RunObserveChildren(0xd63f39, 0x1b, 0xc000332d70, 0xc000469080, 0x18, 0xc0001e56c0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:392 +0x85
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/manager/common.(*CommandListener).ListenTokens.func2.1(0x0, 0x0, 0x0, 0x20, 0xc724a0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/manager/common/token.go:1298 +0xfd
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/common.(*RetryHelper).Run(0xc000911f98, 0xc0001e56c0, 0xc000469020)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/common/retry_helper.go:36 +0x8f
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/manager/common.(*CommandListener).ListenTokens.func2(0xc0001dd2c0, 0xc000332d70)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/manager/common/token.go:1303 +0xe4
14:16:06 [cen-s723.perf.couchbase.com] out: created by github.com/couchbase/indexing/secondary/manager/common.(*CommandListener).ListenTokens
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/manager/common/token.go:1291 +0x74
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 179 [IO wait]:
14:16:06 [cen-s723.perf.couchbase.com] out: internal/poll.runtime_pollWait(0x7efddced9e30, 0x72, 0xc00079b9a0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/runtime/netpoll.go:173 +0x66
14:16:06 [cen-s723.perf.couchbase.com] out: internal/poll.(*pollDesc).wait(0xc0001e7698, 0x72, 0xffffffffffffff00, 0xe3d120, 0x1827148)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/internal/poll/fd_poll_runtime.go:85 +0x9a
14:16:06 [cen-s723.perf.couchbase.com] out: internal/poll.(*pollDesc).waitRead(0xc0001e7698, 0xc000916000, 0x1000, 0x1000)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d
14:16:06 [cen-s723.perf.couchbase.com] out: internal/poll.(*FD).Read(0xc0001e7680, 0xc000916000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/internal/poll/fd_unix.go:169 +0x179
14:16:06 [cen-s723.perf.couchbase.com] out: net.(*netFD).Read(0xc0001e7680, 0xc000916000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/fd_unix.go:202 +0x4f
14:16:06 [cen-s723.perf.couchbase.com] out: net.(*conn).Read(0xc00074a0a8, 0xc000916000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/net.go:177 +0x68
14:16:06 [cen-s723.perf.couchbase.com] out: net/http.(*persistConn).Read(0xc00041a480, 0xc000916000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/transport.go:1497 +0x75
14:16:06 [cen-s723.perf.couchbase.com] out: bufio.(*Reader).fill(0xc0007387e0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/bufio/bufio.go:100 +0x10f
14:16:06 [cen-s723.perf.couchbase.com] out: bufio.(*Reader).ReadSlice(0xc0007387e0, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/bufio/bufio.go:341 +0x36
14:16:06 [cen-s723.perf.couchbase.com] out: net/http/internal.readChunkLine(0xc0007387e0, 0x0, 0x0, 0x0, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/internal/chunked.go:122 +0x34
14:16:06 [cen-s723.perf.couchbase.com] out: net/http/internal.(*chunkedReader).beginChunk(0xc0001df650)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/internal/chunked.go:48 +0x32
14:16:06 [cen-s723.perf.couchbase.com] out: net/http/internal.(*chunkedReader).Read(0xc0001df650, 0xc000742a00, 0x200, 0x200, 0x0, 0xc0004ffd30, 0x42eb12)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/internal/chunked.go:93 +0x125
14:16:06 [cen-s723.perf.couchbase.com] out: net/http.(*body).readLocked(0xc000310500, 0xc000742a00, 0x200, 0x200, 0x200, 0xc000742a00, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/transfer.go:803 +0x6c
14:16:06 [cen-s723.perf.couchbase.com] out: net/http.(*body).Read(0xc000310500, 0xc000742a00, 0x200, 0x200, 0x0, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/transfer.go:795 +0xde
14:16:06 [cen-s723.perf.couchbase.com] out: net/http.(*bodyEOFSignal).Read(0xc000310540, 0xc000742a00, 0x200, 0x200, 0x0, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/transport.go:2255 +0xd6
14:16:06 [cen-s723.perf.couchbase.com] out: encoding/json.(*Decoder).refill(0xc000770dc0, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/encoding/json/stream.go:159 +0x102
14:16:06 [cen-s723.perf.couchbase.com] out: encoding/json.(*Decoder).readValue(0xc000770dc0, 0x0, 0x0, 0xc965e0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/encoding/json/stream.go:134 +0x222
14:16:06 [cen-s723.perf.couchbase.com] out: encoding/json.(*Decoder).Decode(0xc000770dc0, 0xbc55c0, 0xc00074e2d0, 0xe3c1e0, 0xc000310540)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/encoding/json/stream.go:63 +0x78
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/cbauth/metakv.doRunObserveChildren.func1(0xc00073c240, 0xc0007388a0, 0xc00073e9c0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:288 +0x137
14:16:06 [cen-s723.perf.couchbase.com] out: created by github.com/couchbase/cbauth/metakv.doRunObserveChildren
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/godeps/src/github.com/couchbase/cbauth/metakv/metakv.go:284 +0x1e8
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 81 [select]:
14:16:06 [cen-s723.perf.couchbase.com] out: net/http.(*persistConn).readLoop(0xc00041a480)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/transport.go:1761 +0x6b9
14:16:06 [cen-s723.perf.couchbase.com] out: created by net/http.(*Transport).dialConn
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/transport.go:1338 +0x941
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 166 [select]:
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/gometa/protocol.syncWithPeer(0xc0007aa700, 0xe4bd60, 0xc00073c3f0, 0xe4b8c0, 0x1895ea0, 0xc000132230, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/gometa/protocol/watcherServer.go:208 +0x2ff
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/gometa/protocol.runOnce(0xc000908900, 0x20, 0xe3fd40, 0xc00073c3f0, 0xe4bd60, 0xc00073c3f0, 0xe4b8c0, 0x1895ea0, 0xc000132230, 0xc000468c60, ...)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/gometa/protocol/watcherServer.go:172 +0x309
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/gometa/protocol.RunWatcherServerWithRequest(0xc000908900, 0x20, 0xe3fd40, 0xc00073c3f0, 0xe4bd60, 0xc00073c3f0, 0xe4b8c0, 0x1895ea0, 0xc000132230, 0xc000468c60, ...)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/gometa/protocol/watcherServer.go:49 +0x166
14:16:06 [cen-s723.perf.couchbase.com] out: created by github.com/couchbase/indexing/secondary/manager/client.(*MetadataProvider).startWatcher
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/manager/client/metadata_provider.go:3473 +0x2a1
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 178 [select]:
14:16:06 [cen-s723.perf.couchbase.com] out: net/http.(*persistConn).writeLoop(0xc00041a480)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/transport.go:1885 +0x113
14:16:06 [cen-s723.perf.couchbase.com] out: created by net/http.(*Transport).dialConn
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/transport.go:1339 +0x966
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 142 [select]:
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/manager/client.(*watcher).waitForReady(0xc00073c3f0, 0xc000468c60, 0x0, 0xc000132000, 0xc000730000)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/manager/client/metadata_provider.go:4909 +0x28c
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/manager/client.(*MetadataProvider).retryHelper(0xc0001906e0, 0xc00073c3f0, 0xc000468c60, 0xc000908900, 0x20, 0xc0010d41e0, 0x2d, 0xc000132000, 0xc0003327d0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/manager/client/metadata_provider.go:3407 +0x20a
14:16:06 [cen-s723.perf.couchbase.com] out: created by github.com/couchbase/indexing/secondary/manager/client.(*MetadataProvider).WatchMetadata
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/manager/client/metadata_provider.go:309 +0x52f
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 171 [IO wait]:
14:16:06 [cen-s723.perf.couchbase.com] out: internal/poll.runtime_pollWait(0x7efddced9af0, 0x72, 0xc000117ce0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/runtime/netpoll.go:173 +0x66
14:16:06 [cen-s723.perf.couchbase.com] out: internal/poll.(*pollDesc).wait(0xc0001e6918, 0x72, 0xffffffffffffff00, 0xe3d120, 0x1827148)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/internal/poll/fd_poll_runtime.go:85 +0x9a
14:16:06 [cen-s723.perf.couchbase.com] out: internal/poll.(*pollDesc).waitRead(0xc0001e6918, 0xc0010ba700, 0x8, 0x8)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d
14:16:06 [cen-s723.perf.couchbase.com] out: internal/poll.(*FD).Read(0xc0001e6900, 0xc0010ba7e8, 0x8, 0x8, 0x0, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/internal/poll/fd_unix.go:169 +0x179
14:16:06 [cen-s723.perf.couchbase.com] out: net.(*netFD).Read(0xc0001e6900, 0xc0010ba7e8, 0x8, 0x8, 0xc000117df0, 0x4469c7, 0x8)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/fd_unix.go:202 +0x4f
14:16:06 [cen-s723.perf.couchbase.com] out: net.(*conn).Read(0xc000320638, 0xc0010ba7e8, 0x8, 0x8, 0x0, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/net.go:177 +0x68
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/gometa/common.(*PeerPipe).readBytes(0xc0007aa700, 0x8, 0x0, 0x0, 0x0, 0x1, 0x0, 0x40, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/gometa/common/peerPipe.go:284 +0xc7
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/gometa/common.(*PeerPipe).doReceive(0xc0007aa700)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/gometa/common/peerPipe.go:213 +0xce
14:16:06 [cen-s723.perf.couchbase.com] out: created by github.com/couchbase/gometa/common.NewPeerPipe
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/gometa/common/peerPipe.go:64 +0x13a
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 172 [select]:
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/gometa/protocol.(*FollowerSyncProxy).Start(0xc00034c5f0, 0xc000332800)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/gometa/protocol/discovery.go:898 +0x18f
14:16:06 [cen-s723.perf.couchbase.com] out: created by github.com/couchbase/gometa/protocol.syncWithPeer
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/gometa/protocol/watcherServer.go:204 +0x248
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 173 [runnable]:
14:16:06 [cen-s723.perf.couchbase.com] out: encoding/json.(*decodeState).scanWhile(0xc0010cf400, 0x9)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/encoding/json/decode.go:347 +0xf6
14:16:06 [cen-s723.perf.couchbase.com] out: encoding/json.(*decodeState).object(0xc0010cf400, 0xc926e0, 0xc0009c34a0, 0x16, 0xc0010cf428, 0x7b)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/encoding/json/decode.go:761 +0x6d8
14:16:06 [cen-s723.perf.couchbase.com] out: encoding/json.(*decodeState).value(0xc0010cf400, 0xc926e0, 0xc0009c34a0, 0x16, 0xc0007d9b30, 0x70648d)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/encoding/json/decode.go:383 +0x6d
14:16:06 [cen-s723.perf.couchbase.com] out: encoding/json.(*decodeState).unmarshal(0xc0010cf400, 0xc926e0, 0xc0009c34a0, 0xc0010cf428, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/encoding/json/decode.go:180 +0x204
14:16:06 [cen-s723.perf.couchbase.com] out: encoding/json.Unmarshal(0xc000d6ad80, 0x164, 0x180, 0xc926e0, 0xc0009c34a0, 0x25, 0xffffffffffffffff)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/encoding/json/decode.go:107 +0x122
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/common.UnmarshallIndexDefn(0xc000d6ad80, 0x164, 0x180, 0x47abf0288bb00146, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/common/index.go:584 +0x6e
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/manager/client.(*metadataRepo).unmarshallAndAddDefn(0xc00034c730, 0xc000d6ad80, 0x164, 0x180, 0x0, 0x1)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/manager/client/metadata_provider.go:4447 +0x43
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/manager/client.(*watcher).processChange(0xc00073c3f0, 0x930, 0x2, 0xc000d3acc0, 0x25, 0xc000d6ad80, 0x164, 0x180, 0x0, 0x0, ...)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/manager/client/metadata_provider.go:5769 +0x2d4
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/manager/client.(*watcher).LogAndCommit(0xc00073c3f0, 0x930, 0x2, 0xc000d3acc0, 0x25, 0xc000d6ad80, 0x164, 0x180, 0xd55701, 0x0, ...)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/manager/client/metadata_provider.go:5743 +0xd0
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/gometa/protocol.(*FollowerSyncProxy).syncReceive(0xc00034c5f0, 0x0, 0x0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/gometa/protocol/discovery.go:1202 +0x453
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/gometa/protocol.(*FollowerSyncProxy).execute(0xc00034c5f0, 0xc0001324d0)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/gometa/protocol/discovery.go:1011 +0x1c5
14:16:06 [cen-s723.perf.couchbase.com] out: created by github.com/couchbase/gometa/protocol.(*FollowerSyncProxy).Start
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/gometa/protocol/discovery.go:896 +0xd2
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 212 [select]:
14:16:06 [cen-s723.perf.couchbase.com] out: github.com/couchbase/indexing/secondary/queryport/client.(*GsiClient).logstats(0xc0001e6100, 0xc0007ac000)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/client.go:1805 +0x250
14:16:06 [cen-s723.perf.couchbase.com] out: created by github.com/couchbase/indexing/secondary/queryport/client.makeWithMetaProvider
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/queryport/client/client.go:1780 +0x249
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 [cen-s723.perf.couchbase.com] out: goroutine 174 [select]:
14:16:06 [cen-s723.perf.couchbase.com] out: net/http.setRequestCancel.func3(0x0, 0xc000c44060, 0xc0010820f0, 0xc000d54450, 0xc00073e060)
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/client.go:321 +0xcf
14:16:06 [cen-s723.perf.couchbase.com] out: created by net/http.setRequestCancel
14:16:06 [cen-s723.perf.couchbase.com] out: 	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.11.4/go/src/net/http/client.go:320 +0x24e
14:16:06 [cen-s723.perf.couchbase.com] out: 
14:16:06 
14:16:06 Fatal error: run() received nonzero return code 2 while executing!
14:16:06 
14:16:06 Requested: /opt/couchbase/bin/cbindex -auth=Administrator:password -server cen-s705.perf.couchbase.com:8091 -type create -bucket bucket-1 -scope scope-1 -collection collection-389 -fields \\\`coins\\\`  -using memdb -index i389-6 -with {\\\"defer_build\\\":true} -refresh_settings=true
14:16:06 Executed: /bin/bash -l -c -o pipefail "/opt/couchbase/bin/cbindex -auth=Administrator:password -server cen-s705.perf.couchbase.com:8091 -type create -bucket bucket-1 -scope scope-1 -collection collection-389 -fields \\\`coins\\\`  -using memdb -index i389-6 -with {\\\"defer_build\\\":true} -refresh_settings=true"