Details
-
Bug
-
Resolution: Fixed
-
Critical
-
Cheshire-Cat
-
Untriaged
-
-
1
-
Yes
Description
Build : 7.0.0-4951
Test : -test tests/2i/cheshirecat/test_idx_clusterops_cheshire_cat_moi.yml -scope tests/2i/cheshirecat/scope_idx_cheshire_cat_moi.yml (MOI)
Scale : 2
Iteration : 1st
In the MOI system test, in the first iteration, an indexer crash is observed on 172.23.107.3
2021-04-15T20:02:27.691-07:00 [Info] MemDBSlice Slice Id 0, IndexInstId 17798637152331761337, PartitionId 1 Removing disk snapshot /data/@2i/bucket3_idx3_uas0KwW5_17798637152331761337_1.index/snapshot.2021-04-15.194226.935. Num snapshots 3.
|
2021-04-15T20:02:27.692-07:00 [Info] MemDBSlice Slice Id 0, Threads 1, IndexInstId 17798637152331761337, PartitionId 1 created ondisk snapshot /data/@2i/bucket3_idx3_uas0KwW5_17798637152331761337_1.index/snapshot.2021-04-15.200226.947. Took 745.573962ms
|
2021-04-15T20:02:27.693-07:00 [Info] memdb.StoreToDisk: Done dir [/data/@2i/bucket3_idx2_CpYIZL_16464859440971911668_1.index/.tmp] disk snapshot - count per shard [[0 0 0 0 0 0 0 0]] total count [0] snap count [48001] size per shard [[0 0 0 0 0 0 0 0]] total size [0] memoryInUse [0]
|
panic: iterator cannot be nil
|
|
goroutine 170523783 [running]:
|
github.com/couchbase/indexing/secondary/memdb.(*MemDB).Visitor.func1(0xc004e21760, 0xc004e21810, 0xc031571bc0, 0x8)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/memdb/memdb.go:1034 +0x516
|
github.com/couchbase/indexing/secondary/memdb.(*MemDB).Visitor(0xc004e21760, 0xc004e21810, 0xc02d5e9bc0, 0x8, 0x1, 0x1b0, 0x0)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/memdb/memdb.go:1056 +0xd1
|
github.com/couchbase/indexing/secondary/memdb.(*MemDB).StoreToDisk(0xc004e21760, 0xc026fdf8c0, 0x3f, 0xc004e21810, 0x1, 0x0, 0x0, 0x0)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/memdb/memdb.go:1327 +0xb9d
|
github.com/couchbase/indexing/secondary/indexer.(*memdbSlice).doPersistSnapshot(0xc00a441b00, 0xc0095322c0)
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/memdb_slice_impl.go:993 +0x435
|
created by github.com/couchbase/indexing/secondary/indexer.(*memdbSlice).OpenSnapshot
|
/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/memdb_slice_impl.go:934 +0x16e
|
Initializing write barrier = 8000
|
2021-04-15T20:02:28.269-07:00 [Info] Indexer started with command line: [/opt/couchbase/bin/indexer -adminPort=9100 -scanPort=9101 -httpPort=9102 -streamInitPort=9103 -streamCatchupPort=9104 -streamMaintPort=9105 --httpsPort=19102 --certFile=/opt/couchbase/var/lib/couchbase/config/memcached-cert.pem --keyFile=/opt/couchbase/var/lib/couchbase/config/memcached-key.pem -vbuckets=1024 -cluster=127.0.0.1:8091 -storageDir=/data/@2i -diagDir=/opt/couchbase/var/lib/couchbase/crash -logDir=/opt/couchbase/var/lib/couchbase/logs -nodeUUID=d5a520b5bd340b4732385f0c87322ca0 -ipv6=false -isEnterprise=true]
|
2021-04-15T20:02:28.275-07:00 [Info] Indexer::NewIndexer Status Warmup
|
This wasnt seen in any of the earlier runs with MOI test, as recent as 7.0.0-4854. Could be an intermittent issue as well.