Details
-
Bug
-
Resolution: Duplicate
-
Critical
-
4.0.0
-
Security Level: Public
-
400-3570 RC1
▪ Machine configuration:
⁃ SSDs
⁃ 24GB Ram
⁃ 250GB /data and /index disks
⁃ 8 Core CPU
⁃ Individual VMs ( non-shared)
▪ Indexer Settings :
⁃ Index Ram Quota on Index Nodes : 18GB,
⁃ 8 Indexer Threads
⁃ indexer.settings.scan_timeout:1400000 No index settings changed at any point on the test.
▪ 8 Node cluster
⁃ 4 Index nodes,
⁃ 2 Query nodes,
⁃ 4 KV nodes
▪ 10 Buckets
▪ 100 warehouses for tppc(9) buckets and 10M items for default.
▪ 36 GSI indexes across 10 buckets ( 18 active, 18 replica)
Index Compaction ON
400-3570 RC1 ▪ Machine configuration: ⁃ SSDs ⁃ 24GB Ram ⁃ 250GB /data and /index disks ⁃ 8 Core CPU ⁃ Individual VMs ( non-shared) ▪ Indexer Settings : ⁃ Index Ram Quota on Index Nodes : 18GB, ⁃ 8 Indexer Threads ⁃ indexer.settings.scan_timeout:1400000 No index settings changed at any point on the test. ▪ 8 Node cluster ⁃ 4 Index nodes, ⁃ 2 Query nodes, ⁃ 4 KV nodes ▪ 10 Buckets ▪ 100 warehouses for tppc(9) buckets and 10M items for default. ▪ 36 GSI indexes across 10 buckets ( 18 active, 18 replica) Index Compaction ON
-
Untriaged
-
Unknown
Description
1. Load data + indexes
2. Seeing indexer crash during the incremental index building.
Added New Snapshot Index: 2298622948065610328 PartitionId: 0 SliceId: 0 Crc64: 5621589856138892193 (SnapshotInfo: seqnos: 274, 274, 133 committed:true) SnapCreateDur 111.890243ms SnapOpenDur 45.811µs
|
fatal error: unexpected signal during runtime execution
|
[signal 0xb code=0x1 addr=0x8 pc=0x7fdfed707f96]
|
|
|
runtime stack:
|
runtime.gothrow(0xf439f0, 0x2a)
|
/usr/local/go/src/runtime/panic.go:503 +0x8e
|
runtime.sigpanic()
|
/usr/local/go/src/runtime/sigpanic_unix.go:14 +0x5e
|
|
|
goroutine 274 [syscall, locked to thread]:
|
runtime.cgocall_errno(0x402f7f, 0xc20d8d4748, 0x0)
|
/usr/local/go/src/runtime/cgocall.go:130 +0xf5 fp=0xc20d8d4728 sp=0xc20d8d4700
|
github.com/couchbase/indexing/secondary/fdb._Cfunc_fdb_estimate_space_used(0x7fdd07823840, 0x0)
|
/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/indexing/secondary/fdb/:106 +0x44 fp=0xc20d8d4748 sp=0xc20d8d4728
|
github.com/couchbase/indexing/secondary/fdb.(*File).EstimateSpaceUsed(0xc212562180, 0x0)
|
/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/indexing/secondary/fdb/file.go:116 +0x178 fp=0xc20d8d47c0 sp=0xc20d8d4748
|
github.com/couchbase/indexing/secondary/indexer.(*fdbSlice).Statistics(0xc20e771080, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
|
/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/forestdb_slice_writer.go:1003 +0x107 fp=0xc20d8d4840 sp=0xc20d8d47c0
|
github.com/couchbase/indexing/secondary/indexer.(*storageMgr).getIndexStorageStats(0xc20eab9500, 0x0, 0x0, 0x0)
|
/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/storage_manager.go:775 +0x343 fp=0xc20d8d4bf8 sp=0xc20d8d4840
|
github.com/couchbase/indexing/secondary/indexer.(*storageMgr).handleStats(0xc20eab9500, 0x7fdd087f2fd0, 0xc211e35780)
|
/home/couchbase/jenkins/workspace/sherlock-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/storage_manager.go:729 +0xe5 fp=0xc20d8d4f60 sp=0xc20d8d4bf8
|
...
|
Attaching logs from the nodes.
Index Nodes: 233, 234, 237, 238