Indexer runtime: out of memory

Build: 7.0.0 build 5219

Steps:

• Multi-node cluster with KV, N1ql+index, FTS
• 3 Couchbase buckets (Each bucket with 3 scopes and 2 collections under each scope. Total 18 collections on 3 buckets)
• Creating 50 GSI on each available collection with gsi_index_replica=1

Observation:

1. On bucket "6RQl-30-113000" 150 total GSI indexes are created
2. On bucket  "Eu0JTbzu37-30-110000" scope "bH0NXefmzmkbkYBCuf-30-112000" all 100 GSIs are created
3. While trying to create indexes on "Eu0JTbzu37-30-110000" scope "default", indexer crashed with exit status 2

4. Service 'indexer' exited with status 2. Restarting. Messages:

   /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/common/retry_helper.go:36 +0x8f

   github.com/couchbase/indexing/secondary/indexer.(*kvSender).openMutationStream(0xc0041a6cf0, 0xc0022b0003, 0xc03d3d5da0, 0x14, 0x210dfd0, 0x1, 0xc06b457400, 0x1, 0x1, 0x0, ...)

   /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/kv_sender.go:396 +0x8a4

   created by github.com/couchbase/indexing/secondary/indexer.(*kvSender).handleOpenStream

   /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/kv_sender.go:171 +0x267

   goroutine 618624 [chan receive]:

   github.com/couchbase/plasma.(*deallocCtx).smrWorker(0xc060f43d40)

   /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/plasma/smr.go:323 +0xea

   created by github.com/couchbase/plasma.(*deallocCtx).startSMRWorker

   /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/plasma/smr.go:315 +0xc5