Uploaded image for project: 'Couchbase Gateway'
  1. Couchbase Gateway
  2. CBG-620

Concurrent attempts to create indexes trigger SG restart

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Blocker
    • 2.7.0
    • None
    • SyncGateway
    • Security Level: Public
    • None
    • CBG Sprint 37
    • 2

    Description

      When multiple Sync Gateway nodes concurrently attempt to initialize indexes, the Sync Gateway node can fail with an error like:

      2019-12-06T15:11:05.136-08:00 [WRN] Error when querying index using statement: [SELECT meta(`bucket-1`).xattrs._sync.access.foo as val FROM `bucket-1` WHERE ANY op in OBJECT_PAIRS(meta(`bucket-1`).xattrs._sync.access) SATISFIES op.name = 'foo' end LIMIT 1] parameters: [<nil>] error:[5000] Index sg_access_x1 will retry building in the background for reason: Build Already In Progress. Bucket bucket-1.
       - cause: Index sg_access_x1 will retry building in the background for reason: Build Already In Progress. Bucket bucket-1.
       -- base.(*CouchbaseBucketGoCB).Query() at bucket_n1ql.go:76
      2019-12-06T15:11:05.136-08:00 [WRN] Query error for statement [SELECT meta(`$_bucket`).xattrs._sync.access.foo as val FROM `$_bucket` WHERE ANY op in OBJECT_PAIRS(meta(`$_bucket`).xattrs._sync.access) SATISFIES op.name = 'foo' end LIMIT 1], err:[5000] Index sg_access_x1 will retry building in the background for reason: Build Already In Progress. Bucket bucket-1.
       - cause: Index sg_access_x1 will retry building in the background for reason: Build Already In Progress. Bucket bucket-1.
       -- db.waitForIndexes.func1() at indexes.go:335
      2019-12-06T15:11:05.145-08:00 [ERR] Error opening database db: [5000] Index sg_access_x1 will retry building in the background for reason: Build Already In Progress. Bucket bucket-1.
       - cause: Index sg_access_x1 will retry building in the background for reason: Build Already In Progress. Bucket bucket-1.
       
      github.com/couchbase/sync_gateway/base.(*CouchbaseBucketGoCB).Query
              /home/couchbase/jenkins/workspace/sgw-toy-build/2.7.0/enterprise/godeps/src/github.com/couchbase/sync_gateway/base/bucket_n1ql.go:77
      github.com/couchbase/sync_gateway/db.waitForIndex
              /home/couchbase/jenkins/workspace/sgw-toy-build/2.7.0/enterprise/godeps/src/github.com/couchbase/sync_gateway/db/indexes.go:358
      github.com/couchbase/sync_gateway/db.waitForIndexes.func1
              /home/couchbase/jenkins/workspace/sgw-toy-build/2.7.0/enterprise/godeps/src/github.com/couchbase/sync_gateway/db/indexes.go:333
      runtime.goexit
              /usr/local/go/1.13.4/go/src/runtime/asm_amd64.s:1357 -- rest.RunServer() at config.go:1028
      

      We should detect and retry/wait based on this type of error, instead of exiting.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            ben.brooks Ben Brooks
            adamf Adam Fraser
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty