Uploaded image for project: 'Couchbase Gateway'
  1. Couchbase Gateway
  2. CBG-2181

In-flight BLIP connections don't get closed if the database disappears

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Critical
    • Deuterium
    • 3.0.3
    • SyncGateway
    • Security Level: Public
    • None
    • 5

    Description

      In CBG-1878 we added logic to tell BLIP clients to disconnect and reconnect if the BLIP handlers panic and we see that the database has disappeared (because the panic is likely caused by the handlers trying to access a nil database).

      It appears that, even after this case is hit, it's possible for go-blip to process further requests, which will also panic. In most cases this is benign - logs get some noise but nothing meaningful happens - but it's possible in some cases the panic will happen somewhere outside the BlipSyncContext, e.g. MultiChangesFeed, which would bubble up to base.FatalPanicHandler and bring down SGW.

      It sounds like we need to do a few things here:

      1. When we send clients an ErrDatabaseWentAway, we need to promptly close the connection
      2. We should also tell go-blip to stop processing further requests (and specifically calling handlers) after we start the teardown

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              The One The One
              marks.polakovs Marks Polakovs (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty