Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-54300

[BP to 6.6.6 MB-51318] Potential for request stall if stream operator fails to notify request that it has terminated

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • 6.6.6
    • 6.5.2, 6.6.5, 7.0.3, 7.1.0
    • query
    • Untriaged
    • 1
    • No

    Description

      In order to avoid panics and garbled json responses, the main ServeHTTP request needs to know that the Stream operator is dead before continuing after a network close / timeout / request termination / assert failure.
      Although there is no testcase, and code inspection has found nothing, if Stream fails to notify for whatever reason, this code path will lead to a hang, and there is at least one CBSE showing these symptoms.
      Proactively find ways to remove this weakness, while not affecting throughput.

      Symptoms include all the active requests stuck like this:

      32 @ 0xee57d0 0xeb9538 0xeb950e 0xeb91fb 0x204ed81 0x1f9d596 0x1f9be0c 0x1f9b1c7 0x204198d 0x14efdd2 0x12034a4 0x11fed45 0xf135a1
      		 
      #	0x204ed80	github.com/couchbase/query/server/http.(*httpRequest).Execute+0x280	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/server/http/service_response.go:156
      		 
      #	0x1f9d595	github.com/couchbase/query/server.(*Server).serviceRequest+0x7a5	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/server/server.go:1037
      		 
      #	0x1f9be0b	github.com/couchbase/query/server.(*Server).handleRequest+0x6b		/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/server/server.go:622
      		 
      #	0x1f9b1c6	github.com/couchbase/query/server.(*Server).ServiceRequest+0x76		/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/server/server.go:561
      		 
      #	0x204198c	github.com/couchbase/query/server/http.(*HttpEndpoint).ServeHTTP+0x35c	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/query/server/http/service_endpoint.go:323
      		 
      #	0x14efdd1	github.com/gorilla/mux.(*Router).ServeHTTP+0xe1				/home/couchbase/jenkins/workspace/couchbase-server-unix/server_build/gopkg/go-1.13.7/pkg/mod/github.com/gorilla/mux@v1.7.4/mux.go:210
      		 
      #	0x12034a3	net/http.serverHandler.ServeHTTP+0xa3					/home/couchbase/.cbdepscache/exploded/x86_64/go-1.13.7/go/src/net/http/server.go:2802
      		 
      #	0x11fed44	net/http.(*conn).serve+0x874						/home/couchbase/.cbdepscache/exploded/x86_64/go-1.13.7/go/src/net/http/server.go:1890

      Attachments

        Issue Links

          backports to

          Bug - A problem which impairs or prevents the functions of the product. MB-54043 [BP 7.1.3/7.2.0 MB-51318] Potential for request stall if stream operator fails to notify request that it has terminated

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MB-54236 [BP 7.0.5 MB-51318] Potential for request stall if stream operator fails to notify request that it has terminated

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MB-54417 [BP 7.1.x MB-51318] Potential for request stall if stream operator fails to notify request that it has terminated

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          is a backport of

          Bug - A problem which impairs or prevents the functions of the product. MB-51318 Potential for request stall if stream operator fails to notify request that it has terminated

          • Major - Major loss of function.
          • Closed
          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              pavan.pb Pavan PB
              Sitaram.Vemulapalli Sitaram Vemulapalli
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty