Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-42819

Memory leak within gRPC's resetTransport(..) due to context.WithCancel(..)

    XMLWordPrintable

    Details

    • Triage:
      Untriaged
    • Story Points:
      1
    • Is this a Regression?:
      Unknown

      Description

      N1QL/FTS use grpc version 1.17.

      There seems to be a leak within grpc's resetTransport as pointed out here -

      which is addressed by https://github.com/grpc/grpc-go/pull/2985 for version 1.24

      Consider upgrading grpc within these projects to v1.24.0?

      The leak is coming from context.WithCancel as pointed out here: https://github.com/grpc/grpc-go/issues/2976.

      Setting a timeout with context is a workaround for this - by setting a deadline for the datastore.IndexConnection which is handed to n1fty's Search(..). But WithTimeout for context was introduced in 7.0, so this is not a viable workaround for older builds.

        Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

          abhinav Abhinav Dangeti created issue -
          abhinav Abhinav Dangeti made changes -
          Field Original Value New Value
          Description N1QL/FTS use grpc version 1.17.

          There seems to be a leak within grpc's resetTransport as pointed out here - [https://github.com/grpc/grpc-go/issues/3667] and [https://github.com/etcd-io/etcd/issues/11371] which is addressed by [https://github.com/grpc/grpc-go/pull/2985] for version 1.24

          Consider upgrading grpc within these projects to v1.24.0?
           * [https://github.com/couchbase/cbft|http://github.com/couchbase/cbft]
           * [https://github.com/couchbase/cbftx]
           * [https://github.com/couchbase/query|http://github.com/couchbase/query]
           * [https://github.com/couchbase/n1fty|http://github.com/couchbase/n1fty]

           
          N1QL/FTS use grpc version 1.17.

          There seems to be a leak within grpc's resetTransport as pointed out here -
           * [https://github.com/grpc/grpc-go/issues/3667
           * [https://github.com/etcd-io/etcd/issues/11371
           * [https://github.com/grpc/grpc-go/issues/2976]

          which is addressed by [https://github.com/grpc/grpc-go/pull/2985] for version 1.24

          Consider upgrading grpc within these projects to v1.24.0?
           * [https://github.com/couchbase/cbft|http://github.com/couchbase/cbft]
           * [https://github.com/couchbase/cbftx]
           * [https://github.com/couchbase/query|http://github.com/couchbase/query]
           * [https://github.com/couchbase/n1fty|http://github.com/couchbase/n1fty]

           
          abhinav Abhinav Dangeti made changes -
          Link This issue relates to CBSE-9208 [ CBSE-9208 ]
          Sitaram.Vemulapalli Sitaram Vemulapalli made changes -
          Fix Version/s 6.6.2 [ 17103 ]
          Show
          abhinav Abhinav Dangeti added a comment - For 7.0 builds .. http://review.couchbase.org/c/cbft/+/140595 http://review.couchbase.org/c/cbftx/+/140596 http://review.couchbase.org/c/n1fty/+/140597 http://review.couchbase.org/c/query/+/140598  
          abhinav Abhinav Dangeti made changes -
          Summary Connection leak within gRPC's resetTransport Memory leak within gRPC's resetTransport
          abhinav Abhinav Dangeti made changes -
          Summary Memory leak within gRPC's resetTransport Memory leak within gRPC's resetTransport(..)
          Hide
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-7.0.0-3792 contains cbft commit b704118 with commit message:
          MB-42819: Upgrade grpc version from 1.17.0 -> 1.24.0

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-3792 contains cbft commit b704118 with commit message: MB-42819 : Upgrade grpc version from 1.17.0 -> 1.24.0
          Hide
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-7.0.0-3792 contains cbftx commit 6ce92cd with commit message:
          MB-42819: grpc 1.17.0 -> 1.24.0

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-3792 contains cbftx commit 6ce92cd with commit message: MB-42819 : grpc 1.17.0 -> 1.24.0
          Hide
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-7.0.0-3792 contains n1fty commit cc88733 with commit message:
          MB-42819: grpc 1.17.0 -> 1.24.0

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-3792 contains n1fty commit cc88733 with commit message: MB-42819 : grpc 1.17.0 -> 1.24.0
          Hide
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-7.0.0-3792 contains query commit e8dcce7 with commit message:
          MB-42819: grpc 1.17.0 -> 1.24.0

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-3792 contains query commit e8dcce7 with commit message: MB-42819 : grpc 1.17.0 -> 1.24.0
          Hide
          abhinav Abhinav Dangeti added a comment - - edited

          Here's a proposal for a manifest update for 6.6.2 builds .. http://review.couchbase.org/c/manifest/+/140601 to ..

          • grpc version to v1.24.0
          • protobuf version to v1.3.5
          • net version update to same as master

          (Pending approval for 6.6.2)

          Show
          abhinav Abhinav Dangeti added a comment - - edited Here's a proposal for a manifest update for 6.6.2 builds .. http://review.couchbase.org/c/manifest/+/140601  to .. grpc version to v1.24.0 protobuf version to v1.3.5 net version update to same as master (Pending approval for 6.6.2)
          abhinav Abhinav Dangeti made changes -
          Summary Memory leak within gRPC's resetTransport(..) Memory leak within gRPC's resetTransport(..) due to context without Timeout
          abhinav Abhinav Dangeti made changes -
          Summary Memory leak within gRPC's resetTransport(..) due to context without Timeout Memory leak within gRPC's resetTransport(..)
          abhinav Abhinav Dangeti made changes -
          Description N1QL/FTS use grpc version 1.17.

          There seems to be a leak within grpc's resetTransport as pointed out here -
           * [https://github.com/grpc/grpc-go/issues/3667
           * [https://github.com/etcd-io/etcd/issues/11371
           * [https://github.com/grpc/grpc-go/issues/2976]

          which is addressed by [https://github.com/grpc/grpc-go/pull/2985] for version 1.24

          Consider upgrading grpc within these projects to v1.24.0?
           * [https://github.com/couchbase/cbft|http://github.com/couchbase/cbft]
           * [https://github.com/couchbase/cbftx]
           * [https://github.com/couchbase/query|http://github.com/couchbase/query]
           * [https://github.com/couchbase/n1fty|http://github.com/couchbase/n1fty]

           
          N1QL/FTS use grpc version 1.17.

          There seems to be a leak within grpc's resetTransport as pointed out here -
           * [https://github.com/grpc/grpc-go/issues/3667
           * [https://github.com/etcd-io/etcd/issues/11371
           * [https://github.com/grpc/grpc-go/issues/2976]

          which is addressed by [https://github.com/grpc/grpc-go/pull/2985] for version 1.24

          Consider upgrading grpc within these projects to v1.24.0?
           * [https://github.com/couchbase/cbft|http://github.com/couchbase/cbft]
           * [https://github.com/couchbase/cbftx]
           * [https://github.com/couchbase/query|http://github.com/couchbase/query]
           * [https://github.com/couchbase/n1fty|http://github.com/couchbase/n1fty]

          The leak is coming from context.WithCancel as pointed out here: [https://github.com/grpc/grpc-go/issues/2976].

          Setting a timeout with context is a workaround for this - by setting a deadline for the datastore.IndexConnection which is handed to n1fty's Search(..).
          abhinav Abhinav Dangeti made changes -
          Description N1QL/FTS use grpc version 1.17.

          There seems to be a leak within grpc's resetTransport as pointed out here -
           * [https://github.com/grpc/grpc-go/issues/3667
           * [https://github.com/etcd-io/etcd/issues/11371
           * [https://github.com/grpc/grpc-go/issues/2976]

          which is addressed by [https://github.com/grpc/grpc-go/pull/2985] for version 1.24

          Consider upgrading grpc within these projects to v1.24.0?
           * [https://github.com/couchbase/cbft|http://github.com/couchbase/cbft]
           * [https://github.com/couchbase/cbftx]
           * [https://github.com/couchbase/query|http://github.com/couchbase/query]
           * [https://github.com/couchbase/n1fty|http://github.com/couchbase/n1fty]

          The leak is coming from context.WithCancel as pointed out here: [https://github.com/grpc/grpc-go/issues/2976].

          Setting a timeout with context is a workaround for this - by setting a deadline for the datastore.IndexConnection which is handed to n1fty's Search(..).
          N1QL/FTS use grpc version 1.17.

          There seems to be a leak within grpc's resetTransport as pointed out here -
           * [https://github.com/grpc/grpc-go/issues/3667
           * [https://github.com/etcd-io/etcd/issues/11371
           * [https://github.com/grpc/grpc-go/issues/2976]

          which is addressed by [https://github.com/grpc/grpc-go/pull/2985] for version 1.24

          Consider upgrading grpc within these projects to v1.24.0?
           * [https://github.com/couchbase/cbft|http://github.com/couchbase/cbft]
           * [https://github.com/couchbase/cbftx]
           * [https://github.com/couchbase/query|http://github.com/couchbase/query]
           * [https://github.com/couchbase/n1fty|http://github.com/couchbase/n1fty]

          The leak is coming from context.WithCancel as pointed out here: [https://github.com/grpc/grpc-go/issues/2976].

          Setting a timeout with context is a workaround for this - by setting a deadline for the datastore.IndexConnection which is handed to n1fty's Search(..). But WithTimeout for context was introduced in 7.0, so this is not a viable workaround for older builds.
          abhinav Abhinav Dangeti made changes -
          Summary Memory leak within gRPC's resetTransport(..) Memory leak within gRPC's resetTransport(..) due to context.WithCancel(..)
          wayne Wayne Siu made changes -
          Fix Version/s 6.6.2 [ 17103 ]
          Fix Version/s 6.6.1 [ 17002 ]
          wayne Wayne Siu made changes -
          Labels approved-for-6.6.1
          wayne Wayne Siu made changes -
          Link This issue blocks MB-40528 [ MB-40528 ]
          abhinav Abhinav Dangeti made changes -
          Resolution Fixed [ 1 ]
          Status Open [ 1 ] Resolved [ 5 ]
          wayne Wayne Siu made changes -
          Assignee Abhinav Dangeti [ abhinav ] Mihir Kamdar [ mihir.kamdar ]
          Hide
          girish.benakappa Girish Benakappa added a comment -

          Closing as there were no regressions seen with build 6.6.1-9198 w.r.t gRPC.

          Show
          girish.benakappa Girish Benakappa added a comment - Closing as there were no regressions seen with build 6.6.1-9198 w.r.t gRPC.
          girish.benakappa Girish Benakappa made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          lynn.straus Lynn Straus made changes -
          Fix Version/s 7.0.0 [ 17233 ]
          lynn.straus Lynn Straus made changes -
          Fix Version/s Cheshire-Cat [ 15915 ]

            People

            Assignee:
            mihir.kamdar Mihir Kamdar
            Reporter:
            abhinav Abhinav Dangeti
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Gerrit Reviews

                There are no open Gerrit changes

                  PagerDuty