Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-31049

Another instance of concurrent map read and write...

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: Mad-Hatter, 6.0.0, 5.5.0
    • Fix Version/s: Mad-Hatter, 6.0.0, 5.5.2
    • Component/s: query
    • Labels:
      None
    • Triage:
      Untriaged
    • Is this a Regression?:
      No

      Description

      This has been reported in the field.
      No statement has been provided, but I believe this is another instance of MB-28443

      Relevant stack:

      fatal error: concurrent map read and map write
       
      goroutine 113160 [running]:
      runtime.throw(0xfba33d, 0x21)
      	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.8.5/go/src/runtime/panic.go:596 +0x95 fp=0xc424dff4b8 sp=0xc424dff498
      runtime.mapaccess1_faststr(0xf313a0, 0xc422e95410, 0xc420fc5181, 0xd, 0xc4238ace00)
      	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.8.5/go/src/runtime/hashmap_fast.go:217 +0x4cf fp=0xc424dff518 sp=0xc424dff4b8
      github.com/couchbase/query/value.objectValue.MarshalJSON(0xc422e95410, 0x0, 0x0, 0x0, 0x0, 0x0)
      	goproj/src/github.com/couchbase/query/value/object.go:51 +0x1d6 fp=0xc424dff5d8 sp=0xc424dff518
      encoding/json.marshalerEncoder(0xc4213dbce0, 0xf313a0, 0xc422e95410, 0x15, 0xc422e90100)
      	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.8.5/go/src/encoding/json/encode.go:451 +0x9f fp=0xc424dff648 sp=0xc424dff5d8
      encoding/json.(*encodeState).reflectValue(0xc4213dbce0, 0xf313a0, 0xc422e95410, 0x15, 0xc422e90100)
      	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.8.5/go/src/encoding/json/encode.go:323 +0x82 fp=0xc424dff680 sp=0xc424dff648
      encoding/json.interfaceEncoder(0xc4213dbce0, 0xe1d9e0, 0xc4250c8b00, 0x94, 0xc4250c0100)
      	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.8.5/go/src/encoding/json/encode.go:617 +0xdb fp=0xc424dff6c0 sp=0xc424dff680
      encoding/json.(*mapEncoder).encode(0xc42017c170, 0xc4213dbce0, 0xe31340, 0xc423fa5650, 0x15, 0xe30100)
      	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.8.5/go/src/encoding/json/encode.go:690 +0x589 fp=0xc424dff830 sp=0xc424dff6c0
      encoding/json.(*mapEncoder).(encoding/json.encode)-fm(0xc4213dbce0, 0xe31340, 0xc423fa5650, 0x15, 0xc423fa0100)
      	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.8.5/go/src/encoding/json/encode.go:706 +0x64 fp=0xc424dff870 sp=0xc424dff830
      encoding/json.(*encodeState).reflectValue(0xc4213dbce0, 0xe31340, 0xc423fa5650, 0x15, 0x100)
      	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.8.5/go/src/encoding/json/encode.go:323 +0x82 fp=0xc424dff8a8 sp=0xc424dff870
      encoding/json.(*encodeState).marshal(0xc4213dbce0, 0xe31340, 0xc423fa5650, 0xc422460100, 0x0, 0x0)
      	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.8.5/go/src/encoding/json/encode.go:296 +0xb8 fp=0xc424dff8e0 sp=0xc424dff8a8
      encoding/json.Marshal(0xe31340, 0xc423fa5650, 0x3, 0xc420024650, 0xc420024600, 0xc42246fa00, 0xc424c38a00)
      	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.8.5/go/src/encoding/json/encode.go:161 +0x6e fp=0xc424dff928 sp=0xc424dff8e0
      github.com/couchbase/go-couchbase.(*Bucket).WriteCasWithMT(0xc4201881c0, 0xc4233837d0, 0x23, 0x2000000, 0x0, 0x154ddaa8e5cc0000, 0xe31340, 0xc423fa5650, 0x0, 0x0, ...)
      	godeps/src/github.com/couchbase/go-couchbase/client.go:853 +0x3b7 fp=0xc424dffa00 sp=0xc424dff928
      github.com/couchbase/go-couchbase.(*Bucket).CasWithMeta(0xc4201881c0, 0xc4233837d0, 0x23, 0x2000000, 0x0, 0x154ddaa8e5cc0000, 0xe31340, 0xc423fa5650, 0x154ddaadfc1c0000, 0x0, ...)
      	godeps/src/github.com/couchbase/go-couchbase/client.go:835 +0x93 fp=0xc424dffa78 sp=0xc424dffa00
      github.com/couchbase/query/datastore/couchbase.(*keyspace).performOp(0xc420f0cd20, 0x2, 0xc42097e000, 0x10, 0x40, 0x2f4dcaee0000000a, 0xed3121720, 0x1, 0x6e38, 0x1905dc0)
      	goproj/src/github.com/couchbase/query/datastore/couchbase/couchbase.go:1263 +0xaa9 fp=0xc424dffc28 sp=0xc424dffa78
      github.com/couchbase/query/datastore/couchbase.(*keyspace).Update(0xc420f0cd20, 0xc42097e000, 0x10, 0x40, 0xc423fb26c0, 0x0, 0x0, 0x19116c0, 0x19116d8)
      	goproj/src/github.com/couchbase/query/datastore/couchbase/couchbase.go:1295 +0x56 fp=0xc424dffc88 sp=0xc424dffc28
      github.com/couchbase/query/execution.(*SendUpdate).flushBatch(0xc424370b40, 0xc4201598c0, 0xed3121700)
      	goproj/src/github.com/couchbase/query/execution/update_send.go:154 +0x75d fp=0xc424dffd80 sp=0xc424dffc88
      github.com/couchbase/query/execution.(*base).enbatchSize(0xc424370b40, 0x18d0980, 0xc423388b40, 0x1899f00, 0xc424370b40, 0x10, 0xc4201598c0, 0x18d0980)
      	goproj/src/github.com/couchbase/query/execution/base.go:707 +0x1a2 fp=0xc424dffde0 sp=0xc424dffd80
      github.com/couchbase/query/execution.(*base).enbatch(0xc424370b40, 0x18d0980, 0xc423388b40, 0x1899f00, 0xc424370b40, 0xc4201598c0, 0xaef201)
      	goproj/src/github.com/couchbase/query/execution/base.go:721 +0x69 fp=0xc424dffe30 sp=0xc424dffde0
      github.com/couchbase/query/execution.(*SendUpdate).processItem(0xc424370b40, 0x18d0980, 0xc423388b40, 0xc4201598c0, 0x1)
      	goproj/src/github.com/couchbase/query/execution/update_send.go:56 +0x90 fp=0xc424dffe78 sp=0xc424dffe30
      github.com/couchbase/query/execution.(*base).runConsumer.func1()
      	goproj/src/github.com/couchbase/query/execution/base.go:547 +0x25d fp=0xc424dfff00 sp=0xc424dffe78
      github.com/couchbase/query/util.(*Once).Do(0xc424370c38, 0xc420301738)
      	goproj/src/github.com/couchbase/query/util/sync.go:51 +0x68 fp=0xc424dfff28 sp=0xc424dfff00
      github.com/couchbase/query/execution.(*base).runConsumer(0xc424370b40, 0x1897940, 0xc424370b40, 0xc4201598c0, 0x0, 0x0)
      	goproj/src/github.com/couchbase/query/execution/base.go:552 +0xaf fp=0xc424dfff80 sp=0xc424dfff28
      github.com/couchbase/query/execution.(*SendUpdate).RunOnce(0xc424370b40, 0xc4201598c0, 0x0, 0x0)
      	goproj/src/github.com/couchbase/query/execution/update_send.go:52 +0x5c fp=0xc424dfffc0 sp=0xc424dfff80
      runtime.goexit()
      	/home/couchbase/.cbdepscache/exploded/x86_64/go-1.8.5/go/src/runtime/asm_amd64.s:2197 +0x1 fp=0xc424dfffc8 sp=0xc424dfffc0
      created by github.com/couchbase/query/execution.(*Sequence).RunOnce.func1
      	goproj/src/github.com/couchbase/query/execution/sequence.go:95 +0x404
      

      What is happening is that go-couchbase is marshalling an object value for the purpose of writing it, and as it cycles through the elements, if finds that one element is being written to.

      Of particular concern is the fact that the panic in value - going through decode/json and go couchbase, does not trickle down to context and therefore we don't go through the assertion failure code.

        Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

          Hide
          lynn.straus Lynn Straus added a comment -

          September 7 : Consistent with recent email notices, this ticket is being moved from Alice to Mad-Hatter given Alice deadlines for any non-critical changes.

          Show
          lynn.straus Lynn Straus added a comment - September 7 : Consistent with recent email notices, this ticket is being moved from Alice to Mad-Hatter given Alice deadlines for any non-critical changes.
          Hide
          lynn.straus Lynn Straus added a comment -

          added to Alice tracking ticket as approved

          Show
          lynn.straus Lynn Straus added a comment - added to Alice tracking ticket as approved
          Hide
          marco.greco Marco Greco added a comment -

          Fixed in both Alice and Mad Hatter

          Show
          marco.greco Marco Greco added a comment - Fixed in both Alice and Mad Hatter
          Hide
          marco.greco Marco Greco added a comment -

          In 5.5.2 too

          Show
          marco.greco Marco Greco added a comment - In 5.5.2 too
          Hide
          korrigan.clark Korrigan Clark added a comment -

          Verified in build 5.5.2-3733

          Show
          korrigan.clark Korrigan Clark added a comment - Verified in build 5.5.2-3733

            People

            • Assignee:
              ritam.sharma Ritam Sharma
              Reporter:
              marco.greco Marco Greco
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Due:
                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                There are no open Gerrit changes

                  PagerDuty

                  Error rendering 'com.pagerduty.jira-server-plugin:PagerDuty'. Please contact your Jira administrators.