Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-19543

MOI Secondary Indexes : panic: runtime error: invalid memory address or nil pointer dereference

    XMLWordPrintable

Details

    • Untriaged
    • Unknown

    Description

      Error reported:
      statement=MERGE+INTO+simple_table_3+USING+copy_simple_table+ON+KEY+copy_simple_table.primary_key_id+WHEN+MATCHED+THEN+DELETE+WHERE+NOT%28%28simple_table_3.primary_key_id+%3C%3E+%22549%22+AND+simple_table_3.decimal_field1+%3D+5188%29%29%0A+++ error 500 reason: unknown {
      "requestID": "e8d138c8-0aba-47f6-be51-ccd1ee5bb400",
      "signature": null,
      "results": [
      ],
      "errors": [

      { "code": 5000, "msg": "dial tcp 127.0.0.1:9101: connection refused from 127.0.0.1:9101 - cause: dial tcp 127.0.0.1:9101: connection refused from 127.0.0.1:9101" }

      ],
      "status": "errors",
      "metrics":

      { "elapsedTime": "895.761852ms", "executionTime": "895.638202ms", "resultCount": 0, "resultSize": 0, "errorCount": 1 }

      }

      Job failed:
      cen006-n1ql-vset08-01-rqg-crud-merge-delete-primary-secondary-tests

      Test which failed:
      ./testrunner -i ${resource_tuq} -p gsi_type=${gsi_type} -t rqg.test_rqg.RQGTests.test_rqg_crud_ops,test_file_path=b/resources/rqg/simple_table_db/query_tests_using_templates/crud_delete_ops_50000.txt.zip,database=crud_db,reset_database=True,concurreny_count=10,index_quota_percent=30,password=password,use_mysql=True,replicas=0,merge_operation=True,crud_type=merge_delete,total_queries=${total_queries},record_failure=False,failure_record_path=/tmp/,skip_cleanup=${skip_cleanup}

      Panic error seen in indexer.log:
      Info: count:736 committed:false) SnapCreateDur 34.261µs SnapOpenDur 85.322µs
      panic: runtime error: invalid memory address or nil pointer dereference
      [signal 0xb code=0x1 addr=0x0 pc=0x82c0cb]

      goroutine 4423920 [running]:
      panic(0xf1fe80, 0xc8200140c0)
      /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/runtime/panic.go:464 +0x3e6 fp=0xc82ccb1b50 sp=0xc82ccb1ad0
      runtime.panicmem()
      /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/runtime/panic.go:45 +0x49 fp=0xc82ccb1b78 sp=0xc82ccb1b50
      runtime.sigpanic()
      /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/runtime/sigpanic_unix.go:24 +0x2ba fp=0xc82ccb1bc8 sp=0xc82ccb1b78
      github.com/couchbase/indexing/secondary/memdb.(*Iterator).skipUnwanted(0xc82dbd3ec0)
      /home/couchbase/jenkins/workspace/watson-unix/goproj/src/github.com/couchbase/indexing/secondary/memdb/iterator.go:23 +0x5b fp=0xc82ccb1bd8 sp=0xc82ccb1bc8
      github.com/couchbase/indexing/secondary/memdb.(*Iterator).Next(0xc82dbd3ec0)
      /home/couchbase/jenkins/workspace/watson-unix/goproj/src/github.com/couchbase/indexing/secondary/memdb/iterator.go:56 +0x3c fp=0xc82ccb1be8 sp=0xc82ccb1bd8
      github.com/couchbase/indexing/secondary/indexer.(*memdbSnapshot).Iterate(0xc824331020, 0x7f8c9c3baf30, 0x1965250, 0x7f8c9c3baf30, 0x1965260, 0x3, 0x1314aa0, 0xc8301b8e60, 0x0, 0x0)
      /home/couchbase/jenkins/workspace/watson-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/memdb_slice_impl.go:1285 +0x455 fp=0xc82ccb1ce8 sp=0xc82ccb1be8
      github.com/couchbase/indexing/secondary/indexer.(*memdbSnapshot).Range(0xc824331020, 0x7f8c9c3baf30, 0x1965250, 0x7f8c9c3baf30, 0x1965260, 0x3, 0xc8301b8e60, 0x0, 0x0)
      /home/couchbase/jenkins/workspace/watson-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/memdb_slice_impl.go:1239 +0x95 fp=0xc82ccb1d40 sp=0xc82ccb1ce8
      github.com/couchbase/indexing/secondary/indexer.(*memdbSnapshot).All(0xc824331020, 0xc8301b8e60, 0x0, 0x0)
      /home/couchbase/jenkins/workspace/watson-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/memdb_slice_impl.go:1243 +0x98 fp=0xc82ccb1da8 sp=0xc82ccb1d40
      github.com/couchbase/indexing/secondary/indexer.(*IndexScanSource).Routine(0xc832c2d260, 0x0, 0x0)
      /home/couchbase/jenkins/workspace/watson-unix/goproj/src/github.com/couchbase/indexing/secondary/indexer/scan_pipeline.go:109 +0x255 fp=0xc82ccb1f08 sp=0xc82ccb1da8
      github.com/couchbase/indexing/secondary/pipeline.(*Pipeline).runIt.func1(0xc82dbcef80, 0xc832c2d200)
      /home/couchbase/jenkins/workspace/watson-unix/goproj/src/github.com/couchbase/indexing/secondary/pipeline/pipeline.go:75 +0x5d fp=0xc82ccb1fa0 sp=0xc82ccb1f08
      runtime.goexit()
      /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/runtime/asm_amd64.s:1998 +0x1 fp=0xc82ccb1fa8 sp=0xc82ccb1fa0
      created by github.com/couchbase/indexing/secondary/pipeline.(*Pipeline).runIt
      goroutine 35 [syscall, 23 minutes]:
      runtime.notetsleepg(0x19ac6a0, 0xffffffffffffffff, 0x1)
      /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/runtime/lock_futex.go:205 +0x4e fp=0xc82016e740 sp=0xc82016e718
      os/signal.signal_recv(0x0)
      /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/runtime/sigqueue.go:116 +0x132 fp=0xc82016e778 sp=0xc82016e740
      os/signal.loop()
      /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/os/signal/signal_unix.go:22 +0x18 fp=0xc82016e7c0 sp=0xc82016e778
      runtime.goexit()
      /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/runtime/asm_amd64.s:1998 +0x1 fp=0xc82016e7c8 sp=0xc82016e7c0
      created by os/signal.init.1
      /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/os/signal/signal_unix.go:28 +0x37

      goroutine 36 [syscall]:
      runtime.notetsleepg(0x198ac38, 0x463712, 0x0)
      /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/runtime/lock_futex.go:205 +0x4e fp=0xc82016ef38 sp=0xc82016ef10
      runtime.timerproc()
      /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/runtime/time.go:209 +0xde fp=0xc82016efc0 sp=0xc82016ef38
      runtime.goexit()
      /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/runtime/asm_amd64.s:1998 +0x1 fp=0xc82016efc8 sp=0xc82016efc0
      created by runtime.addtimerLocked
      /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/runtime/time.go:116 +0x11f

      goroutine 37 [IO wait, 23 minutes]:
      runtime.gopark(0x1318838, 0x7f8ca8e97910, 0x10aafe0, 0x7, 0x62a51b, 0x5)
      /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/runtime/proc.go:262 +0x163 fp=0xc82004f378 sp=0xc82004f350
      runtime.netpollblock(0x7f8ca8e978e8, 0xffffff0000000072, 0x415879)
      /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/runtime/netpoll.go:341 +0x15e fp=0xc82004f3c0 sp=0xc82004f378
      net.runtime_pollWait(0x7f8ca8e978e8, 0x72, 0xc8201c6000)
      /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/runtime/netpoll.go:160 +0x60 fp=0xc82004f3e0 sp=0xc82004f3c0
      net.(*pollDesc).Wait(0xc820072140, 0x72, 0x0, 0x0)
      /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/net/fd_poll_runtime.go:73 +0x3a fp=0xc82004f400 sp=0xc82004f3e0
      net.(*pollDesc).WaitRead(0xc820072140, 0x0, 0x0)
      /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/net/fd_poll_runtime.go:78 +0x36 fp=0xc82004f428 sp=0xc82004f400
      net.(*netFD).Read(0xc8200720e0, 0xc8201c6000, 0x1000, 0x1000, 0x0, 0x7f8ca8e8e028, 0xc8200780a0)
      /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/net/fd_unix.go:250 +0x23a fp=0xc82004f4f8 sp=0xc82004f428
      net.(*conn).Read(0xc820088078, 0xc8201c6000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
      /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/net/net.go:172 +0xe4 fp=0xc82004f560 sp=0xc82004f4f8
      bufio.(*Reader).fill(0xc820076240)
      /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/bufio/bufio.go:97 +0x1e9 fp=0xc82004f620 sp=0xc82004f560
      bufio.(*Reader).Read(0xc820076240, 0xc821162001, 0x5ff, 0x5ff, 0x3dc, 0x0, 0x0)
      /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/bufio/bufio.go:207 +0x260 fp=0xc82004f6d0 sp=0xc82004f620
      github.com/couchbase/cbauth/revrpc.(*minirwc).Read(0xc8200e5080, 0xc821162001, 0x5ff, 0x5ff, 0xc8233ff560, 0x0, 0x0)
      /home/couchbase/jenkins/workspace/watson-unix/godeps/src/github.com/couchbase/cbauth/revrpc/revrpc.go:100 +0x51 fp=0xc82004f710 sp=0xc82004f6d0
      encoding/json.(*Decoder).refill(0xc8200811e0, 0x0, 0x0)
      /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/encoding/json/stream.go:152 +0x287 fp=0xc82004f7c8 sp=0xc82004f710
      encoding/json.(*Decoder).readValue(0xc8200811e0, 0x1, 0x0, 0x0)
      /home/couchbase/.cbdepscache/exploded/x86_64/go-1.6/go/src/encoding/json/stream.go:128 +0x413 fp=0xc82004f878 sp=0xc82004f7c8
      encoding/json.(*Decoder).Decode(0xc8200811e0, 0xeb0be0, 0xc8200762c0, 0x0, 0x0)

      What the test does?
      This test creates 10 primary indexes using MOI each on a different bucket.
      It then issues various queries like merge into...when matched then delete.., select *....order by, insert into..select *, insert into...select meta().id....,

      The test is supposed to run 10000 queries.
      800 queries are run successfully.
      Error is seen on query:
      MERGE INTO simple_table_4 USING copy_simple_table ON KEY copy_simple_table.primary_key_id WHEN MATCHED THEN DELETE WHERE (simple_table_4.decimal_field1 = 5188 AND simple_table_4.varchar_field1 IS NOT NULL AND simple_table_4.bool_field1 != true)

      Same test passes with forestdb storage mode for an earlier build.

      Attaching screenshots.
      There are 10 buckets, each having 1000 documents.
      Attaching screenshot for memory settings also.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            siri Sriram Melkote (Inactive)
            Prerna.Manaktala Prerna Manaktala (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty