Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-4513

ep-engine fails with assertion еrror during/right-after warmup

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.0-developer-preview-3
    • Fix Version/s: None
    • Component/s: couchbase-bucket
    • Security Level: Public
    • Labels:
      None

      Description

      Start couchbase server. 1 node is enough.

      Load few items. (2 was enough here).

      Stop it. Start it back.

      Observe how memcached will repeatedly crash.

      With the following in core dump:

      Program terminated with signal 6, Aborted.
      #0 0xf76f7430 in __kernel_vsyscall ()
      (gdb) up
      #1 0xf750b911 in *__GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
      64 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
      in ../nptl/sysdeps/unix/sysv/linux/raise.c
      (gdb)
      #2 0xf750ed42 in *__GI_abort () at abort.c:92
      92 abort.c: No such file or directory.
      in abort.c
      (gdb)
      #3 0xf7504b28 in *_GI__assert_fail (assertion=0xf3c5e49c "ht.cacheSize.get() < ((size_t)1<<(sizeof(size_t)*8-1))", file=0xf3c5e3df "stored-value.cc", line=377,
      function=0xf3c5e6a0 "static void StoredValue::reduceCacheSize(HashTable&, size_t, bool)") at assert.c:81
      81 assert.c: No such file or directory.
      in assert.c
      (gdb)
      #4 0xf3afb4fd in StoredValue::reduceCacheSize (residentOnly=<optimized out>, by=<optimized out>, ht=<optimized out>) at stored-value.cc:377
      377 assert(ht.cacheSize.get() < GIGANTOR);
      (gdb)
      #5 0xf3b70e64 in StoredValue::reduceCacheSize (ht=..., by=91, residentOnly=false) at stored-value.cc:377
      377 assert(ht.cacheSize.get() < GIGANTOR);
      (gdb) p ht.cacheSize.value
      $1 = 4294967292
      (gdb) p/x ht.cacheSize.value
      $2 = 0xfffffffc
      (gdb)

      # Subject Project Status CR V
      For Gerrit Dashboard: &For+MB-4513=message:MB-4513

        Activity

        Hide
        trond Trond Norbye added a comment -

        You've done some stuff around cache size lately? does this look familiar?

        Show
        trond Trond Norbye added a comment - You've done some stuff around cache size lately? does this look familiar?
        Hide
        alkondratenko Aleksey Kondratenko (Inactive) added a comment -

        BTW, I've tried rewinding ep-engine few commits (we made 1.8 merge recently) without success.

        Show
        alkondratenko Aleksey Kondratenko (Inactive) added a comment - BTW, I've tried rewinding ep-engine few commits (we made 1.8 merge recently) without success.
        Hide
        mikew Mike Wiederhold added a comment -

        Yeah, I have been doing work in this area, but the big change I made hasn't gotten into 1.8 yet. My plan is to merge it into 1.8.1 when we have more resources for QA since I don't want to create issue like these.

        Show
        mikew Mike Wiederhold added a comment - Yeah, I have been doing work in this area, but the big change I made hasn't gotten into 1.8 yet. My plan is to merge it into 1.8.1 when we have more resources for QA since I don't want to create issue like these.
        Hide
        keith Keith Batten (Inactive) added a comment -

        centos 5.4 64bit
        r-384
        1 node

        1) create default bucket 200M
        2) load in 2 items
        3) run "service couchbase-server restart"
        4) watch the logs/processes to see memcached repeatedly failing

        its possible this only happens at low item counts. running with 100k and 10M items did not repro this, while 2 items repeatedly reproduced it

        Show
        keith Keith Batten (Inactive) added a comment - centos 5.4 64bit r-384 1 node 1) create default bucket 200M 2) load in 2 items 3) run "service couchbase-server restart" 4) watch the logs/processes to see memcached repeatedly failing its possible this only happens at low item counts. running with 100k and 10M items did not repro this, while 2 items repeatedly reproduced it
        Hide
        dipti Dipti Borkar added a comment -

        Mike, is this merged?

        Show
        dipti Dipti Borkar added a comment - Mike, is this merged?
        Hide
        thuan Thuan Nguyen added a comment -

        Integrated in github-ep-engine-2-0 #131 (See http://qa.hq.northscale.net/job/github-ep-engine-2-0/131/)

        Show
        thuan Thuan Nguyen added a comment - Integrated in github-ep-engine-2-0 #131 (See http://qa.hq.northscale.net/job/github-ep-engine-2-0/131/ )

          People

          • Assignee:
            mikew Mike Wiederhold
            Reporter:
            alkondratenko Aleksey Kondratenko (Inactive)
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes