Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-6238

disk commit failed with error : cannot save CouchDB docs for vbucket = 77 when loading 100M items blobs

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Critical
    • 2.0
    • None
    • storage-engine
    • Security Level: Public
    • None

    Description

      Perhaps some race because it was working in my previous run with exactly same code.

      I was running some single node write throughput experiment. With goal of loading 100M short (~256 bytes) blobs into single node 128 vbuckets cluster.

      Everything starts like this:

      [ns_server:debug,2012-08-15T20:00:21.224,n_10@127.0.0.1:<0.6728.0>:ns_pubsub:do_subscribe_link:149]Deleting

      {ns_stats_event,<0.742.0>} event handler: ok
      [ns_server:debug,2012-08-15T20:00:21.291,n_10@127.0.0.1:<0.6734.0>:ns_pubsub:do_subscribe_link:120]Started subscription {ns_stats_event,<0.742.0>}

      [ns_server:debug,2012-08-15T20:00:22.224,n_10@127.0.0.1:<0.6734.0>:ns_pubsub:do_subscribe_link:149]Deleting

      {ns_stats_event,<0.742.0>} event handler: ok
      [ns_server:debug,2012-08-15T20:00:22.266,n_10@127.0.0.1:<0.6743.0>:ns_pubsub:do_subscribe_link:120]Started subscription {ns_stats_event,<0.742.0>}

      [ns_server:info,2012-08-15T20:00:22.972,n_10@127.0.0.1:ns_port_memcached:ns_port_server:log:169]memcached<0.769.0>: 537 3: Warning: failed to save docs to database, error=error reading file errno=Resource temporarily unavailable
      memcached<0.769.0>: 537 3: Warning: commit failed, cannot save CouchDB docs for vbucket = 77 rev = 2
      memcached<0.769.0>: 537 3: Fatal error in persisting SET ``10001711'' on vb 77!!! Requeue it...
      memcached<0.769.0>: 537 3: Fatal error in persisting SET ``10002964'' on vb 77!!! Requeue it...
      memcached<0.769.0>: 537 3: Fatal error in persisting SET ``10015799'' on vb 77!!! Requeue it...

      And then it doesn't seem to recover.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            alkondratenko Aleksey Kondratenko (Inactive)
            alkondratenko Aleksey Kondratenko (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty