Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-7832

cbtransfer gives error while transferring data between 2.0.1 nodes

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.0.1
    • Fix Version/s: 2.0.1
    • Component/s: tools
    • Security Level: Public
    • Labels:
      None
    • Environment:
      build 2.0.1-164-rel

      Description

      Steps to reproduce:

      Install build 2.0.1-164-rel on 2 nodes
      Create default bucket on node1 and create 5 json docs
      Create default bucket on node2
      Run cbtransfer on node1 to transfer data to node2

      /opt/couchbase/bin/cbtransfer couchstore-files:///opt/couchbase/var/lib/couchbase/data/ http://Administrator:password@10.3.3.100:8091 -b default -B default -v -v -v

      Below exception is thrown and the command hangs. Diagnostics are attached.

      The node has Python 2.4.3 and Centos 64bit

      [root@caper-012 ~]# /opt/couchbase/bin/cbtransfer couchstore-files:///opt/couchbase/var/lib/couchbase/data/ http://Administrator:password@10.3.3.100:8091 -b default -B default -v -v -v
      2013-02-27 03:33:48,435: mt cbtransfer...
      2013-02-27 03:33:48,435: mt source : couchstore-files:///opt/couchbase/var/lib/couchbase/data/
      2013-02-27 03:33:48,436: mt sink : http://Administrator:password@10.3.3.100:8091
      2013-02-27 03:33:48,436: mt opts : {'username': None, 'source_vbucket_state': 'active', 'destination_vbucket_state': 'active', 'verbose': 3, 'dry_run': False, 'extra':

      {'max_retry': 10.0, 'rehash': 0.0, 'nmv_retry': 1.0, 'cbb_max_mb': 100000.0, 'try_xwm': 1.0, 'batch_max_bytes': 400000.0, 'report_full': 2000.0, 'batch_max_size': 1000.0, 'report': 5.0, 'recv_min_bytes': 4096.0}

      , 'single_node': False, 'bucket_destination': 'default', 'destination_operation': None, 'threads': 4, 'key': None, 'password': None, 'id': None, 'bucket_source': 'default'}
      2013-02-27 03:33:48,437: mt source_class: <class 'pump_sfd.SFDSource'>
      2013-02-27 03:33:48,471: mt sink_class: <class 'pump_cb.CBSink'>
      2013-02-27 03:33:48,472: mt rest_request: Administrator@10.3.3.100:8091/pools/default/buckets
      2013-02-27 03:33:48,491: mt source_bucket: default
      2013-02-27 03:33:48,492: mt sink_bucket: default
      2013-02-27 03:33:48,492: mt source_buckets: default
      2013-02-27 03:33:48,492: mt bucket_source: default
      2013-02-27 03:33:48,493: mt source_buckets filtered: default
      2013-02-27 03:33:48,493: mt bucket: default
      2013-02-27 03:33:48,493: mt source_nodes: N/A
      2013-02-27 03:33:48,508: mt enqueueing node: N/A
      2013-02-27 03:33:48,508: w0 node: N/A
      Exception in thread w0:
      Traceback (most recent call last):
      File "/usr/lib64/python2.4/threading.py", line 442, in __bootstrap
      self.run()
      File "/usr/lib64/python2.4/threading.py", line 422, in run
      self._target(*self.args, **self._kwargs)
      File "/opt/couchbase/lib/python/pump.py", line 265, in run_worker
      source_map, sink_map, self.ctl, curx).run()
      File "/opt/couchbase/lib/python/pump.py", line 322, in run
      rv_batch, batch = self.source.provide_batch()
      File "/opt/couchbase/lib/python/pump_sfd.py", line 158, in provide_batch
      self.queue.task_done()
      AttributeError: Queue instance has no attribute 'task_done'

      interrupted.

      1. 10.3.3.100-8091-diag.txt.gz
        108 kB
        Deepkaran Salooja
      2. 10.3.3.95-8091-diag.txt.gz
        109 kB
        Deepkaran Salooja
      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        Hide
        bcui Bin Cui added a comment -

        what is username/pwd to access?

        Show
        bcui Bin Cui added a comment - what is username/pwd to access?
        Hide
        bcui Bin Cui added a comment -
        Show
        bcui Bin Cui added a comment - http://review.couchbase.org/#/c/24900/ Test on 10.3.3.100
        Hide
        deepkaran.salooja Deepkaran Salooja added a comment -

        Tried with build 170.

        Getting the below error with slightly larger load(e.g. 100k). Tried on VM 10.3.3.95. There is sufficient space in /tmp. Loaded 100k on default bucket using mcsoda and executed the below command.

        [root@caper-012 ~]# /opt/couchbase/bin/cbtransfer -v -v -v couchstore-files:///opt/couchbase/var/lib/couchbase/data/ /tmp/backup
        2013-03-01 03:19:31,359: mt cbtransfer...
        2013-03-01 03:19:31,360: mt source : couchstore-files:///opt/couchbase/var/lib/couchbase/data/
        2013-03-01 03:19:31,360: mt sink : /tmp/backup
        2013-03-01 03:19:31,360: mt opts : {'username': None, 'source_vbucket_state': 'active', 'destination_vbucket_state': 'active', 'verbose': 3, 'dry_run': False, 'extra':

        {'max_retry': 10.0, 'rehash': 0.0, 'nmv_retry': 1.0, 'cbb_max_mb': 100000.0, 'try_xwm': 1.0, 'batch_max_bytes': 400000.0, 'report_full': 2000.0, 'batch_max_size': 1000.0, 'report': 5.0, 'recv_min_bytes': 4096.0}

        , 'single_node': False, 'bucket_destination': None, 'destination_operation': None, 'threads': 4, 'key': None, 'password': None, 'id': None, 'bucket_source': None}
        2013-03-01 03:19:31,361: mt source_class: <class 'pump_sfd.SFDSource'>
        2013-03-01 03:19:31,395: mt sink_class: <class 'pump_bfd.BFDSink'>
        2013-03-01 03:19:31,395: mt source_buckets: default
        2013-03-01 03:19:31,395: mt bucket: default
        2013-03-01 03:19:31,396: mt source_nodes: N/A
        2013-03-01 03:19:31,411: mt enqueueing node: N/A
        2013-03-01 03:19:31,411: w0 node: N/A
        2013-03-01 03:19:31,509: s0 create_db: /tmp/backup/bucket-default/node-N%2FA/data-0000.cbb
        2013-03-01 03:19:31,509: s0 connect_db: /tmp/backup/bucket-default/node-N%2FA/data-0000.cbb
        ....................python: Objects/obmalloc.c:765: PyObject_Malloc: Assertion `bp != ((void *)0)' failed.
        Aborted

        Show
        deepkaran.salooja Deepkaran Salooja added a comment - Tried with build 170. Getting the below error with slightly larger load(e.g. 100k). Tried on VM 10.3.3.95. There is sufficient space in /tmp. Loaded 100k on default bucket using mcsoda and executed the below command. [root@caper-012 ~] # /opt/couchbase/bin/cbtransfer -v -v -v couchstore-files:///opt/couchbase/var/lib/couchbase/data/ /tmp/backup 2013-03-01 03:19:31,359: mt cbtransfer... 2013-03-01 03:19:31,360: mt source : couchstore-files:///opt/couchbase/var/lib/couchbase/data/ 2013-03-01 03:19:31,360: mt sink : /tmp/backup 2013-03-01 03:19:31,360: mt opts : {'username': None, 'source_vbucket_state': 'active', 'destination_vbucket_state': 'active', 'verbose': 3, 'dry_run': False, 'extra': {'max_retry': 10.0, 'rehash': 0.0, 'nmv_retry': 1.0, 'cbb_max_mb': 100000.0, 'try_xwm': 1.0, 'batch_max_bytes': 400000.0, 'report_full': 2000.0, 'batch_max_size': 1000.0, 'report': 5.0, 'recv_min_bytes': 4096.0} , 'single_node': False, 'bucket_destination': None, 'destination_operation': None, 'threads': 4, 'key': None, 'password': None, 'id': None, 'bucket_source': None} 2013-03-01 03:19:31,361: mt source_class: <class 'pump_sfd.SFDSource'> 2013-03-01 03:19:31,395: mt sink_class: <class 'pump_bfd.BFDSink'> 2013-03-01 03:19:31,395: mt source_buckets: default 2013-03-01 03:19:31,395: mt bucket: default 2013-03-01 03:19:31,396: mt source_nodes: N/A 2013-03-01 03:19:31,411: mt enqueueing node: N/A 2013-03-01 03:19:31,411: w0 node: N/A 2013-03-01 03:19:31,509: s0 create_db: /tmp/backup/bucket-default/node-N%2FA/data-0000.cbb 2013-03-01 03:19:31,509: s0 connect_db: /tmp/backup/bucket-default/node-N%2FA/data-0000.cbb ....................python: Objects/obmalloc.c:765: PyObject_Malloc: Assertion `bp != ((void *)0)' failed. Aborted
        Hide
        deepkaran.salooja Deepkaran Salooja added a comment -

        Files a separate issue as the error is coming from python.

        http://www.couchbase.com/issues/browse/MB-7849

        Show
        deepkaran.salooja Deepkaran Salooja added a comment - Files a separate issue as the error is coming from python. http://www.couchbase.com/issues/browse/MB-7849
        Hide
        deepkaran.salooja Deepkaran Salooja added a comment -

        original bug verified with 2.0.1-170-rel

        Show
        deepkaran.salooja Deepkaran Salooja added a comment - original bug verified with 2.0.1-170-rel

          People

          • Assignee:
            bcui Bin Cui
            Reporter:
            deepkaran.salooja Deepkaran Salooja
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes