Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-7879

cbtransfer crashes on 32-bit Linux with couchstore-files as a source

    Details

    • Type: Task
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.0, 2.0.1
    • Fix Version/s: 2.1.0
    • Component/s: tools
    • Security Level: Public
    • Labels:
      None
    • Environment:
      Ubuntu 12.04.1 LTS, 32bit, python 2.7.3
      Build - 2.0.1-170-rel
    • Sprint:
      PCI Team - Sprint 4, PCI Team - Sprint 6

      Description

      Steps to reproduce:
      1. Create default bucket and load 100k items.
      2. Run below command
      sudo /opt/couchbase/bin/cbtransfer -v -v -v couchstore-files:///opt/couchbase/var/lib/couchbase/data/ /tmp/backup

      ubuntu@ip-10-128-247-201:~$ sudo /opt/couchbase/bin/cbtransfer -v -v -v couchstore-files:///opt/couchbase/var/lib/couchbase/data/ /tmp/backup
      2013-03-07 12:36:09,485: mt cbtransfer...
      2013-03-07 12:36:09,486: mt source : couchstore-files:///opt/couchbase/var/lib/couchbase/data/
      2013-03-07 12:36:09,486: mt sink : /tmp/backup
      2013-03-07 12:36:09,486: mt opts : {'username': None, 'source_vbucket_state': 'active', 'destination_vbucket_state': 'active', 'verbose': 3, 'dry_run': False, 'extra':

      {'max_retry': 10.0, 'rehash': 0.0, 'nmv_retry': 1.0, 'cbb_max_mb': 100000.0, 'try_xwm': 1.0, 'batch_max_bytes': 400000.0, 'report_full': 2000.0, 'batch_max_size': 1000.0, 'report': 5.0, 'recv_min_bytes': 4096.0}

      , 'single_node': False, 'bucket_destination': None, 'destination_operation': None, 'threads': 4, 'key': None, 'password': None, 'id': None, 'bucket_source': None}
      2013-03-07 12:36:09,486: mt source_class: <class 'pump_sfd.SFDSource'>
      2013-03-07 12:36:09,649: mt sink_class: <class 'pump_bfd.BFDSink'>
      2013-03-07 12:36:09,650: mt source_buckets: default
      2013-03-07 12:36:09,650: mt bucket: default
      2013-03-07 12:36:09,650: mt source_nodes: N/A
      2013-03-07 12:36:09,651: mt enqueueing node: N/A
      2013-03-07 12:36:09,651: w0 node: N/A
      Segmentation fault (core dumped)

      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        Hide
        maria Maria McDuff (Inactive) added a comment -

        per bug scrub, ready for QE. fix is merged. changed to resolved and assigning to Deep.

        Show
        maria Maria McDuff (Inactive) added a comment - per bug scrub, ready for QE. fix is merged. changed to resolved and assigning to Deep.
        Hide
        deepkaran.salooja Deepkaran Salooja added a comment -

        Now I am hitting the assertion error:
        Objects/obmalloc.c:765: PyObject_Malloc: Assertion `bp != ((void *)0)' failed.
        as filed in bug MB-7849.

        Will be able to verify this conclusively once MB-7849 is fixed as well.

        Show
        deepkaran.salooja Deepkaran Salooja added a comment - Now I am hitting the assertion error: Objects/obmalloc.c:765: PyObject_Malloc: Assertion `bp != ((void *)0)' failed. as filed in bug MB-7849 . Will be able to verify this conclusively once MB-7849 is fixed as well.
        Hide
        pavelpaulau Pavel Paulau added a comment - - edited

        Are you hitting this on EC2 instance with Ubuntu LTS as originally reported?

        Since I have absolutely clean setup with 2.0.2-789, I loaded 1M items and successfully transferred them many times:

        root@ip-10-32-243-102:~# /opt/couchbase/bin/cbtransfer -v -v -v couchstore-files:///opt/couchbase/var/lib/couchbase/data/ /tmp/backup
        2013-05-08 08:35:14,712: mt cbtransfer...
        2013-05-08 08:35:14,712: mt source : couchstore-files:///opt/couchbase/var/lib/couchbase/data/
        2013-05-08 08:35:14,712: mt sink : /tmp/backup
        2013-05-08 08:35:14,712: mt opts : {'username': None, 'source_vbucket_state': 'active', 'destination_vbucket_state': 'active', 'verbose': 3, 'dry_run': False, 'extra':

        {'max_retry': 10.0, 'rehash': 0.0, 'data_only': 0.0, 'nmv_retry': 1.0, 'cbb_max_mb': 100000.0, 'try_xwm': 1.0, 'batch_max_bytes': 400000.0, 'report_full': 2000.0, 'batch_max_size': 1000.0, 'report': 5.0, 'design_doc_only': 0.0, 'recv_min_bytes': 4096.0}

        , 'single_node': False, 'bucket_destination': None, 'destination_operation': None, 'vbucket_list': None, 'threads': 4, 'key': None, 'password': None, 'id': None, 'bucket_source': None}
        2013-05-08 08:35:14,713: mt source_class: <class 'pump_sfd.SFDSource'>
        2013-05-08 08:35:14,904: mt sink_class: <class 'pump_bfd.BFDSink'>
        2013-05-08 08:35:14,905: mt source_buckets: default
        2013-05-08 08:35:14,905: mt bucket: default
        2013-05-08 08:35:14,905: mt source_nodes: N/A
        2013-05-08 08:35:14,906: mt enqueueing node: N/A
        2013-05-08 08:35:14,907: w0 node: N/A
        2013-05-08 08:35:14,988: s0 create_db: /tmp/backup/bucket-default/node-N%2FA/data-0000.cbb
        2013-05-08 08:35:14,988: s0 connect_db: /tmp/backup/bucket-default/node-N%2FA/data-0000.cbb
        ........................................................................................................................................................................................................2013-05-08 08:36:36,424: w0 pump (couchstore-files:///opt/couchbase/var/lib/couchbase/data/(default@N/A)->/tmp/backup(default@N/A)) done.
        2013-05-08 08:36:36,424: w0 source : couchstore-files:///opt/couchbase/var/lib/couchbase/data/(default@N/A)
        2013-05-08 08:36:36,424: w0 sink : /tmp/backup(default@N/A)
        2013-05-08 08:36:36,425: w0 : total | last | per sec
        2013-05-08 08:36:36,425: w0 batch : 1000 | 1000 | 12.3
        2013-05-08 08:36:36,425: w0 byte : 10000000 | 10000000 | 122672.8
        2013-05-08 08:36:36,425: w0 msg : 1000000 | 1000000 | 12267.3
        2013-05-08 08:36:36,425: w0 node: N/A, done; rv: 0
        .
        bucket: default, msgs transferred...
        : total | last | per sec
        batch : 1000 | 1000 | 12.1
        byte : 10000000 | 10000000 | 121190.9
        msg : 1000000 | 1000000 | 12119.1
        done

        1. uname -a
          Linux ip-10-32-243-102 3.2.0-36-virtual #57-Ubuntu SMP Tue Jan 8 22:21:19 UTC 2013 i686 i686 i386 GNU/Linux
        1. lsb_release -a
          No LSB modules are available.
          Distributor ID: Ubuntu
          Description: Ubuntu 12.04.1 LTS
          Release: 12.04
          Codename: precise
        1. python -V
          Python 2.7.3
        Show
        pavelpaulau Pavel Paulau added a comment - - edited Are you hitting this on EC2 instance with Ubuntu LTS as originally reported? Since I have absolutely clean setup with 2.0.2-789, I loaded 1M items and successfully transferred them many times: root@ip-10-32-243-102:~# /opt/couchbase/bin/cbtransfer -v -v -v couchstore-files:///opt/couchbase/var/lib/couchbase/data/ /tmp/backup 2013-05-08 08:35:14,712: mt cbtransfer... 2013-05-08 08:35:14,712: mt source : couchstore-files:///opt/couchbase/var/lib/couchbase/data/ 2013-05-08 08:35:14,712: mt sink : /tmp/backup 2013-05-08 08:35:14,712: mt opts : {'username': None, 'source_vbucket_state': 'active', 'destination_vbucket_state': 'active', 'verbose': 3, 'dry_run': False, 'extra': {'max_retry': 10.0, 'rehash': 0.0, 'data_only': 0.0, 'nmv_retry': 1.0, 'cbb_max_mb': 100000.0, 'try_xwm': 1.0, 'batch_max_bytes': 400000.0, 'report_full': 2000.0, 'batch_max_size': 1000.0, 'report': 5.0, 'design_doc_only': 0.0, 'recv_min_bytes': 4096.0} , 'single_node': False, 'bucket_destination': None, 'destination_operation': None, 'vbucket_list': None, 'threads': 4, 'key': None, 'password': None, 'id': None, 'bucket_source': None} 2013-05-08 08:35:14,713: mt source_class: <class 'pump_sfd.SFDSource'> 2013-05-08 08:35:14,904: mt sink_class: <class 'pump_bfd.BFDSink'> 2013-05-08 08:35:14,905: mt source_buckets: default 2013-05-08 08:35:14,905: mt bucket: default 2013-05-08 08:35:14,905: mt source_nodes: N/A 2013-05-08 08:35:14,906: mt enqueueing node: N/A 2013-05-08 08:35:14,907: w0 node: N/A 2013-05-08 08:35:14,988: s0 create_db: /tmp/backup/bucket-default/node-N%2FA/data-0000.cbb 2013-05-08 08:35:14,988: s0 connect_db: /tmp/backup/bucket-default/node-N%2FA/data-0000.cbb ........................................................................................................................................................................................................2013-05-08 08:36:36,424: w0 pump (couchstore-files:///opt/couchbase/var/lib/couchbase/data/(default@N/A)->/tmp/backup(default@N/A)) done. 2013-05-08 08:36:36,424: w0 source : couchstore-files:///opt/couchbase/var/lib/couchbase/data/(default@N/A) 2013-05-08 08:36:36,424: w0 sink : /tmp/backup(default@N/A) 2013-05-08 08:36:36,425: w0 : total | last | per sec 2013-05-08 08:36:36,425: w0 batch : 1000 | 1000 | 12.3 2013-05-08 08:36:36,425: w0 byte : 10000000 | 10000000 | 122672.8 2013-05-08 08:36:36,425: w0 msg : 1000000 | 1000000 | 12267.3 2013-05-08 08:36:36,425: w0 node: N/A, done; rv: 0 . bucket: default, msgs transferred... : total | last | per sec batch : 1000 | 1000 | 12.1 byte : 10000000 | 10000000 | 121190.9 msg : 1000000 | 1000000 | 12119.1 done uname -a Linux ip-10-32-243-102 3.2.0-36-virtual #57-Ubuntu SMP Tue Jan 8 22:21:19 UTC 2013 i686 i686 i386 GNU/Linux lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 12.04.1 LTS Release: 12.04 Codename: precise python -V Python 2.7.3
        Hide
        deepkaran.salooja Deepkaran Salooja added a comment -

        No, I actually got this with Centos 32bit VM (10.1.4.10) and build 2.0.2-787.

        [root@localhost ~]# uname -a
        Linux localhost.localdomain 2.6.18-308.el5xen #1 SMP Tue Feb 21 21:26:03 EST 2012 i686 i686 i386 GNU/Linux
        [root@localhost ~]# python -V
        Python 2.7.1
        [root@localhost ~]# lsb_release -a
        LSB Version: :core-4.0-ia32:core-4.0-noarch:graphics-4.0-ia32:graphics-4.0-noarch:printing-4.0-ia32:printing-4.0-noarch
        Distributor ID: CentOS
        Description: CentOS release 5.8 (Final)
        Release: 5.8
        Codename: Final
        [root@localhost ~]#

        Tested on both inhouse Ubuntu32 and EC2 ubuntu32. Works fine.

        Show
        deepkaran.salooja Deepkaran Salooja added a comment - No, I actually got this with Centos 32bit VM (10.1.4.10) and build 2.0.2-787. [root@localhost ~] # uname -a Linux localhost.localdomain 2.6.18-308.el5xen #1 SMP Tue Feb 21 21:26:03 EST 2012 i686 i686 i386 GNU/Linux [root@localhost ~] # python -V Python 2.7.1 [root@localhost ~] # lsb_release -a LSB Version: :core-4.0-ia32:core-4.0-noarch:graphics-4.0-ia32:graphics-4.0-noarch:printing-4.0-ia32:printing-4.0-noarch Distributor ID: CentOS Description: CentOS release 5.8 (Final) Release: 5.8 Codename: Final [root@localhost ~] # Tested on both inhouse Ubuntu32 and EC2 ubuntu32. Works fine.
        Hide
        deepkaran.salooja Deepkaran Salooja added a comment -

        verified with 2.0.2-807-rel

        Show
        deepkaran.salooja Deepkaran Salooja added a comment - verified with 2.0.2-807-rel

          People

          • Assignee:
            deepkaran.salooja Deepkaran Salooja
            Reporter:
            deepkaran.salooja Deepkaran Salooja
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Agile

                Gerrit Reviews

                There are no open Gerrit changes