Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-7878

cbstats does not work on port 11210 in non-root installation

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.1
    • Fix Version/s: 2.5.0
    • Component/s: installer, tools
    • Security Level: Public
    • Labels:
    • Environment:
      centos 5.6 64bit in ec2

      Description

      Download couchbase server 2.0.1-170
      Login as non-root user (test)
      Follow instruction in https://github.com/couchbase/voltron/blob/2.0.1/README-non-root.md to install couchbase server
      Start couchbase server
      Open web browser to do initial setup.
      Go to opt/couchbase/bin/tools
      Run cbstats at port 11210, failed to get info

      [test@ip-10-33-3-109 ~]$ ./opt/couchbase/bin/cbstats 10.33.3.109:11210 all
      Stats '' are not available from the requested engine.
      [test@ip-10-33-3-109 ~]$ ./opt/couchbase/bin/cbstats 10.33.3.109:11210 all -b default
      Traceback (most recent call last):
      File "/home/test/opt/couchbase/lib/python/cbstats", line 428, in ?
      main()
      File "/home/test/opt/couchbase/lib/python/cbstats", line 425, in main
      c.execute()
      File "/home/test/opt/couchbase/lib/python/clitool.py", line 58, in execute
      f[0](mc, *args[2:], **opts._dict_)
      File "/home/test/opt/couchbase/lib/python/cbstats", line 32, in g
      mc.sasl_auth_plain(bucket, password)
      File "/home/test/opt/couchbase/lib/python/mc_bin_client.py", line 262, in sasl_auth_plain
      return self.sasl_auth_start('PLAIN', '\0'.join([foruser, user, password]))
      File "/home/test/opt/couchbase/lib/python/mc_bin_client.py", line 258, in sasl_auth_start
      return self._doCmd(memcacheConstants.CMD_SASL_AUTH, mech, data)
      File "/home/test/opt/couchbase/lib/python/mc_bin_client.py", line 104, in _doCmd
      return self._handleSingleResponse(opaque)
      File "/home/test/opt/couchbase/lib/python/mc_bin_client.py", line 97, in _handleSingleResponse
      cmd, opaque, cas, keylen, extralen, data = self._handleKeyedResponse(myopaque)
      File "/home/test/opt/couchbase/lib/python/mc_bin_client.py", line 89, in _handleKeyedResponse
      cmd, errcode, opaque, cas, keylen, extralen, rv = self._recvMsg()
      File "/home/test/opt/couchbase/lib/python/mc_bin_client.py", line 71, in _recvMsg
      raise exceptions.EOFError("Got empty data (remote died?).")
      EOFError: Got empty data (remote died?).
      [test@ip-10-33-3-109 ~]$ ./opt/couchbase/bin/cbstats 10.33.3.109:11210 all -b sasl -p password
      Traceback (most recent call last):
      File "/home/test/opt/couchbase/lib/python/cbstats", line 428, in ?
      main()
      File "/home/test/opt/couchbase/lib/python/cbstats", line 425, in main
      c.execute()
      File "/home/test/opt/couchbase/lib/python/clitool.py", line 58, in execute
      f[0](mc, *args[2:], **opts._dict_)
      File "/home/test/opt/couchbase/lib/python/cbstats", line 32, in g
      mc.sasl_auth_plain(bucket, password)
      File "/home/test/opt/couchbase/lib/python/mc_bin_client.py", line 262, in sasl_auth_plain
      return self.sasl_auth_start('PLAIN', '\0'.join([foruser, user, password]))
      File "/home/test/opt/couchbase/lib/python/mc_bin_client.py", line 258, in sasl_auth_start
      return self._doCmd(memcacheConstants.CMD_SASL_AUTH, mech, data)
      File "/home/test/opt/couchbase/lib/python/mc_bin_client.py", line 104, in _doCmd
      return self._handleSingleResponse(opaque)
      File "/home/test/opt/couchbase/lib/python/mc_bin_client.py", line 97, in _handleSingleResponse
      cmd, opaque, cas, keylen, extralen, data = self._handleKeyedResponse(myopaque)
      File "/home/test/opt/couchbase/lib/python/mc_bin_client.py", line 89, in _handleKeyedResponse
      cmd, errcode, opaque, cas, keylen, extralen, rv = self._recvMsg()
      File "/home/test/opt/couchbase/lib/python/mc_bin_client.py", line 71, in _recvMsg
      raise exceptions.EOFError("Got empty data (remote died?).")
      EOFError: Got empty data (remote died?).

      Load items to both bucket, items loaed

      grep process

      [test@ip-10-33-3-109 ~]$ ps aux | grep memcached
      test 2047 0.3 6.0 706412 474232 ? Ssl 02:09 3:21 /home/test/opt/couchbase/bin/memcached.bin -X /home/test/opt/couchbase/lib/memcached/stdin_term_handler.so -X /home/test/opt/couchbase/lib/memcached/file_logger.so,cyclesize=104857600;sleeptime=19;filename=/home/test/opt/couchbase/var/lib/couchbase/logs/memcached.log -l 0.0.0.0:11210,0.0.0.0:11209:1000 -p 11210 -E /home/test/opt/couchbase/lib/memcached/bucket_engine.so -B binary -r -c 10000 -e admin=_admin;default_bucket_name=default;auto_create=false
      test 6963 0.0 0.0 61180 812 pts/3 S+ 18:09 0:00 grep memcached
      [test@ip-10-33-3-109 ~]$ ps aux | grep couchbase
      test 1881 8.1 5.8 850812 456204 pts/1 Sl+ 02:07 78:47 /home/test/opt/couchbase/lib/erlang/erts-5.8.5/bin/beam.smp -A 16 -sbt u -P 327680 -K true -MMmcs 30 – -root /home/test/opt/couchbase/lib/erlang -progname erl – -home /home/test – -smp enable -setcookie nocookie -kernel inet_dist_listen_min 21100 inet_dist_listen_max 21299 error_logger false -sasl sasl_error_logger false -run ns_bootstrap – -couch_ini /home/test/opt/couchbase/etc/couchdb/default.ini /home/test/opt/couchbase/etc/couchdb/default.d/capi.ini /home/test/opt/couchbase/etc/couchdb/default.d/geocouch.ini /home/test/opt/couchbase/etc/couchdb/local.ini -ns_server config_path "/home/test/opt/couchbase/etc/couchbase/static_config" -ns_server pidfile "/home/test/opt/couchbase/var/lib/couchbase/couchbase-server.pid" -ns_server nodefile "/home/test/opt/couchbase/var/lib/couchbase/couchbase-server.node" -ns_server cookiefile "/home/test/opt/couchbase/var/lib/couchbase/couchbase-server.cookie" -ns_server enable_mlockall true
      test 1892 0.0 0.0 10572 568 ? S 02:07 0:00 /home/test/opt/couchbase/lib/erlang/erts-5.8.5/bin/epmd -daemon
      test 1931 0.0 0.0 3800 548 ? Ss 02:07 0:00 /home/test/opt/couchbase/lib/erlang/lib/os_mon-2.2.7/priv/bin/memsup
      test 1932 0.0 0.0 3796 412 ? Ss 02:07 0:00 /home/test/opt/couchbase/lib/erlang/lib/os_mon-2.2.7/priv/bin/cpu_sup
      test 1933 0.0 0.0 38348 2016 ? Ss 02:07 0:00 /home/test/opt/couchbase/lib/erlang/lib/ssl-4.1.6/priv/bin/ssl_esock
      test 2046 0.0 0.0 158172 3504 ? Ssl 02:09 0:34 /home/test/opt/couchbase/bin/moxi.bin -Z port_listen=11211,default_bucket_name=default,downstream_max=1024,downstream_conn_max=4,connect_max_errors=5,connect_retry_interval=30000,connect_timeout=400,auth_timeout=100,cycle=200,downstream_conn_queue_timeout=200,downstream_timeout=5000,wait_queue_timeout=200 -z url=http://127.0.0.1:8091/pools/default/saslBucketsStreaming -p 0 -Y y -O stderr
      test 2047 0.3 6.0 706412 474232 ? Ssl 02:09 3:21 /home/test/opt/couchbase/bin/memcached.bin -X /home/test/opt/couchbase/lib/memcached/stdin_term_handler.so -X /home/test/opt/couchbase/lib/memcached/file_logger.so,cyclesize=104857600;sleeptime=19;filename=/home/test/opt/couchbase/var/lib/couchbase/logs/memcached.log -l 0.0.0.0:11210,0.0.0.0:11209:1000 -p 11210 -E /home/test/opt/couchbase/lib/memcached/bucket_engine.so -B binary -r -c 10000 -e admin=_admin;default_bucket_name=default;auto_create=false
      test 2073 0.0 0.0 6016 580 ? Ss 02:09 0:07 /home/test/opt/couchbase/bin/sigar_port.bin
      test 6032 0.0 0.0 13612 920 pts/0 T 15:03 0:00 /home/test/opt/couchbase/bin/sqlite3.bin
      test 6965 0.0 0.0 61180 812 pts/3 S+ 18:09 0:00 grep couchbase
      [test@ip-10-

      Moxi port works ok

      [test@ip-10-33-3-109 ~]$ ./opt/couchbase/bin/cbstats 10.118.163.79:11211 all -b sasl -p password
      accepting_conns: 1
      auth_cmds: 3078
      auth_errors: 0
      bucket_active_conns: 1
      bucket_conns: 12
      bytes: 222778816
      bytes_read: 184970587
      bytes_written: 2074537677
      cas_badval: 0
      cas_hits: 0
      cas_misses: 0
      cmd_flush: 0
      cmd_get: 1
      cmd_set: 14174
      conn_yields: 14895
      connection_structures: 1006
      curr_connections: 26
      curr_conns_on_port_11209: 19
      curr_conns_on_port_11210: 5
      curr_items: 164042
      curr_items_tot: 314174
      curr_temp_items: 0
      daemon_connections: 4
      decr_hits:

      Link to collect info from 2 nodes

      https://s3.amazonaws.com/packages.couchbase/collect_info/2_0_1/201303/2nodes_201-170_cbstats_failed_11210_non_root_20130306.tgz

      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        thuan Thuan Nguyen created issue -
        Hide
        farshid Farshid Ghods (Inactive) added a comment -

        most likelty memcached is not listening in 0.0.0.0

        Tony,

        please post instructions on how anyone can install with non-root and then assign to ep-engine team

        Show
        farshid Farshid Ghods (Inactive) added a comment - most likelty memcached is not listening in 0.0.0.0 Tony, please post instructions on how anyone can install with non-root and then assign to ep-engine team
        anil Anil Kumar made changes -
        Field Original Value New Value
        Rank Ranked higher
        maria Maria McDuff (Inactive) made changes -
        Assignee Bin Cui [ bcui ] Thuan Nguyen [ thuan ]
        maria Maria McDuff (Inactive) made changes -
        Fix Version/s 2.0.2 [ 10418 ]
        maria Maria McDuff (Inactive) made changes -
        Summary cbstats does not work on port 11210 in non-root installation [2.0.2 DOC] cbstats does not work on port 11210 in non-root installation
        maria Maria McDuff (Inactive) made changes -
        Assignee Thuan Nguyen [ thuan ] Bin Cui [ bcui ]
        Hide
        maria Maria McDuff (Inactive) added a comment -

        Bin, can you take a look at this issue?

        Show
        maria Maria McDuff (Inactive) added a comment - Bin, can you take a look at this issue?
        maria Maria McDuff (Inactive) made changes -
        Summary [2.0.2 DOC] cbstats does not work on port 11210 in non-root installation [2.0.2 RN] cbstats does not work on port 11210 in non-root installation
        maria Maria McDuff (Inactive) made changes -
        Component/s documentation [ 10012 ]
        Hide
        kzeller kzeller added a comment -

        Added to RN 2.0.2 Can update if becomes fix:

        <command>cbstats</command> will not work on port 11210 if you installed Couchbase Server without root-level permissions.

        Show
        kzeller kzeller added a comment - Added to RN 2.0.2 Can update if becomes fix: <command>cbstats</command> will not work on port 11210 if you installed Couchbase Server without root-level permissions.
        kzeller kzeller made changes -
        Summary [2.0.2 RN] cbstats does not work on port 11210 in non-root installation [Doc'd] cbstats does not work on port 11210 in non-root installation
        kzeller kzeller made changes -
        Rank Ranked higher
        anil Anil Kumar made changes -
        Fix Version/s 2.1 [ 10414 ]
        Fix Version/s 2.0.2 [ 10418 ]
        kzeller kzeller made changes -
        Component/s documentation [ 10012 ]
        Hide
        kzeller kzeller added a comment -

        removing doc component as already in doc for 2.1.0

        Save as open issue for Eng/QA

        Show
        kzeller kzeller added a comment - removing doc component as already in doc for 2.1.0 Save as open issue for Eng/QA
        wayne Wayne Siu made changes -
        Assignee Bin Cui [ bcui ] Thuan Nguyen [ thuan ]
        Hide
        wayne Wayne Siu added a comment - - edited

        Tony, can you rerun the test using the steps listed below? Thanks.

        Installation steps:

        • rpm2cpio couchbase-server-enterprise_x86_64_2.1.0-718-rel.rpm | cpio --extract --make-directories --no-absolute-filenames
        • cd opt/couchbase/
        • copy couchbase_init.d to etc/couchbase_init.d, and remove command used by root.
        • copy reloc_wrapper.sh to bin/install/reloc_wrapper.sh, and remove references to pwd.
        • ./bin/install/reloc.sh /home/abhinav/opt/couchbase
        • rm -f /home/username/etc/init.d/couchbase-server
        • ln -s /home/username/opt/couchbase/etc/couchbase_init.d /home/username/etc/init.d/couchbase-server
        • ln -s /home/username/couchbase/logs /home/username/opt/couchbase/var/lib/couchbase/logs

        Then to start couchbase-server:

        • /home/username/etc/init.d/couchbase-server start
          Starting couchbase-serverchown: changing ownership of `/home/username/opt/couchbase/var/lib/couchbase/couchbase-server.pid': Operation not permitted
          chown: changing ownership of `/home/username/opt/couchbase/var/lib/couchbase/couchbase-server.node': Operation not permitted
          chown: changing ownership of `/home/username/opt/couchbase/var/lib/couchbase/couchbase-server.cookie': Operation not permitted
          [ OK ]

        Running cbstats:

        [username@orange-11602 ~]$ opt/couchbase/bin/cbstats localhost:11210 raw memory
        bytes: 31727960
        ep_kv_size: 0
        ep_max_data_size: 10485760000
        ep_mem_high_wat: 8912896000
        ep_mem_low_wat: 7864320000
        ep_mem_tracker_enabled: true
        ep_oom_errors: 0
        ep_overhead: 27347928
        ep_tmp_oom_errors: 0
        ep_value_size: 0
        mem_used: 31727960
        tcmalloc_current_thread_cache_bytes: 962520
        tcmalloc_max_thread_cache_bytes: 4194304
        tcmalloc_unmapped_bytes: 0
        total_allocated_bytes: 50362584
        total_fragmentation_bytes: 3573544
        total_free_bytes: 589824
        total_heap_bytes: 54525952

        Show
        wayne Wayne Siu added a comment - - edited Tony, can you rerun the test using the steps listed below? Thanks. Installation steps: rpm2cpio couchbase-server-enterprise_x86_64_2.1.0-718-rel.rpm | cpio --extract --make-directories --no-absolute-filenames cd opt/couchbase/ copy couchbase_init.d to etc/couchbase_init.d, and remove command used by root. copy reloc_wrapper.sh to bin/install/reloc_wrapper.sh, and remove references to pwd. ./bin/install/reloc.sh /home/abhinav/opt/couchbase rm -f /home/username/etc/init.d/couchbase-server ln -s /home/username/opt/couchbase/etc/couchbase_init.d /home/username/etc/init.d/couchbase-server ln -s /home/username/couchbase/logs /home/username/opt/couchbase/var/lib/couchbase/logs Then to start couchbase-server: /home/username/etc/init.d/couchbase-server start Starting couchbase-serverchown: changing ownership of `/home/username/opt/couchbase/var/lib/couchbase/couchbase-server.pid': Operation not permitted chown: changing ownership of `/home/username/opt/couchbase/var/lib/couchbase/couchbase-server.node': Operation not permitted chown: changing ownership of `/home/username/opt/couchbase/var/lib/couchbase/couchbase-server.cookie': Operation not permitted [ OK ] Running cbstats: [username@orange-11602 ~] $ opt/couchbase/bin/cbstats localhost:11210 raw memory bytes: 31727960 ep_kv_size: 0 ep_max_data_size: 10485760000 ep_mem_high_wat: 8912896000 ep_mem_low_wat: 7864320000 ep_mem_tracker_enabled: true ep_oom_errors: 0 ep_overhead: 27347928 ep_tmp_oom_errors: 0 ep_value_size: 0 mem_used: 31727960 tcmalloc_current_thread_cache_bytes: 962520 tcmalloc_max_thread_cache_bytes: 4194304 tcmalloc_unmapped_bytes: 0 total_allocated_bytes: 50362584 total_fragmentation_bytes: 3573544 total_free_bytes: 589824 total_heap_bytes: 54525952
        perry Perry Krug made changes -
        Labels non-root
        maria Maria McDuff (Inactive) made changes -
        Fix Version/s 2.5.0 [ 11200 ]
        Fix Version/s 3.0 [ 10414 ]
        maria Maria McDuff (Inactive) made changes -
        Summary [Doc'd] cbstats does not work on port 11210 in non-root installation cbstats does not work on port 11210 in non-root installation
        maria Maria McDuff (Inactive) made changes -
        Assignee Thuan Nguyen [ thuan ] Aruna Piravi [ apiravi ]
        Hide
        maria Maria McDuff (Inactive) added a comment -

        aruna,
        pls try in latest 2.5.0 build using the steps above and update this bug.

        Show
        maria Maria McDuff (Inactive) added a comment - aruna, pls try in latest 2.5.0 build using the steps above and update this bug.
        Hide
        wayne Wayne Siu added a comment -

        Aruna,
        The steps listed above were for releases prior to officially supporting the non-root/non-sudo feature. We should now test using the instructions documented in our doc to see if all the tools work (under non-root installation).

        Show
        wayne Wayne Siu added a comment - Aruna, The steps listed above were for releases prior to officially supporting the non-root/non-sudo feature. We should now test using the instructions documented in our doc to see if all the tools work (under non-root installation).
        Hide
        apiravi Aruna Piravi added a comment -

        Wayne and Maria, I figured out the correct documentation for the non sudo, non root installation (there are many versions available on JIRA) but this seems to be latest and is working - http://docs.couchbase.com/couchbase-manual-2.2/#installing-on-centosrhel-as-non-root-non-sudo. I arrived at this through the install script automated by Abhinav.

        Results -

        Tested on both v2.2.0 (build821) and 2.5.0 (build 839). cbstats works fine for non-root user without sudo privileges.

        [nonroot@olive-012 ~]$ opt/couchbase/bin/cbstats 10.3.3.118:11210 raw memory
        bytes: 33036360
        ep_kv_size: 10866064
        ep_max_data_size: 3145728000
        ep_mem_high_wat: 2673868800
        ep_mem_low_wat: 2359296000
        ep_mem_tracker_enabled: true
        ep_oom_errors: 0
        ep_overhead: 18214476
        ep_tmp_oom_errors: 0
        ep_value_size: 5659470
        mem_used: 33036360
        tcmalloc_current_thread_cache_bytes: 3180680
        tcmalloc_max_thread_cache_bytes: 4194304
        tcmalloc_unmapped_bytes: 0
        total_allocated_bytes: 107614656
        total_fragmentation_bytes: 6688320
        total_free_bytes: 2088960
        total_heap_bytes: 116391936

        Hence marking this item as resolved. Thanks.

        Show
        apiravi Aruna Piravi added a comment - Wayne and Maria, I figured out the correct documentation for the non sudo, non root installation (there are many versions available on JIRA) but this seems to be latest and is working - http://docs.couchbase.com/couchbase-manual-2.2/#installing-on-centosrhel-as-non-root-non-sudo . I arrived at this through the install script automated by Abhinav. Results - Tested on both v2.2.0 (build821) and 2.5.0 (build 839). cbstats works fine for non-root user without sudo privileges. [nonroot@olive-012 ~] $ opt/couchbase/bin/cbstats 10.3.3.118:11210 raw memory bytes: 33036360 ep_kv_size: 10866064 ep_max_data_size: 3145728000 ep_mem_high_wat: 2673868800 ep_mem_low_wat: 2359296000 ep_mem_tracker_enabled: true ep_oom_errors: 0 ep_overhead: 18214476 ep_tmp_oom_errors: 0 ep_value_size: 5659470 mem_used: 33036360 tcmalloc_current_thread_cache_bytes: 3180680 tcmalloc_max_thread_cache_bytes: 4194304 tcmalloc_unmapped_bytes: 0 total_allocated_bytes: 107614656 total_fragmentation_bytes: 6688320 total_free_bytes: 2088960 total_heap_bytes: 116391936 Hence marking this item as resolved. Thanks.
        Hide
        apiravi Aruna Piravi added a comment -

        Verified on couchbase server v2.5.0 and v2.2.0

        Show
        apiravi Aruna Piravi added a comment - Verified on couchbase server v2.5.0 and v2.2.0
        apiravi Aruna Piravi made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        apiravi Aruna Piravi made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Sprint Status Current Sprint [ 10027 ]

          People

          • Assignee:
            apiravi Aruna Piravi
            Reporter:
            thuan Thuan Nguyen
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes