Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-42291

Cbcollect_info dumps all output into stderr by default

    XMLWordPrintable

Details

    • Bug
    • Resolution: Won't Fix
    • Minor
    • 7.0.0
    • Cheshire-Cat
    • ns_server
    • Enterprise Edition 7.0.0 build 3444
    • Untriaged
    • Centos 64-bit
    • 1
    • No

    Description

      Build: 7.0.0-3444

      Triggering cbcollect_info dumps all console logs into stderr.

      Expectation:

      • collection status with [OK] should be redirected to stdout
      • status only those are !=0 should be printed via stderr.

      Sample logs collection outputs:

       

      # Cbcollect with stderr re-direction to /dev/null (No outputs seen on console)
      [root@node2-cb601-centos7 bin]# ./cbcollect_info /tmp/test.zip 2>/dev/null
      Checking for server guts in /opt/couchbase/var/lib/couchbase/initargs...
       
      # Cbcollect with stdout only redirection to /dev/null (All logs exposed via stderr)
      [root@node2-cb601-centos7 bin]# ./cbcollect_info /tmp/test.zip 1>/dev/null
      Using temporary dir /tmp
      Log Redaction () - OK
      uname (uname -a) - OK
      time and TZ (date; date -u) - OK
      ntp time (ntpdate -q pool.ntp.org || nc time.nist.gov 13 || netcat time.nist.gov 13) - Exit code 127
      ntp peers (ntpq -p) - Exit code 127
      raw /etc/sysconfig/clock (cat /etc/sysconfig/clock) - Exit code 1
      raw /etc/timezone (cat /etc/timezone) - Exit code 1
      System Hardware (lshw -json || lshw) - Exit code 127
      Process list snapshot (export TERM=''; top -Hb -n1 || top -H n1) - OK
      Process list (ps -AwwL -o user,pid,lwp,ppid,nlwp,pcpu,maj_flt,min_flt,pri,nice,vsize,rss,tty,stat,wchan:12,start,bsdtime,comm,command) - OK
      Raw /proc/buddyinfo (cat /proc/buddyinfo) - OK
      Raw /proc/meminfo (cat /proc/meminfo) - OK
      Raw /proc/pagetypeinfo (cat /proc/pagetypeinfo) - OK
      Raw /proc/zoneinfo (cat /proc/zoneinfo) - OK
      Raw /proc/vmstat (cat /proc/vmstat) - OK
      Raw /proc/mounts (cat /proc/mounts) - OK
      Raw /proc/partitions (cat /proc/partitions) - OK
      Raw /proc/diskstats (cat /proc/diskstats; echo '') - OK
      Taking sample 2 after 1.000000 seconds - OK
      Taking sample 3 after 1.000000 seconds - OK
      Taking sample 4 after 1.000000 seconds - OK
      Taking sample 5 after 1.000000 seconds - OK
      Taking sample 6 after 1.000000 seconds - OK
      Taking sample 7 after 1.000000 seconds - OK
      Taking sample 8 after 1.000000 seconds - OK
      Taking sample 9 after 1.000000 seconds - OK
      Taking sample 10 after 1.000000 seconds - OK
      Raw /proc/interrupts (cat /proc/interrupts) - OK
      Swap configuration (free -t) - OK
      Swap configuration (swapon -s) - OK
      Kernel modules (lsmod) - OK
      Distro version (cat /etc/redhat-release) - OK
      Distro version (cat /etc/oracle-release) - Exit code 1
      Distro version (lsb_release -a) - Exit code 127
      Distro version (cat /etc/SuSE-release) - Exit code 1
      Distro version (cat /etc/issue) - OK
      Distro version (cat /etc/os-release) - OK
      Distro version (cat /etc/system-release) - OK
      Installed software (rpm -qa) - OK
      Ksplice updates (uptrack-show) - Exit code 127
      Hot fix list (rpm -V couchbase-server) - Exit code 1
      Installed software (COLUMNS=300 dpkg -l) - Exit code 127
      Hot fix list (COLUMNS=300 dpkg -V couchbase-server) - Exit code 127
      Extended iostat (iostat -x -p ALL 1 10 || iostat -x 1 10) - Exit code 127
      Core dump settings (find /proc/sys/kernel -type f -name '*core*' -print -exec cat '{}' ';') - OK
      sysctl settings (sysctl -a) - OK
      Relevant lsof output (echo moxi memcached beam.smp couch_compact godu sigar_port cbq-engine indexer projector goxdcr cbft eventing-producer eventing-consumer | xargs -n1 pgrep | xargs -n1 -r -- lsof -n -p) - Exit code 127
      LVM info (lvdisplay) - OK
      LVM info (vgdisplay) - OK
      LVM info (pvdisplay) - OK
      Block device queue settings (find /sys/block/*/queue -type f | xargs grep -vH xxxx | sort) - OK
      Network configuration (ifconfig -a) - OK
      Taking sample 2 after 10.000000 seconds - OK
      Network configuration (echo link addr neigh rule route netns | xargs -n1 -- sh -x -c 'ip $1 list' --) - OK
      Raw /proc/net/dev (cat /proc/net/dev) - OK
      Network link statistics (ip -s link) - OK
      Network status (netstat -anp || netstat -an) - OK
      Network routing table (netstat -rn) - OK
      Network socket statistics (ss -an) - OK
      Extended socket statistics (ss -an --info --processes --memory --options) - OK
      Arp cache (arp -na) - OK
      Iptables dump (iptables-save) - OK
      Raw /etc/hosts (cat /etc/hosts) - OK
      Raw /etc/resolv.conf (cat /etc/resolv.conf) - OK
      Raw /etc/nsswitch.conf (cat /etc/nsswitch.conf) - OK
      Filesystem (df -ha) - OK
      System activity reporter (sar 1 10) - Exit code 127
      System paging activity (vmstat 1 10) - OK
      System uptime (uptime) - OK
      Last logins of users and ttys (last -x || last) - OK
      couchbase user definition (getent passwd couchbase) - OK
      couchbase user limits (su couchbase -s /bin/sh -c "ulimit -a") - OK
      Interrupt status (intrstat 1 10) - Exit code 127
      Processor status (mpstat 1 10) - Exit code 127
      System log (cat /var/adm/messages) - Exit code 1
      Raw /proc/uptime (cat /proc/uptime) - OK
      Systemd journal (journalctl | gzip -c > systemd_journal.gz) - OK
      All logs (tar cz /var/log/syslog* /var/log/dmesg /var/log/messages* /var/log/daemon* /var/log/debug* /var/log/kern.log* 2>/dev/null) - Exit code 2
      Relevant proc data (echo moxi memcached beam.smp couch_compact godu sigar_port cbq-engine indexer projector goxdcr cbft eventing-producer eventing-consumer | xargs -n1 pgrep | xargs -n1 -- sh -c 'echo $1; cat /proc/$1/status; cat /proc/$1/limits; cat /proc/$1/smaps; cat /proc/$1/numa_maps; cat /proc/$1/task/*/sched; echo' --) - OK
      Processes' environment (echo moxi memcached beam.smp couch_compact godu sigar_port cbq-engine indexer projector goxdcr cbft eventing-producer eventing-consumer | xargs -n1 pgrep | xargs -n1 -- sh -c 'echo $1; ( cat /proc/$1/environ | tr \\0 \\n | egrep -v ^CB_MASTER_PASSWORD=\|^CBAUTH_REVRPC_URL=); echo' --) - OK
      Processes' stack (for program in moxi memcached beam.smp couch_compact godu sigar_port cbq-engine indexer projector goxdcr cbft eventing-producer eventing-consumer; do for thread in $(pgrep --lightweight $program); do echo $program/$thread:; cat /proc/$thread/stack; echo; done; done) - OK
      NUMA data (numactl --hardware) - Exit code 127
      NUMA data (numactl --show) - Exit code 127
      NUMA data (cat /sys/devices/system/node/node*/numastat) - OK
      Kernel log buffer (dmesg -T || dmesg -H || dmesg) - OK
      Transparent Huge Pages data (cat /sys/kernel/mm/transparent_hugepage/enabled) - OK
      Transparent Huge Pages data (cat /sys/kernel/mm/transparent_hugepage/defrag) - OK
      Transparent Huge Pages data (cat /sys/kernel/mm/redhat_transparent_hugepage/enabled) - Exit code 1
      Transparent Huge Pages data (cat /sys/kernel/mm/redhat_transparent_hugepage/defrag) - Exit code 1
      Network statistics (netstat -s) - OK
      Full raw netstat (cat /proc/net/netstat) - OK
      CPU throttling info (echo /sys/devices/system/cpu/cpu*/thermal_throttle/* | xargs -n1 -- sh -c 'echo $1; cat $1' --) - Exit code 123
      Raw PID 1 scheduler /proc/1/sched (cat /proc/1/sched | head -n 1) - OK
      Raw PID 1 control groups /proc/1/cgroup (cat /proc/1/cgroup) - OK
      Found dump-guts: /opt/couchbase/bin/dump-guts
      product diag header () - OK
      Found escript-wrapper: /opt/couchbase/bin/escript-wrapper
      Found dump-stats: /opt/couchbase/bin/dump-stats
      Name Service Switch hosts database info for 10.112.191.101 (['getent', 'ahosts', '10.112.191.101']) - OK
      Name Service Switch hosts database info for 10.112.191.102 (['getent', 'ahosts', '10.112.191.102']) - OK
      Result of query statement 'SELECT * FROM system:datastores' (['curl', '-sS', '--proxy', '', '-K-', 'http://127.0.0.1:8093/query/service?statement=SELECT%20%2A%20FROM%20system%3Adatastores']) - OK
      Result of query statement 'SELECT * FROM system:namespaces' (['curl', '-sS', '--proxy', '', '-K-', 'http://127.0.0.1:8093/query/service?statement=SELECT%20%2A%20FROM%20system%3Anamespaces']) - OK
      Result of query statement 'SELECT * FROM system:keyspaces' (['curl', '-sS', '--proxy', '', '-K-', 'http://127.0.0.1:8093/query/service?statement=SELECT%20%2A%20FROM%20system%3Akeyspaces']) - OK
      Result of query statement 'SELECT * FROM system:indexes' (['curl', '-sS', '--proxy', '', '-K-', 'http://127.0.0.1:8093/query/service?statement=SELECT%20%2A%20FROM%20system%3Aindexes']) - OK
      Index definitions are:  (['curl', '-sS', '--proxy', '', '-K-', 'http://127.0.0.1:9102/getIndexStatus']) - OK
      Indexer settings are:  (['curl', '-sS', '--proxy', '', '-K-', 'http://127.0.0.1:9102/settings']) - OK
      Indexer stats are:  (['curl', '-sS', '--proxy', '', '-K-', 'http://127.0.0.1:9102/stats?partition=true']) - OK
      Index storage stats are:  (['curl', '-sS', '--proxy', '', '-K-', 'http://127.0.0.1:9102/stats/storage']) - OK
      MOI allocator stats are:  (['curl', '-sS', '--proxy', '', '-K-', 'http://127.0.0.1:9102/stats/storage/mm']) - OK
      Indexer Go routine dump:  (['curl', '-sS', '--proxy', '', '-K-', 'http://127.0.0.1:9102/debug/pprof/goroutine?debug=1']) - OK
      Indexer Rebalance Tokens:  (['curl', '-sS', '--proxy', '', '-K-', 'http://127.0.0.1:9102/listRebalanceTokens']) - OK
      Indexer Metadata Tokens:  (['curl', '-sS', '--proxy', '', '-K-', 'http://127.0.0.1:9102/listMetadataTokens']) - OK
      Indexer CPU Profile:  (['curl', '-sS', '--proxy', '', '-K-', 'http://127.0.0.1:9102/debug/pprof/profile']) - OK
      Indexer Memory Profile:  (['curl', '-sS', '--proxy', '', '-K-', 'http://127.0.0.1:9102/debug/pprof/heap?debug=1']) - OK
      GoXDCR Go routine dump  (['curl', '-sS', '--proxy', '', '-K-', 'http://127.0.0.1:9998/debug/pprof/goroutine?debug=1']) - OK
      Directory structure (['ls', '-lRai', '/opt/couchbase']) - OK
      Database directory structure (['ls', '-lRai', '/opt/couchbase/var/lib/couchbase/data']) - OK
      Index directory structure (['ls', '-lRai', '/opt/couchbase/var/lib/couchbase/data']) - OK
      couch_dbinfo (['find', '/opt/couchbase/var/lib/couchbase/data', '-type', 'f', '-name', '*.couch.*', '-exec', 'couch_dbinfo', '{}', '+']) - OK
      Database directory filefrag info (['find', '/opt/couchbase/var/lib/couchbase/data', '-type', 'f', '-exec', 'filefrag', '-v', '{}', '+']) - OK
      Index directory filefrag info (['find', '/opt/couchbase/var/lib/couchbase/data', '-type', 'f', '-exec', 'filefrag', '-v', '{}', '+']) - OK
      Version file (cat '/opt/couchbase/VERSION.txt') - OK
      Variant file (cat '/opt/couchbase/VARIANT.txt') - OK
      Manifest file (cat '/opt/couchbase/manifest.txt') - Exit code 1
      Manifest file (cat '/opt/couchbase/manifest.xml') - OK
      Couchbase config () - OK
      Couchbase static config () - OK
      Memcached logs (['sh', '-c', 'cd "$1"; for file in $(ls -tr memcached.log.*); do cat "$file"; done', '--', '/opt/couchbase/var/lib/couchbase/logs']) - OK
      Ini files (['sh', '-c', 'for i in "$@"; do echo "file: $i"; cat "$i"; done', '--', '/opt/couchbase/etc/couchdb/default.ini', '/opt/couchbase/etc/couchdb/default.d/capi.ini', '/opt/couchbase/etc/couchdb/default.d/geocouch.ini', '/opt/couchbase/etc/couchdb/local.ini']) - OK
      couchbase diags (['curl', '-sS', '--proxy', '', '-K-', 'http://127.0.0.1:8091/diag']) - OK
      master events (['curl', '-sS', '--proxy', '', '-K-', 'http://127.0.0.1:8091/diag/masterEvents?o=1']) - OK
      ale configuration (['curl', '-sS', '--proxy', '', '-K-', 'http://127.0.0.1:8091/diag/ale']) - OK
      couchbase logs (debug.log) (cbbrowse_logs debug.log) - OK
      couchbase logs (info.log) (cbbrowse_logs info.log) - OK
      couchbase logs (error.log) (cbbrowse_logs error.log) - OK
      couchbase logs (couchdb.log) (cbbrowse_logs couchdb.log) - OK
      couchbase logs (xdcr_target.log) (cbbrowse_logs xdcr_target.log) - OK
      couchbase logs (prometheus.log) (cbbrowse_logs prometheus.log) - OK
      couchbase logs (views.log) (cbbrowse_logs views.log) - OK
      couchbase logs (mapreduce_errors.log) (cbbrowse_logs mapreduce_errors.log) - OK
      couchbase logs (stats.log) (cbbrowse_logs stats.log) - OK
      couchbase logs (babysitter.log) (cbbrowse_logs babysitter.log) - OK
      couchbase logs (reports.log) (cbbrowse_logs reports.log) - OK
      couchbase logs (http_access.log) (cbbrowse_logs http_access.log) - OK
      couchbase logs (http_access_internal.log) (cbbrowse_logs http_access_internal.log) - OK
      couchbase logs (ns_couchdb.log) (cbbrowse_logs ns_couchdb.log) - OK
      couchbase logs (goxdcr.log) (cbbrowse_logs goxdcr.log) - OK
      couchbase logs (query.log) (cbbrowse_logs query.log) - OK
      couchbase logs (projector.log) (cbbrowse_logs projector.log) - OK
      couchbase logs (indexer.log) (cbbrowse_logs indexer.log) - OK
      couchbase logs (fts.log) (cbbrowse_logs fts.log) - Exit code 1
      couchbase logs (metakv.log) (cbbrowse_logs metakv.log) - OK
      couchbase logs (json_rpc.log) (cbbrowse_logs json_rpc.log) - OK
      couchbase logs (eventing.log) (cbbrowse_logs eventing.log) - Exit code 1
      couchbase logs (analytics_info.log) (cbbrowse_logs analytics_info.log) - Exit code 1
      couchbase logs (analytics_debug.log) (cbbrowse_logs analytics_debug.log) - Exit code 1
      couchbase logs (analytics_shutdown.log) (cbbrowse_logs analytics_shutdown.log) - Exit code 1
      couchbase logs (analytics_error.log) (cbbrowse_logs analytics_error.log) - Exit code 1
      couchbase logs (analytics_warn.log) (cbbrowse_logs analytics_warn.log) - Exit code 1
      couchbase logs (analytics_dcpdebug.log) (cbbrowse_logs analytics_dcpdebug.log) - Exit code 1
      couchbase logs (analytics_trace.json) (cbbrowse_logs analytics_trace.json) - Exit code 1
      couchbase logs (analytics_access.log) (cbbrowse_logs analytics_access.log) - Exit code 1
      couchbase logs (analytics_cbas_debug.log) (cbbrowse_logs analytics_cbas_debug.log) - Exit code 1
      couchbase logs (indexer_stats.log) (cbbrowse_logs indexer_stats.log) - OK
      couchbase logs (backup_service.log) (cbbrowse_logs backup_service.log) - Exit code 1
      memcached stats all (['cbstats', '-a', '127.0.0.1:11209', 'all', '-u', '@ns_server']) - OK
      memcached stats checkpoint (['cbstats', '-a', '127.0.0.1:11209', 'checkpoint', '-u', '@ns_server']) - OK
      memcached stats collections (['cbstats', '-a', '127.0.0.1:11209', 'collections', '-u', '@ns_server']) - OK
      memcached stats collections-details (['cbstats', '-a', '127.0.0.1:11209', 'collections-details', '-u', '@ns_server']) - OK
      memcached stats config (['cbstats', '-a', '127.0.0.1:11209', 'config', '-u', '@ns_server']) - OK
      memcached stats dcp (['cbstats', '-a', '127.0.0.1:11209', 'dcp', '-u', '@ns_server']) - OK
      memcached stats dcpagg (['cbstats', '-a', '127.0.0.1:11209', 'dcpagg', '-u', '@ns_server']) - OK
      memcached stats ['diskinfo', 'detail'] (['cbstats', '-a', '127.0.0.1:11209', 'diskinfo', 'detail', '-u', '@ns_server']) - OK
      memcached stats ['dispatcher', 'logs'] (['cbstats', '-a', '127.0.0.1:11209', 'dispatcher', 'logs', '-u', '@ns_server']) - OK
      memcached stats eviction (['cbstats', '-a', '127.0.0.1:11209', 'eviction', '-u', '@ns_server']) - OK
      memcached stats failovers (['cbstats', '-a', '127.0.0.1:11209', 'failovers', '-u', '@ns_server']) - OK
      memcached stats ['hash', 'detail'] (['cbstats', '-a', '127.0.0.1:11209', 'hash', 'detail', '-u', '@ns_server']) - OK
      memcached stats kvstore (['cbstats', '-a', '127.0.0.1:11209', 'kvstore', '-u', '@ns_server']) - OK
      memcached stats kvtimings (['cbstats', '-a', '127.0.0.1:11209', 'kvtimings', '-u', '@ns_server']) - OK
      memcached stats memory (['cbstats', '-a', '127.0.0.1:11209', 'memory', '-u', '@ns_server']) - OK
      memcached stats prev-vbucket (['cbstats', '-a', '127.0.0.1:11209', 'prev-vbucket', '-u', '@ns_server']) - OK
      memcached stats ['responses', 'all'] (['cbstats', '-a', '127.0.0.1:11209', 'responses', 'all', '-u', '@ns_server']) - OK
      memcached stats runtimes (['cbstats', '-a', '127.0.0.1:11209', 'runtimes', '-u', '@ns_server']) - OK
      memcached stats scheduler (['cbstats', '-a', '127.0.0.1:11209', 'scheduler', '-u', '@ns_server']) - OK
      memcached stats scopes (['cbstats', '-a', '127.0.0.1:11209', 'scopes', '-u', '@ns_server']) - OK
      memcached stats tasks (['cbstats', '-a', '127.0.0.1:11209', 'tasks', '-u', '@ns_server']) - OK
      memcached stats timings (['cbstats', '-a', '127.0.0.1:11209', 'timings', '-u', '@ns_server']) - OK
      memcached stats uuid (['cbstats', '-a', '127.0.0.1:11209', 'uuid', '-u', '@ns_server']) - OK
      memcached stats vbucket-details (['cbstats', '-a', '127.0.0.1:11209', 'vbucket-details', '-u', '@ns_server']) - OK
      memcached stats vbucket-seqno (['cbstats', '-a', '127.0.0.1:11209', 'vbucket-seqno', '-u', '@ns_server']) - OK
      memcached stats warmup (['cbstats', '-a', '127.0.0.1:11209', 'warmup', '-u', '@ns_server']) - OK
      memcached stats workload (['cbstats', '-a', '127.0.0.1:11209', 'workload', '-u', '@ns_server']) - OK
      memcached mcstat allocator (['mcstat', '-h', '127.0.0.1:11209', '-u', '@ns_server', 'allocator']) - OK
      memcached mcstat connections (['mcstat', '-h', '127.0.0.1:11209', '-u', '@ns_server', 'connections']) - OK
      memcached mcstat tracing (['mcstat', '-h', '127.0.0.1:11209', '-u', '@ns_server', 'tracing']) - OK
      moxi stats (port 0) (echo stats proxy | nc 127.0.0.1 0) - Exit code 127
      mctimings [] (['mctimings', '-u', '@ns_server', '-h', '127.0.0.1:11209', '-v']) - Exit code 1
      mctimings ['subdoc_execute'] (['mctimings', '-u', '@ns_server', '-h', '127.0.0.1:11209', '-v', 'subdoc_execute']) - OK
      Users storage () - OK
      Dist configuration (dist_cfg) () - OK
      Rebalance Report: /opt/couchbase/var/lib/couchbase/logs/rebalance/rebalance_report_2020-10-23T11:04:06Z.json () - OK
      Rebalance Report: /opt/couchbase/var/lib/couchbase/logs/rebalance/rebalance_report_2020-10-23T11:21:16Z.json () - OK
      Rebalance Report: /opt/couchbase/var/lib/couchbase/logs/rebalance/rebalance_report_2020-10-23T11:27:06Z.json () - OK
      Rebalance Report: /opt/couchbase/var/lib/couchbase/logs/rebalance/rebalance_report_2020-10-23T11:26:46Z.json () - OK
      Rebalance Report: /opt/couchbase/var/lib/couchbase/logs/rebalance/rebalance_report_2020-10-23T11:20:56Z.json () - OK
      NS Log () - OK
      stats archives (['escript', '/opt/couchbase/bin/escript-wrapper', '--initargs-path', '/opt/couchbase/var/lib/couchbase/initargs', '--', '/opt/couchbase/bin/dump-stats', '/opt/couchbase/var/lib/couchbase/stats', 'stats_archives.json']) - OK
      Phosphor Trace (['kv_trace_dump', '-H', '127.0.0.1:11209', '-u', '@ns_server', 'kv_trace.json']) - OK
      Removing existing snapshot directory '/opt/couchbase/var/lib/couchbase/stats_data/snapshots'
      cbcollect_info log () - OK
      Removing existing snapshot directory '/opt/couchbase/var/lib/couchbase/stats_data/snapshots'
      [root@node2-cb601-centos7 bin]#

       

       

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            dfinlay Dave Finlay
            ashwin.govindarajulu Ashwin Govindarajulu
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty