Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-60323

[Install] : Chronicle fails to start up post installation

    XMLWordPrintable

Details

    Description

       Installed Couchbase with the following command

      $DEBIAN_FRONTEND='noninteractive' apt-get -y -f install /tmp/couchbase-server-enterprise_7.2.1-5932-linux_amd64.deb > /dev/null && echo 1 || echo 0 

      Couchbase starts up after installation 

      $service couchbase-server status
       
      ● couchbase-server.service - Couchbase Server
           Loaded: loaded (/lib/systemd/system/couchbase-server.service; enabled; vendor preset: enabled)
           Active: active (running) since Wed 2024-01-10 06:43:05 PST; 33min ago
             Docs: https://docs.couchbase.com
         Main PID: 2393003 (beam.smp)
            Tasks: 123 (limit: 4632)
           Memory: 300.7M
              CPU: 30min 50.057s
           CGroup: /system.slice/couchbase-server.service
                   ├─2393003 /opt/couchbase/lib/erlang/erts-13.2.2.3/bin/beam.smp -A 16 -sbwt none -- -root /opt/couchbase/lib/erlang -bindir /opt/couchbase/lib/erlang/erts-13.2.2.3/bin -progname erl -- -home /home/couchbase -- -smp enable -kernel lo>
                   ├─2393038 erl_child_setup 200000
                   ├─2393063 /opt/couchbase/bin/gosecrets --config /opt/couchbase/var/lib/couchbase/config/gosecrets.cfg
                   ├─2418329 /opt/couchbase/lib/erlang/erts-13.2.2.3/bin/beam.smp -A 16 -sbt u -P 327680 -K true -swt low -sbwt none -MMmcs 30 -e102400 -S 4:4 -- -root /opt/couchbase/lib/erlang -bindir /opt/couchbase/lib/erlang/erts-13.2.2.3/bin -pro>
                   ├─2418353 erl_child_setup 200000
                   ├─2418376 sh -s disksup
                   ├─2418378 /opt/couchbase/lib/erlang/lib/os_mon-2.8.2/priv/bin/memsup
                   ├─2418379 /opt/couchbase/lib/erlang/lib/os_mon-2.8.2/priv/bin/cpu_sup
                   ├─2418408 portsigar for ns_1@cb.local --babysitter_pid 2393003 --logfile /opt/couchbase/var/lib/couchbase/logs/sigar_port.log --config /opt/couchbase/var/lib/couchbase/config/sigar_port.json
                   ├─2418409 /opt/couchbase/lib/erlang/erts-13.2.2.3/bin/beam.smp -S 4:4 -P 327680 -K true -- -root /opt/couchbase/lib/erlang -bindir /opt/couchbase/lib/erlang/erts-13.2.2.3/bin -progname erl -- -home /home/couchbase -- -pa /opt/couch>
                   ├─2418418 erl_child_setup 200000
                   ├─2418447 sh -s disksup
                   ├─2418449 /opt/couchbase/lib/erlang/lib/os_mon-2.8.2/priv/bin/memsup
                   ├─2418450 /opt/couchbase/lib/erlang/lib/os_mon-2.8.2/priv/bin/cpu_sup
                   ├─2418455 /opt/couchbase/lib/erlang/erts-13.2.2.3/bin/inet_gethost 4
                   └─2418456 /opt/couchbase/lib/erlang/erts-13.2.2.3/bin/inet_gethost 4
       
       
      Jan 10 06:43:05 sa1514-deb11 systemd[1]: Started Couchbase Server.
      Jan 10 06:43:34 sa1514-deb11 systemd[1]: /lib/systemd/system/couchbase-server.service:17: Unknown key name 'ExitType' in section 'Service', ignoring.
      Jan 10 06:43:34 sa1514-deb11 systemd[1]: /lib/systemd/system/couchbase-server.service:21: Unknown key name 'StartLimitIntervalSec' in section 'Service', ignoring. 

      Not all couchbase processes are up - The output of "ps -aef" - https://cb-engineering.s3.amazonaws.com/MB-60323/ps.txt

      Observing a lot of CRASH REPORTS in ns_server.debug.log and babysitter.log post failure to start chronicle

      [ns_server:debug,2024-01-10T06:45:06.951-08:00,ns_1@cb.local:<0.9.0>:child_erlang:child_start:104]Can't start process: error:{badmatch,                            {error,                             {{shutdown,                               {failed_to_start_child,ns_server_cluster_sup,                                {shutdown,                                 {failed_to_start_child,chronicle_local,                                  {timeout,                                   {gen_server,call,                                    [chronicle_secondary_sup,sync,20000]}}}}}},                              {ns_server,start,[normal,[]]}}}}[{ns_bootstrap,start,1,[{file,"src/ns_bootstrap.erl"},{line,32}]}, {child_erlang,do_child_start,1,[{file,"src/child_erlang.erl"},{line,130}]}, {child_erlang,child_start,1,[{file,"src/child_erlang.erl"},{line,101}]}, {init,start_em,1,[]}, {init,do_boot,3,[]}] 

      [ns_server:info,2024-01-10T06:45:07.152-08:00,babysitter_of_ns_1@cb.local:<0.132.0>:ns_port_server:log:226]ns_server<0.132.0>: Can't start process error:{badmatch,ns_server<0.132.0>:                            {error,ns_server<0.132.0>:                             {{shutdown,ns_server<0.132.0>:                               {failed_to_start_child,ns_server_cluster_sup,ns_server<0.132.0>:                                {shutdown,ns_server<0.132.0>:                                 {failed_to_start_child,chronicle_local,ns_server<0.132.0>:                                  {timeout,ns_server<0.132.0>:                                   {gen_server,call,ns_server<0.132.0>:                                    [chronicle_secondary_sup,sync,20000]}}}}}},ns_server<0.132.0>:                              {ns_server,start,[normal,[]]}}}}ns_server<0.132.0>: [{ns_bootstrap,start,1,[{file,"src/ns_bootstrap.erl"},{line,32}]},ns_server<0.132.0>:  {child_erlang,do_child_start,1,[{file,"src/child_erlang.erl"},{line,130}]},ns_server<0.132.0>:  {child_erlang,child_start,1,[{file,"src/child_erlang.erl"},{line,101}]},ns_server<0.132.0>:  {init,start_em,1,[]},ns_server<0.132.0>:  {init,do_boot,3,[]}] 

       

      This issue is not consistently observed.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            raghav.sk Raghav S K
            raghav.sk Raghav S K
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty