Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-51262

Log human readable message if failing to start with enospc

    XMLWordPrintable

Details

    • 1

    Description

      Fairly often, we see issues where the primary symptom is Couchbase Server failing to start due to a lack of space on a device. This produces a very Erlang error of the form:

      {"init terminating in do_boot",{{badmatch,{error,{bad_return,{{ns_babysitter,start,[normal,[]]},{'EXIT',{{badmatch,{error,enospc}},[{ns_babysitter,do_write_file,3,[{file,"src/ns_babysitter.erl"},{line,93}]},{ns_babysitter,start,2,[{file,"src/ns_babysitter.erl"},{line,64}]},{application_master,start_it_old,4,[{file,"application_master.erl"},{line,273}]}]}}}}}},[{ns_babysitter_bootstrap,start,0,[{file,"src/ns_babysitter_bootstrap.erl"},{line,30}]},{init,start_em,1,[]},{init,do_boot,3,[]}]}}
      

      If you know what to look for, you can spot the enospc hidden in there and draw a conclusion, but for end users it would be much easier to have a clearer message so they can fix the issue on their own.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          I've picked out enospc as it's one of the most common ones we see, but there's probably a few more that would be low hanging fruit if we were to add this, such as eaccess

          James Flather James Flather added a comment - I've picked out enospc as it's one of the most common ones we see, but there's probably a few more that would be low hanging fruit if we were to add this, such as eaccess
          peter.searby Peter Searby added a comment -

          Not worth the effort to fix as memcached gives us a human readable error message already:

          [ns_server:info,2022-11-04T10:57:51.475Z,babysitter_of_n_0@cb.local:<0.136.0>:ns_port_server:log:226]memcached<0.136.0>: [*** LOG ERROR #0001 ***] [2022-11-04 10:57:51] [spdlog_file_logger] {Failed writing to file /Volumes/Test/logs/n_0/memcached.log.000000.txt: No space left on device}

           

          The error message ns_server produces when failing to start with enospc has changed since 6.6.4 and now the main error is logged in a dependency which makes it impractical to modify.

          peter.searby Peter Searby added a comment - Not worth the effort to fix as memcached gives us a human readable error message already: [ns_server:info, 2022 - 11 -04T10: 57 : 51 .475Z,babysitter_of_n_0 @cb .local:< 0.136 . 0 >:ns_port_server:log: 226 ]memcached< 0.136 . 0 >: [*** LOG ERROR # 0001 ***] [ 2022 - 11 - 04 10 : 57 : 51 ] [spdlog_file_logger] {Failed writing to file /Volumes/Test/logs/n_0/memcached.log. 000000 .txt: No space left on device}   The error message ns_server produces when failing to start with enospc has changed since 6.6.4 and now the main error is logged in a dependency which makes it impractical to modify.

          People

            peter.searby Peter Searby
            James Flather James Flather
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty