Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-12076

Internal moxi misconfiguration

    XMLWordPrintable

Details

    • Bug
    • Resolution: Won't Fix
    • Major
    • bug-backlog
    • 2.5.1
    • moxi
    • Security Level: Public
    • None
    • centos 6.5
    • Untriaged
    • Centos 64-bit
    • Unknown

    Description

      I have 4 servers cluster. Four buckets. One of them is default couchbase bucket with replica=1.

      On one server moxi behavior is very strange. Third server's moxi lives its own live.
      I've telneted to 11211 moxi's port and tried to set test keys. This key apperared only on that server, not on whole cluster. Also couchbase monitoring tool doesn't show any activity on cluster.

      I've noticed that problem moxi process listens only three TCP port's:

      netstat -nlpt | grep 30070
      tcp        0      0 0.0.0.0:11211               0.0.0.0:*                   LISTEN      30070/moxi          
      tcp        0      0 :::11211                    :::*                        LISTEN      30070/moxi          
      tcp        0      0 :::6696                     :::*                        LISTEN      30070/moxi
      

      Other servers' moxies have four listen ports:

      netstat -nltp | grep 2577
      tcp        0      0 0.0.0.0:11211               0.0.0.0:*                   LISTEN      2577/moxi           
      tcp        0      0 0.0.0.0:60593               0.0.0.0:*                   LISTEN      2577/moxi           
      tcp        0      0 :::11211                    :::*                        LISTEN      2577/moxi           
      tcp        0      0 :::18347                    :::*                        LISTEN      2577/moxi
       
      netstat -nlpt | grep 23001
      tcp        0      0 0.0.0.0:11211               0.0.0.0:*                   LISTEN      23001/moxi          
      tcp        0      0 0.0.0.0:11339               0.0.0.0:*                   LISTEN      23001/moxi          
      tcp        0      0 :::11211                    :::*                        LISTEN      23001/moxi          
      tcp        0      0 :::5191                     :::*                        LISTEN      23001/moxi
       
      netstat -nlpt | grep 31535
      tcp        0      0 0.0.0.0:11211               0.0.0.0:*                   LISTEN      31535/moxi          
      tcp        0      0 0.0.0.0:33578               0.0.0.0:*                   LISTEN      31535/moxi          
      tcp        0      0 :::11211                    :::*                        LISTEN      31535/moxi          
      tcp        0      0 :::53475                    :::*                        LISTEN      31535/moxi
      

      So it seems that moxi on problem server was not able to listen one TCP port.

      I've attached debug logs for two servers: problem and normal server.

      The problem process is still running. Please let me know which logs do you need for further problem investigation.

      Attachments

        1. normal.log
          10.00 MB
        2. problem.log
          10.00 MB
        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            avsej Sergey Avseyev
            kay kay
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty