Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-3474

Moxi hang

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Major
    • None
    • 1.6.5
    • moxi
    • Security Level: Public
    • None

    Description

      Unclear exactly what causes this, but the symptoms are that Moxi won't respond to any requests. It accepts connections, but any request hangs (including 'quit')

      Stack trace info below:
      gdb:
      (gdb) where
      #0 0x0000003f36ad4358 in epoll_wait () from /lib64/libc.so.6
      #1 0x00000000004472d4 in epoll_dispatch (base=0x5995be0, tv=<value optimized out>) at epoll.c:314
      #2 0x00000000004419a1 in event_base_loop (base=0x5995be0, flags=0) at event.c:1536
      #3 0x0000000000409146 in main (argc=8, argv=0x7fff16cb9298) at memcached.c:4969
      strace:

      1. strace -p 17629
        Process 17629 attached - interrupt to quit
        clock_gettime(CLOCK_MONOTONIC, {4666384, 868335267}

        ) = 0
        epoll_wait(4, {}, 32, 705) = 0
        clock_gettime(CLOCK_MONOTONIC,

        {4666385, 573471267}

        ) = 0
        epoll_wait(4, {}, 32, 1000) = 0
        clock_gettime(CLOCK_MONOTONIC,

        {4666386, 573502267}

        ) = 0
        epoll_wait(4, {}, 32, 1000) = 0
        clock_gettime(CLOCK_MONOTONIC,

        {4666387, 573597267}

        ) = 0
        epoll_wait(4, {}, 32, 1000) = 0
        clock_gettime(CLOCK_MONOTONIC,

        {4666388, 573691267}

        ) = 0
        epoll_wait(4, {}, 32, 1000) = 0
        clock_gettime(CLOCK_MONOTONIC,

        {4666389, 573943267}

        ) = 0
        epoll_wait(4, {}, 32, 1000) = 0
        clock_gettime(CLOCK_MONOTONIC,

        {4666390, 573916267}

        ) = 0
        epoll_wait(4, {}, 32, 1) = 0
        clock_gettime(CLOCK_MONOTONIC,

        {4666390, 574838267}

        ) = 0
        epoll_wait(4, {}, 32, 1000) = 0
        pstack:
        Thread 7 (Thread 0x41562940 (LWP 17630)):
        #0 0x0000003f36acd212 in select () from /lib64/libc.so.6
        #1 0x000000000042d874 in mcs_io_read ()
        #2 0x0000000000416830 in cproxy_auth_downstream ()
        #3 0x000000000041ad53 in downstream_connect_init ()
        #4 0x000000000041be50 in cproxy_on_connect_downstream_conn ()
        #5 0x000000000040c90b in drive_machine ()
        #6 0x0000000000441d11 in event_base_loop ()
        #7 0x0000000000413ab5 in worker_libevent ()
        #8 0x00002b333da9d73d in start_thread () from /lib64/libpthread.so.0
        #9 0x0000003f36ad3f6d in clone () from /lib64/libc.so.6
        Thread 6 (Thread 0x41f63940 (LWP 17631)):
        #0 0x0000003f36acd212 in select () from /lib64/libc.so.6
        #1 0x000000000042d874 in mcs_io_read ()
        #2 0x0000000000416830 in cproxy_auth_downstream ()
        #3 0x000000000041ad53 in downstream_connect_init ()
        #4 0x000000000041be50 in cproxy_on_connect_downstream_conn ()
        #5 0x000000000040c90b in drive_machine ()
        #6 0x0000000000441d11 in event_base_loop ()
        #7 0x0000000000413ab5 in worker_libevent ()
        #8 0x00002b333da9d73d in start_thread () from /lib64/libpthread.so.0
        #9 0x0000003f36ad3f6d in clone () from /lib64/libc.so.6
        Thread 5 (Thread 0x42964940 (LWP 17632)):
        #0 0x0000003f36acd212 in select () from /lib64/libc.so.6
        #1 0x000000000042d874 in mcs_io_read ()
        #2 0x0000000000416830 in cproxy_auth_downstream ()
        #3 0x000000000041ad53 in downstream_connect_init ()
        #4 0x000000000041be50 in cproxy_on_connect_downstream_conn ()
        #5 0x000000000040c90b in drive_machine ()
        #6 0x0000000000441d11 in event_base_loop ()
        #7 0x0000000000413ab5 in worker_libevent ()
        #8 0x00002b333da9d73d in start_thread () from /lib64/libpthread.so.0
        #9 0x0000003f36ad3f6d in clone () from /lib64/libc.so.6
        Thread 4 (Thread 0x43365940 (LWP 17633)):
        #0 0x0000003f36acd212 in select () from /lib64/libc.so.6
        #1 0x000000000042d874 in mcs_io_read ()
        #2 0x0000000000416830 in cproxy_auth_downstream ()
        #3 0x000000000041ad53 in downstream_connect_init ()
        #4 0x000000000041be50 in cproxy_on_connect_downstream_conn ()
        #5 0x000000000040c90b in drive_machine ()
        #6 0x0000000000441d11 in event_base_loop ()
        #7 0x0000000000413ab5 in worker_libevent ()
        #8 0x00002b333da9d73d in start_thread () from /lib64/libpthread.so.0
        #9 0x0000003f36ad3f6d in clone () from /lib64/libc.so.6
        Thread 3 (Thread 0x43d66940 (LWP 17634)):
        #0 0x00002b333daa1ee9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
        #1 0x0000000000412d20 in assoc_maintenance_thread ()
        #2 0x00002b333da9d73d in start_thread () from /lib64/libpthread.so.0
        #3 0x0000003f36ad3f6d in clone () from /lib64/libc.so.6
        Thread 2 (Thread 0x44767940 (LWP 17635)):
        #0 0x0000003f36acb186 in poll () from /lib64/libc.so.6
        #1 0x000000000044edc6 in Curl_socket_ready ()
        #2 0x0000000000465547 in Curl_do_perform ()
        #3 0x0000000000449007 in run_rest_conflate ()
        #4 0x00002b333da9d73d in start_thread () from /lib64/libpthread.so.0
        #5 0x0000003f36ad3f6d in clone () from /lib64/libc.so.6
        Thread 1 (Thread 0x2b333dcb4a10 (LWP 17629)):
        #0 0x0000003f36ad4358 in epoll_wait () from /lib64/libc.so.6
        #1 0x00000000004472d4 in epoll_dispatch ()
        #2 0x00000000004419a1 in event_base_loop ()
        #3 0x0000000000409146 in main ()
        #0 0x0000003f36acd212 in select () from /lib64/libc.so.6
        #1 0x000000000042d874 in mcs_io_read ()
        #2 0x0000000000416830 in cproxy_auth_downstream ()
        #3 0x000000000041ad53 in downstream_connect_init ()
        #4 0x000000000041be50 in cproxy_on_connect_downstream_conn ()
        #5 0x000000000040c90b in drive_machine ()
        #6 0x0000000000441d11 in event_base_loop ()
        #7 0x0000000000413ab5 in worker_libevent ()
        #8 0x00002b333da9d73d in start_thread () from /lib64/libpthread.so.0
        #9 0x0000003f36ad3f6d in clone () from /lib64/libc.so.6
        Thread 6 (Thread 0x41f63940 (LWP 17631)):
        #0 0x0000003f36acd212 in select () from /lib64/libc.so.6
        #1 0x000000000042d874 in mcs_io_read ()
        #2 0x0000000000416830 in cproxy_auth_downstream ()
        #3 0x000000000041ad53 in downstream_connect_init ()
        #4 0x000000000041be50 in cproxy_on_connect_downstream_conn ()
        #5 0x000000000040c90b in drive_machine ()
        #6 0x0000000000441d11 in event_base_loop ()
        #7 0x0000000000413ab5 in worker_libevent ()
        #8 0x00002b333da9d73d in start_thread () from /lib64/libpthread.so.0
        #9 0x0000003f36ad3f6d in clone () from /lib64/libc.so.6
        Thread 5 (Thread 0x42964940 (LWP 17632)):
        #0 0x0000003f36acd212 in select () from /lib64/libc.so.6
        #1 0x000000000042d874 in mcs_io_read ()
        #2 0x0000000000416830 in cproxy_auth_downstream ()
        #3 0x000000000041ad53 in downstream_connect_init ()
        #4 0x000000000041be50 in cproxy_on_connect_downstream_conn ()
        #5 0x000000000040c90b in drive_machine ()
        #6 0x0000000000441d11 in event_base_loop ()
        #7 0x0000000000413ab5 in worker_libevent ()
        #8 0x00002b333da9d73d in start_thread () from /lib64/libpthread.so.0
        #9 0x0000003f36ad3f6d in clone () from /lib64/libc.so.6
        Thread 4 (Thread 0x43365940 (LWP 17633)):
        #0 0x0000003f36acd212 in select () from /lib64/libc.so.6
        #1 0x000000000042d874 in mcs_io_read ()
        #2 0x0000000000416830 in cproxy_auth_downstream ()
        #3 0x000000000041ad53 in downstream_connect_init ()
        #4 0x000000000041be50 in cproxy_on_connect_downstream_conn ()
        #5 0x000000000040c90b in drive_machine ()
        #6 0x0000000000441d11 in event_base_loop ()
        #7 0x0000000000413ab5 in worker_libevent ()
        #8 0x00002b333da9d73d in start_thread () from /lib64/libpthread.so.0
        #9 0x0000003f36ad3f6d in clone () from /lib64/libc.so.6
        Thread 3 (Thread 0x43d66940 (LWP 17634)):
        #0 0x00002b333daa1ee9 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
        #1 0x0000000000412d20 in assoc_maintenance_thread ()
        #2 0x00002b333da9d73d in start_thread () from /lib64/libpthread.so.0
        #3 0x0000003f36ad3f6d in clone () from /lib64/libc.so.6
        Thread 2 (Thread 0x44767940 (LWP 17635)):
        #0 0x0000003f36acb186 in poll () from /lib64/libc.so.6
        #1 0x000000000044edc6 in Curl_socket_ready ()
        #2 0x0000000000465547 in Curl_do_perform ()
        #3 0x0000000000449007 in run_rest_conflate ()
        #4 0x00002b333da9d73d in start_thread () from /lib64/libpthread.so.0
        #5 0x0000003f36ad3f6d in clone () from /lib64/libc.so.6
        Thread 1 (Thread 0x2b333dcb4a10 (LWP 17629)):
        #0 0x0000003f36ad4358 in epoll_wait () from /lib64/libc.so.6
        #1 0x00000000004472d4 in epoll_dispatch ()
        #2 0x00000000004419a1 in event_base_loop ()
        #3 0x0000000000409146 in main ()

      Attachments

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. MB-3482 Turn on auth timeouts for Moxi by default

          • Major - Major loss of function.
          • Resolved
          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              steve Steve Yen
              perry Perry Krug
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty