Uploaded image for project: 'Spymemcached Java Client'
  1. Spymemcached Java Client
  2. SPY-81

spike in requests causes large amount of java exceptions and tomcat crash

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Major
    • .next
    • None
    • docs
    • Security Level: Public
    • None
    • MB 1.7.1-community
      Spy 2.7.0

    Description

      user is trying to understand why they are seeing these messages in the logs, application servers and membase are on the same local subnet and value sizes are under 1k

      spy is connected to membase through a server side moxi.

      2012-02-23 13:55:02.462 ERROR net.spy.memcached.protocol.binary.StoreOperationImpl: Error: Too large
      2012-02-23 13:55:02.462 INFO net.spy.memcached.MemcachedConnection: Reconnection due to exception handling a memcached operation on

      {QA sa=apimemcache1a01/10.90.46.208:11211, #Rops=1, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@43452e96, topWop=null, toWrite=0, interested=1}

      . This may be due to an authentication failure.
      OperationException: SERVER: Too large
      at net.spy.memcached.protocol.BaseOperationImpl.handleError(BaseOperationImpl.java:136)
      at net.spy.memcached.protocol.binary.OperationImpl.finishedPayload(OperationImpl.java:154)
      at net.spy.memcached.protocol.binary.OperationImpl.readFromBuffer(OperationImpl.java:140)
      at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:493)
      at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:425)
      at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:278)
      at net.spy.memcached.MemcachedClient.run(MemcachedClient.java:1981)
      2012-02-23 13:55:02.463 WARN net.spy.memcached.MemcachedConnection: Closing, and reopening

      {QA sa=apimemcache1a01/10.90.46.208:11211, #Rops=1, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@43452e96, topWop=null, toWrite=0, interested=1}

      , attempt 0.
      2012-02-23 13:55:02.463 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.StoreOperationImpl@43452e96
      Feb 23, 2012 1:55:02 PM org.apache.tomcat.util.http.Parameters processParameters
      INFO: Parameters: Invalid chunk '' ignored.
      Feb 23, 2012 1:55:02 PM org.apache.tomcat.util.http.Parameters processParameters
      INFO: Parameters: Invalid chunk '' ignored.
      2012-02-23 13:55:03.132 ERROR net.spy.memcached.protocol.binary.StoreOperationImpl: Error: Too large
      2012-02-23 13:55:03.132 INFO net.spy.memcached.MemcachedConnection: Reconnection due to exception handling a memcached operation on

      {QA sa=apimemcache1a02/10.90.46.209:11211, #Rops=1, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@f4c9792, topWop=null, toWrite=0, interested=1}

      . This may be due to an authentication failure.
      OperationException: SERVER: Too large
      at net.spy.memcached.protocol.BaseOperationImpl.handleError(BaseOperationImpl.java:136)
      at net.spy.memcached.protocol.binary.OperationImpl.finishedPayload(OperationImpl.java:154)
      at net.spy.memcached.protocol.binary.OperationImpl.readFromBuffer(OperationImpl.java:140)
      at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:493)
      at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:425)
      at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:278)
      at net.spy.memcached.MemcachedClient.run(MemcachedClient.java:1981)
      2012-02-23 13:55:03.133 WARN net.spy.memcached.MemcachedConnection: Closing, and reopening

      {QA sa=apimemcache1a02/10.90.46.209:11211, #Rops=1, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@f4c9792, topWop=null, toWrite=0, interested=1}

      , attempt 0.
      2012-02-23 13:55:03.133 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.StoreOperationImpl@f4c9792
      Feb 23, 2012 1:55:03 PM org.apache.tomcat.util.http.Parameters processParameters
      INFO: Parameters: Invalid chunk '' ignored.
      2012-02-23 13:55:03.348 ERROR net.spy.memcached.protocol.binary.StoreOperationImpl: Error: Too large
      2012-02-23 13:55:03.348 INFO net.spy.memcached.MemcachedConnection: Reconnection due to exception handling a memcached operation on

      {QA sa=apimemcache1a03/10.90.46.210:11211, #Rops=3, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@2307e11c, topWop=null, toWrite=0, interested=1}

      . This may be due to an authentication failure.
      OperationException: SERVER: Too large
      at net.spy.memcached.protocol.BaseOperationImpl.handleError(BaseOperationImpl.java:136)
      at net.spy.memcached.protocol.binary.OperationImpl.finishedPayload(OperationImpl.java:154)
      at net.spy.memcached.protocol.binary.OperationImpl.readFromBuffer(OperationImpl.java:140)
      at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:493)
      at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:425)
      at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:278)
      at net.spy.memcached.MemcachedClient.run(MemcachedClient.java:1981)
      2012-02-23 13:55:03.349 WARN net.spy.memcached.MemcachedConnection: Closing, and reopening

      {QA sa=apimemcache1a03/10.90.46.210:11211, #Rops=3, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@2307e11c, topWop=null, toWrite=0, interested=1}

      , attempt 0.
      2012-02-23 13:55:03.349 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.StoreOperationImpl@2307e11c
      2012-02-23 13:55:03.350 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.OptimizedGetImpl@7a8100c5
      2012-02-23 13:55:03.350 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.StoreOperationImpl@44423a6b

      2012-02-23 13:55:03.788 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for ihr_tomcat/authkeys/7yV9qppJ38flpgQqm0Dw31bNYdlyruj_bbj7mHGzQyo.
      2012-02-23 13:55:03.789 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for ihr_tomcat/authkeys/7yV9qppJ38flpgQqm0Dw31bNYdlyruj_bbj7mHGzQyo[Ljava.lang.String;@9dbde1b.
      2012-02-23 13:55:03.790 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for ihr_tomcatclient_config_iPhone_4.0.0.
      2012-02-23 13:55:03.824 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for ihr_tomcat/authkeys/cTGxajvGtJg_sO0cQfTqMr3dwOJDbAwv0ecbJJGzgcM.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            alex Alex Ma [X] (Inactive)
            alex Alex Ma [X] (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty