Uploaded image for project: 'Spymemcached Java Client'
  1. Spymemcached Java Client
  2. SPY-81

spike in requests causes large amount of java exceptions and tomcat crash

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: .next
    • Component/s: docs
    • Security Level: Public
    • Labels:
      None
    • Environment:
      MB 1.7.1-community
      Spy 2.7.0

      Description

      user is trying to understand why they are seeing these messages in the logs, application servers and membase are on the same local subnet and value sizes are under 1k

      spy is connected to membase through a server side moxi.

      2012-02-23 13:55:02.462 ERROR net.spy.memcached.protocol.binary.StoreOperationImpl: Error: Too large
      2012-02-23 13:55:02.462 INFO net.spy.memcached.MemcachedConnection: Reconnection due to exception handling a memcached operation on

      {QA sa=apimemcache1a01/10.90.46.208:11211, #Rops=1, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@43452e96, topWop=null, toWrite=0, interested=1}

      . This may be due to an authentication failure.
      OperationException: SERVER: Too large
      at net.spy.memcached.protocol.BaseOperationImpl.handleError(BaseOperationImpl.java:136)
      at net.spy.memcached.protocol.binary.OperationImpl.finishedPayload(OperationImpl.java:154)
      at net.spy.memcached.protocol.binary.OperationImpl.readFromBuffer(OperationImpl.java:140)
      at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:493)
      at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:425)
      at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:278)
      at net.spy.memcached.MemcachedClient.run(MemcachedClient.java:1981)
      2012-02-23 13:55:02.463 WARN net.spy.memcached.MemcachedConnection: Closing, and reopening

      {QA sa=apimemcache1a01/10.90.46.208:11211, #Rops=1, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@43452e96, topWop=null, toWrite=0, interested=1}

      , attempt 0.
      2012-02-23 13:55:02.463 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.StoreOperationImpl@43452e96
      Feb 23, 2012 1:55:02 PM org.apache.tomcat.util.http.Parameters processParameters
      INFO: Parameters: Invalid chunk '' ignored.
      Feb 23, 2012 1:55:02 PM org.apache.tomcat.util.http.Parameters processParameters
      INFO: Parameters: Invalid chunk '' ignored.
      2012-02-23 13:55:03.132 ERROR net.spy.memcached.protocol.binary.StoreOperationImpl: Error: Too large
      2012-02-23 13:55:03.132 INFO net.spy.memcached.MemcachedConnection: Reconnection due to exception handling a memcached operation on

      {QA sa=apimemcache1a02/10.90.46.209:11211, #Rops=1, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@f4c9792, topWop=null, toWrite=0, interested=1}

      . This may be due to an authentication failure.
      OperationException: SERVER: Too large
      at net.spy.memcached.protocol.BaseOperationImpl.handleError(BaseOperationImpl.java:136)
      at net.spy.memcached.protocol.binary.OperationImpl.finishedPayload(OperationImpl.java:154)
      at net.spy.memcached.protocol.binary.OperationImpl.readFromBuffer(OperationImpl.java:140)
      at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:493)
      at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:425)
      at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:278)
      at net.spy.memcached.MemcachedClient.run(MemcachedClient.java:1981)
      2012-02-23 13:55:03.133 WARN net.spy.memcached.MemcachedConnection: Closing, and reopening

      {QA sa=apimemcache1a02/10.90.46.209:11211, #Rops=1, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@f4c9792, topWop=null, toWrite=0, interested=1}

      , attempt 0.
      2012-02-23 13:55:03.133 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.StoreOperationImpl@f4c9792
      Feb 23, 2012 1:55:03 PM org.apache.tomcat.util.http.Parameters processParameters
      INFO: Parameters: Invalid chunk '' ignored.
      2012-02-23 13:55:03.348 ERROR net.spy.memcached.protocol.binary.StoreOperationImpl: Error: Too large
      2012-02-23 13:55:03.348 INFO net.spy.memcached.MemcachedConnection: Reconnection due to exception handling a memcached operation on

      {QA sa=apimemcache1a03/10.90.46.210:11211, #Rops=3, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@2307e11c, topWop=null, toWrite=0, interested=1}

      . This may be due to an authentication failure.
      OperationException: SERVER: Too large
      at net.spy.memcached.protocol.BaseOperationImpl.handleError(BaseOperationImpl.java:136)
      at net.spy.memcached.protocol.binary.OperationImpl.finishedPayload(OperationImpl.java:154)
      at net.spy.memcached.protocol.binary.OperationImpl.readFromBuffer(OperationImpl.java:140)
      at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:493)
      at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:425)
      at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:278)
      at net.spy.memcached.MemcachedClient.run(MemcachedClient.java:1981)
      2012-02-23 13:55:03.349 WARN net.spy.memcached.MemcachedConnection: Closing, and reopening

      {QA sa=apimemcache1a03/10.90.46.210:11211, #Rops=3, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@2307e11c, topWop=null, toWrite=0, interested=1}

      , attempt 0.
      2012-02-23 13:55:03.349 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.StoreOperationImpl@2307e11c
      2012-02-23 13:55:03.350 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.OptimizedGetImpl@7a8100c5
      2012-02-23 13:55:03.350 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.StoreOperationImpl@44423a6b

      2012-02-23 13:55:03.788 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for ihr_tomcat/authkeys/7yV9qppJ38flpgQqm0Dw31bNYdlyruj_bbj7mHGzQyo.
      2012-02-23 13:55:03.789 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for ihr_tomcat/authkeys/7yV9qppJ38flpgQqm0Dw31bNYdlyruj_bbj7mHGzQyo[Ljava.lang.String;@9dbde1b.
      2012-02-23 13:55:03.790 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for ihr_tomcatclient_config_iPhone_4.0.0.
      2012-02-23 13:55:03.824 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for ihr_tomcat/authkeys/cTGxajvGtJg_sO0cQfTqMr3dwOJDbAwv0ecbJJGzgcM.

      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        alex Alex Ma created issue -
        alex Alex Ma made changes -
        Field Original Value New Value
        Description user is trying to understand why they are seeing these messages in the logs, application servers and membase are on the same local subnet and value sizes are under 1k



        2012-02-23 13:55:02.462 ERROR net.spy.memcached.protocol.binary.StoreOperationImpl: Error: Too large
        2012-02-23 13:55:02.462 INFO net.spy.memcached.MemcachedConnection: Reconnection due to exception handling a memcached operation on {QA sa=apimemcache1a01/10.90.46.208:11211, #Rops=1, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@43452e96, topWop=null, toWrite=0, interested=1}. This may be due to an authentication failure.
        OperationException: SERVER: Too large
                at net.spy.memcached.protocol.BaseOperationImpl.handleError(BaseOperationImpl.java:136)
                at net.spy.memcached.protocol.binary.OperationImpl.finishedPayload(OperationImpl.java:154)
                at net.spy.memcached.protocol.binary.OperationImpl.readFromBuffer(OperationImpl.java:140)
                at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:493)
                at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:425)
                at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:278)
                at net.spy.memcached.MemcachedClient.run(MemcachedClient.java:1981)
        2012-02-23 13:55:02.463 WARN net.spy.memcached.MemcachedConnection: Closing, and reopening {QA sa=apimemcache1a01/10.90.46.208:11211, #Rops=1, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@43452e96, topWop=null, toWrite=0, interested=1}, attempt 0.
        2012-02-23 13:55:02.463 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.StoreOperationImpl@43452e96
        Feb 23, 2012 1:55:02 PM org.apache.tomcat.util.http.Parameters processParameters
        INFO: Parameters: Invalid chunk '' ignored.
        Feb 23, 2012 1:55:02 PM org.apache.tomcat.util.http.Parameters processParameters
        INFO: Parameters: Invalid chunk '' ignored.
        2012-02-23 13:55:03.132 ERROR net.spy.memcached.protocol.binary.StoreOperationImpl: Error: Too large
        2012-02-23 13:55:03.132 INFO net.spy.memcached.MemcachedConnection: Reconnection due to exception handling a memcached operation on {QA sa=apimemcache1a02/10.90.46.209:11211, #Rops=1, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@f4c9792, topWop=null, toWrite=0, interested=1}. This may be due to an authentication failure.
        OperationException: SERVER: Too large
                at net.spy.memcached.protocol.BaseOperationImpl.handleError(BaseOperationImpl.java:136)
                at net.spy.memcached.protocol.binary.OperationImpl.finishedPayload(OperationImpl.java:154)
                at net.spy.memcached.protocol.binary.OperationImpl.readFromBuffer(OperationImpl.java:140)
                at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:493)
                at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:425)
                at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:278)
                at net.spy.memcached.MemcachedClient.run(MemcachedClient.java:1981)
        2012-02-23 13:55:03.133 WARN net.spy.memcached.MemcachedConnection: Closing, and reopening {QA sa=apimemcache1a02/10.90.46.209:11211, #Rops=1, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@f4c9792, topWop=null, toWrite=0, interested=1}, attempt 0.
        2012-02-23 13:55:03.133 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.StoreOperationImpl@f4c9792
        Feb 23, 2012 1:55:03 PM org.apache.tomcat.util.http.Parameters processParameters
        INFO: Parameters: Invalid chunk '' ignored.
        2012-02-23 13:55:03.348 ERROR net.spy.memcached.protocol.binary.StoreOperationImpl: Error: Too large
        2012-02-23 13:55:03.348 INFO net.spy.memcached.MemcachedConnection: Reconnection due to exception handling a memcached operation on {QA sa=apimemcache1a03/10.90.46.210:11211, #Rops=3, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@2307e11c, topWop=null, toWrite=0, interested=1}. This may be due to an authentication failure.
        OperationException: SERVER: Too large
                at net.spy.memcached.protocol.BaseOperationImpl.handleError(BaseOperationImpl.java:136)
                at net.spy.memcached.protocol.binary.OperationImpl.finishedPayload(OperationImpl.java:154)
                at net.spy.memcached.protocol.binary.OperationImpl.readFromBuffer(OperationImpl.java:140)
                at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:493)
                at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:425)
                at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:278)
                at net.spy.memcached.MemcachedClient.run(MemcachedClient.java:1981)
        2012-02-23 13:55:03.349 WARN net.spy.memcached.MemcachedConnection: Closing, and reopening {QA sa=apimemcache1a03/10.90.46.210:11211, #Rops=3, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@2307e11c, topWop=null, toWrite=0, interested=1}, attempt 0.
        2012-02-23 13:55:03.349 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.StoreOperationImpl@2307e11c
        2012-02-23 13:55:03.350 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.OptimizedGetImpl@7a8100c5
        2012-02-23 13:55:03.350 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.StoreOperationImpl@44423a6b


        2012-02-23 13:55:03.788 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for ihr_tomcat/authkeys/7yV9qppJ38flpgQqm0Dw31bNYdlyruj_bbj7mHGzQyo.
        2012-02-23 13:55:03.789 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for ihr_tomcat/authkeys/7yV9qppJ38flpgQqm0Dw31bNYdlyruj_bbj7mHGzQyo[Ljava.lang.String;@9dbde1b.
        2012-02-23 13:55:03.790 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for ihr_tomcatclient_config_iPhone_4.0.0.
        2012-02-23 13:55:03.824 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for ihr_tomcat/authkeys/cTGxajvGtJg_sO0cQfTqMr3dwOJDbAwv0ecbJJGzgcM.

        user is trying to understand why they are seeing these messages in the logs, application servers and membase are on the same local subnet and value sizes are under 1k



        2012-02-23 13:55:02.462 ERROR net.spy.memcached.protocol.binary.StoreOperationImpl: Error: Too large
        2012-02-23 13:55:02.462 INFO net.spy.memcached.MemcachedConnection: Reconnection due to exception handling a memcached operation on {QA sa=apimemcache1a01/10.90.46.208:11211, #Rops=1, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@43452e96, topWop=null, toWrite=0, interested=1}. This may be due to an authentication failure.
        OperationException: SERVER: Too large
                at net.spy.memcached.protocol.BaseOperationImpl.handleError(BaseOperationImpl.java:136)
                at net.spy.memcached.protocol.binary.OperationImpl.finishedPayload(OperationImpl.java:154)
                at net.spy.memcached.protocol.binary.OperationImpl.readFromBuffer(OperationImpl.java:140)
                at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:493)
                at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:425)
                at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:278)
                at net.spy.memcached.MemcachedClient.run(MemcachedClient.java:1981)
        2012-02-23 13:55:02.463 WARN net.spy.memcached.MemcachedConnection: Closing, and reopening {QA sa=apimemcache1a01/10.90.46.208:11211, #Rops=1, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@43452e96, topWop=null, toWrite=0, interested=1}, attempt 0.
        2012-02-23 13:55:02.463 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.StoreOperationImpl@43452e96
        Feb 23, 2012 1:55:02 PM org.apache.tomcat.util.http.Parameters processParameters
        INFO: Parameters: Invalid chunk '' ignored.
        Feb 23, 2012 1:55:02 PM org.apache.tomcat.util.http.Parameters processParameters
        INFO: Parameters: Invalid chunk '' ignored.
        2012-02-23 13:55:03.132 ERROR net.spy.memcached.protocol.binary.StoreOperationImpl: Error: Too large
        2012-02-23 13:55:03.132 INFO net.spy.memcached.MemcachedConnection: Reconnection due to exception handling a memcached operation on {QA sa=apimemcache1a02/10.90.46.209:11211, #Rops=1, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@f4c9792, topWop=null, toWrite=0, interested=1}. This may be due to an authentication failure.
        OperationException: SERVER: Too large
                at net.spy.memcached.protocol.BaseOperationImpl.handleError(BaseOperationImpl.java:136)
                at net.spy.memcached.protocol.binary.OperationImpl.finishedPayload(OperationImpl.java:154)
                at net.spy.memcached.protocol.binary.OperationImpl.readFromBuffer(OperationImpl.java:140)
                at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:493)
                at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:425)
                at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:278)
                at net.spy.memcached.MemcachedClient.run(MemcachedClient.java:1981)
        2012-02-23 13:55:03.133 WARN net.spy.memcached.MemcachedConnection: Closing, and reopening {QA sa=apimemcache1a02/10.90.46.209:11211, #Rops=1, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@f4c9792, topWop=null, toWrite=0, interested=1}, attempt 0.
        2012-02-23 13:55:03.133 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.StoreOperationImpl@f4c9792
        Feb 23, 2012 1:55:03 PM org.apache.tomcat.util.http.Parameters processParameters
        INFO: Parameters: Invalid chunk '' ignored.
        2012-02-23 13:55:03.348 ERROR net.spy.memcached.protocol.binary.StoreOperationImpl: Error: Too large
        2012-02-23 13:55:03.348 INFO net.spy.memcached.MemcachedConnection: Reconnection due to exception handling a memcached operation on {QA sa=apimemcache1a03/10.90.46.210:11211, #Rops=3, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@2307e11c, topWop=null, toWrite=0, interested=1}. This may be due to an authentication failure.
        OperationException: SERVER: Too large
                at net.spy.memcached.protocol.BaseOperationImpl.handleError(BaseOperationImpl.java:136)
                at net.spy.memcached.protocol.binary.OperationImpl.finishedPayload(OperationImpl.java:154)
                at net.spy.memcached.protocol.binary.OperationImpl.readFromBuffer(OperationImpl.java:140)
                at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:493)
                at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:425)
                at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:278)
                at net.spy.memcached.MemcachedClient.run(MemcachedClient.java:1981)
        2012-02-23 13:55:03.349 WARN net.spy.memcached.MemcachedConnection: Closing, and reopening {QA sa=apimemcache1a03/10.90.46.210:11211, #Rops=3, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@2307e11c, topWop=null, toWrite=0, interested=1}, attempt 0.
        2012-02-23 13:55:03.349 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.StoreOperationImpl@2307e11c
        2012-02-23 13:55:03.350 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.OptimizedGetImpl@7a8100c5
        2012-02-23 13:55:03.350 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.StoreOperationImpl@44423a6b


        2012-02-23 13:55:03.788 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for ihr_tomcat/authkeys/7yV9qppJ38flpgQqm0Dw31bNYdlyruj_bbj7mHGzQyo.
        2012-02-23 13:55:03.789 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for ihr_tomcat/authkeys/7yV9qppJ38flpgQqm0Dw31bNYdlyruj_bbj7mHGzQyo[Ljava.lang.String;@9dbde1b.
        2012-02-23 13:55:03.790 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for ihr_tomcatclient_config_iPhone_4.0.0.
        2012-02-23 13:55:03.824 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for ihr_tomcat/authkeys/cTGxajvGtJg_sO0cQfTqMr3dwOJDbAwv0ecbJJGzgcM.




        spy is connected to membase through a server side moxi.
        Component/s docs [ 10121 ]
        Component/s library [ 10120 ]
        alex Alex Ma made changes -
        Description user is trying to understand why they are seeing these messages in the logs, application servers and membase are on the same local subnet and value sizes are under 1k



        2012-02-23 13:55:02.462 ERROR net.spy.memcached.protocol.binary.StoreOperationImpl: Error: Too large
        2012-02-23 13:55:02.462 INFO net.spy.memcached.MemcachedConnection: Reconnection due to exception handling a memcached operation on {QA sa=apimemcache1a01/10.90.46.208:11211, #Rops=1, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@43452e96, topWop=null, toWrite=0, interested=1}. This may be due to an authentication failure.
        OperationException: SERVER: Too large
                at net.spy.memcached.protocol.BaseOperationImpl.handleError(BaseOperationImpl.java:136)
                at net.spy.memcached.protocol.binary.OperationImpl.finishedPayload(OperationImpl.java:154)
                at net.spy.memcached.protocol.binary.OperationImpl.readFromBuffer(OperationImpl.java:140)
                at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:493)
                at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:425)
                at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:278)
                at net.spy.memcached.MemcachedClient.run(MemcachedClient.java:1981)
        2012-02-23 13:55:02.463 WARN net.spy.memcached.MemcachedConnection: Closing, and reopening {QA sa=apimemcache1a01/10.90.46.208:11211, #Rops=1, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@43452e96, topWop=null, toWrite=0, interested=1}, attempt 0.
        2012-02-23 13:55:02.463 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.StoreOperationImpl@43452e96
        Feb 23, 2012 1:55:02 PM org.apache.tomcat.util.http.Parameters processParameters
        INFO: Parameters: Invalid chunk '' ignored.
        Feb 23, 2012 1:55:02 PM org.apache.tomcat.util.http.Parameters processParameters
        INFO: Parameters: Invalid chunk '' ignored.
        2012-02-23 13:55:03.132 ERROR net.spy.memcached.protocol.binary.StoreOperationImpl: Error: Too large
        2012-02-23 13:55:03.132 INFO net.spy.memcached.MemcachedConnection: Reconnection due to exception handling a memcached operation on {QA sa=apimemcache1a02/10.90.46.209:11211, #Rops=1, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@f4c9792, topWop=null, toWrite=0, interested=1}. This may be due to an authentication failure.
        OperationException: SERVER: Too large
                at net.spy.memcached.protocol.BaseOperationImpl.handleError(BaseOperationImpl.java:136)
                at net.spy.memcached.protocol.binary.OperationImpl.finishedPayload(OperationImpl.java:154)
                at net.spy.memcached.protocol.binary.OperationImpl.readFromBuffer(OperationImpl.java:140)
                at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:493)
                at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:425)
                at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:278)
                at net.spy.memcached.MemcachedClient.run(MemcachedClient.java:1981)
        2012-02-23 13:55:03.133 WARN net.spy.memcached.MemcachedConnection: Closing, and reopening {QA sa=apimemcache1a02/10.90.46.209:11211, #Rops=1, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@f4c9792, topWop=null, toWrite=0, interested=1}, attempt 0.
        2012-02-23 13:55:03.133 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.StoreOperationImpl@f4c9792
        Feb 23, 2012 1:55:03 PM org.apache.tomcat.util.http.Parameters processParameters
        INFO: Parameters: Invalid chunk '' ignored.
        2012-02-23 13:55:03.348 ERROR net.spy.memcached.protocol.binary.StoreOperationImpl: Error: Too large
        2012-02-23 13:55:03.348 INFO net.spy.memcached.MemcachedConnection: Reconnection due to exception handling a memcached operation on {QA sa=apimemcache1a03/10.90.46.210:11211, #Rops=3, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@2307e11c, topWop=null, toWrite=0, interested=1}. This may be due to an authentication failure.
        OperationException: SERVER: Too large
                at net.spy.memcached.protocol.BaseOperationImpl.handleError(BaseOperationImpl.java:136)
                at net.spy.memcached.protocol.binary.OperationImpl.finishedPayload(OperationImpl.java:154)
                at net.spy.memcached.protocol.binary.OperationImpl.readFromBuffer(OperationImpl.java:140)
                at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:493)
                at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:425)
                at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:278)
                at net.spy.memcached.MemcachedClient.run(MemcachedClient.java:1981)
        2012-02-23 13:55:03.349 WARN net.spy.memcached.MemcachedConnection: Closing, and reopening {QA sa=apimemcache1a03/10.90.46.210:11211, #Rops=3, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@2307e11c, topWop=null, toWrite=0, interested=1}, attempt 0.
        2012-02-23 13:55:03.349 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.StoreOperationImpl@2307e11c
        2012-02-23 13:55:03.350 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.OptimizedGetImpl@7a8100c5
        2012-02-23 13:55:03.350 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.StoreOperationImpl@44423a6b


        2012-02-23 13:55:03.788 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for ihr_tomcat/authkeys/7yV9qppJ38flpgQqm0Dw31bNYdlyruj_bbj7mHGzQyo.
        2012-02-23 13:55:03.789 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for ihr_tomcat/authkeys/7yV9qppJ38flpgQqm0Dw31bNYdlyruj_bbj7mHGzQyo[Ljava.lang.String;@9dbde1b.
        2012-02-23 13:55:03.790 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for ihr_tomcatclient_config_iPhone_4.0.0.
        2012-02-23 13:55:03.824 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for ihr_tomcat/authkeys/cTGxajvGtJg_sO0cQfTqMr3dwOJDbAwv0ecbJJGzgcM.




        spy is connected to membase through a server side moxi.
        user is trying to understand why they are seeing these messages in the logs, application servers and membase are on the same local subnet and value sizes are under 1k

        spy is connected to membase through a server side moxi.

        2012-02-23 13:55:02.462 ERROR net.spy.memcached.protocol.binary.StoreOperationImpl: Error: Too large
        2012-02-23 13:55:02.462 INFO net.spy.memcached.MemcachedConnection: Reconnection due to exception handling a memcached operation on {QA sa=apimemcache1a01/10.90.46.208:11211, #Rops=1, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@43452e96, topWop=null, toWrite=0, interested=1}. This may be due to an authentication failure.
        OperationException: SERVER: Too large
                at net.spy.memcached.protocol.BaseOperationImpl.handleError(BaseOperationImpl.java:136)
                at net.spy.memcached.protocol.binary.OperationImpl.finishedPayload(OperationImpl.java:154)
                at net.spy.memcached.protocol.binary.OperationImpl.readFromBuffer(OperationImpl.java:140)
                at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:493)
                at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:425)
                at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:278)
                at net.spy.memcached.MemcachedClient.run(MemcachedClient.java:1981)
        2012-02-23 13:55:02.463 WARN net.spy.memcached.MemcachedConnection: Closing, and reopening {QA sa=apimemcache1a01/10.90.46.208:11211, #Rops=1, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@43452e96, topWop=null, toWrite=0, interested=1}, attempt 0.
        2012-02-23 13:55:02.463 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.StoreOperationImpl@43452e96
        Feb 23, 2012 1:55:02 PM org.apache.tomcat.util.http.Parameters processParameters
        INFO: Parameters: Invalid chunk '' ignored.
        Feb 23, 2012 1:55:02 PM org.apache.tomcat.util.http.Parameters processParameters
        INFO: Parameters: Invalid chunk '' ignored.
        2012-02-23 13:55:03.132 ERROR net.spy.memcached.protocol.binary.StoreOperationImpl: Error: Too large
        2012-02-23 13:55:03.132 INFO net.spy.memcached.MemcachedConnection: Reconnection due to exception handling a memcached operation on {QA sa=apimemcache1a02/10.90.46.209:11211, #Rops=1, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@f4c9792, topWop=null, toWrite=0, interested=1}. This may be due to an authentication failure.
        OperationException: SERVER: Too large
                at net.spy.memcached.protocol.BaseOperationImpl.handleError(BaseOperationImpl.java:136)
                at net.spy.memcached.protocol.binary.OperationImpl.finishedPayload(OperationImpl.java:154)
                at net.spy.memcached.protocol.binary.OperationImpl.readFromBuffer(OperationImpl.java:140)
                at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:493)
                at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:425)
                at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:278)
                at net.spy.memcached.MemcachedClient.run(MemcachedClient.java:1981)
        2012-02-23 13:55:03.133 WARN net.spy.memcached.MemcachedConnection: Closing, and reopening {QA sa=apimemcache1a02/10.90.46.209:11211, #Rops=1, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@f4c9792, topWop=null, toWrite=0, interested=1}, attempt 0.
        2012-02-23 13:55:03.133 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.StoreOperationImpl@f4c9792
        Feb 23, 2012 1:55:03 PM org.apache.tomcat.util.http.Parameters processParameters
        INFO: Parameters: Invalid chunk '' ignored.
        2012-02-23 13:55:03.348 ERROR net.spy.memcached.protocol.binary.StoreOperationImpl: Error: Too large
        2012-02-23 13:55:03.348 INFO net.spy.memcached.MemcachedConnection: Reconnection due to exception handling a memcached operation on {QA sa=apimemcache1a03/10.90.46.210:11211, #Rops=3, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@2307e11c, topWop=null, toWrite=0, interested=1}. This may be due to an authentication failure.
        OperationException: SERVER: Too large
                at net.spy.memcached.protocol.BaseOperationImpl.handleError(BaseOperationImpl.java:136)
                at net.spy.memcached.protocol.binary.OperationImpl.finishedPayload(OperationImpl.java:154)
                at net.spy.memcached.protocol.binary.OperationImpl.readFromBuffer(OperationImpl.java:140)
                at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:493)
                at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:425)
                at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:278)
                at net.spy.memcached.MemcachedClient.run(MemcachedClient.java:1981)
        2012-02-23 13:55:03.349 WARN net.spy.memcached.MemcachedConnection: Closing, and reopening {QA sa=apimemcache1a03/10.90.46.210:11211, #Rops=3, #Wops=0, #iq=0, topRop=net.spy.memcached.protocol.binary.StoreOperationImpl@2307e11c, topWop=null, toWrite=0, interested=1}, attempt 0.
        2012-02-23 13:55:03.349 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.StoreOperationImpl@2307e11c
        2012-02-23 13:55:03.350 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.OptimizedGetImpl@7a8100c5
        2012-02-23 13:55:03.350 WARN net.spy.memcached.protocol.binary.BinaryMemcachedNodeImpl: Discarding partially completed op: net.spy.memcached.protocol.binary.StoreOperationImpl@44423a6b


        2012-02-23 13:55:03.788 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for ihr_tomcat/authkeys/7yV9qppJ38flpgQqm0Dw31bNYdlyruj_bbj7mHGzQyo.
        2012-02-23 13:55:03.789 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for ihr_tomcat/authkeys/7yV9qppJ38flpgQqm0Dw31bNYdlyruj_bbj7mHGzQyo[Ljava.lang.String;@9dbde1b.
        2012-02-23 13:55:03.790 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for ihr_tomcatclient_config_iPhone_4.0.0.
        2012-02-23 13:55:03.824 WARN net.spy.memcached.MemcachedConnection: Could not redistribute to another node, retrying primary node for ihr_tomcat/authkeys/cTGxajvGtJg_sO0cQfTqMr3dwOJDbAwv0ecbJJGzgcM.



        alex Alex Ma made changes -
        Environment MB 1.7.1
        Spy 2.7.0
        MB 1.7.1-community
        Spy 2.7.0
        alex Alex Ma made changes -
        Assignee Raghavan Srinivas [ rags ] Mike Wiederhold [ mikew ]
        mikew Mike Wiederhold made changes -
        Assignee Mike Wiederhold [ mikew ] Raghavan Srinivas [ rags ]
        ingenthr Matt Ingenthron made changes -
        Assignee Raghavan Srinivas [ rags ] Michael Nitschinger [ daschl ]
        daschl Michael Nitschinger made changes -
        Assignee Michael Nitschinger [ daschl ] Alex Ma [ alex ]
        daschl Michael Nitschinger made changes -
        Fix Version/s .next [ 10205 ]

          People

          • Assignee:
            alex Alex Ma
            Reporter:
            alex Alex Ma
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:

              Gerrit Reviews

              There are no open Gerrit changes