Uploaded image for project: 'Couchbase PHP client library'
  1. Couchbase PHP client library
  2. PCBC-104

PHP Client Need to Catch and Process "Object Too Large"errors from libcouchbase

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.1.0
    • Component/s: None
    • Security Level: Public
    • Labels:
      None
    • Environment:
      PHP client DP3

      Description

      Problem Summary:

      When writing more than 1MB of data using php libcouch client, the client itself does not process the object too large error and the libcouchbase library logs a PHP warning as "PHP Warning: Couchbase::set(): Failed to store a value to server: Object too big in.....".

      As result, it's impossible to write a strict warning free high performance php app using the current setup when we can't control what data is attempted to be cached.

      1) We cannot catch this error before passing data in since more >1MB data might be compressible to less than <1MB so we have to rely on the PHP Client to do this for us.
      2) We cannot catch this on the client site since it's not an exception but a PHP error/warning.
      3) We can use set_error_handler() in php to catch this but it is extremely slow and negates the performance advantage of having libcouchbase in the first place.

      Recommendation:

      1) Return FALSE from PHP Client if post-compressed data is more than 1MB.

      /* INSERT CODE test for payload_len size here before calling line below, if more, than return FALSE */

      if (!(payload = php_couchbase_zval_to_payload(value, &payload_len, &flags, couchbase_res->serializer, couchbase_res->compressor TSRMLS_CC)))

      { RETURN_FALSE; }

      Notes: pecl-memcached using libmemcached lib does not throw PHP warnings when object size is over limit. Error is logged internally and is up to the programmer to read the error post-fail to determine what happened.

      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        xing xing created issue -
        Hide
        xing xing added a comment -

        1) Typos. The test code should happen after, not before, the zval_to_payload conversion which is when we know exactly is data is over limit.

        2) Since libcouchbase supports I believe 10MB for disk based buckets, my report only deals with ram buckets. Perhaps there is a test somewhere for bucket type before size compare.

        Show
        xing xing added a comment - 1) Typos. The test code should happen after, not before, the zval_to_payload conversion which is when we know exactly is data is over limit. 2) Since libcouchbase supports I believe 10MB for disk based buckets, my report only deals with ram buckets. Perhaps there is a test somewhere for bucket type before size compare.
        Hide
        daschl Michael Nitschinger added a comment -

        Hi xing,

        thanks for taking the time to open an issue!

        Just to recap - you are using the following setup:

        • PHP SDK 1.1-dp3
        • Couchbase 2.0-dp4 (am I right?)

        Are you using a couchbase bucket or a memcached bucket? The couchbase bucket is definitely capable of handling files larger than 1MB.

        There is some restructuring going on regarding the warning/exception handling in the PHP SDK, but let's first sort out your particular issue here.

        Thanks, Michael

        Show
        daschl Michael Nitschinger added a comment - Hi xing, thanks for taking the time to open an issue! Just to recap - you are using the following setup: PHP SDK 1.1-dp3 Couchbase 2.0-dp4 (am I right?) Are you using a couchbase bucket or a memcached bucket? The couchbase bucket is definitely capable of handling files larger than 1MB. There is some restructuring going on regarding the warning/exception handling in the PHP SDK, but let's first sort out your particular issue here. Thanks, Michael
        Hide
        ingenthr Matt Ingenthron added a comment -

        Trond: assigning to you to determine if still an issue and triage.

        Show
        ingenthr Matt Ingenthron added a comment - Trond: assigning to you to determine if still an issue and triage.
        ingenthr Matt Ingenthron made changes -
        Field Original Value New Value
        Assignee Matt Ingenthron [ ingenthr ] Trond Norbye [ trond ]
        Fix Version/s 1.1.0 [ 10242 ]
        Hide
        trond Trond Norbye added a comment -

        This is no longer a problem. You can either catch the exception thrown by the object oriented interface, or you can call couchbase_get_result_code() and it'll contain e2big

        Show
        trond Trond Norbye added a comment - This is no longer a problem. You can either catch the exception thrown by the object oriented interface, or you can call couchbase_get_result_code() and it'll contain e2big
        trond Trond Norbye made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        kzeller kzeller added a comment -

        PHP Client now catches and processes 'object too larger' errors
        Added to RN coming from underlying C library.

        Show
        kzeller kzeller added a comment - PHP Client now catches and processes 'object too larger' errors Added to RN coming from underlying C library.
        brett19 Brett Lawson made changes -
        Workflow jira [ 20353 ] Couchbase SDK Workflow [ 44319 ]

          People

          • Assignee:
            trond Trond Norbye
            Reporter:
            xing xing
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes