Uploaded image for project: 'Couchbase PHP client library'
  1. Couchbase PHP client library
  2. PCBC-56

Couchbase.so - prepend/append integer subsequently only returns last value on "get"

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.0
    • Fix Version/s: 1.0.3, 1.1.0-dp1
    • Component/s: library
    • Security Level: Public
    • Labels:
      None
    • Environment:
      OS X 10.7, PHP 5.3.8, Apache 2.2.21, Couchbase.so 1.0.0, Couchbase Server 1.8.0

      Description

      Using couchbase.so, when I prepend or append an integer to an existing integer value, and subsequently "get" that key, only the most recently appended/prepended value is returned instead of the entire value of the key. Using the console I can verify that the actual server held value for the key is correct, it's just couchbase.so that doesn't return the correct value on "get". This problem seems to be isolated to integer values (which is why it hasn't been caught by your PHP Tests, which only tests string value appending/prepending).

      Example PHP:

      if(!$cb->get("Test6"))

      { $cb->add("Test6",rand()); }

      $cb->prepend("Test6", rand().",");
      echo "<pre>";
      $vars = $cb->get("Test6");
      print_r($vars);

      only outputs a single value, no matter how often you reload and re-execute...

      However, when you check via console:

      get Test6
      VALUE Test6 1 41
      1407971623,1474693532,101417852,331145725
      END
      get Test6
      VALUE Test6 1 41
      1407971623,1474693532,101417852,331145725
      END
      get Test6
      VALUE Test6 1 230
      1582106666,1487511791,1969168774,1956718300,643137240,1558088382,719325115,1737977192,922644553,1697835667,363523321,175374370,2004964137,92468065,1381011077,1588161021,1237581990,31693720,1407971623,1474693532,101417852,331145725
      END

      See also http://www.couchbase.com/forums/thread/php-ext-get-after-prepend-or-append-only-returns-last-value-prependedappended

      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        Hide
        ingenthr Matt Ingenthron added a comment -

        Please try to identify the cause.

        Show
        ingenthr Matt Ingenthron added a comment - Please try to identify the cause.
        Hide
        avsej Sergey Avseyev added a comment -

        Actually this behaviour is OK:

        1) the server isn't aware about your data formats, it just concatenating binary streams
        2) you have chosen "," separator for values and I considering it is quite freeform, what are you expecting to get in the following values:

        • 111,,3333
        • aaa,1111
          and so on. What I'm trying to say is that this issue should be considered as a feature request. Because when you set the value first time, the client detected that you are setting integer value and selected appropriate flags combination for future get (to convert it back into integer). If you are trying to implement lists, you should split and convert them explicitly and set back after update (here you can use CAS value for optimistic lock). BTW there is nice article about implementing Set structure on application side using append, prepend operations: http://dustin.github.com/2011/02/17/memcached-set.html

        Another case is do not do guessing of types like the php-memcached client does (and this client too). In ruby client I have only three data modes and all of them won't cause any problems:

        • PLAIN mode, is just a bytestream, it is handy for implementing custom data structures or performing specific server-side operations like append
        • DOCUMENT mode, when all values should be primitives which can be encoded into JSON automatically by using parser, arrays, hashes, strings, numbers. Obviously append/prepend could break this format
        • MARSHAL mode, this is standard serialization mechanism, I guess such a thing is available in almost any language. In this mode more values could be serialized transparently (comparing to DOCUMENT mode).

        The layout above shows how easily to avoid issues with guessing values. But implementing them here of course will break compatibility here. So regarding the issue topic: yes, it is possible to implement such parser, but in this case the library will be in charge of the format rules (separators, empty values etc.)

        Show
        avsej Sergey Avseyev added a comment - Actually this behaviour is OK: 1) the server isn't aware about your data formats, it just concatenating binary streams 2) you have chosen "," separator for values and I considering it is quite freeform, what are you expecting to get in the following values: 111,,3333 aaa,1111 and so on. What I'm trying to say is that this issue should be considered as a feature request. Because when you set the value first time, the client detected that you are setting integer value and selected appropriate flags combination for future get (to convert it back into integer). If you are trying to implement lists, you should split and convert them explicitly and set back after update (here you can use CAS value for optimistic lock). BTW there is nice article about implementing Set structure on application side using append, prepend operations: http://dustin.github.com/2011/02/17/memcached-set.html Another case is do not do guessing of types like the php-memcached client does (and this client too). In ruby client I have only three data modes and all of them won't cause any problems: PLAIN mode, is just a bytestream, it is handy for implementing custom data structures or performing specific server-side operations like append DOCUMENT mode, when all values should be primitives which can be encoded into JSON automatically by using parser, arrays, hashes, strings, numbers. Obviously append/prepend could break this format MARSHAL mode, this is standard serialization mechanism, I guess such a thing is available in almost any language. In this mode more values could be serialized transparently (comparing to DOCUMENT mode). The layout above shows how easily to avoid issues with guessing values. But implementing them here of course will break compatibility here. So regarding the issue topic: yes, it is possible to implement such parser, but in this case the library will be in charge of the format rules (separators, empty values etc.)
        Hide
        mssteuer Michael Steuer added a comment -

        Shouldn't an append that I suppose changes the data type reset the flags on the client? Ie. I set 123 and then append "abc". I don't see how it's a feature that when I get that key, I don't get the server value but some assumption of the client as to what the server value should be? If an append can change the nature of a value, which it can, then it should also be able to change the client's perception of that value.

        I still think that when your client does NOT return the actual server value for a key, for whatever reason, it is a bug, not a feature. When I issue get(Key1), I should get the server value of Key1...

        Show
        mssteuer Michael Steuer added a comment - Shouldn't an append that I suppose changes the data type reset the flags on the client? Ie. I set 123 and then append "abc". I don't see how it's a feature that when I get that key, I don't get the server value but some assumption of the client as to what the server value should be? If an append can change the nature of a value, which it can, then it should also be able to change the client's perception of that value. I still think that when your client does NOT return the actual server value for a key, for whatever reason, it is a bug, not a feature. When I issue get(Key1), I should get the server value of Key1...
        Hide
        mssteuer Michael Steuer added a comment -

        Aside from that, thanks for the link re Set structure... good write up

        Show
        mssteuer Michael Steuer added a comment - Aside from that, thanks for the link re Set structure... good write up
        Show
        avsej Sergey Avseyev added a comment - - edited No, memcached protocol doesn't allow to pass flags with append/prepend commands. Compare https://github.com/membase/memcached/blob/engine/include/memcached/protocol_binary.h#L385-L390 to https://github.com/membase/memcached/blob/engine/include/memcached/protocol_binary.h#L302-L315
        Hide
        avsej Sergey Avseyev added a comment -

        > I still think that when your client does NOT return the actual server value for a key, for whatever reason, it is a bug, not a feature. When I issue get(Key1), I should get the server value of Key1...

        You are right, but php-couchbase (like php-memcached) has chosen the way with encoding datatypes in flags (not only serialization formats). In ruby client there is an option which allow to force serialization format (useful to workaround getting data in heterogeneous environments where another lib could write JSON without setting flags). I think adding such option here will be good compromise.

        In your case the GET function just reads only the first integer up to the comma (with atol), because it knows from flags that initial value was the integer.

        Show
        avsej Sergey Avseyev added a comment - > I still think that when your client does NOT return the actual server value for a key, for whatever reason, it is a bug, not a feature. When I issue get(Key1), I should get the server value of Key1... You are right, but php-couchbase (like php-memcached) has chosen the way with encoding datatypes in flags (not only serialization formats). In ruby client there is an option which allow to force serialization format (useful to workaround getting data in heterogeneous environments where another lib could write JSON without setting flags). I think adding such option here will be good compromise. In your case the GET function just reads only the first integer up to the comma (with atol), because it knows from flags that initial value was the integer.
        Hide
        jan Jan Lehnardt (Inactive) added a comment -

        Just a heads up: I'm in the final stages of making a release with the fix. Thanks for your support and patience!

        Show
        jan Jan Lehnardt (Inactive) added a comment - Just a heads up: I'm in the final stages of making a release with the fix. Thanks for your support and patience!

          People

          • Assignee:
            avsej Sergey Avseyev
            Reporter:
            mssteuer Michael Steuer
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes