Uploaded image for project: 'Couchbase Python Client Library'
  1. Couchbase Python Client Library
  2. PYCBC-451

Python SDK Documentation could use example of upsert involving JSON text

    XMLWordPrintable

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.3.4
    • docs
    • None

    Description

      I have a Python script that retrieves JSON documents from Google AppEngine and needs to upsert them into a Couchbase bucket. The documentation on 'upsert', found here:

      https://developer.couchbase.com/documentation/server/current/sdk/python/start-using-sdk.html

      provides an example of 'upsert' using Python objects, e.g., 

      cb.upsert('u:king_arthur', {'name': 'Arthur', 'email': 'kingarthur@couchbase.com', 'interests': ['Holy Grail', 'African Swallows']})

      The more detailed API documentation, found here:

      http://pythonhosted.org/couchbase/api/couchbase.html

      Also uses Python objects. But there is no example showing me what to do if I have a string of JSON. What I'd like to see is something like:

      JSONstr = "{\"key1\": \"value1\", \"key2\": 123}"

      JSONobj = json.loads(JSONstr)

      cb.upsert("documentID",JSONobj)

       

      Attachments

        For Gerrit Dashboard: PYCBC-451
        # Subject Branch Project Status CR V

        Activity

          I do not think it's a very common ask as most libraries automatically convert any JSON from REST APIs to dictionaries (e.g. http://docs.python-requests.org/en/master/).

          For most purposes you would want to load some data from a source, e.g. a file, a REST API and then perform some manipulation on it before storing elsewhere. If you're only passing around the json string this makes that very impractical other than for the most very basic use cases (e.g. load string from a file and then put it into CB without any changes).

          matt.carabine Matt Carabine added a comment - I do not think it's a very common ask as most libraries automatically convert any JSON from REST APIs to dictionaries (e.g. http://docs.python-requests.org/en/master/ ). For most purposes you would want to load some data from a source, e.g. a file, a REST API and then perform some manipulation on it before storing elsewhere. If you're only passing around the json string this makes that very impractical other than for the most very basic use cases (e.g. load string from a file and then put it into CB without any changes).
          Ellis.Breen Ellis Breen added a comment - - edited

          Hi, it appears you can already do this using the 'format=FMT_UTF8' setting, which tells the transcoder what the intended encoding is, e.g.:

          cb.upsert("documentID","{\"key1\": \"value1\", \"key2\": 123}", format=FMT_UTF8)

          I will add a line to the 'upsert' documentation in the Python API and site docs to make this clearer.

          On the get side, you can use 'no_format=True' to ensure the result is not decoded, as documented. You can then use whatever decoder you wish to use.

           

          self.cb.get("documentID", no_format=True)
          b'{"key1": "value1", "key2": 123}'
          

           

           

           

          Ellis.Breen Ellis Breen added a comment - - edited Hi, it appears you can already do this using the 'format=FMT_UTF8' setting, which tells the transcoder what the intended encoding is, e.g.: cb.upsert( "documentID" , "{\"key1\": \"value1\", \"key2\": 123}" , format=FMT_UTF8) I will add a line to the 'upsert' documentation in the Python API and site docs to make this clearer. On the get side, you can use 'no_format=True' to ensure the result is not decoded, as documented. You can then use whatever decoder you wish to use.   self.cb.get( "documentID" , no_format=True) b '{"key1": "value1", "key2": 123}'      

          This gets close, but not quite to the same place Ellis Breen, I think. What does the transcoder do with the flags in this case? Also, your documentation change doesn't show JSON. It shows XML?

          I've not run it, but I suspect Eben Haber's case may come up again, and showing JSON in an arbitrary string, converted to an object and stored is what's needed here. That'd set the flags.

          Using UTF-8 will get it in there and then we'll have JSON detection in the cluster, but with incorrect flags for clients, I suspect.

          ingenthr Matt Ingenthron added a comment - This gets close, but not quite to the same place Ellis Breen , I think. What does the transcoder do with the flags in this case? Also, your documentation change doesn't show JSON. It shows XML? I've not run it, but I suspect Eben Haber 's case may come up again, and showing JSON in an arbitrary string, converted to an object and stored is what's needed here. That'd set the flags. Using UTF-8 will get it in there and then we'll have JSON detection in the cluster, but with incorrect flags for clients, I suspect.
          Ellis.Breen Ellis Breen added a comment -

          Apologies, I did not appreciate that there are persisted flags regarding the type of the data. Will add documentation to the suggested effect ASAP.

          Ellis.Breen Ellis Breen added a comment - Apologies, I did not appreciate that there are persisted flags regarding the type of the data. Will add documentation to the suggested effect ASAP.
          Ellis.Breen Ellis Breen added a comment -

          Have added example as requested.

          Ellis.Breen Ellis Breen added a comment - Have added example as requested.

          People

            Ellis.Breen Ellis Breen
            eben Eben Haber
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty