Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-4511

JSON encoding during View Indexing over escapes double quotes in values

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.0-developer-preview-3
    • Fix Version/s: 2.0-developer-preview-3
    • Component/s: couchbase-bucket
    • Security Level: Public
    • Labels:
      None
    • Flagged:
      Impediment

      Description

      I'm getting "invalid_json" warnings during indexing/view runs for JSON docs that are indeed valid both prior to being added to the bucket and after retrieval in the UI.

      It seems to center around escaped double quotes in content.

      Steps to reproduce:
      1. add both of the attached .json docs to a new, empty bucket.
      2. access any of the production views. The two that should return results with this single, content document in place are "brewery_beers" and "by_key_value"

      After clicking "Show Results" for one of these views (or otherwise triggering view generation), Couchbase Server will generate an invalid_json error message (see the "json_encoding_issue" attached).

      An example of the invalidated JSON is below (abbreviated "description" value from the IPA doc):
      \"Esquire Magazine calls our 90 Minute .IPA., \"\"\\\"\"\"perhaps the best I.P.A. in America.\"\"\\\"\"\" An Imperial I.P.A. brewed to be savored from a snifter....\"

      The invalid_json error is specifically:
      "parse error: after key and value, inside map, I expect ',' or '}'\n"

      Additionally, we should add a test for this, but that likely deserves it's own ticket.

      1. beer_90_Minute_IPA.json
        0.5 kB
        BigBlueHat
      2. beer.json
        1 kB
        BigBlueHat
      3. json_encoding_issue
        12 kB
        BigBlueHat
      # Subject Project Status CR V
      For Gerrit Dashboard: &For+MB-4511=message:MB-4511

        Activity

        BigBlueHat Benjamin Young created issue -
        alkondratenko Aleksey Kondratenko (Inactive) made changes -
        Field Original Value New Value
        Assignee Aleksey Kondratenko [ alkondratenko ]
        Hide
        alkondratenko Aleksey Kondratenko (Inactive) added a comment -

        traced down to ejson and/or mccouch issue. By printing original value and value from validate here's what I get:

        Value: <<"

        {\"brewery\":\"Dogfish Head Craft Brewery\",\"name\":\"90 Minute IPA\",\"abv\":\"9\",\"description\":\"Esquire Magazine calls our 90 Minute .IPA., \\\"perhaps the best I.P.A. in America.\\\" An Imperial I.P.A. brewed to be savored from a snifter. A big beer with a great malt backbone that stands up to the extreme hopping rate. This beer is an excellent candidate for use with Randall The Enamel Animal!\",\"category\":\"North American Ale\",\"style\":\"Imperial or Double India Pale Ale\",\"updated\":\"2010-07-22 20:00:20\"}

        ">>
        Json:<<"\"brewery\":\"Dogfish Head Craft Brewery\",\"name\":\"90 Minute IPA\",\"abv\":\"9\",\"description\":\"Esquire Magazine calls our 90 Minute .IPA., \"\"\\\"\"\"perhaps the best I.P.A. in America.\"\"\\\"\"\" An Imperial I.P.A. brewed to be savored from a snifter. A big beer with a great malt backbone that stands up to the extreme hopping rate. This beer is an excellent candidate for use with Randall The Enamel Animal!\",\"category\":\"North American Ale\",\"style\":\"Imperial or Double India Pale Ale\",\"updated\":\"2010-07-22 20:00:20\"}">>

        So ejson:validate is messing with escaped quotes. Assigning to Filipe, who is familiar with ejson.

        Show
        alkondratenko Aleksey Kondratenko (Inactive) added a comment - traced down to ejson and/or mccouch issue. By printing original value and value from validate here's what I get: Value: <<" {\"brewery\":\"Dogfish Head Craft Brewery\",\"name\":\"90 Minute IPA\",\"abv\":\"9\",\"description\":\"Esquire Magazine calls our 90 Minute .IPA., \\\"perhaps the best I.P.A. in America.\\\" An Imperial I.P.A. brewed to be savored from a snifter. A big beer with a great malt backbone that stands up to the extreme hopping rate. This beer is an excellent candidate for use with Randall The Enamel Animal!\",\"category\":\"North American Ale\",\"style\":\"Imperial or Double India Pale Ale\",\"updated\":\"2010-07-22 20:00:20\"} ">> Json:<<"\"brewery\":\"Dogfish Head Craft Brewery\",\"name\":\"90 Minute IPA\",\"abv\":\"9\",\"description\":\"Esquire Magazine calls our 90 Minute .IPA., \"\"\\\"\"\"perhaps the best I.P.A. in America.\"\"\\\"\"\" An Imperial I.P.A. brewed to be savored from a snifter. A big beer with a great malt backbone that stands up to the extreme hopping rate. This beer is an excellent candidate for use with Randall The Enamel Animal!\",\"category\":\"North American Ale\",\"style\":\"Imperial or Double India Pale Ale\",\"updated\":\"2010-07-22 20:00:20\"}">> So ejson:validate is messing with escaped quotes. Assigning to Filipe, who is familiar with ejson.
        alkondratenko Aleksey Kondratenko (Inactive) made changes -
        Assignee Aleksey Kondratenko [ alkondratenko ] Filipe Manana [ filipe manana ]
        Hide
        filipe manana filipe manana added a comment -

        Assigning to Aaron, as he wrote the validation feature for ejson recently. It's only enabled bia mccouch afaik.

        Show
        filipe manana filipe manana added a comment - Assigning to Aaron, as he wrote the validation feature for ejson recently. It's only enabled bia mccouch afaik.
        filipe manana filipe manana made changes -
        Assignee Filipe Manana [ filipe manana ] Aaron Miller [ aaron ]
        Hide
        alkondratenko Aleksey Kondratenko (Inactive) added a comment -

        (n_0@127.0.0.1)11> io:format("~s~n", [ejson:decode(<<"\"a\\\"sd\"">>)]).
        a"sd
        ok
        (n_0@127.0.0.1)12> io:format("~s~n", [element(2, ejson:validate(<<"\"a\\\"sd\"">>, <<"_">>))]).
        "a""\"""sd"
        ok

        Show
        alkondratenko Aleksey Kondratenko (Inactive) added a comment - (n_0@127.0.0.1)11> io:format("~s~n", [ejson:decode(<<"\"a\\\"sd\"">>)] ). a"sd ok (n_0@127.0.0.1)12> io:format("~s~n", [element(2, ejson:validate(<<"\"a\\\"sd\"">>, <<"_">>))] ). "a""\"""sd" ok
        alkondratenko Aleksey Kondratenko (Inactive) made changes -
        Fix Version/s 2.0-developer-preview-3 [ 10247 ]
        Affects Version/s 2.0-developer-preview-3 [ 10247 ]
        Priority Major [ 3 ] Critical [ 2 ]
        Hide
        filipe manana filipe manana added a comment -

        A side note, ejson:validate functionality should definitely have unit tests. Unfortunately it's not the current case.
        Features so self contained like this, should always have unit tests.

        Show
        filipe manana filipe manana added a comment - A side note, ejson:validate functionality should definitely have unit tests. Unfortunately it's not the current case. Features so self contained like this, should always have unit tests.
        sharon Sharon Barr (Inactive) made changes -
        Assignee Aaron Miller [ aaron ] Damien Katz [ damien ]
        Hide
        sharon Sharon Barr (Inactive) added a comment -

        Damien, can you fix that along your changes?

        Show
        sharon Sharon Barr (Inactive) added a comment - Damien, can you fix that along your changes?
        Show
        sharon Sharon Barr (Inactive) added a comment - http://review.couchbase.org/#change,11528
        sharon Sharon Barr (Inactive) made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Show
        alkondratenko Aleksey Kondratenko (Inactive) added a comment - http://review.couchbase.org/11552
        Hide
        thuan Thuan Nguyen added a comment -

        Integrated in github-couchdb-preview #221 (See http://qa.hq.northscale.net/job/github-couchdb-preview/221/)

        Show
        thuan Thuan Nguyen added a comment - Integrated in github-couchdb-preview #221 (See http://qa.hq.northscale.net/job/github-couchdb-preview/221/ )
        Hide
        thuan Thuan Nguyen added a comment -

        Integrated in github-couchdb-preview #226 (See http://qa.hq.northscale.net/job/github-couchdb-preview/226/)

        Show
        thuan Thuan Nguyen added a comment - Integrated in github-couchdb-preview #226 (See http://qa.hq.northscale.net/job/github-couchdb-preview/226/ )
        peter peter made changes -
        Component/s bucket-engine [ 10010 ]
        Component/s mccouch [ 10071 ]
        maria Maria McDuff (Inactive) made changes -
        Component/s couchbase-bucket [ 10173 ]
        Component/s view-engine [ 10060 ]
        Component/s bucket-engine [ 10010 ]

          People

          • Assignee:
            damien damien
            Reporter:
            BigBlueHat Benjamin Young
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes