Major Description
The analytics situational test I added that restarts the couchbase service fails consistently with an assertion in the lcb json library
/home/***/jenkins/workspace/c-cpp/lcb/qe-situational-lcb/lib***/contrib/lcb-jsoncpp/lcb-jsoncpp.cpp:2366: void Json::throwLogicError(const string&): Assertion `false' failed.
After investigating, the analytics service is returning a 503 error with a plaintext body of "503 Service Unavailable" at the moment the service is restarted (possible a separate issue).
lcb_ANALYTICS_HANDLE_::invoke_row does not check the content type so tries to decode it as json. "503 Service Unavailable" should be invalid JSON
jakerawsthorne@REML0715 libcouchbase % node
Welcome to Node.js v14.10.0.
Type ".help" for more information.
> JSON.parse("503 Service Unavailable")
Uncaught SyntaxError: Unexpected token S in JSON at position 4
however lcb-jsoncpp parses it as the number 503. Failing test case: http://review.couchbase.org/c/libcouchbase/+/160062
When invoke_row calls const Json::Value &errors = meta["errors"]; the assertion is hit because you can't index into a number
Attachments
Issue Links
- relates to
-
MB-48154 Analytics server returns errors as plain text instead of JSON
-
- Closed
-
Activity
| Status | New [ 10003 ] | Open [ 1 ] |
| Fix Version/s | 3.2.2 [ 17647 ] |
| Link | This issue relates to CBD-4306 [ CBD-4306 ] |
| Resolution | Fixed [ 1 ] | |
| Status | Open [ 1 ] | Resolved [ 5 ] |
Still seeing lcb crash but now in a different place http://sdk.jenkins.couchbase.com/blue/organizations/jenkins/c-cpp%2Flcb%2Fcentos-lcb-sdk-server-situational-tests/detail/centos-lcb-sdk-server-situational-tests/882/pipeline#log-1189
[785.07 DEBUG] (SSHCommand close:147) Closing channel com.jcraft.jsch.ChannelExec@156426d6
[785.86 INFO] (SDKD log:137) terminate called after throwing an instance of 'Json::LogicError'
[785.86 INFO] (SDKD log:137) what(): in Json::Value::resolveReference(key, end): requires objectValue
[785.90 DEBUG] (RunContext run:361) Caught exception in run
com.****.sdkdclient.context.HarnessException: Harness error of type: DRIVER (Driver crashed)
| Resolution | Fixed [ 1 ] | |
| Status | Resolved [ 5 ] | Reopened [ 4 ] |
Jake Rawsthorne are we sure that it does not come from sdkd-cpp? It also uses Json::Reader. The way how I patched it should ensure the exception will not bubble from the parser (we use embedded version of json-cpp): http://review.couchbase.org/c/libcouchbase/+/161750/1/contrib/lcb-jsoncpp/lcb-jsoncpp.cpp#335
Basically with this patch it catches all exceptions internally and returns false.
sdkd-cpp uses the parser only in three places, could you make sure in these places we catch exceptions around Json::Reader.parse() method invocation?: https://github.com/couchbase/sdkd-cpp/search?q=Json%3A%3AReader
| Resolution | Fixed [ 1 ] | |
| Status | Reopened [ 4 ] | Resolved [ 5 ] |
Build couchbase-server-7.1.0-1450 contains libcouchbase commit 1dedda9 with commit message:
CCBC-1472: ensure strict JSON parsing mode for query meta
Thanks for reporting this issue, I've opened ticket for MB to clarify this behaviour. Meanwhile I will make sure that bad JSON response will not break response handlers of HTTP services inside libcouchbase.