Details
Description
We had been experimenting with using the streaming=True for our view iteration as we do not want to load the entire result set into memory before beginning processing and noticed that if a value is present in the view emission, we get an error in the json scan of the return object. This is the error that python spits back:
Traceback (most recent call last):
File "/Users/belbis/Desktop/stream_test.py", line 20, in <module>
for q in query1:
File "build/bdist.macosx-10.8-intel/egg/couchbase/views/iterator.py", line 390, in _iter_
File "build/bdist.macosx-10.8-intel/egg/couchbase/views/iterator.py", line 361, in _get_page
File "build/bdist.macosx-10.8-intel/egg/couchbase/views/iterator.py", line 361, in <genexpr>
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/_init_.py", line 326, in loads
return _default_decoder.decode(s)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 382, in raw_decode
obj, end = self.scan_once(s, idx)
ValueError: Expecting object: line 1 column 108 (char 108)
We did some investigation into the matter and if we catch this exception in the json.loads method and append a '}' to the end of the object being scanned, the error goes away. We are not sure how this occurs but we believe that the streaming=True parameter being passed to the view causes it to drop a the closing json character (or something else we're not exactly sure).
We are able to replicate this behavior in the sample bucket 'beer-sample' with a simple test.
I created a design doc 'b' and added two views 'stream_test1' and 'stream_test2'. The attached stream_test.py will attempt to iterate over the two views (both with streaming enabled and disabled). What you will see is that the view that emits a value will fail with streaming=True enabled. This error will also happen if you use the view iterator constructor so we assume that the problem is in the handling of the result object and not the iterator itself.
We would appreciate any input you guys could give on what's going on here. For the time being we are able to work around this by using only keys in views we need to stream. However certain processes are being bogged down by this issue.