Uploaded image for project: 'Couchbase Java Client'
  1. Couchbase Java Client
  2. JCBC-241

Paginator object doesn't move onto next "page" correctly

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 1.1.2
    • Fix Version/s: 1.1.6
    • Component/s: Core
    • Security Level: Public
    • Labels:
      None
    • Sprint:
      Sprint 1 - CW 19 & 20

      Description

      Customer found his own issue:
      I had a look at the source code for the java driver and found the error there. It is in:

      com.couchbase.client.protocol.views.Paginator.getNextPage()

      Right after calculating the number of "remaining" keys, it sets the limit of the query to the remaining number of keys and re-runs the query. What is missing here is:

      q.setSkip(totalDocs);

      To actually start from the entry after the last entry on the current page. I have tried this and it works. But there is more code in this method and it is not very easy to follow so important to make sure that this fits.

      I am enclosing a diff.

      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        perry Perry Krug created issue -
        perry Perry Krug made changes -
        Field Original Value New Value
        Description Customer found his own issue:
        I had a look at the source code for the java driver and found the error there. It is in:

        com.couchbase.client.protocol.views.Paginator.getNextPage()

        Right after calculating the number of "remaining" keys, it sets the limit of the query to the remaining number of keys and re-runs the query. What is missing here is:

            q.setSkip(totalDocs);

        To actually start from the entry after the last entry on the current page. I have tried this and it works. But there is more code in this method and it is not very easy to follow so important to make sure that this fits.

        Got response containing: 50
        Done iterating through page, #missing keys: 51
        Got response containing: 50
        Done iterating through page, #missing keys: 1
        Got response containing: 1
        Done iterating through page, #missing keys: 0

        I am enclosing a diff.
        Customer found his own issue:
        I had a look at the source code for the java driver and found the error there. It is in:

        com.couchbase.client.protocol.views.Paginator.getNextPage()

        Right after calculating the number of "remaining" keys, it sets the limit of the query to the remaining number of keys and re-runs the query. What is missing here is:

            q.setSkip(totalDocs);

        To actually start from the entry after the last entry on the current page. I have tried this and it works. But there is more code in this method and it is not very easy to follow so important to make sure that this fits.


        I am enclosing a diff.
        Hide
        perry Perry Krug added a comment -

        Update, but the bug still stands as described.

        It seems that my fix is not the correct one. The bug seem to be that although the Paginator specifies setStartkeyDocID, the query doesn't start from this key.

        I have debugged the Paginator class – and when resolving the next page it instructs the query to start from a specific doc id, e.g:

        Query: ?limit=50&startkey=144&skip=0&stale=false&startkey_docid=144

        But as a result of this query I get:

        Got key: 1
        Got key: 10
        Got key: 100
        Got key: 101
        Got key: 102
        Got key: 103

        Got key: 141
        Got key: 142
        Got key: 143

        Every time. So the bug is in the Query context somewhere.

        Also – a note of inefficiency in the Paginator class: why perform two queries every time? One for the content of the "page" and one to find the row for the next page? Would be better to query for page + 1 and keep the last separate from the ViewResponse.

        So, what I can conclude is that when the Paginator bug is resolved it should work fine for us. The additional query for the last row could be an issue though, since it uses setSkip(). So could this be treated as a bug as well?

        Show
        perry Perry Krug added a comment - Update, but the bug still stands as described. It seems that my fix is not the correct one. The bug seem to be that although the Paginator specifies setStartkeyDocID, the query doesn't start from this key. I have debugged the Paginator class – and when resolving the next page it instructs the query to start from a specific doc id, e.g: Query: ?limit=50&startkey=144&skip=0&stale=false&startkey_docid=144 But as a result of this query I get: Got key: 1 Got key: 10 Got key: 100 Got key: 101 Got key: 102 Got key: 103 … Got key: 141 Got key: 142 Got key: 143 Every time. So the bug is in the Query context somewhere. Also – a note of inefficiency in the Paginator class: why perform two queries every time? One for the content of the "page" and one to find the row for the next page? Would be better to query for page + 1 and keep the last separate from the ViewResponse. So, what I can conclude is that when the Paginator bug is resolved it should work fine for us. The additional query for the last row could be an issue though, since it uses setSkip(). So could this be treated as a bug as well?
        perry Perry Krug made changes -
        Link This issue depends on CBSE-394 [ CBSE-394 ]
        daschl Michael Nitschinger made changes -
        Fix Version/s 1.1.3 [ 10496 ]
        Priority Major [ 3 ] Blocker [ 1 ]
        daschl Michael Nitschinger made changes -
        Planned Start (set to new fixed version's start date)
        Planned End (set to new fixed version's start date)
        ingenthr Matt Ingenthron made changes -
        Assignee Michael Nitschinger [ daschl ] Matt Ingenthron [ ingenthr ]
        ingenthr Matt Ingenthron made changes -
        Planned End (re-schedule end date based on new assignee)
        ingenthr Matt Ingenthron made changes -
        Assignee Matt Ingenthron [ ingenthr ] Michael Nitschinger [ daschl ]
        ingenthr Matt Ingenthron made changes -
        Planned End (re-schedule end date based on new assignee)
        daschl Michael Nitschinger made changes -
        Fix Version/s 1.1.4 [ 10514 ]
        Fix Version/s 1.1.3 [ 10496 ]
        daschl Michael Nitschinger made changes -
        Planned Start (set to new fixed version's start date)
        Planned End (set to new fixed version's start date)
        daschl Michael Nitschinger made changes -
        Fix Version/s 1.1.5 [ 10515 ]
        Fix Version/s 1.1.4 [ 10514 ]
        daschl Michael Nitschinger made changes -
        Planned Start (set to new fixed version's start date)
        Planned End (set to new fixed version's start date)
        daschl Michael Nitschinger made changes -
        Fix Version/s 1.1.6 [ 10531 ]
        Fix Version/s 1.1.5 [ 10515 ]
        daschl Michael Nitschinger made changes -
        Rank Ranked higher
        daschl Michael Nitschinger made changes -
        Rank Ranked higher
        daschl Michael Nitschinger made changes -
        Sprint Sprint 1 - CW 19 & 20 [ 13 ]
        daschl Michael Nitschinger made changes -
        Status Open [ 1 ] In Progress [ 3 ]
        daschl Michael Nitschinger made changes -
        Status In Progress [ 3 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        ingenthr Matt Ingenthron made changes -
        Workflow jira [ 24446 ] Couchbase SDK Workflow [ 38521 ]

          People

          • Assignee:
            daschl Michael Nitschinger
            Reporter:
            perry Perry Krug
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Agile

                Gerrit Reviews

                There are no open Gerrit changes