Uploaded image for project: 'Couchbase Lite'
  1. Couchbase Lite
  2. CBL-106

Replicator C++ exception: "properties excessively large"

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.5.1
    • Fix Version/s: 2.6.0
    • Component/s: LiteCore
    • Security Level: Public
    • Labels:

      Description

      From a forum post:

      I am using the replicator in Couchbase Lite 2.5.1 on iOS with Objective C.

      It all seems to work fine for the most part but recently it has started failing and it gives the following error in the Xcode log

      CouchbaseLite Replicator ERROR: {DBWorker#3}==> litecore::repl::DBWorker ->wss://{sync-gateway-address}
      CouchbaseLite Replicator ERROR: {DBWorker#3} Threw C++ exception: properties excessively large

      BLIP will throw this exception when a message is created or received if the size of its properties (headers) would be > 100KB.

      The question is, what would cause the properties to be that large? I'll look at the protocol docs and see what might trigger this.

        Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

          jens Jens Alfke created issue -
          Hide
          jens Jens Alfke added a comment -

          Protocol messages that have properties which could conceivably be large:

          • "rev" — the "history" property (list of rev IDs)
          • "subChanges" — each filter parameter is a property
          Show
          jens Jens Alfke added a comment - Protocol messages that have properties which could conceivably be large: "rev" — the "history" property (list of rev IDs) "subChanges" — each filter parameter is a property
          Hide
          jens Jens Alfke added a comment -

          I've asked the OP if he's perhaps giving a very large list of docIDs to `ReplicatorConfiguration.documentIDs`.

          As for the rev history: CBL doesn't store or send much history, so if this is the culprit it would have to be on the SG side (with LiteCore complaining when receiving the BLIP message.) But LiteCore sets the `maxHistory` property to 20 in every `changes` reply, so SG shouldn't be sending back any huge history.

          Show
          jens Jens Alfke added a comment - I've asked the OP if he's perhaps giving a very large list of docIDs to `ReplicatorConfiguration.documentIDs`. As for the rev history: CBL doesn't store or send much history, so if this is the culprit it would have to be on the SG side (with LiteCore complaining when receiving the BLIP message.) But LiteCore sets the `maxHistory` property to 20 in every `changes` reply, so SG shouldn't be sending back any huge history.
          Hide
          jens Jens Alfke added a comment -

          From the OP, on the forum:

          My only thought is that due to how the program works the documents get updated relatively regularly. It seems to be an issue that only lasts for a little while so my thought is this. (Based off a very limited understanding on how all of this works). The document properties get very large because there are a large number of revisions and when those are attempted to be pushed it fails. As time goes on, CBL automatically purges the older revisions and eventually reduces the size so that it can be synced. Again, I’m just spitballing here cause I don’t really know how all of this works.

          My reply:

          Hmmm… CBL is only supposed to keep track of the last 20 revisions. Even if it were keeping more, it would take a lot of revisions to trigger this; something like 2,000. Is it possible your app modifies a single document this many times between replications?

          Show
          jens Jens Alfke added a comment - From the OP, on the forum: My only thought is that due to how the program works the documents get updated relatively regularly. It seems to be an issue that only lasts for a little while so my thought is this. (Based off a very limited understanding on how all of this works). The document properties get very large because there are a large number of revisions and when those are attempted to be pushed it fails. As time goes on, CBL automatically purges the older revisions and eventually reduces the size so that it can be synced. Again, I’m just spitballing here cause I don’t really know how all of this works. My reply: Hmmm… CBL is only supposed to keep track of the last 20 revisions. Even if it were keeping more, it would take a lot of revisions to trigger this; something like 2,000. Is it possible your app modifies a single document this many times between replications?
          Hide
          jens Jens Alfke added a comment -

          I can reproduce the error in a unit test that pushes a document, modifies it 5000 times, then pushes it again.

          Show
          jens Jens Alfke added a comment - I can reproduce the error in a unit test that pushes a document, modifies it 5000 times, then pushes it again.
          daniel.petersen Daniel Petersen made changes -
          Field Original Value New Value
          Fix Version/s Mercury [ 16188 ]
          Hide
          jens Jens Alfke added a comment -
          Show
          jens Jens Alfke added a comment - Fixed and in review:  https://github.com/couchbase/couchbase-lite-core/pull/802
          jens Jens Alfke made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          jens Jens Alfke made changes -
          Actual Start 2019-06-24 18:36 (issue has been started)
          Hide
          build-team Couchbase Build Team added a comment -

          Build couchbase-lite-android-2.7.0-2 contains couchbase-lite-core commit 71d4a67 with commit message:
          Fixed C++ exception pushing huge numbers of revs

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-lite-android-2.7.0-2 contains couchbase-lite-core commit 71d4a67 with commit message: Fixed C++ exception pushing huge numbers of revs
          Hide
          build-team Couchbase Build Team added a comment -

          Build couchbase-lite-core-2.7.0-2 contains couchbase-lite-core commit 71d4a67 with commit message:
          Fixed C++ exception pushing huge numbers of revs

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-lite-core-2.7.0-2 contains couchbase-lite-core commit 71d4a67 with commit message: Fixed C++ exception pushing huge numbers of revs
          jens Jens Alfke made changes -
          Resolution Fixed [ 1 ]
          Status In Progress [ 3 ] Resolved [ 5 ]
          jens Jens Alfke made changes -
          Actual End 2019-06-25 20:35 (issue has been resolved)
          Hide
          build-team Couchbase Build Team added a comment -

          Build couchbase-lite-log-2.6.0-133 contains couchbase-lite-core commit 71d4a67 with commit message:
          Fixed C++ exception pushing huge numbers of revs

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-lite-log-2.6.0-133 contains couchbase-lite-core commit 71d4a67 with commit message: Fixed C++ exception pushing huge numbers of revs
          Hide
          build-team Couchbase Build Team added a comment -

          Build couchbase-lite-cblite-2.6.0-102 contains couchbase-lite-core commit 71d4a67 with commit message:
          Fixed C++ exception pushing huge numbers of revs

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-lite-cblite-2.6.0-102 contains couchbase-lite-core commit 71d4a67 with commit message: Fixed C++ exception pushing huge numbers of revs
          Hide
          build-team Couchbase Build Team added a comment -

          Build couchbase-lite-ios-2.7.0-7 contains couchbase-lite-core commit 71d4a67 with commit message:
          Fixed C++ exception pushing huge numbers of revs

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-lite-ios-2.7.0-7 contains couchbase-lite-core commit 71d4a67 with commit message: Fixed C++ exception pushing huge numbers of revs
          daniel.petersen Daniel Petersen made changes -
          Fix Version/s Cobalt [ 16173 ]
          Fix Version/s Mercury [ 16188 ]

            People

            • Assignee:
              jens Jens Alfke
              Reporter:
              jens Jens Alfke
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Gerrit Reviews

                There are no open Gerrit changes

                  PagerDuty

                  Error rendering 'com.pagerduty.jira-server-plugin:PagerDuty'. Please contact your Jira administrators.