Uploaded image for project: 'Couchbase Lite'
  1. Couchbase Lite
  2. CBL-106

Replicator C++ exception: "properties excessively large"

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.6.0, 2.5.1
    • 2.7.0
    • LiteCore
    • Security Level: Public

    Description

      From a forum post:

      I am using the replicator in Couchbase Lite 2.5.1 on iOS with Objective C.

      It all seems to work fine for the most part but recently it has started failing and it gives the following error in the Xcode log

      CouchbaseLite Replicator ERROR: {DBWorker#3}==> litecore::repl::DBWorker ->wss://{sync-gateway-address}
      CouchbaseLite Replicator ERROR: {DBWorker#3} Threw C++ exception: properties excessively large

      BLIP will throw this exception when a message is created or received if the size of its properties (headers) would be > 100KB.

      The question is, what would cause the properties to be that large? I'll look at the protocol docs and see what might trigger this.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          jens Jens Alfke created issue -
          jens Jens Alfke added a comment -

          Protocol messages that have properties which could conceivably be large:

          • "rev" — the "history" property (list of rev IDs)
          • "subChanges" — each filter parameter is a property
          jens Jens Alfke added a comment - Protocol messages that have properties which could conceivably be large: "rev" — the "history" property (list of rev IDs) "subChanges" — each filter parameter is a property
          jens Jens Alfke added a comment -

          I've asked the OP if he's perhaps giving a very large list of docIDs to `ReplicatorConfiguration.documentIDs`.

          As for the rev history: CBL doesn't store or send much history, so if this is the culprit it would have to be on the SG side (with LiteCore complaining when receiving the BLIP message.) But LiteCore sets the `maxHistory` property to 20 in every `changes` reply, so SG shouldn't be sending back any huge history.

          jens Jens Alfke added a comment - I've asked the OP if he's perhaps giving a very large list of docIDs to `ReplicatorConfiguration.documentIDs`. As for the rev history: CBL doesn't store or send much history, so if this is the culprit it would have to be on the SG side (with LiteCore complaining when receiving the BLIP message.) But LiteCore sets the `maxHistory` property to 20 in every `changes` reply, so SG shouldn't be sending back any huge history.
          jens Jens Alfke added a comment -

          From the OP, on the forum:

          My only thought is that due to how the program works the documents get updated relatively regularly. It seems to be an issue that only lasts for a little while so my thought is this. (Based off a very limited understanding on how all of this works). The document properties get very large because there are a large number of revisions and when those are attempted to be pushed it fails. As time goes on, CBL automatically purges the older revisions and eventually reduces the size so that it can be synced. Again, I’m just spitballing here cause I don’t really know how all of this works.

          My reply:

          Hmmm… CBL is only supposed to keep track of the last 20 revisions. Even if it were keeping more, it would take a lot of revisions to trigger this; something like 2,000. Is it possible your app modifies a single document this many times between replications?

          jens Jens Alfke added a comment - From the OP, on the forum: My only thought is that due to how the program works the documents get updated relatively regularly. It seems to be an issue that only lasts for a little while so my thought is this. (Based off a very limited understanding on how all of this works). The document properties get very large because there are a large number of revisions and when those are attempted to be pushed it fails. As time goes on, CBL automatically purges the older revisions and eventually reduces the size so that it can be synced. Again, I’m just spitballing here cause I don’t really know how all of this works. My reply: Hmmm… CBL is only supposed to keep track of the last 20 revisions. Even if it were keeping more, it would take a lot of revisions to trigger this; something like 2,000. Is it possible your app modifies a single document this many times between replications?
          jens Jens Alfke added a comment -

          I can reproduce the error in a unit test that pushes a document, modifies it 5000 times, then pushes it again.

          jens Jens Alfke added a comment - I can reproduce the error in a unit test that pushes a document, modifies it 5000 times, then pushes it again.
          daniel.petersen Daniel Petersen made changes -
          Field Original Value New Value
          Fix Version/s Mercury [ 16188 ]
          jens Jens Alfke added a comment - Fixed and in review:  https://github.com/couchbase/couchbase-lite-core/pull/802
          jens Jens Alfke made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          jens Jens Alfke made changes -
          Actual Start 2019-06-24 18:36 (issue has been started)

          Build couchbase-lite-android-2.7.0-2 contains couchbase-lite-core commit 71d4a67 with commit message:
          Fixed C++ exception pushing huge numbers of revs

          build-team Couchbase Build Team added a comment - Build couchbase-lite-android-2.7.0-2 contains couchbase-lite-core commit 71d4a67 with commit message: Fixed C++ exception pushing huge numbers of revs

          Build couchbase-lite-core-2.7.0-2 contains couchbase-lite-core commit 71d4a67 with commit message:
          Fixed C++ exception pushing huge numbers of revs

          build-team Couchbase Build Team added a comment - Build couchbase-lite-core-2.7.0-2 contains couchbase-lite-core commit 71d4a67 with commit message: Fixed C++ exception pushing huge numbers of revs
          jens Jens Alfke made changes -
          Resolution Fixed [ 1 ]
          Status In Progress [ 3 ] Resolved [ 5 ]
          jens Jens Alfke made changes -
          Actual End 2019-06-25 20:35 (issue has been resolved)

          Build couchbase-lite-log-2.6.0-133 contains couchbase-lite-core commit 71d4a67 with commit message:
          Fixed C++ exception pushing huge numbers of revs

          build-team Couchbase Build Team added a comment - Build couchbase-lite-log-2.6.0-133 contains couchbase-lite-core commit 71d4a67 with commit message: Fixed C++ exception pushing huge numbers of revs

          Build couchbase-lite-cblite-2.6.0-102 contains couchbase-lite-core commit 71d4a67 with commit message:
          Fixed C++ exception pushing huge numbers of revs

          build-team Couchbase Build Team added a comment - Build couchbase-lite-cblite-2.6.0-102 contains couchbase-lite-core commit 71d4a67 with commit message: Fixed C++ exception pushing huge numbers of revs

          Build couchbase-lite-ios-2.7.0-7 contains couchbase-lite-core commit 71d4a67 with commit message:
          Fixed C++ exception pushing huge numbers of revs

          build-team Couchbase Build Team added a comment - Build couchbase-lite-ios-2.7.0-7 contains couchbase-lite-core commit 71d4a67 with commit message: Fixed C++ exception pushing huge numbers of revs
          daniel.petersen Daniel Petersen made changes -
          Fix Version/s Cobalt [ 16173 ]
          Fix Version/s Mercury [ 16188 ]
          jimb Jim Borden added a comment -

          Hold on why is this fix version marked as 2.6.0?  I don't see any 2.6.0 builds containing it except for the mistaken cblite....

          jimb Jim Borden added a comment - Hold on why is this fix version marked as 2.6.0?  I don't see any 2.6.0 builds containing it except for the mistaken cblite....

          good catch - marking as mercury.

          daniel.petersen Daniel Petersen added a comment - good catch - marking as mercury.
          daniel.petersen Daniel Petersen made changes -
          Fix Version/s Mercury [ 16188 ]
          Fix Version/s 2.6.0 [ 16173 ]
          daniel.petersen Daniel Petersen made changes -
          Affects Version/s 2.6.0 [ 16173 ]
          daniel.petersen Daniel Petersen made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

          People

            jens Jens Alfke
            jens Jens Alfke
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty