Uploaded image for project: 'Couchbase Lite'
  1. Couchbase Lite
  2. CBL-4506

Replicator starts up slow for big database

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • Beryllium
    • 3.1.0
    • LiteCore
    • Security Level: Public
    • None
    • LiteCore 111, LiteCore 112
    • 8

    Description

      This is to follow up with an external PR, https://github.com/couchbase/couchbase-lite-core/pull/1415.

      This PR intends to addresses two performance bottle-necks,

      1. {{speedup nextExpiration query }}
      2. speedup unresolvedDocsEnumerator

      #1 has been found by other customer, and already fixed by CBL-4325.

      #2 is due to a query we invoke at the beginning of replicating, 

      Replicator::findExistingConflicts() calling unresolvedDocsEnumerator(). For database as large as 3GB, as the PR told, it could take several minutes. The author tried SQLite's "INDEXED BY" clause to have boosted the performance to "almost instantly." However, In SQLite's document, it is suggested  that, "the INDEXED BY clause is _not intended for use in tuning the performance of a query." 

      Jens Alfke pointed out that the query should not use ORDER BY. My reading of the code also shows it shouldn't use it in the Lithium. However, also as Jens Alfke pointed out, in Helium, because of BothKeyStores, we will do sort. (ORDER BY key causes SQLite not use the index of conflicts flag).

      This ticket is to explore if we can get SQLite to use the index. "Several minutes" is quite long before replicator starts the messaging.

      Attachments

        Activity

          People

            jianmin.zhao Jianmin Zhao
            jianmin.zhao Jianmin Zhao
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              PagerDuty