Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-16194

Accelerate GSI index and scan using RAM

    XMLWordPrintable

Details

    • Task
    • Resolution: Fixed
    • Critical
    • 4.5.0
    • 4.0.0
    • secondary-index
    • Security Level: Public
    • None

    Description

      Currently, GSI operations such as maximum supported mutation rate, stale=false query throughput and latency, and minimum snapshot generation interval fall below initial design targets. Address this by utilizing RAM differently to allow operations to start and end with RAM based structures alone. In this task, let us focus on strictly less than DGM indexes only.

      Attachments

        Issue Links

          is duplicated by

          Improvement - An improvement or enhancement to an existing feature or task. MB-16358 In-memory GSI indexes

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          For Gerrit Dashboard: MB-16194
          # Subject Branch Project Status CR V
          54931,2 MB-16194 Import memdb implementation unstable indexing Status: MERGED +2 +1
          54932,2 MB-16194 Add memdb slice integration unstable indexing Status: MERGED +2 +1
          54933,2 MB-16194 memdb: Create backindex snapshot only for non primary index unstable indexing Status: MERGED +2 +1
          54934,2 MB-16194 memdb: Fix backindex comparator unstable indexing Status: MERGED +2 +1
          54935,2 MB-16194 memdb Reuse the main index_entry for backindex item unstable indexing Status: MERGED +2 +1
          54936,2 MB-16194 Fix slice API to add mutation meta field for supporting multiwriters unstable indexing Status: MERGED +2 +1
          54937,2 MB-16194 memdb: Fix multiwriter out-of-order mutation distribution unstable indexing Status: MERGED +2 +1
          54938,2 MB-16194 memdb: Implement ondisk snapshots, recovery and rollback unstable indexing Status: MERGED +2 +1
          54939,2 MB-16194 memdb: Remove snapshotting for back index unstable indexing Status: MERGED +2 +1
          54940,2 MB-16194 memdb: RollbackToZero should delete ondisk snapshots unstable indexing Status: MERGED +2 +1
          54941,2 MB-16194 memdb: Add logging for ondisk snapshot timing unstable indexing Status: MERGED +2 +1
          54942,2 MB-16194 memdb: Use fixed length filename for snapshot file names unstable indexing Status: MERGED +2 +1
          54960,3 MB-16194 memdb: Periodically dump stats to log unstable indexing Status: MERGED +2 +1
          55168,1 MB-16194 memdb: import 2i copy of goforestdb unstable indexing Status: MERGED +2 +1
          55328,2 MB-16194 Update memdb unstable indexing Status: MERGED +2 +1
          55484,3 MB-16194 memdb: Add MemoryInUse() API unstable indexing Status: MERGED +2 +1
          55485,3 MB-16194 indexer: Account for memdb memory usage unstable indexing Status: MERGED +2 +1
          55689,2 MB-16194 Update memdb unstable indexing Status: MERGED +2 +1
          55690,2 MB-16194 memdb: Close memdb slice unstable indexing Status: MERGED +2 +1
          55896,1 MB-16194 memdb: Add perf test for memdb slice unstable indexing Status: MERGED +2 +1
          55897,1 MB-16194 memdb: Add snapshots in memdb slice test unstable indexing Status: MERGED +2 +1
          55898,2 MB-16194 Update memdb unstable indexing Status: MERGED +2 +1
          55899,2 MB-16194 memdb: Optimize incremental update performance unstable indexing Status: MERGED +2 +1
          55900,2 MB-16194 indexer: Use max ncpus as numSliceWriters unstable indexing Status: MERGED +2 +1
          55952,2 MB-16194 Add configurable num_docs for memdb slice test unstable indexing Status: MERGED +2 +1
          55953,2 MB-16194 memdb: Fix back index node linking for indexer restart unstable indexing Status: MERGED +2 +1
          56042,2 MB-16194 Enhance memdb slice test unstable indexing Status: MERGED +2 +1
          56044,2 MB-16194 memdb: Optimize primary index insert unstable indexing Status: MERGED +2 +1
          56593,1 MB-16194 Add index_type parameter in deployment plan unstable indexing Status: MERGED +2 +1
          56594,1 MB-16194 indexer: Add indexer.settings.default_index_type setting unstable indexing Status: ABANDONED 0 0
          57120,3 MB-16194 Add index_type definition in getIndexStatus API unstable indexing Status: MERGED +2 +1
          57231,2 MB-16194 Add REST endpoint to force freeing memory to OS unstable indexing Status: MERGED +2 +1
          57293,1 MB-16194 memdb: Use time.Sleep instead of ticker for flush wait unstable indexing Status: MERGED +2 +1
          57502,3 MB-16194 memdb: Partition backindex to one per writer unstable indexing Status: MERGED +2 +1
          58032,2 MB-16194 Update memdb - memory overhead reduction improvements unstable indexing Status: MERGED +2 +1
          58212,3 MB-16194 memdb: Fix backindex multimapping during bottom-up build unstable indexing Status: MERGED +2 +1

          Activity

            People

              sarath Sarath Lakshman
              sarath Sarath Lakshman
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty