Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-10273

View compaction should be triggered predictably, especially with parallel compaction enabled

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Blocker
    • 3.0
    • 3.0
    • ns_server, view-engine
    • Security Level: Public
    • Build 3.0.0-378.

      Platform = Physical
      OS = CentOS 6.5
      CPU = Intel Xeon E5-2630
      Memory = 64 GB
      Disk = 2 x SSD

    Description

      ORIGINAL TITLE: View compaction doesn't catch up in basic non-DGM tests with view queries

      4 nodes, 1 bucket x 20M x 2KB, non-DGM, 4 ddocs, 1 view per ddoc.
      10K op/sec (read-heavy), 400 queries/sec.

      Data and index are on different SSD drives, parallel compaction is enabled.

      See attached screenshot that compares index fragmentation between 2.5 and 3.0.

      I observe similar issue with data compaction where way higher drain rate causes IO saturation. However in this case:
      1. Isolated index compaction is known to be faster.
      2. There is enough CPU and disk bandwidth.

      Attachments

        1. 3.0.0-558_couch_views_fragmentation_dgm.png
          59 kB
          Pavel Paulau
        2. 3.0.0-558_couch_views_fragmentation_non_dgm.png
          68 kB
          Pavel Paulau
        3. couch_total_disk_size_b731.png
          49 kB
          Pavel Paulau
        4. couch_views_fragmentation_b731.png
          51 kB
          Pavel Paulau
        5. couch_views_fragmentation.png
          62 kB
          Pavel Paulau
        6. disk_data_size_2.5.1-1083.png
          48 kB
          Pavel Paulau
        7. disk_data_size_3.0.0-859.png
          60 kB
          Pavel Paulau
        8. fragmentation_2.5.1-1083.png
          97 kB
          Pavel Paulau
        9. fragmentation_3.0.0-819.png
          91 kB
          Pavel Paulau
        10. fragmentation_3.0.0-859.png
          88 kB
          Pavel Paulau
        11. fragmentation_gaps.png
          60 kB
          Pavel Paulau
        12. latency_query_histo.png
          33 kB
          Pavel Paulau
        13. latency_query_lt90.png
          40 kB
          Pavel Paulau
        14. manual_compaction.png
          87 kB
          Pavel Paulau
        15. view_queries_comp_2.5.0_vs_3.0.0.pdf
          4.12 MB
          Pavel Paulau

        Issue Links

          depends on

          Bug - A problem which impairs or prevents the functions of the product. MB-11486 Erlang memory usage increases to 50GB in 10 minutes after "unexpected_binary", node obviously fails afterwards

          • Test Blocker - QE-related tests can't proceed
          • Closed
          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. MB-11523 [System Test] Very uneven view compaction across cluster [ranging from 0 to 92% on different nodes.]

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              pavelpaulau Pavel Paulau (Inactive)
              pavelpaulau Pavel Paulau (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                PagerDuty