Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-62404

[Windows][Vector Search] : Data ingestion to index hung for 15+ hours

    XMLWordPrintable

Details

    Description

      Steps to reproduce the issue

      1. Created a 1 node kv, 1 node fts cluster
      2. Created a bucket and loaded 2.1M docs onto it(4096 dims)
      3. Created a vector index with 1 partition 0 replica
      4. Data ingestion stops at around 790k docs

      For windows cluster

      1. fts doc count is stuck since 14 hours

      2. The memory used is very high and does not reduce. The quota is around 30 GB and it doesn't use it all up

      3. The total gc increases linearly 

      For Linux cluster

      Did a comparison with a cluster on Ubuntu24 with the same set of steps and scenarios

      1. fts doc count is 2.1M within 1 hour

      2. The memory used reduces after indexing is complete

      Attachments

        1. image-2024-06-19-13-46-35-634.png
          image-2024-06-19-13-46-35-634.png
          124 kB
        2. image-2024-06-19-13-47-06-455.png
          image-2024-06-19-13-47-06-455.png
          166 kB
        3. image-2024-06-19-13-48-18-573.png
          image-2024-06-19-13-48-18-573.png
          370 kB
        4. image-2024-06-19-13-53-40-148.png
          image-2024-06-19-13-53-40-148.png
          125 kB
        5. image-2024-06-19-13-54-34-671.png
          image-2024-06-19-13-54-34-671.png
          139 kB
        6. image-2024-06-19-13-55-11-382.png
          image-2024-06-19-13-55-11-382.png
          1.67 MB
        7. image-2024-06-19-13-55-34-021.png
          image-2024-06-19-13-55-34-021.png
          1.62 MB
        8. Screenshot 2024-06-19 at 3.32.34 PM.png
          Screenshot 2024-06-19 at 3.32.34 PM.png
          57 kB
        9. Screenshot 2024-06-27 at 1.51.05 PM.png
          Screenshot 2024-06-27 at 1.51.05 PM.png
          357 kB
        10. image-2024-06-27-19-28-02-838.png
          image-2024-06-27-19-28-02-838.png
          125 kB
        11. image-2024-06-27-19-30-42-692.png
          image-2024-06-27-19-30-42-692.png
          125 kB
        12. image-2024-06-27-19-31-28-175.png
          image-2024-06-27-19-31-28-175.png
          192 kB
        13. image.png
          image.png
          113 kB
        14. image (1).png
          image (1).png
          78 kB
        15. image-2024-07-01-07-02-42-769.png
          image-2024-07-01-07-02-42-769.png
          154 kB

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. MB-62494 [Windows][Vector Search] : Unable to create and use multiple indexes

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MB-62393 [Windows][Vector Search] : Memory/CPU utilization does not reduce even after dropping all indexes

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            People

              raghav.sk Raghav S K
              raghav.sk Raghav S K
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                PagerDuty