Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-62404

[Windows][Vector Search] : Data ingestion to index hung for 15+ hours

    XMLWordPrintable

Details

    Description

      Steps to reproduce the issue

      1. Created a 1 node kv, 1 node fts cluster
      2. Created a bucket and loaded 2.1M docs onto it(4096 dims)
      3. Created a vector index with 1 partition 0 replica
      4. Data ingestion stops at around 790k docs

      For windows cluster

      1. fts doc count is stuck since 14 hours

      2. The memory used is very high and does not reduce. The quota is around 30 GB and it doesn't use it all up

      3. The total gc increases linearly 

      For Linux cluster

      Did a comparison with a cluster on Ubuntu24 with the same set of steps and scenarios

      1. fts doc count is 2.1M within 1 hour

      2. The memory used reduces after indexing is complete

      Attachments

        1. image.png
          113 kB
          Rahul Rampure
        2. image (1).png
          78 kB
          Rahul Rampure
        3. image-2024-06-19-13-46-35-634.png
          124 kB
          Raghav S K
        4. image-2024-06-19-13-47-06-455.png
          166 kB
          Raghav S K
        5. image-2024-06-19-13-48-18-573.png
          370 kB
          Raghav S K
        6. image-2024-06-19-13-53-40-148.png
          125 kB
          Raghav S K
        7. image-2024-06-19-13-54-34-671.png
          139 kB
          Raghav S K
        8. image-2024-06-19-13-55-11-382.png
          1.67 MB
          Raghav S K
        9. image-2024-06-19-13-55-34-021.png
          1.62 MB
          Raghav S K
        10. image-2024-06-27-19-28-02-838.png
          125 kB
          Rahul Rampure
        11. image-2024-06-27-19-30-42-692.png
          125 kB
          Rahul Rampure
        12. image-2024-06-27-19-31-28-175.png
          192 kB
          Rahul Rampure
        13. image-2024-07-01-07-02-42-769.png
          154 kB
          Rahul Rampure
        14. Screenshot 2024-06-19 at 3.32.34 PM.png
          57 kB
          Abhi Dangeti
        15. Screenshot 2024-06-27 at 1.51.05 PM.png
          357 kB
          Raghav S K

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. MB-62494 [Windows][Vector Search] : Unable to create and use multiple indexes

          • Blocker - Blocks development and/or testing work, production could not run.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MB-62393 [Windows][Vector Search] : Memory/CPU utilization does not reduce even after dropping all indexes

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            People

              raghav.sk Raghav S K
              raghav.sk Raghav S K
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                PagerDuty