Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-63084

Incomplete results while filtering vector data with scalar field

    XMLWordPrintable

Details

    • Untriaged
    • 0
    • Unknown

    Description

      I am using this cluster 

          172.23.110.53:kv
          172.23.110.54:kv
          172.23.110.72:index
          172.23.110.55:kv
          172.23.110.56:n1ql
          172.23.110.71:kv 

      While querying on 100M SIFT vector data "dimension": 128, using description: IVF,PQ128x8, similarity : L2, with 

      12:36:23 2024-08-08T00:06:23 [INFO] query_statement: SELECT meta().id from `bucket-1`.`scope-1`.`collection-1` where id<1000 ORDER BY ANN(emb, [3.0, 9.0, 17.0, 78.0, 83.0, 15.0, 10.0, 8.0, 101.0, 109.0, 21.0, 8.0, 3.0, 2.0, 9.0, 64.0, 39.0, 31.0, 18.0, 80.0, 55.0, 10.0, 2.0, 12.0, 7.0, 7.0, 26.0, 58.0, 32.0, 6.0, 4.0, 3.0, 14.0, 2.0, 13.0, 28.0, 37.0, 19.0, 47.0, 59.0, 109.0, 22.0, 2.0, 6.0, 18.0, 15.0, 20.0, 109.0, 30.0, 8.0, 11.0, 44.0, 109.0, 54.0, 19.0, 32.0, 17.0, 21.0, 15.0, 22.0, 12.0, 28.0, 101.0, 35.0, 66.0, 11.0, 9.0, 30.0, 68.0, 35.0, 30.0, 75.0, 106.0, 103.0, 26.0, 50.0, 76.0, 20.0, 8.0, 13.0, 51.0, 41.0, 63.0, 109.0, 40.0, 2.0, 3.0, 15.0, 36.0, 49.0, 21.0, 13.0, 12.0, 9.0, 36.0, 37.0, 52.0, 37.0, 24.0, 34.0, 19.0, 3.0, 13.0, 23.0, 21.0, 8.0, 3.0, 20.0, 68.0, 56.0, 79.0, 60.0, 99.0, 36.0, 7.0, 28.0, 78.0, 41.0, 7.0, 21.0, 74.0, 26.0, 3.0, 15.0, 34.0, 15.0, 12.0, 27.0],'L2',30) LIMIT 10

      12:36:23 2024-08-08T00:06:23 [INFO] result['725', '245', '506', '682'] 

      getting only 4 nearest results while limiting top10 results.

      and for some queries:

      12:36:23 2024-08-08T00:06:23 [INFO] query_statement: SELECT meta().id from `bucket-1`.`scope-1`.`collection-1` where id<1000 ORDER BY ANN(emb, [98.0, 3.0, 0.0, 0.0, 0.0, 1.0, 9.0, 13.0, 176.0, 38.0, 0.0, 0.0, 0.0, 0.0, 0.0, 3.0, 112.0, 18.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 10.0, 8.0, 20.0, 19.0, 1.0, 0.0, 0.0, 0.0, 80.0, 1.0, 0.0, 0.0, 0.0, 0.0, 21.0, 22.0, 176.0, 23.0, 0.0, 0.0, 0.0, 0.0, 0.0, 9.0, 167.0, 12.0, 0.0, 1.0, 2.0, 2.0, 0.0, 7.0, 5.0, 1.0, 14.0, 44.0, 43.0, 47.0, 10.0, 2.0, 42.0, 0.0, 0.0, 0.0, 0.0, 0.0, 40.0, 31.0, 176.0, 9.0, 0.0, 0.0, 0.0, 0.0, 2.0, 25.0, 176.0, 16.0, 0.0, 0.0, 0.0, 1.0, 0.0, 3.0, 17.0, 4.0, 0.0, 0.0, 5.0, 43.0, 20.0, 2.0, 13.0, 0.0, 0.0, 0.0, 0.0, 0.0, 52.0, 26.0, 176.0, 8.0, 0.0, 0.0, 0.0, 0.0, 8.0, 34.0, 152.0, 19.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 25.0, 2.0, 0.0, 0.0, 1.0, 1.0, 0.0, 3.0],'L2',30) LIMIT 10
      12:36:23 2024-08-08T00:06:23 [INFO] result[]
      12:36:23 2024-08-08T00:06:23 [INFO] ids: {'requestID': 'b821d8bc-2079-45c5-b5e2-bf71796b6a6c', 'signature': {'id': 'json'}, 'results': [], 'status': 'success', 'metrics': {'elapsedTime': '36.306438ms', 'executionTime': '36.146432ms', 'resultCount': 0, 'resultSize': 0, 'serviceLoad': 0}} 

      query is returning 0 results.

      https://perf.jenkins.couchbase.com/job/aether_with_remote/68/console

      Logs : 

      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-aether_with_remote-68/172.23.110.53.zip - kv
      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-aether_with_remote-68/172.23.110.54.zip - kv 
      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-aether_with_remote-68/172.23.110.55.zip - index
      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-aether_with_remote-68/172.23.110.56.zip - kv
      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-aether_with_remote-68/172.23.110.71.zip - n1ql
      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-aether_with_remote-68/172.23.110.72.zip - kv

       

       

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              sai.teja Sai Krishna Teja
              nikhil.anandkumar Nikhil Anand Kumar
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty