Loading...

Details

Type: Bug
Resolution: Fixed
Priority: Critical
Fix Version/s: 7.6.4
Affects Version/s: 7.6.4
Component/s: fts
Labels:
- FTS
- Windows
- Windows10
- fts
- os-certification
- vector-search
- windows
Environment:
Operating System : Microsoft Windows Server 2022
Couchbase Enterprise Edition build 7.6.2-3720

Triage:
Untriaged
Operating System:
Windows 64-bit
Link to Log File, atop/blg, CBCollectInfo, Core dump:

Hide
Index def : https://cb-engineering.s3.amazonaws.com/MB-62616/index_def.json

Windows cluster with 16 vCPUs
https://cb-engineering.s3.amazonaws.com/MB-62610/collectinfo-2024-07-06T122524-ns_1%40ec2-3-91-45-121.compute-1.amazonaws.com.zip
https://cb-engineering.s3.amazonaws.com/MB-62610/collectinfo-2024-07-06T122524-ns_1%40ec2-35-173-49-119.compute-1.amazonaws.com.zip
Windows cluster with 8 vCPUs
https://cb-engineering.s3.amazonaws.com/MB-62616/collectinfo-2024-07-07T060700-ns_1%40ec2-18-214-40-150.compute-1.amazonaws.com.zip
https://cb-engineering.s3.amazonaws.com/MB-62616/collectinfo-2024-07-07T060700-ns_1%40ec2-44-199-247-80.compute-1.amazonaws.com.zip
Linux cluster with 16 vCPUs
https://cb-engineering.s3.amazonaws.com/MB-62616/collectinfo-2024-07-07T133104-ns_1%40ec2-3-80-108-22.compute-1.amazonaws.com.zip
https://cb-engineering.s3.amazonaws.com/MB-62616/collectinfo-2024-07-07T133104-ns_1%40ec2-54-167-54-27.compute-1.amazonaws.com.zip
Linux cluster with 8 vCPUs
https://cb-engineering.s3.amazonaws.com/MB-62616/collectinfo-2024-07-07T154610-ns_1%40ec2-3-239-22-162.compute-1.amazonaws.com.zip
https://cb-engineering.s3.amazonaws.com/MB-62616/collectinfo-2024-07-07T154610-ns_1%40ec2-52-203-187-107.compute-1.amazonaws.com.zip

Show
Index def : https://cb-engineering.s3.amazonaws.com/MB-62616/index_def.json Windows cluster with 16 vCPUs https://cb-engineering.s3.amazonaws.com/MB-62610/collectinfo-2024-07-06T122524-ns_1%40ec2-3-91-45-121.compute-1.amazonaws.com.zip https://cb-engineering.s3.amazonaws.com/MB-62610/collectinfo-2024-07-06T122524-ns_1%40ec2-35-173-49-119.compute-1.amazonaws.com.zip Windows cluster with 8 vCPUs https://cb-engineering.s3.amazonaws.com/MB-62616/collectinfo-2024-07-07T060700-ns_1%40ec2-18-214-40-150.compute-1.amazonaws.com.zip https://cb-engineering.s3.amazonaws.com/MB-62616/collectinfo-2024-07-07T060700-ns_1%40ec2-44-199-247-80.compute-1.amazonaws.com.zip Linux cluster with 16 vCPUs https://cb-engineering.s3.amazonaws.com/MB-62616/collectinfo-2024-07-07T133104-ns_1%40ec2-3-80-108-22.compute-1.amazonaws.com.zip https://cb-engineering.s3.amazonaws.com/MB-62616/collectinfo-2024-07-07T133104-ns_1%40ec2-54-167-54-27.compute-1.amazonaws.com.zip Linux cluster with 8 vCPUs https://cb-engineering.s3.amazonaws.com/MB-62616/collectinfo-2024-07-07T154610-ns_1%40ec2-3-239-22-162.compute-1.amazonaws.com.zip https://cb-engineering.s3.amazonaws.com/MB-62616/collectinfo-2024-07-07T154610-ns_1%40ec2-52-203-187-107.compute-1.amazonaws.com.zip
Story Points:
0
Is this a Regression?:
Yes

Description

Steps taken for this issue

Created a 2 node cluster - 1 kv, 1 fts node
Created a magma bucket and loaded 10M docs onto it (4096 dims)
Created a vector index - definition attached

The above set of steps was carried out on 16 vCPU machines and 8 vCPU machines in windows. For comparison, the same was also executed on Linux systems.

For 16 vCPU systems - c6a.4xlarge EC2 instances were used

For 8 vCPU systems - t3.2xlarge EC2 instances were used.

The same random data was used in all the clusters. It was loaded onto one of the clusters and was replicated onto all the others using XDCR.

Observed the indexing stats at this point of time

Listing down the observations

In windows systems the sysproc CPU utilization is around 200% regardless of the number of vCPUs.
In linux machines, the sysproc CPU utilization goes to almost 1600% in a 16 vCPU node cluster and 800% in a 8 vCPU node cluster
In Linux clusters, overall CPU utilization always oscillates around 80-100% during indexing indicating good resource utilization
In Windows clusters, the resource utilization is very low.

Windows - 16 vCPU machines