[Provisioned/Colocated Services]: FTS index build is extremely slow.

Test steps

1. Deploy an GCP cluster having 3 KV, GSI, N1QL, Query and FTS nodes:
2. Create a Magma bucket having single replica and 1 scope + 2 collections in addition to _default._default keyspace.
3. Load 50M docs in each of the 2 collections.
4. Create CBAS datasets and let them build complete.
5. Create 5 FTS Indexes per collection and wait for the Indexes to come online. FTS index build is taking very long time. Its still building since 8 hours.

   FTS Build progress
   2023-06-12 20:43:29,573 | test | DEBUG | MainThread | [hostedFTS:wait_for_fts_index_online:114] index: default0VolumeCollection0_fts_idx_3, status: 1, count: 46217866, expected: 50000000
   2023-06-12 20:43:34,927 | test | DEBUG | MainThread | [hostedFTS:wait_for_fts_index_online:114] index: default0VolumeCollection0_fts_idx_3, status: 1, count: 46219066, expected: 50000000
   2023-06-12 20:43:40,283 | test | DEBUG | MainThread | [hostedFTS:wait_for_fts_index_online:114] index: default0VolumeCollection0_fts_idx_3, status: 1, count: 46219866, expected: 50000000
   2023-06-12 20:43:45,632 | test | DEBUG | MainThread | [hostedFTS:wait_for_fts_index_online:114] index: default0VolumeCollection0_fts_idx_3, status: 1, count: 46220066, expected: 50000000
   2023-06-12 20:43:50,980 | test | DEBUG | MainThread | [hostedFTS:wait_for_fts_index_online:114] index: default0VolumeCollection0_fts_idx_3, status: 1, count: 46220466, expected: 50000000
   2023-06-12 20:43:56,331 | test | DEBUG | MainThread | [hostedFTS:wait_for_fts_index_online:114] index: default0VolumeCollection0_fts_idx_3, status: 1, count: 46220666, expected: 50000000
   2023-06-12 20:44:01,683 | test | DEBUG | MainThread | [hostedFTS:wait_for_fts_index_online:114] index: default0VolumeCollection0_fts_idx_3, status: 1, count: 46220866, expected: 50000000
   2023-06-12 20:44:07,039 | test | DEBUG | MainThread | [hostedFTS:wait_for_fts_index_online:114] index: default0VolumeCollection0_fts_idx_3, status: 1, count: 46225066, expected: 50000000
   2023-06-12 20:44:12,391 | test | DEBUG | MainThread | [hostedFTS:wait_for_fts_index_online:114] index: default0VolumeCollection0_fts_idx_3, status: 1, count: 46226266, expected: 50000000
   2023-06-12 20:44:17,737 | test | DEBUG | MainThread | [hostedFTS:wait_for_fts_index_online:114] index: default0VolumeCollection0_fts_idx_3, status: 1, count: 46226666, expected: 50000000
   2023-06-12 20:44:23,089 | test | DEBUG | MainThread | [hostedFTS:wait_for_fts_index_online:114] index: default0VolumeCollection0_fts_idx_3, status: 1, count: 46227866, expected: 50000000
   2023-06-12 20:44:28,448 | test | DEBUG | MainThread | [hostedFTS:wait_for_fts_index_online:114] index: default0VolumeCollection0_fts_idx_3, status: 1, count: 46228666, expected: 50000000
   2023-06-12 20:44:33,799 | test | DEBUG | MainThread | [hostedFTS:wait_for_fts_index_online:114] index: default0VolumeCollection0_fts_idx_3, status: 1, count: 46229466, expected: 50000000

FTS DCP Batches Blocked 18446744T

sudo guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/couchbase_capella_volume_2.ini -p sdk_timeout=60,log_level=debug,infra_log_level=debug,mutation_perc=100,key_type=RandomKey,capella_run=true,bucket_storage=magma,bucket_eviction_policy=fullEviction,rerun=False -t aGoodDoctor.hostedHospital.Murphy.test_rebalance,skip_cleanup=True,num_buckets=1,bucket_names=GleamBook,bucket_type=membase,iterations=1,key_size=18,randomize_doc_size=False,randomize_value=True,maxttl=10,pc=30,gsi_nodes=3,cbas_nodes=3,fts_nodes=3,kv_nodes=3,n1ql_nodes=3,services=data:query:index:cbas:fts,max_rebl_nodes=27,kv_compute=n2-standard-8,gsi_compute=n2-standard-8,n1ql_compute=n2-standard-8,fts_compute=n2-standard-8,cbas_compute=n2-standard-8,kv_disk=500,n1ql_disk=50,gsi_disk=500,fts_disk=500,cbas_disk=500,provider=GCP,region=us-east1,type=PD-SSD,skip_teardown_cleanup=true,wait_timeout=14400,index_timeout=28800,runtype=dedicated,track_failures=False,skip_init=false,sanity=true -m rest'