Details
-
Bug
-
Resolution: Fixed
-
Blocker
-
7.6.2
-
Operating System : Microsoft Windows Server 2022
Couchbase Enterprise Edition build 7.6.2-3691
-
Untriaged
-
Windows 64-bit
-
-
0
-
Yes
Description
Steps to reproduce
- Created a 3 node kv cluster in Windows 2022
- Created a magma bucket named vector_bucket
- Created 3 collections under _default scope. The collection name as as follows
- steady_state
- mutation
- mutation_expiry
- Loaded 10M docs onto each collection. 30M docs in total in the bucket
- The state of the doc is as follows
{
|
"content" : a text content, |
"vec" : A vector of dims 4096, |
"doc_type" : "vector", |
"key" : key |
}
|
6. Added 5 fts nodes onto the cluster
7. Created 2 indexes. The definitions are attached
8. Service fts crashed immediately after
Service 'fts' exited with status 1. Restarting. Messages: |
2024-06-04T10:07:16.613+00:00 [INFO] pindex_bleve: started runBatchWorker: 2 for pindex: vector_bucket._default.index_mutation_688ab0af2eead4f1_df981270 |
2024-06-04T10:07:16.613+00:00 [INFO] pindex_bleve: started runBatchWorker: 1 for pindex: vector_bucket._default.index_mutation_688ab0af2eead4f1_df981270 |
2024-06-04T10:07:16.613+00:00 [INFO] pindex_bleve: started runBatchWorker: 0 for pindex: vector_bucket._default.index_mutation_688ab0af2eead4f1_df981270 |
2024-06-04T10:07:16.620+00:00 [FATA] scorch AsyncError, path: c:/Program Files/Couchbase/Server/var/lib/couchbase/data/@fts\vector_bucket._default.index_mutation_688ab0af2eead4f1_df981270.pindex\store, treating this as fatal, err: merging err: merging failed: Error in struct faiss::InvertedListsIOHook *__cdecl faiss::InvertedListsIOHook::lookup(int) at C:\Jenkins\workspace\cbdeps-platform-build\faiss\faiss\invlists\InvertedListsIOHook.cpp:63: read_InvertedLists: could not load ArrayInvertedLists as 646f6c69 ("ilod"), stack dump: c:\Program Files\Couchbase\Server\var\lib\couchbase\data\@fts\dumps\1717495636.fts.stack.dump.txt -- main.initBleveOptions.func2() at init_bleve.go:113 |
Service 'fts' exited with status 1. Restarting. Messages: |
2024-06-04T10:08:17.368+00:00 [INFO] vector_bucket._default.index_mutation_688ab0af2eead4f1_43f87f2c |
2024-06-04T10:08:17.368+00:00 [INFO] vector_bucket._default.index_mutation_688ab0af2eead4f1_259380a5 |
2024-06-04T10:08:17.368+00:00 [INFO] vector_bucket._default.index_mutation_688ab0af2eead4f1_137764b5 |
2024-06-04T10:08:17.368+00:00 [INFO] vector_bucket._default.index_mutation_688ab0af2eead4f1_8a53903c |
2024-06-04T10:08:17.368+00:00 [INFO] vector_bucket._default.index_mutation_688ab0af2eead4f1_3b831f22 |
2024-06-04T10:08:17.368+00:00 [INFO] vector_bucket._default.index_mutation_688ab0af2eead4f1_85b5aacd |
2024-06-04T10:08:17.377+00:00 [FATA] scorch AsyncError, path: c:/Program Files/Couchbase/Server/var/lib/couchbase/data/@fts\vector_bucket._default.index_mutation_688ab0af2eead4f1_8a53903c.pindex\store, treating this as fatal, err: merging err: merging failed: rm�, stack dump: c:\Program Files\Couchbase\Server\var\lib\couchbase\data\@fts\dumps\1717495697.fts.stack.dump.txt -- main.initBleveOptions.func2() at init_bleve.go:113 |
Service 'fts' exited with status 1. Restarting. Messages: |
2024-06-04T10:08:49.810+00:00 [INFO] cfg_metakv: metaKVCallback, path: /fts/cbgt/cfg/nodeDefs-wanted/e6c3b5a7bbd5c4759d84c4a32cac52f6, key: nodeDefs-wanted/e6c3b5a7bbd5c4759d84c4a32cac52f6, deletion: false |
2024-06-04T10:08:49.810+00:00 [INFO] cfg_metakv: metaKVCallback, path: /fts/cbgt/cfg/planPIndexesLean/planPIndexesLean-7df40867525b46fa41e2e8e32e3d1629-1717494566128/0-$#$, key: planPIndexesLean/planPIndexesLean-7df40867525b46fa41e2e8e32e3d1629-1717494566128/0-$#$, deletion: false |
2024-06-04T10:08:49.810+00:00 [INFO] cfg_metakv: metaKVCallback, path: /fts/cbgt/cfg/planPIndexesLean/planPIndexesLean-b0fec9356345f77af0e1cd8ae80cf028-1717494622272/0-$#$, key: planPIndexesLean/planPIndexesLean-b0fec9356345f77af0e1cd8ae80cf028-1717494622272/0-$#$, deletion: false |
2024-06-04T10:08:49.817+00:00 [FATA] scorch AsyncError, path: c:/Program Files/Couchbase/Server/var/lib/couchbase/data/@fts\vector_bucket._default.index_steady_state_1a073c12c05399bc_4c1c5584.pindex\store, treating this as fatal, err: merging err: merging failed: 0���, stack dump: c:\Program Files\Couchbase\Server\var\lib\couchbase\data\@fts\dumps\1717495729.fts.stack.dump.txt -- main.initBleveOptions.func2() at init_bleve.go:113 |