Observed Memcached crash after a rebalance operation with N1QL transactions on 7.2.4-7070

Steps:

1. Initialise a 3 node cluster running version 7.2.4-7070 (release build)
   Node 1 - kv, index, n1ql
   Node 2 - kv
   Node 3 - kv
2. Create a CouchStore bucket with 2 replicas and 50 collections.
3. Load 1 million documents into the bucket.
4. Primary and secondary indexes are created on certain collections.
5. Rebalance-in operation is started in which 2 KV nodes are rebalanced-in.
6. N1QL transactions are executed during the rebalance-in operation.
7. More data is loaded after the rebalance-in operation.
8. Observed memcached crash logs, backtrace is attached

9. memcached.log.000003.txt:2024-02-23T03:03:28.243332-08:00 CRITICAL Detected previous crash

   memcached.log.000003.txt:2024-02-23T03:03:28.243361-08:00 CRITICAL Breakpad caught a crash (Couchbase version 7.2.4-7070). Writing crash dump to /opt/couchbase/var/lib/couchbase/crash/2eb7e2a3-97e0-4204-b67010a9-c797bdca.dmp before terminating.

   memcached.log.000003.txt:2024-02-23T03:03:28.243364-08:00 CRITICAL Stack backtrace of crashed thread:

   memcached.log.000003.txt:2024-02-23T03:03:28.243364-08:00 CRITICAL #0 /opt/couchbase/bin/memcached() [0x400000+0x751188]

   memcached.log.000003.txt:2024-02-23T03:03:28.243365-08:00 CRITICAL #1 /opt/couchbase/bin/memcached(_ZN15google_breakpad16ExceptionHandler12GenerateDumpEPNS0_12CrashContextE+0x3ea) [0x400000+0x78797a]

   memcached.log.000003.txt:2024-02-23T03:03:28.243366-08:00 CRITICAL #2 /opt/couchbase/bin/memcached(_ZN15google_breakpad16ExceptionHandler13SignalHandlerEiP9siginfo_tPv+0xb8) [0x400000+0x787cb8]

   memcached.log.000003.txt:2024-02-23T03:03:28.243366-08:00 CRITICAL #3 /lib/x86_64-linux-gnu/libc.so.6() [0x7f59af41f000+0x3bfd0]

   memcached.log.000003.txt:2024-02-23T03:03:28.243366-08:00 CRITICAL #4 /lib/x86_64-linux-gnu/libc.so.6() [0x7f59af41f000+0x8ad3c]

   memcached.log.000003.txt:2024-02-23T03:03:28.243367-08:00 CRITICAL #5 /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x12) [0x7f59af41f000+0x3bf32]

   memcached.log.000003.txt:2024-02-23T03:03:28.243367-08:00 CRITICAL #6 /lib/x86_64-linux-gnu/libc.so.6(abort+0xd3) [0x7f59af41f000+0x26472]

   memcached.log.000003.txt:2024-02-23T03:03:28.243367-08:00 CRITICAL #7 /opt/couchbase/bin/../lib/libstdc++.so.6() [0x7f59ac600000+0xa89ab]

   memcached.log.000003.txt:2024-02-23T03:03:28.243370-08:00 CRITICAL #8 /opt/couchbase/bin/memcached() [0x400000+0x75b6bb]

   memcached.log.000003.txt:2024-02-23T03:03:28.243370-08:00 CRITICAL #9 /opt/couchbase/bin/../lib/libstdc++.so.6() [0x7f59ac600000+0xb82fa]

   memcached.log.000003.txt:2024-02-23T03:03:28.243370-08:00 CRITICAL #10 /opt/couchbase/bin/../lib/libstdc++.so.6() [0x7f59ac600000+0xb8365]

   memcached.log.000003.txt:2024-02-23T03:03:28.243370-08:00 CRITICAL #11 /opt/couchbase/bin/../lib/libstdc++.so.6() [0x7f59ac600000+0xb85b7]

   memcached.log.000003.txt:2024-02-23T03:03:28.243370-08:00 CRITICAL #12 /opt/couchbase/bin/memcached() [0x400000+0x13937e]

   memcached.log.000003.txt:2024-02-23T03:03:28.243371-08:00 CRITICAL #13 /opt/couchbase/bin/memcached() [0x400000+0x17b06e]

   memcached.log.000003.txt:2024-02-23T03:03:28.243372-08:00 CRITICAL #14 /lib/x86_64-linux-gnu/libc.so.6() [0x7f59af41f000+0x271ca]

   memcached.log.000003.txt:2024-02-23T03:03:28.243372-08:00 CRITICAL #15 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x85) [0x7f59af41f000+0x27285]

   memcached.log.000003.txt:2024-02-23T03:03:28.243373-08:00 CRITICAL #16 /opt/couchbase/bin/memcached() [0x400000+0x1733f7]

   root@s43401-deb12:/opt/couchbase/var/lib/couchbase/logs#

1. Cb-collect logs are attached too.