Seeing 'COLLECTION_MAP_REFRESH_IN_PROGRESS' in steady state conditions

Description

Scenario:

  1. Two node cluster, Couchbase bucket (replica=1, scope=3, collections=2 per scope)

  2. Load each collection with 100 docs initially into all collections

  3. Open a separate client for testing sync_write outcome

  4. Stop persistence on one of the node using cbepctl command

  5. Load single doc with durability=MAJORITY_AND_PERSIST_TO_ACTIVE on 5 collections in parallel with timeout=60sec, such that the op won't timeout

  6. Using the client opened during step#3 try the same doc_op on same doc_key to validate  SYNC_WRITE_IN_PROGRESS outcome

Observation:

Seeing COLLECTION_MAP_REFRESH_IN_PROGRESS during one of the five operations. All other operations results in valid KV_SYNC_WRITE_IN_PROGRESS.

Expected behavior:

Should not see MAP_REFRESH_IN_PROGRESS, since the client has been already opened well before the doc_ops and the simulated error here is only STOP_PERSISTENCE(which has no direct effect on client)

Cbcollect logs:
https://cb-engineering.s3.amazonaws.com/c_map_refresh_in_progress/collectinfo-2021-03-23T120441-ns_1%40172.23.105.155.zip 
https://cb-engineering.s3.amazonaws.com/c_map_refresh_in_progress/collectinfo-2021-03-23T120441-ns_1%40172.23.105.211.zip
Attaching test run logs

and pcap

QE test:

Environment

Java client:3.1.0 CB server Enterprise Edition 7.0.0 build 4733

Gerrit Reviews

None

Release Notes Description

None

Attachments

7

Activity

Show:

Ashwin Govindarajulu June 10, 2021 at 12:48 PM

Closing this ticket.

Ashwin Govindarajulu June 10, 2021 at 9:01 AM

Yes this can be closed. Have modified the testcase to handle this state.

Michael Nitschinger June 7, 2021 at 1:54 PM

what's the status of this ticket, with the workaround applied - can it be closed?

Ashwin Govindarajulu May 4, 2021 at 1:27 PM

Updated the testcase to avoid fail fast testing in collection specific tests.

Note: Still fail_fast testing is part of durability suite

Michael Nitschinger April 19, 2021 at 7:56 AM

The change I asked you to run is still helpful since it optimizes a couple things, but overall the states are expected to happen.

Fixed
Pinned fields
Click on the next to a field label to start pinning.

Details

Assignee

Reporter

Labels

Story Points

Priority

Instabug

Open Instabug

PagerDuty

Sentry

Zendesk Support

Created March 23, 2021 at 12:29 PM
Updated June 10, 2021 at 12:48 PM
Resolved June 10, 2021 at 11:21 AM
Instabug