Details
-
Bug
-
Resolution: Fixed
-
Critical
-
2.2.0
-
Security Level: Public
-
Observed on Mac OS X, but presumed to affect all versions
-
Triaged
-
Release Note
-
Yes
Description
By shutting down Couchbase Server, intentionally corrupting one recently stored key, then starting up the server and trying to read said key, an older version of that key is seen. The corruption wasn't logged (that I could find).
Note, the actual component here is couchstore.
Steps to reproduce:
1) Add a new document to a given bucket. Call the key something known, like "corruptme"
2) Edit the document once (so you'll have two versions of it)
3) Shut down the server
4) grep for that string in the vbucket data files
5) Edit the vbucket file for the given key. Change "corruptme" to "corruptm3"
6) Start the server
7) Perform a get for the given key (with cbc or the like)
Expected behavior: either the right key is returned (assumes replicated metadata) or an error is returned.
Observed behavior: the old version of the key is returned.
The probability of encountering this goes up dramatically in environments where there are many nodes, disks.
Related reading:
http://static.googleusercontent.com/media/research.google.com/en/us/archive/disk_failures.pdf