Details
-
Bug
-
Resolution: Cannot Reproduce
-
Major
-
1.1.4
-
None
-
None
-
-java 6
-Couchbase Server 3.0.1
-Couchbase java client api 2.1.4
-ubuntu server
-25 node cluster
Description
We have been running into an issue related to incorrect data while trying to ramp up in production.
In the code, I'm using the non-blocking pattern (rxJava + Twitter Future).
- Fetch a set of data for a set of keys
-subscribe to the results and once completed set the twitter promise.
-Compare the request keys and the result keys and the keys are different. The result appears to be from a different request.
Current traffic is ~180k writes per a min and ~180k read per a min and we find at most one request would hit the scenario above and so it's very rare. We haven't been able to repro this in a test environment (probably since we are not hitting the same QPS or the data is different).
Code to fetch data and subscribe:
private def batchGet(ids: Iterable[String], keySpace: String): Future[Seq[JsonDocument]] = {
val promise = new Promise[Seq[JsonDocument]]()
val errorLogs =
Observable.from(ids.toArray)
.flatMap(new Func1[String, Observable[JsonDocument]]() {
override def call(id: String): Observable[JsonDocument] =
}).toList()
.single()
.subscribe(new Action1[java.util.List[JsonDocument]] {
override def call(t1: java.util.List[JsonDocument]): Unit =
}, errorAction(promise, Seq.empty, keySpace, BATCHGET_STAT_TYPE, errorLogs), completeAction(promise, errorLogs))
promise
}
Calling method:
batchGet(keyMapping.keys, keySpace).map { seq =>
seq.map { jsonDoc =>
(keyMapping.get(jsonDoc.id()).getOrElse
->
(parseDoc(jsonDoc), jsonDoc))
}.toMap
}
All our data types are immutable other than the twitter promise. The keyMapping.keys are used to fetch the items from Couchbase but we found results from Couchbase where the id wasn't in the set of keys that was requested. In the logs, it clearly shows it is unrelated (and not a mismatch in casing or spacing).