Loading...

Details

Type: Bug
Resolution: Cannot Reproduce
Priority: Blocker
Fix Version/s: 4.1.1
Affects Version/s: 4.0.0, 4.1.0
Component/s: secondary-index
Security Level: Public
Labels:
None
Environment:
Tested with OSX on 4.0 and 4.1 Beta build (4904) on Ubuntu 12.04

Triage:
Untriaged
Is this a Regression?:
Unknown

Description

I've been starting to write integration tests for the java client with DML and came across a what I think is a consistency issue.

The following scenario sometimes works and sometimes doesn't (on my machines 50/50):

Flush the bucket so its clean state
Use DML INSERT INTO to create insert two documents
Use KV API on the SDK to ensure the documents are created
Use a DELETE FROM with a WHERE clause which matches one of the documents, but not the other
Use KV API again to determine that one doc is gone and one is still present

The part which occasionally fails is the last part, so KV API tells me that sometimes both docs are still there. Other times, one is gone correctly. Both N1QL statements return with success, but you can spot (see below) that when it fails on the DELETE FROM the mutationCount is not 1 (in fact it doesn't show 0, its just not there).

I've been experimenting with consistency levels, but even under REQUEST_PLUS on the DELETE path it happens.

The following two snippets are the raw request/response flows from client to server so you can exactly see whats going on:

Insert first:

         +-------------------------------------------------+

         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |

+--------+-------------------------------------------------+----------------+

|00000000| 50 4f 53 54 20 2f 71 75 65 72 79 20 48 54 54 50 |POST /query HTTP|

|00000010| 2f 31 2e 31 0d 0a 55 73 65 72 2d 41 67 65 6e 74 |/1.1..User-Agent|

|00000020| 3a 20 63 6f 75 63 68 62 61 73 65 2d 6a 61 76 61 |: couchbase-java|

|00000030| 2d 63 6c 69 65 6e 74 2f 32 2e 32 2e 32 2d 53 4e |-client/2.2.2-SN|

|00000040| 41 50 53 48 4f 54 20 28 67 69 74 3a 20 24 7b 67 |APSHOT (git: ${g|

|00000050| 69 74 2e 63 6f 6d 6d 69 74 2e 69 64 2e 64 65 73 |it.commit.id.des|

|00000060| 63 72 69 62 65 7d 29 20 28 4d 61 63 20 4f 53 20 |cribe}) (Mac OS |

|00000070| 58 2f 31 30 2e 31 30 2e 35 20 78 38 36 5f 36 34 |X/10.10.5 x86_64|

|00000080| 3b 20 4a 61 76 61 20 48 6f 74 53 70 6f 74 28 54 |; Java HotSpot(T|

|00000090| 4d 29 20 36 34 2d 42 69 74 20 53 65 72 76 65 72 |M) 64-Bit Server|

|000000a0| 20 56 4d 20 31 2e 38 2e 30 5f 35 31 2d 62 31 36 | VM 1.8.0_51-b16|

|000000b0| 29 0d 0a 43 6f 6e 74 65 6e 74 2d 54 79 70 65 3a |)..Content-Type:|

|000000c0| 20 61 70 70 6c 69 63 61 74 69 6f 6e 2f 6a 73 6f | application/jso|

|000000d0| 6e 0d 0a 43 6f 6e 74 65 6e 74 2d 4c 65 6e 67 74 |n..Content-Lengt|

|000000e0| 68 3a 20 31 34 36 0d 0a 41 75 74 68 6f 72 69 7a |h: 146..Authoriz|

|000000f0| 61 74 69 6f 6e 3a 20 42 61 73 69 63 20 5a 47 56 |ation: Basic ZGV|

|00000100| 6d 59 58 56 73 64 44 6f 3d 0d 0a 0d 0a 7b 22 73 |mYXVsdDo=....{"s|

|00000110| 74 61 74 65 6d 65 6e 74 22 3a 22 49 4e 53 45 52 |tatement":"INSER|

|00000120| 54 20 49 4e 54 4f 20 60 64 65 66 61 75 6c 74 60 |T INTO `default`|

|00000130| 20 28 4b 45 59 2c 20 56 41 4c 55 45 29 20 56 41 | (KEY, VALUE) VA|

|00000140| 4c 55 45 53 20 28 27 6e 31 71 6c 44 65 6c 57 31 |LUES ('n1qlDelW1|

|00000150| 27 2c 20 7b 5c 22 74 79 70 65 5c 22 3a 5c 22 61 |', {\"type\":\"a|

|00000160| 62 63 5c 22 7d 29 2c 20 28 27 6e 31 71 6c 44 65 |bc\"}), ('n1qlDe|

|00000170| 6c 57 32 27 2c 20 7b 5c 22 74 79 70 65 5c 22 3a |lW2', {\"type\":|

|00000180| 5c 22 64 65 66 5c 22 7d 29 22 2c 22 74 69 6d 65 |\"def\"})","time|

|00000190| 6f 75 74 22 3a 22 37 35 30 30 30 6d 73 22 7d    |out":"75000ms"} |

+--------+-------------------------------------------------+----------------+

         +-------------------------------------------------+

         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |

+--------+-------------------------------------------------+----------------+

|00000000| 48 54 54 50 2f 31 2e 31 20 32 30 30 20 4f 4b 0d |HTTP/1.1 200 OK.|

|00000010| 0a 43 6f 6e 74 65 6e 74 2d 4c 65 6e 67 74 68 3a |.Content-Length:|

|00000020| 20 33 31 34 0d 0a 43 6f 6e 74 65 6e 74 2d 54 79 | 314..Content-Ty|

|00000030| 70 65 3a 20 61 70 70 6c 69 63 61 74 69 6f 6e 2f |pe: application/|

|00000040| 6a 73 6f 6e 3b 20 76 65 72 73 69 6f 6e 3d 31 2e |json; version=1.|

|00000050| 30 2e 30 0d 0a 44 61 74 65 3a 20 46 72 69 2c 20 |0.0..Date: Fri, |

|00000060| 30 36 20 4e 6f 76 20 32 30 31 35 20 30 37 3a 35 |06 Nov 2015 07:5|

|00000070| 31 3a 33 30 20 47 4d 54 0d 0a 0d 0a 7b 0a 20 20 |1:30 GMT....{.  |

|00000080| 20 20 22 72 65 71 75 65 73 74 49 44 22 3a 20 22 |  "requestID": "|

|00000090| 31 36 63 39 32 35 37 66 2d 30 65 62 62 2d 34 62 |16c9257f-0ebb-4b|

|000000a0| 63 65 2d 62 34 35 61 2d 62 33 66 31 63 32 66 35 |ce-b45a-b3f1c2f5|

|000000b0| 31 32 34 38 22 2c 0a 20 20 20 20 22 73 69 67 6e |1248",.    "sign|

|000000c0| 61 74 75 72 65 22 3a 20 6e 75 6c 6c 2c 0a 20 20 |ature": null,.  |

|000000d0| 20 20 22 72 65 73 75 6c 74 73 22 3a 20 5b 0a 20 |  "results": [. |

|000000e0| 20 20 20 5d 2c 0a 20 20 20 20 22 73 74 61 74 75 |   ],.    "statu|

|000000f0| 73 22 3a 20 22 73 75 63 63 65 73 73 22 2c 0a 20 |s": "success",. |

|00000100| 20 20 20 22 6d 65 74 72 69 63 73 22 3a 20 7b 0a |   "metrics": {.|

|00000110| 20 20 20 20 20 20 20 20 22 65 6c 61 70 73 65 64 |        "elapsed|

|00000120| 54 69 6d 65 22 3a 20 22 34 32 34 2e 32 31 35 33 |Time": "424.2153|

|00000130| 35 35 6d 73 22 2c 0a 20 20 20 20 20 20 20 20 22 |55ms",.        "|

|00000140| 65 78 65 63 75 74 69 6f 6e 54 69 6d 65 22 3a 20 |executionTime": |

|00000150| 22 34 32 34 2e 30 32 37 36 32 34 6d 73 22 2c 0a |"424.027624ms",.|

|00000160| 20 20 20 20 20 20 20 20 22 72 65 73 75 6c 74 43 |        "resultC|

|00000170| 6f 75 6e 74 22 3a 20 30 2c 0a 20 20 20 20 20 20 |ount": 0,.      |

|00000180| 20 20 22 72 65 73 75 6c 74 53 69 7a 65 22 3a 20 |  "resultSize": |

|00000190| 30 2c 0a 20 20 20 20 20 20 20 20 22 6d 75 74 61 |0,.        "muta|

|000001a0| 74 69 6f 6e 43 6f 75 6e 74 22 3a 20 32 0a 20 20 |tionCount": 2.  |

|000001b0| 20 20 7d 0a 7d 0a                               |  }.}.          |

+--------+-------------------------------------------------+----------------+

And now the DELETE:

         +-------------------------------------------------+

         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |

+--------+-------------------------------------------------+----------------+

|00000000| 50 4f 53 54 20 2f 71 75 65 72 79 20 48 54 54 50 |POST /query HTTP|

|00000010| 2f 31 2e 31 0d 0a 55 73 65 72 2d 41 67 65 6e 74 |/1.1..User-Agent|

|00000020| 3a 20 63 6f 75 63 68 62 61 73 65 2d 6a 61 76 61 |: couchbase-java|

|00000030| 2d 63 6c 69 65 6e 74 2f 32 2e 32 2e 32 2d 53 4e |-client/2.2.2-SN|

|00000040| 41 50 53 48 4f 54 20 28 67 69 74 3a 20 24 7b 67 |APSHOT (git: ${g|

|00000050| 69 74 2e 63 6f 6d 6d 69 74 2e 69 64 2e 64 65 73 |it.commit.id.des|

|00000060| 63 72 69 62 65 7d 29 20 28 4d 61 63 20 4f 53 20 |cribe}) (Mac OS |

|00000070| 58 2f 31 30 2e 31 30 2e 35 20 78 38 36 5f 36 34 |X/10.10.5 x86_64|

|00000080| 3b 20 4a 61 76 61 20 48 6f 74 53 70 6f 74 28 54 |; Java HotSpot(T|

|00000090| 4d 29 20 36 34 2d 42 69 74 20 53 65 72 76 65 72 |M) 64-Bit Server|

|000000a0| 20 56 4d 20 31 2e 38 2e 30 5f 35 31 2d 62 31 36 | VM 1.8.0_51-b16|

|000000b0| 29 0d 0a 43 6f 6e 74 65 6e 74 2d 54 79 70 65 3a |)..Content-Type:|

|000000c0| 20 61 70 70 6c 69 63 61 74 69 6f 6e 2f 6a 73 6f | application/jso|

|000000d0| 6e 0d 0a 43 6f 6e 74 65 6e 74 2d 4c 65 6e 67 74 |n..Content-Lengt|

|000000e0| 68 3a 20 31 31 30 0d 0a 41 75 74 68 6f 72 69 7a |h: 110..Authoriz|

|000000f0| 61 74 69 6f 6e 3a 20 42 61 73 69 63 20 5a 47 56 |ation: Basic ZGV|

|00000100| 6d 59 58 56 73 64 44 6f 3d 0d 0a 0d 0a 7b 22 73 |mYXVsdDo=....{"s|

|00000110| 74 61 74 65 6d 65 6e 74 22 3a 22 44 45 4c 45 54 |tatement":"DELET|

|00000120| 45 20 46 52 4f 4d 20 60 64 65 66 61 75 6c 74 60 |E FROM `default`|

|00000130| 20 57 48 45 52 45 20 74 79 70 65 20 3d 20 27 64 | WHERE type = 'd|

|00000140| 65 66 27 22 2c 22 74 69 6d 65 6f 75 74 22 3a 22 |ef'","timeout":"|

|00000150| 37 35 30 30 30 6d 73 22 2c 22 73 63 61 6e 5f 63 |75000ms","scan_c|

|00000160| 6f 6e 73 69 73 74 65 6e 63 79 22 3a 22 72 65 71 |onsistency":"req|

|00000170| 75 65 73 74 5f 70 6c 75 73 22 7d                |uest_plus"}     |

+--------+-------------------------------------------------+----------------+

         +-------------------------------------------------+

         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |

+--------+-------------------------------------------------+----------------+

|00000000| 48 54 54 50 2f 31 2e 31 20 32 30 30 20 4f 4b 0d |HTTP/1.1 200 OK.|

|00000010| 0a 43 6f 6e 74 65 6e 74 2d 4c 65 6e 67 74 68 3a |.Content-Length:|

|00000020| 20 32 38 35 0d 0a 43 6f 6e 74 65 6e 74 2d 54 79 | 285..Content-Ty|

|00000030| 70 65 3a 20 61 70 70 6c 69 63 61 74 69 6f 6e 2f |pe: application/|

|00000040| 6a 73 6f 6e 3b 20 76 65 72 73 69 6f 6e 3d 31 2e |json; version=1.|

|00000050| 30 2e 30 0d 0a 44 61 74 65 3a 20 46 72 69 2c 20 |0.0..Date: Fri, |

|00000060| 30 36 20 4e 6f 76 20 32 30 31 35 20 30 37 3a 35 |06 Nov 2015 07:5|

|00000070| 31 3a 33 30 20 47 4d 54 0d 0a 0d 0a 7b 0a 20 20 |1:30 GMT....{.  |

|00000080| 20 20 22 72 65 71 75 65 73 74 49 44 22 3a 20 22 |  "requestID": "|

|00000090| 65 39 65 65 65 63 61 33 2d 66 65 64 31 2d 34 62 |e9eeeca3-fed1-4b|

|000000a0| 61 34 2d 38 31 35 62 2d 66 63 38 65 33 32 31 66 |a4-815b-fc8e321f|

|000000b0| 61 33 65 39 22 2c 0a 20 20 20 20 22 73 69 67 6e |a3e9",.    "sign|

|000000c0| 61 74 75 72 65 22 3a 20 6e 75 6c 6c 2c 0a 20 20 |ature": null,.  |

|000000d0| 20 20 22 72 65 73 75 6c 74 73 22 3a 20 5b 0a 20 |  "results": [. |

|000000e0| 20 20 20 5d 2c 0a 20 20 20 20 22 73 74 61 74 75 |   ],.    "statu|

|000000f0| 73 22 3a 20 22 73 75 63 63 65 73 73 22 2c 0a 20 |s": "success",. |

|00000100| 20 20 20 22 6d 65 74 72 69 63 73 22 3a 20 7b 0a |   "metrics": {.|

|00000110| 20 20 20 20 20 20 20 20 22 65 6c 61 70 73 65 64 |        "elapsed|

|00000120| 54 69 6d 65 22 3a 20 22 33 33 33 2e 37 39 35 34 |Time": "333.7954|

|00000130| 33 39 6d 73 22 2c 0a 20 20 20 20 20 20 20 20 22 |39ms",.        "|

|00000140| 65 78 65 63 75 74 69 6f 6e 54 69 6d 65 22 3a 20 |executionTime": |

|00000150| 22 33 33 33 2e 36 38 39 38 34 6d 73 22 2c 0a 20 |"333.68984ms",. |

|00000160| 20 20 20 20 20 20 20 22 72 65 73 75 6c 74 43 6f |       "resultCo|

|00000170| 75 6e 74 22 3a 20 30 2c 0a 20 20 20 20 20 20 20 |unt": 0,.       |

|00000180| 20 22 72 65 73 75 6c 74 53 69 7a 65 22 3a 20 30 | "resultSize": 0|

|00000190| 0a 20 20 20 20 7d 0a 7d 0a                      |.    }.}.       |

+--------+-------------------------------------------------+----------------+

One final note - it seems to have to do with the WHERE clause, because I have another similar tests which uses the USE KEYS approach in DELETE FROM and there it always passes at least in my runs.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending
- Thumbnails
- List
- Download All

indexer-CB4940-simon.log
530 kB
24/Nov/15 4:28 AM

Issue Links

blocks

MB-17211 4.1.1 Minor Release

Closed

relates to

MB-16957 Reduce scan latency for stale=false after bucket flush

Closed

DELETE with WHERE clause not consistent right after INSERT

Details

Description

Attachments

Attachments

Issue Links

Activity

People

Dates

PagerDuty