Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
6.0.0, 5.5.0, 5.5.1, 5.5.2, 5.5.3, 6.0.1, 6.0.2, 5.5.4
-
KV-Engine MH 2nd Beta
Description
Summary
If a DCP client uses a long (exact length TBD) name when connecting, it can result in
STATS calls (for example performed by ns_server to monitor rebalance) to fail:
462445 2019-05-11T08:18:28.525392+00:00 WARNING 143: exception occurred in runloop during packet execution. Cookie info: {"packet":{"magic":"ClientRequest","opcode":"STAT","keylen":3,"extlen":0,"datatype":"raw","vbucket":0,"bodylen":3,"opaque":"0x0","cas":0},"connection":"[ 127.0.0.1:48735 - 127.0.0.1:11209 (<ud>@ns_server</ud>) ]"} - closing connection: Response::setKeylen: key cannot exceed 1 byte
|
Details
Looks like problem is here - we cap the response keyLen to 1 byte for all response types, not just "Response with flex-framing extras" (0x18):
void setKeylen(uint16_t value) { |
if (value > 0xff) { |
throw std::invalid_argument( |
"Response::setKeylen: key cannot exceed 1 byte"); |
}
|
keylen = uint8_t(value);
|
}
|
This change was introduced via: http://review.couchbase.org/#/c/86393/
See https://github.com/couchbase/kv_engine/blob/master/docs/BinaryProtocol.md#response-header - it should be permitted to return keys up to 65,535 bytes long for "regular" response packets (0x81).
Attachments
Issue Links
- depends on
-
MB-35881 ns_server should generate DCP connection names that are no longer than allowed by memcached
-
- Closed
-
-
MB-37246 Ensure Views does not generate DCP connection names that are no longer than allowed by memcached
-
- Closed
-
-
MB-37283 Ensure XDCR does not generate DCP connection names that are no longer than allowed by memcached
-
- Closed
-
-
MB-49271 Backport MB-35881 to 6.6.4
-
- Closed
-
-
MB-37244 Ensure 2i does not generate DCP connection names that are no longer than allowed by memcached
-
- Open
-
-
CBES-145 Ensure ElastricSearch Connector does not generate DCP connection names that are no longer than allowed by memcached
-
- Resolved
-
-
JDCP-136 Ensure java-DCP-client does not generate DCP connection names that are no longer than allowed by memcached
-
- Resolved
-
-
KAFKAC-160 Ensure Kafka Connector does not generate DCP connection names that are no longer than allowed by memcached
-
- Resolved
-
-
CBG-626 Ensure Sync-Gateway does not generate DCP connection names that are no longer than allowed by memcached
-
- Closed
-
-
MB-37241 Ensure FTS does not generate DCP connection names that are no longer than allowed by memcached
-
- Closed
-
-
MB-37242 Ensure Backup does not generate DCP connection names that are no longer than allowed by memcached
-
- Closed
-
-
MB-37243 Ensure Query does not generate DCP connection names that are no longer than allowed by memcached
-
- Closed
-
-
MB-37245 Ensure Eventing does not generate DCP connection names that are no longer than allowed by memcached
-
- Closed
-
-
MB-35882 The Memcached STATS "dcp" command should have a mode that only retrieves stats on connections owned by the user
-
- Closed
-
- relates to
-
MB-42968 Eventing Enabled Cluster Fails to Recover
-
- Closed
-
-
MB-51513 Upgrades can fail due to the 200 character limit for DCP connections
-
- Closed
-
-
DOC-9381 names used for DCP connections must not be longer than 200 characters
-
- Resolved
-
-
JDCP-137 Increase connection name limit from 148 bytes to 200 bytes
-
- Resolved
-
-
CMOS-154 Add check for DCP names > 200 chars (MB-34280)
-
- Done
-
For Gerrit Dashboard: MB-34280 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
109497,4 | MB-34280: Allow long keys in (normal) return packets | alice | kv_engine | Status: ABANDONED | -1 | +1 |
114481,8 | MB-34280: Set max DCP name to 200 characters | master | kv_engine | Status: MERGED | +2 | +1 |
119330,2 | JDCP-137 Increase connection name limit from 148 bytes to 200 bytes | master | java-dcp-client | Status: MERGED | +2 | +1 |
164809,3 | MB-34280: Set max DCP name to 200 characters | mad-hatter | kv_engine | Status: MERGED | +2 | +1 |
166408,1 | Merge remote-tracking branch 'couchbase/mad-hatter' | cheshire-cat | kv_engine | Status: MERGED | +2 | +1 |
166749,1 | Merge commit mad-hatter 'b5fa131c0' into couchbase/cheshire-cat | cheshire-cat | kv_engine | Status: ABANDONED | +2 | -1 |
167587,1 | Merge remote-tracking branch 'couchbase/cheshire-cat' into 'couchbase/master' | master | kv_engine | Status: MERGED | +2 | +1 |
169962,2 | K8S-2534: Warn users from using long-names | master | couchbase-operator | Status: ABANDONED | -2 | 0 |
172119,1 | MB-51513: Revert "MB-34280: Set max DCP name to 200 characters" | master | kv_engine | Status: ABANDONED | -1 | -1 |
172485,2 | MB-51513: Revert "MB-34280: Set max DCP name to 200 characters" | cheshire-cat | kv_engine | Status: MERGED | +2 | +1 |
172495,1 | MB-51513: Merge branch 'couchbase/cheshire-cat' into 'couchbase/neo' | neo | kv_engine | Status: MERGED | +2 | +1 |
172599,1 | Merge branch 'neo' | master | kv_engine | Status: ABANDONED | 0 | -1 |
172641,1 | Merge branch 'neo' into master | master | kv_engine | Status: MERGED | +2 | +1 |
Activity
Field | Original Value | New Value |
---|---|---|
Description |
Looks like problem is here - we cap the response keyLen to 1 byte for all response types, not just "Response with flex-framing extras" (0x18):
{code:cpp} void setKeylen(uint16_t value) { if (value > 0xff) { throw std::invalid_argument( "Response::setKeylen: key cannot exceed 1 byte"); } keylen = uint8_t(value); } {code} See https://github.com/couchbase/kv_engine/blob/master/docs/BinaryProtocol.md#response-header - it should be permitted to return keys up to 65,535 bytes long for "regular" response packets (0x81). |
h2. Summary
If a DCP client uses a long (exact length TBD) name when connecting, it can result in STATS calls (for example performed by ns_server to monitor rebalance) to fail: {code} 462445 2019-05-11T08:18:28.525392+00:00 WARNING 143: exception occurred in runloop during packet execution. Cookie info: {"packet":{"magic":"ClientRequest","opcode":"STAT","keylen":3,"extlen":0,"datatype":"raw","vbucket":0,"bodylen":3,"opaque":"0x0","cas":0},"connection":"[ 127.0.0.1:48735 - 127.0.0.1:11209 (<ud>@ns_server</ud>) ]"} - closing connection: Response::setKeylen: key cannot exceed 1 byte {code} h2. Details Looks like problem is here - we cap the response keyLen to 1 byte for all response types, not just "Response with flex-framing extras" (0x18): {code:cpp} void setKeylen(uint16_t value) { if (value > 0xff) { throw std::invalid_argument( "Response::setKeylen: key cannot exceed 1 byte"); } keylen = uint8_t(value); } {code} See https://github.com/couchbase/kv_engine/blob/master/docs/BinaryProtocol.md#response-header - it should be permitted to return keys up to 65,535 bytes long for "regular" response packets (0x81). |
Description |
h2. Summary
If a DCP client uses a long (exact length TBD) name when connecting, it can result in STATS calls (for example performed by ns_server to monitor rebalance) to fail: {code} 462445 2019-05-11T08:18:28.525392+00:00 WARNING 143: exception occurred in runloop during packet execution. Cookie info: {"packet":{"magic":"ClientRequest","opcode":"STAT","keylen":3,"extlen":0,"datatype":"raw","vbucket":0,"bodylen":3,"opaque":"0x0","cas":0},"connection":"[ 127.0.0.1:48735 - 127.0.0.1:11209 (<ud>@ns_server</ud>) ]"} - closing connection: Response::setKeylen: key cannot exceed 1 byte {code} h2. Details Looks like problem is here - we cap the response keyLen to 1 byte for all response types, not just "Response with flex-framing extras" (0x18): {code:cpp} void setKeylen(uint16_t value) { if (value > 0xff) { throw std::invalid_argument( "Response::setKeylen: key cannot exceed 1 byte"); } keylen = uint8_t(value); } {code} See https://github.com/couchbase/kv_engine/blob/master/docs/BinaryProtocol.md#response-header - it should be permitted to return keys up to 65,535 bytes long for "regular" response packets (0x81). |
h2. Summary
If a DCP client uses a long (exact length TBD) name when connecting, it can result in STATS calls (for example performed by ns_server to monitor rebalance) to fail: {code} 462445 2019-05-11T08:18:28.525392+00:00 WARNING 143: exception occurred in runloop during packet execution. Cookie info: {"packet":{"magic":"ClientRequest","opcode":"STAT","keylen":3,"extlen":0,"datatype":"raw","vbucket":0,"bodylen":3,"opaque":"0x0","cas":0},"connection":"[ 127.0.0.1:48735 - 127.0.0.1:11209 (<ud>@ns_server</ud>) ]"} - closing connection: Response::setKeylen: key cannot exceed 1 byte {code} h2. Details Looks like problem is here - we cap the response keyLen to 1 byte for all response types, not just "Response with flex-framing extras" (0x18): {code:cpp} void setKeylen(uint16_t value) { if (value > 0xff) { throw std::invalid_argument( "Response::setKeylen: key cannot exceed 1 byte"); } keylen = uint8_t(value); } {code} This change was introduced via: http://review.couchbase.org/#/c/86393/ See https://github.com/couchbase/kv_engine/blob/master/docs/BinaryProtocol.md#response-header - it should be permitted to return keys up to 65,535 bytes long for "regular" response packets (0x81). |
Affects Version/s | 5.5.4 [ 16003 ] | |
Affects Version/s | 6.0.1 [ 15522 ] | |
Affects Version/s | 5.5.3 [ 15520 ] | |
Affects Version/s | 5.5.2 [ 15412 ] | |
Affects Version/s | 5.5.1 [ 15159 ] | |
Affects Version/s | 5.5.0 [ 14610 ] | |
Affects Version/s | 6.0.0 [ 15048 ] |
Is this a Regression? | Unknown [ 10452 ] | Yes [ 10450 ] |
Link | This issue causes CBSE-6804 [ CBSE-6804 ] |
Fix Version/s | 6.0.3 [ 16164 ] |
Status | Open [ 1 ] | In Progress [ 3 ] |
Actual Start | 2019-05-22 06:48 (issue has been started) |
Rank | Ranked lower |
Affects Version/s | 6.0.2 [ 15919 ] |
Sprint | KV-Engine MH Beta Refresh [ 872 ] |
Rank | Ranked lower |
Link | This issue relates to CBSE-7201 [ CBSE-7201 ] |
Assignee | Trond Norbye [ trond ] | Dave Finlay [ dfinlay ] |
Fix Version/s | 6.0.3 [ 16164 ] |
Component/s | ns_server [ 10019 ] |
Summary | ns_server connections may be terminated if a client uses a long DCP_OPEN.name | Memcached should disallow DCP connections with names greater than 200 characters (was: ns_server connections may be terminated if a client uses a long DCP_OPEN.name) |
Component/s | ns_server [ 10019 ] |
Issue Type | Bug [ 1 ] | Improvement [ 4 ] |
Fix Version/s | Cheshire-Cat [ 15915 ] | |
Fix Version/s | Mad-Hatter [ 15037 ] |
Assignee | Dave Finlay [ dfinlay ] | Trond Norbye [ trond ] |
Sprint | KV-Engine MH 2nd Beta [ 872 ] | KV-Engine MH 2nd Beta, KV-Engine MH 2nd Beta 2 [ 872, 910 ] |
Sprint | KV-Engine MH 2nd Beta, KV-Engine Mad-Hatter GA [ 872, 910 ] | KV-Engine MH 2nd Beta [ 872 ] |
Link | This issue causes CBSE-7505 [ CBSE-7505 ] |
Resolution | Fixed [ 1 ] | |
Status | In Progress [ 3 ] | Resolved [ 5 ] |
Status | Resolved [ 5 ] | Closed [ 6 ] |
Link | This issue backports to CBSE-7584 [ CBSE-7584 ] |
Link |
This issue depends on |
Resolution | Fixed [ 1 ] | |
Status | Closed [ 6 ] | Reopened [ 4 ] |
Resolution | Fixed [ 1 ] | |
Status | Reopened [ 4 ] | Closed [ 6 ] |
Link | This issue relates to CBSE-10104 [ CBSE-10104 ] |
Link | This issue relates to CBSP-3701 [ CBSP-3701 ] |
Fix Version/s | 7.0.0 [ 17233 ] |
Fix Version/s | Cheshire-Cat [ 15915 ] |
Labels | releasenote |
Fix Version/s | 7.0.0 [ 17233 ] |
Fix Version/s | 7.0.0 [ 17233 ] |
Resolution | Fixed [ 1 ] | |
Status | Closed [ 6 ] | Reopened [ 4 ] |
Fix Version/s | 6.6.4 [ 17614 ] |
Link | This issue blocks MB-47673 [ MB-47673 ] |
Labels | releasenote | approved-for-6.6.4 releasenote |
Resolution | Fixed [ 1 ] | |
Status | Reopened [ 4 ] | Resolved [ 5 ] |
Status | Resolved [ 5 ] | Closed [ 6 ] |
Assignee | Trond Norbye [ trond ] | Ritam Sharma [ ritam.sharma ] |
Link | This issue is triggering CBSE-11034 [ CBSE-11034 ] |
Link | This issue causes CBSE-11043 [ CBSE-11043 ] |
Link | This issue blocks MB-50048 [ MB-50048 ] |
Link | This issue relates to CBSE-11560 [ CBSE-11560 ] |
Link | This issue relates to CBSE-11564 [ CBSE-11564 ] |