[MB-34280] Memcached should disallow DCP connections with names greater than 200 characters (was: ns_server connections may be terminated if a client uses a long DCP_OPEN.name) - Couchbase database

XML

Word

Printable

Details

Type: Improvement
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: 6.0.0, 5.5.0, 5.5.1, 5.5.2, 5.5.3, 6.0.1, 6.0.2, 5.5.4
Fix Version/s: 7.0.0, 6.6.4
Component/s: memcached
Labels:
- approved-for-6.6.4
- releasenote

Sprint:
KV-Engine MH 2nd Beta

Description

Summary

If a DCP client uses a long (exact length TBD) name when connecting, it can result in
STATS calls (for example performed by ns_server to monitor rebalance) to fail:

 462445 2019-05-11T08:18:28.525392+00:00 WARNING 143: exception occurred in runloop during packet execution. Cookie info: {"packet":{"magic":"ClientRequest","opcode":"STAT","keylen":3,"extlen":0,"datatype":"raw","vbucket":0,"bodylen":3,"opaque":"0x0","cas":0},"connection":"[ 127.0.0.1:48735 - 127.0.0.1:11209 (<ud>@ns_server</ud>) ]"} - closing connection: Response::setKeylen: key cannot exceed 1 byte

Details

Looks like problem is here - we cap the response keyLen to 1 byte for all response types, not just "Response with flex-framing extras" (0x18):

    void setKeylen(uint16_t value) {

        if (value > 0xff) {

            throw std::invalid_argument(

                    "Response::setKeylen: key cannot exceed 1 byte");

        keylen = uint8_t(value);

This change was introduced via: http://review.couchbase.org/#/c/86393/

See https://github.com/couchbase/kv_engine/blob/master/docs/BinaryProtocol.md#response-header - it should be permitted to return keys up to 65,535 bytes long for "regular" response packets (0x81).

Attachments

Issue Links

depends on

MB-49271 Backport MB-35881 to 6.6.4

Resolved

MB-35881 ns_server should generate DCP connection names that are no longer than allowed by memcached

Closed

MB-37246 Ensure Views does not generate DCP connection names that are no longer than allowed by memcached

Closed

MB-37283 Ensure XDCR does not generate DCP connection names that are no longer than allowed by memcached

Closed

MB-37243 Ensure Query does not generate DCP connection names that are no longer than allowed by memcached

Open

MB-37244 Ensure 2i does not generate DCP connection names that are no longer than allowed by memcached

Open

CBES-145 Ensure ElastricSearch Connector does not generate DCP connection names that are no longer than allowed by memcached

Resolved

JDCP-136 Ensure java-DCP-client does not generate DCP connection names that are no longer than allowed by memcached

Resolved

KAFKAC-160 Ensure Kafka Connector does not generate DCP connection names that are no longer than allowed by memcached

Resolved

CBG-626 Ensure Sync-Gateway does not generate DCP connection names that are no longer than allowed by memcached

Closed

MB-37241 Ensure FTS does not generate DCP connection names that are no longer than allowed by memcached

Closed

MB-37242 Ensure Backup does not generate DCP connection names that are no longer than allowed by memcached

Closed

MB-37245 Ensure Eventing does not generate DCP connection names that are no longer than allowed by memcached

Closed

MB-35882 The Memcached STATS "dcp" command should have a mode that only retrieves stats on connections owned by the user

Closed

relates to

DOC-9381 names used for DCP connections must not be longer than 200 characters

Resolved

JDCP-137 Increase connection name limit from 148 bytes to 200 bytes

Resolved

CMOS-154 Add check for DCP names > 200 chars (MB-34280)

In Review

(9 depends on, 3 relates to)

Gerrit Reviews

- Issue Only
- Show All Reviews
- Show Open Reviews
- Show All Issues
- Show Open Issues

No reviews matched the request. Check your Options in the drop-down menu of this sections header.

Activity

Ascending order - Click to sort in descending order

Dave Rigby created issue - 22/May/19 12:54 AM

Dave Rigby made changes - 22/May/19 12:55 AM

Field	Original Value	New Value
Description	Looks like problem is here - we cap the response keyLen to 1 byte for all response types, not just "Response with flex-framing extras" (0x18): {code:cpp} void setKeylen(uint16_t value) { if (value > 0xff) { throw std::invalid_argument( "Response::setKeylen: key cannot exceed 1 byte"); } keylen = uint8_t(value); } {code} See https://github.com/couchbase/kv_engine/blob/master/docs/BinaryProtocol.md#response-header - it should be permitted to return keys up to 65,535 bytes long for "regular" response packets (0x81).	h2. Summary If a DCP client uses a long (exact length TBD) name when connecting, it can result in STATS calls (for example performed by ns_server to monitor rebalance) to fail: {code} 462445 2019-05-11T08:18:28.525392+00:00 WARNING 143: exception occurred in runloop during packet execution. Cookie info: {"packet":{"magic":"ClientRequest","opcode":"STAT","keylen":3,"extlen":0,"datatype":"raw","vbucket":0,"bodylen":3,"opaque":"0x0","cas":0},"connection":"[ 127.0.0.1:48735 - 127.0.0.1:11209 (<ud>@ns_server</ud>) ]"} - closing connection: Response::setKeylen: key cannot exceed 1 byte {code} h2. Details Looks like problem is here - we cap the response keyLen to 1 byte for all response types, not just "Response with flex-framing extras" (0x18): {code:cpp} void setKeylen(uint16_t value) { if (value > 0xff) { throw std::invalid_argument( "Response::setKeylen: key cannot exceed 1 byte"); } keylen = uint8_t(value); } {code} See https://github.com/couchbase/kv_engine/blob/master/docs/BinaryProtocol.md#response-header - it should be permitted to return keys up to 65,535 bytes long for "regular" response packets (0x81).

Dave Rigby made changes - 22/May/19 12:56 AM

Description

h2. Summary

If a DCP client uses a long (exact length TBD) name when connecting, it can result in
STATS calls (for example performed by ns_server to monitor rebalance) to fail:

{code}
462445 2019-05-11T08:18:28.525392+00:00 WARNING 143: exception occurred in runloop during packet execution. Cookie info: {"packet":{"magic":"ClientRequest","opcode":"STAT","keylen":3,"extlen":0,"datatype":"raw","vbucket":0,"bodylen":3,"opaque":"0x0","cas":0},"connection":"[ 127.0.0.1:48735 - 127.0.0.1:11209 (<ud>@ns_server</ud>) ]"} - closing connection: Response::setKeylen: key cannot exceed 1 byte
{code}

h2. Details

Looks like problem is here - we cap the response keyLen to 1 byte for all response types, not just "Response with flex-framing extras" (0x18):

{code:cpp}
    void setKeylen(uint16_t value) {
        if (value > 0xff) {
            throw std::invalid_argument(
                    "Response::setKeylen: key cannot exceed 1 byte");
        }
        keylen = uint8_t(value);
    }
{code}

See https://github.com/couchbase/kv_engine/blob/master/docs/BinaryProtocol.md#response-header - it should be permitted to return keys up to 65,535 bytes long for "regular" response packets (0x81).

h2. Summary

If a DCP client uses a long (exact length TBD) name when connecting, it can result in
STATS calls (for example performed by ns_server to monitor rebalance) to fail:

{code}
462445 2019-05-11T08:18:28.525392+00:00 WARNING 143: exception occurred in runloop during packet execution. Cookie info: {"packet":{"magic":"ClientRequest","opcode":"STAT","keylen":3,"extlen":0,"datatype":"raw","vbucket":0,"bodylen":3,"opaque":"0x0","cas":0},"connection":"[ 127.0.0.1:48735 - 127.0.0.1:11209 (<ud>@ns_server</ud>) ]"} - closing connection: Response::setKeylen: key cannot exceed 1 byte
{code}

h2. Details

Looks like problem is here - we cap the response keyLen to 1 byte for all response types, not just "Response with flex-framing extras" (0x18):

{code:cpp}
    void setKeylen(uint16_t value) {
        if (value > 0xff) {
            throw std::invalid_argument(
                    "Response::setKeylen: key cannot exceed 1 byte");
        }
        keylen = uint8_t(value);
    }
{code}
This change was introduced via: http://review.couchbase.org/#/c/86393/

See https://github.com/couchbase/kv_engine/blob/master/docs/BinaryProtocol.md#response-header - it should be permitted to return keys up to 65,535 bytes long for "regular" response packets (0x81).

Dave Rigby made changes - 22/May/19 12:57 AM

Affects Version/s		5.5.4 [ 16003 ]
Affects Version/s		6.0.1 [ 15522 ]
Affects Version/s		5.5.3 [ 15520 ]
Affects Version/s		5.5.2 [ 15412 ]
Affects Version/s		5.5.1 [ 15159 ]
Affects Version/s		5.5.0 [ 14610 ]
Affects Version/s		6.0.0 [ 15048 ]

Dave Rigby made changes - 22/May/19 12:57 AM

Is this a Regression?

Unknown [ 10452 ]

Yes [ 10450 ]

Dave Rigby made changes - 22/May/19 12:57 AM

Link

This issue causes CBSE-6804 [ CBSE-6804 ]

Dave Rigby made changes - 22/May/19 12:58 AM

Fix Version/s

6.0.3 [ 16164 ]

Trond Norbye made changes - 22/May/19 4:48 AM

Status

Open [ 1 ]

In Progress [ 3 ]

Trond Norbye made changes - 22/May/19 4:48 AM

Actual Start

2019-05-22 06:48 (issue has been started)

Daniel Owen made changes - 17/Jun/19 5:28 AM

Rank

Ranked lower

Dave Rigby made changes - 28/Jun/19 9:34 AM

Affects Version/s

6.0.2 [ 15919 ]

Dave Rigby made changes - 12/Aug/19 6:34 AM

Sprint

KV-Engine MH Beta Refresh [ 872 ]

Dave Rigby made changes - 12/Aug/19 6:34 AM

Rank

Ranked lower

David Haikney made changes - 13/Aug/19 2:11 PM

Link

This issue relates to CBSE-7201 [ CBSE-7201 ]

Dave Finlay made changes - 13/Aug/19 8:45 PM

Assignee

Trond Norbye [ trond ]

Dave Finlay [ dfinlay ]

Dave Finlay made changes - 13/Aug/19 8:45 PM

Fix Version/s

6.0.3 [ 16164 ]

Dave Finlay made changes - 13/Aug/19 8:45 PM

Component/s

ns_server [ 10019 ]

Dave Finlay made changes - 09/Sep/19 9:33 PM

Summary

ns_server connections may be terminated if a client uses a long DCP_OPEN.name

Memcached should disallow DCP connections with names greater than 200 characters (was: ns_server connections may be terminated if a client uses a long DCP_OPEN.name)

Dave Finlay made changes - 09/Sep/19 9:33 PM

Component/s

ns_server [ 10019 ]

Dave Finlay made changes - 09/Sep/19 9:33 PM

Issue Type

Bug [ 1 ]

Improvement [ 4 ]

Dave Finlay made changes - 09/Sep/19 9:34 PM

Fix Version/s		Cheshire-Cat [ 15915 ]
Fix Version/s	Mad-Hatter [ 15037 ]

Dave Finlay made changes - 09/Sep/19 9:34 PM

Assignee

Dave Finlay [ dfinlay ]

Trond Norbye [ trond ]

Dave Finlay made changes - 09/Sep/19 9:37 PM

Link

This issue depends on ~~MB-35881~~ [ ~~MB-35881~~ ]

Dave Finlay made changes - 09/Sep/19 9:46 PM

Link

This issue depends on ~~MB-35882~~ [ ~~MB-35882~~ ]

Dave Rigby made changes - 09/Oct/19 1:55 AM

Sprint

KV-Engine MH 2nd Beta [ 872 ]

KV-Engine MH 2nd Beta, KV-Engine MH 2nd Beta 2 [ 872, 910 ]

Dave Rigby made changes - 09/Oct/19 1:58 AM

Sprint

KV-Engine MH 2nd Beta, KV-Engine Mad-Hatter GA [ 872, 910 ]

KV-Engine MH 2nd Beta [ 872 ]

Daniel Owen made changes - 23/Oct/19 3:49 AM

Link

This issue causes CBSE-7505 [ CBSE-7505 ]

Trond Norbye made changes - 24/Oct/19 4:26 AM

Resolution		Fixed [ 1 ]
Status	In Progress [ 3 ]	Resolved [ 5 ]

Trond Norbye made changes - 25/Oct/19 12:36 AM

Status

Resolved [ 5 ]

Closed [ 6 ]

Michael Zilbertstein (Inactive) made changes - 06/Nov/19 2:30 AM

Link

This issue backports to CBSE-7584 [ CBSE-7584 ]

Daniel Owen made changes - 12/Dec/19 1:57 AM

Link

This issue depends on ~~MB-37241~~ [ ~~MB-37241~~ ]

Daniel Owen made changes - 12/Dec/19 2:05 AM

Link

This issue depends on ~~MB-37242~~ [ ~~MB-37242~~ ]

Daniel Owen made changes - 12/Dec/19 2:08 AM

Link

This issue depends on MB-37243 [ MB-37243 ]

Daniel Owen made changes - 12/Dec/19 2:19 AM

Link

This issue depends on MB-37244 [ MB-37244 ]

Daniel Owen made changes - 12/Dec/19 2:21 AM

Link

This issue depends on ~~MB-37245~~ [ ~~MB-37245~~ ]

Daniel Owen made changes - 12/Dec/19 2:33 AM

Link

This issue depends on ~~MB-37246~~ [ ~~MB-37246~~ ]

Daniel Owen made changes - 12/Dec/19 2:47 AM

Link

This issue depends on ~~CBG-626~~ [ ~~CBG-626~~ ]

Daniel Owen made changes - 12/Dec/19 2:50 AM

Link

This issue depends on ~~JDCP-136~~ [ ~~JDCP-136~~ ]

Daniel Owen made changes - 12/Dec/19 3:15 AM

Link

This issue depends on ~~KAFKAC-160~~ [ ~~KAFKAC-160~~ ]

Daniel Owen made changes - 12/Dec/19 3:20 AM

Link

This issue depends on ~~CBES-145~~ [ ~~CBES-145~~ ]

David Nault made changes - 12/Dec/19 4:51 PM

Link

This issue relates to ~~JDCP-137~~ [ ~~JDCP-137~~ ]

Daniel Owen made changes - 16/Dec/19 5:15 AM

Link

This issue depends on ~~MB-37283~~ [ ~~MB-37283~~ ]

Daniel Owen made changes - 16/Dec/19 5:20 AM

Resolution	Fixed [ 1 ]
Status	Closed [ 6 ]	Reopened [ 4 ]

Daniel Owen made changes - 16/Dec/19 5:20 AM

Resolution		Fixed [ 1 ]
Status	Reopened [ 4 ]	Closed [ 6 ]

Daniel Owen made changes - 30/Apr/21 5:51 AM

Link

This issue relates to CBSE-10104 [ CBSE-10104 ]

Elliot Goodwin made changes - 20/May/21 8:45 AM

Link

This issue relates to CBSP-3701 [ CBSP-3701 ]

Lynn Straus made changes - 17/Jun/21 2:34 PM

Fix Version/s

7.0.0 [ 17233 ]

Lynn Straus made changes - 17/Jun/21 2:39 PM

Fix Version/s

Cheshire-Cat [ 15915 ]

Dave Rigby made changes - 08/Jul/21 7:45 AM

Labels

releasenote

Daniel Owen made changes - 01/Nov/21 11:08 AM

Fix Version/s

7.0.0 [ 17233 ]

Daniel Owen made changes - 01/Nov/21 11:08 AM

Fix Version/s

7.0.0 [ 17233 ]

Daniel Owen made changes - 01/Nov/21 11:22 AM

Resolution	Fixed [ 1 ]
Status	Closed [ 6 ]	Reopened [ 4 ]

Daniel Owen made changes - 01/Nov/21 11:22 AM

Fix Version/s

6.6.4 [ 17614 ]

Wayne Siu made changes - 01/Nov/21 8:03 PM

Link

This issue blocks MB-47673 [ MB-47673 ]

Wayne Siu made changes - 01/Nov/21 8:08 PM

Labels

releasenote

approved-for-6.6.4 releasenote

Meni Hillel made changes - 01/Nov/21 9:40 PM

Link

This issue depends on ~~MB-49271~~ [ ~~MB-49271~~ ]

Trond Norbye made changes - 02/Nov/21 3:59 AM

Resolution		Fixed [ 1 ]
Status	Reopened [ 4 ]	Resolved [ 5 ]

Trond Norbye made changes - 02/Nov/21 3:59 AM

Status

Resolved [ 5 ]

Closed [ 6 ]

Wayne Siu made changes - 05/Nov/21 9:31 PM

Assignee

Trond Norbye [ trond ]

Ritam Sharma [ ritam.sharma ]

Tony Hillman made changes - 08/Nov/21 10:59 AM

Link

This issue relates to ~~DOC-9381~~ [ ~~DOC-9381~~ ]

Marks Polakovs made changes - 12/Nov/21 8:11 AM

Link

This issue is triggering CBSE-11034 [ CBSE-11034 ]

Marks Polakovs made changes - 12/Nov/21 8:21 AM

Link

This issue relates to CMOS-154 [ CMOS-154 ]

Callum Majumdar made changes - 16/Nov/21 2:52 AM

Link

This issue causes CBSE-11043 [ CBSE-11043 ]

People

Assignee:: Ritam Sharma

Reporter:: Dave Rigby

Votes:: 0 Vote for this issue

Watchers:: 16 Start watching this issue

Dates

Created:: 22/May/19 12:54 AM

Updated:: 16/Nov/21 2:52 AM

Resolved:: 02/Nov/21 3:59 AM

Gerrit Reviews

There are no open Gerrit changes

Show There are 6 closed Gerrit changes

Hide There are 6 closed Gerrit changes

MB-34280: Allow long keys in (normal) return packets: Gerrit Review:

MB-34280: Set max DCP name to 200 characters: Gerrit Review:

JDCP-137 Increase connection name limit from 148 bytes to 200 bytes: Gerrit Review:

MB-34280: Set max DCP name to 200 characters: Gerrit Review:

Merge remote-tracking branch 'couchbase/mad-hatter': Gerrit Review:

Merge commit mad-hatter 'b5fa131c0' into couchbase/cheshire-cat: Gerrit Review:

Couchbase Server

Memcached should disallow DCP connections with names greater than 200 characters (was: ns_server connections may be terminated if a client uses a long DCP_OPEN.name)

Details

Description

Summary

Details

Attachments

Issue Links

Gerrit Reviews

Activity

People

Dates

Gerrit Reviews

PagerDuty