Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

[BP to 7.1.x]- Optimise projector CPU during XATTR processing

Description

In one of the customer issues, it has been observed that projector was spending lot of time during XATTR processing. This ticket is to track the improvements in CPU at projector for XATTR processing

1050: if (event.Opcode == transport.DCP_MUTATION || . . 1051: event.Opcode == transport.DCP_DELETION) && event.HasXATTR() { . . 1052: xattrLen := int(binary.BigEndian.Uint32(rq.Body)) . . 1053: xattrData := rq.Body[4 : 4+xattrLen] . 29.05s 1054: event.RawXATTR = make(map[string][]byte, xattrLen) . . 1055: for len(xattrData) > 0 { . . 1056: pairLen := binary.BigEndian.Uint32(xattrData[0:]) . . 1057: xattrData = xattrData[4:] . . 1058: binaryPair := xattrData[:pairLen-1] . . 1059: xattrData = xattrData[pairLen:] . 60ms 1060: kvPair := bytes.Split(binaryPair, []byte{0x00}) . . 1061: event.RawXATTR[string(kvPair[0])] = kvPair[1] . . 1062: }
492: for _, xattr := range ie.xattrs { . . 493: if _, ok := m.ParsedXATTR[xattr]; !ok { . . 494: var val interface{} . . 495: if len(m.RawXATTR[xattr]) != 0 { . 21.66s 496: if err := json.Unmarshal(m.RawXATTR[xattr], &val); err != nil { . . 497: arg1 := logging.TagStrUD(xattr) . . 498: arg2 := logging.TagStrUD(m.Key) . . 499: logging.Errorf("Error parsing XATTR %s for %s: %v", . . 500: arg1, arg2, err) . . 501: } else { . 10ms 502: m.ParsedXATTR[xattr] = val . . 503: } . . 504: } . . 505: } . . 506: }

Components

Affects versions

Fix versions

Labels

Environment

None

Link to Log File, atop/blg, CBCollectInfo, Core dump

None

Release Notes Description

None

Attachments

3
  • 12 Jan 2023, 05:31 AM
  • 12 Jan 2023, 05:29 AM
  • 12 Jan 2023, 05:29 AM

Activity

Show:

Dragos Taraban February 23, 2023 at 4:23 PM
Edited

On the perf side, we have also seen large improvements in projector cpu and memory usage. For tests with 1,000 channels per document, projector rss is roughly half after the patch, while projector cpu usage is also halved. For larger tests, with 10,000 channels per documents, the improvements are even larger, with projector rss being 4 to 5 times smaller after the fixes and projector cpu being 3 times smaller, with less spikes. We have currently tested 7.1.3 and 7.1.4 and 7.2.0.

For 7.1.4:

  • 1,000 channels per document:

For 7.1.3:

For 7.2.0:

 

CB robot January 13, 2023 at 5:01 AM

Build couchbase-server-7.1.4-3557 contains indexing commit 1bbbb20 with commit message:
https://couchbasecloud.atlassian.net/browse/MB-54560#icft=MB-54560 Do not allocate 'xattrLen' number of map entries

CB robot January 13, 2023 at 5:01 AM

Build couchbase-server-7.1.4-3557 contains indexing commit 96c6ce5 with commit message:
https://couchbasecloud.atlassian.net/browse/MB-54560#icft=MB-54560 Use NewParsedValueWithOptions insted of json unmarshall for xattr processing

CB robot January 13, 2023 at 3:19 AM

Build couchbase-server-7.2.0-5077 contains indexing commit 1bbbb20 with commit message:
https://couchbasecloud.atlassian.net/browse/MB-54560#icft=MB-54560 Do not allocate 'xattrLen' number of map entries

CB robot January 13, 2023 at 3:19 AM

Build couchbase-server-7.2.0-5077 contains indexing commit 96c6ce5 with commit message:
https://couchbasecloud.atlassian.net/browse/MB-54560#icft=MB-54560 Use NewParsedValueWithOptions insted of json unmarshall for xattr processing

Fixed
Pinned fields
Click on the next to a field label to start pinning.

Details

Assignee

Reporter

Is this a Regression?

Unknown

Triage

Untriaged

Story Points

Priority

Instabug

Open Instabug

PagerDuty

Sentry

Zendesk Support

Created November 15, 2022 at 8:24 PM
Updated February 28, 2023 at 7:01 AM
Resolved January 13, 2023 at 1:14 AM
Instabug