Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-38978

STAT "dcp" and "dcpagg" adversely affect front-end operation latency

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 6.5.0, 6.0.0, 5.5.0, 5.5.1, 5.5.2, 5.5.3, 6.0.1, 6.0.2, 5.5.4, 6.0.3, 5.5.5, 5.5.6, 6.0.4, 6.5.1, 7.0.0
    • 7.0.2
    • couchbase-bucket
    • 1
    • KV-Engine Sprint 2021 July

    Description

      Problem

      Currently the "dcp" and "dcpagg" stats are collated by the front-end thread. This can be an expensive operation, and hence the STAT command can take a significant time to return. This means the STAT operation is tying-up a front-and thread which means that operations that happen to be allocated to the same thread get delayed for an unacceptable length of time.

      Solutions

      1. Move the expensive part(s) of these stat groups to a background task, allowing the client operation to EWOULDBLOCK and yield execution back to the front-end thread.
      2. Investigate what part(s) of these stat groups ns_server needs. If the stat(s) needed are cheap then consider new stat groups which only contain the specific elements ns_server needs.

      Note we might want both of these; given that dcp and dcpagg are recorded by cbcollect_info and we wouldn't want to remove them there.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            owend Daniel Owen created issue -
            owend Daniel Owen made changes -
            Field Original Value New Value
            Link This issue relates to CBSE-8260 [ CBSE-8260 ]
            owend Daniel Owen made changes -
            Fix Version/s Cheshire-Cat [ 15915 ]
            owend Daniel Owen made changes -
            Issue Type Improvement [ 4 ] Bug [ 1 ]
            owend Daniel Owen made changes -
            Description Currently the "dcp" and "dcpagg" stats are collated by the front-end thread. This can be an expensive operation, and hence the STAT command can take a significant time to return. This means the STAT operation is tying-up a front-and thread which means that operations that happen to be allocated to the same thread get delayed for an unacceptable length of time.
            drigby Dave Rigby made changes -
            Description Currently the "dcp" and "dcpagg" stats are collated by the front-end thread. This can be an expensive operation, and hence the STAT command can take a significant time to return. This means the STAT operation is tying-up a front-and thread which means that operations that happen to be allocated to the same thread get delayed for an unacceptable length of time. +Problem+

            Currently the "dcp" and "dcpagg" stats are collated by the front-end thread. This can be an expensive operation, and hence the STAT command can take a significant time to return. This means the STAT operation is tying-up a front-and thread which means that operations that happen to be allocated to the same thread get delayed for an unacceptable length of time.

            +Solutions+

            # Move the expensive part(s) of these stat groups to a background task, allowing the client operation to EWOULDBLOCK and yield execution back to the front-end thread.
            # Investigate what part(s) of these stat groups ns_server needs. If the stat(s) needed are cheap then consider new stat groups which only contain the specific elements ns_server needs.

            Note we might want both of these; given that {{dcp}} and {{dcpagg}} are recorded by {{cbcollect_info}} and we wouldn't want to remove them there.
            drigby Dave Rigby made changes -
            Summary Move STAT "dcp" and "dcpagg" to background threads Reduce impact of STAT "dcp" and "dcpagg" on other front-end operations
            owend Daniel Owen made changes -
            Fix Version/s backlog [ 15925 ]
            Fix Version/s Cheshire-Cat [ 15915 ]
            owend Daniel Owen made changes -
            Issue Type Bug [ 1 ] Improvement [ 4 ]
            owend Daniel Owen made changes -
            Fix Version/s CheshireCat.Next [ 16908 ]
            Fix Version/s backlog [ 15925 ]
            owend Daniel Owen made changes -
            Link This issue blocks MB-44024 [ MB-44024 ]
            owend Daniel Owen made changes -
            Priority Major [ 3 ] Critical [ 2 ]
            jeelan.poola Jeelan Poola made changes -
            Link This issue blocks MB-45125 [ MB-45125 ]
            drigby Dave Rigby made changes -
            Fix Version/s 7.0.1 [ 17104 ]
            Fix Version/s CheshireCat.Next [ 16908 ]
            amarantha.kulkarni Amarantha Kulkarni (Inactive) made changes -
            Labels releasenote
            amarantha.kulkarni Amarantha Kulkarni (Inactive) made changes -
            Link This issue is triggering DOC-8403 [ DOC-8403 ]
            drigby Dave Rigby made changes -
            Rank Ranked higher
            drigby Dave Rigby made changes -
            Sprint KV-Engine Sprint 2021 July [ 1660 ]
            drigby Dave Rigby made changes -
            Rank Ranked lower
            drigby Dave Rigby made changes -
            Assignee Daniel Owen [ owend ]
            drigby Dave Rigby made changes -
            Issue Type Improvement [ 4 ] Bug [ 1 ]
            Summary Reduce impact of STAT "dcp" and "dcpagg" on other front-end operations STAT "dcp" and "dcpagg" adversely affect front-end operation latency
            drigby Dave Rigby made changes -
            Rank Ranked higher
            drigby Dave Rigby made changes -
            Rank Ranked higher
            owend Daniel Owen made changes -
            Assignee James Harrison [ james.harrison ]
            vinayaka.kamath Vinayaka Kamath (Inactive) made changes -
            Link This issue causes MB-47048 [ MB-47048 ]
            james.harrison James Harrison made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            james.harrison James Harrison made changes -
            VERIFICATION STEPS Stat correctness guarded by existing unit tests
            Resolution Fixed [ 1 ]
            Status In Progress [ 3 ] Resolved [ 5 ]
            james.harrison James Harrison made changes -
            Assignee James Harrison [ james.harrison ] Daniel Owen [ owend ]
            drigby Dave Rigby made changes -
            Affects Version/s 7.0.0 [ 17233 ]
            ashwin.govindarajulu Ashwin Govindarajulu made changes -
            Assignee Daniel Owen [ owend ] Ashwin Govindarajulu [ ashwin.govindarajulu ]
            Status Resolved [ 5 ] Closed [ 6 ]
            lynn.straus Lynn Straus made changes -
            Fix Version/s 7.0.2 [ 18012 ]
            lynn.straus Lynn Straus made changes -
            Fix Version/s 7.0.1 [ 17104 ]

            People

              ashwin.govindarajulu Ashwin Govindarajulu
              owend Daniel Owen
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  PagerDuty