Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-37991

DCP performance failed because more arguments are needed in call to dcpFeed.DcpRequestStream

    XMLWordPrintable

    Details

    • Triage:
      Untriaged
    • Is this a Regression?:
      Unknown

      Description

      DCP performance tests failed because more arguments are needed in call to dcpFeed.DcpRequestStream. 

      Build: 7.0.0-1375

      Job: http://perf.jenkins.couchbase.com/job/triton/30537/

      Error Message:
      go/dcptest/main.go:183: not enough arguments in call to dcpFeed.DcpRequestStream
      have (uint16, uint16, uint32, uint64, uint64, uint64, uint64, uint64)
      want (uint16, uint16, uint32, uint64, uint64, uint64, uint64, uint64, string, string, []string)
      Makefile:48: recipe for target 'dcptest' failed

       

      Currently, perfrunner is passing these arguments to DcpRequestStream.
      https://github.com/couchbase/perfrunner/blob/master/go/dcptest/main.go#L176

      func startDcp(dcpFeed *couchbase.DcpFeed, flogs couchbase.FailoverLog, wg *sync.WaitGroup) {
         start, end := uint64(0), uint64(0xFFFFFFFFFFFFFFFF)
         snapStart, snapEnd := uint64(0), uint64(0)
         manifest, scope, collection := "", "", []string{}
         for vbno, flog := range flogs {
            x := flog[len(flog)-1] // map[uint16][][2]uint64
            opaque, flags, vbuuid := uint16(vbno), uint32(0), x[0]
            err := dcpFeed.DcpRequestStream(
               vbno, opaque, flags, vbuuid, start, end, snapStart, snapEnd, manifest, scope, collection)
            mf(err, fmt.Sprintf("stream-req for %v failed", vbno))
         }
         logging.Infof("Done startDCP\n")
         defer wg.Done()
      }

      What values should we pass to these new arguments in DcpRequestStream? Are there any default values? Will these values be different after collections are enabled?

      https://github.com/couchbase/indexing/blob/master/secondary/dcp/transport/client/dcp_feed.go#L161

      // DcpRequestStream for a single vbucket.
      func (feed *DcpFeed) DcpRequestStream(vbno, opaqueMSB uint16, flags uint32,
      	vuuid, startSequence, endSequence, snapStart, snapEnd uint64,
      	manifestUID, scopeId string, collectionIds []string) error {
       
      	respch := make(chan []interface{}, 1)
      	cmd := []interface{}{
      		dfCmdRequestStream, vbno, opaqueMSB, flags, vuuid,
      		startSequence, endSequence, snapStart, snapEnd,
      		manifestUID, scopeId, collectionIds,
      		respch}
      	resp, err := failsafeOp(feed.reqch, respch, cmd, feed.finch)
      	return opError(err, resp, 0)
      }
      

        Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

          Hide
          drigby Dave Rigby added a comment -

          This isn’t anything to do with KV-Engine - it looks to be due to perfrunner not updating to the changes in indexing’s DcpFeed: https://github.com/couchbase/indexing/commit/b0a9233b3686812ea095cd16042d740ed08d40c0

          Show
          drigby Dave Rigby added a comment - This isn’t anything to do with KV-Engine - it looks to be due to perfrunner not updating to the changes in indexing’s DcpFeed: https://github.com/couchbase/indexing/commit/b0a9233b3686812ea095cd16042d740ed08d40c0
          Hide
          bo-chun.wang Bo-Chun Wang added a comment -

          Dave Rigby

          We need to update perfrunner to the changes in indexing's DcpFeed. My question is what values we should pass to these new arguments, manifest, scope, and collection. Should we pass empty string for initialization? Are there any default values?

          Show
          bo-chun.wang Bo-Chun Wang added a comment - Dave Rigby We need to update perfrunner to the changes in indexing's DcpFeed. My question is what values we should pass to these new arguments, manifest, scope, and collection. Should we pass empty string for initialization? Are there any default values?
          Hide
          drigby Dave Rigby added a comment - - edited

          I don't know the details of now the library you are using have abstracted things, but from the commit message I linked it sounds like you can pass an empty scopeId and nil collectionIds to get the previous behaviour:

          For all collection aware dcp feed's, projector will open streams for the scope/collections specified in the request timestamps. If
          no scopeId/collectionIds are specified i.e. scopeId is empty and collectionIds is nil, projector will open streams for the entire bucket

          Show
          drigby Dave Rigby added a comment - - edited I don't know the details of now the library you are using have abstracted things, but from the commit message I linked it sounds like you can pass an empty scopeId and nil collectionIds to get the previous behaviour: For all collection aware dcp feed's, projector will open streams for the scope/collections specified in the request timestamps. If no scopeId/collectionIds are specified i.e. scopeId is empty and collectionIds is nil, projector will open streams for the entire bucket
          Hide
          bo-chun.wang Bo-Chun Wang added a comment -

          I passed an empty scopeId and nil collectionIds and did a dcp test. The test passed. 

          http://perf.jenkins.couchbase.com/job/triton/30605/

           

           

          Show
          bo-chun.wang Bo-Chun Wang added a comment - I passed an empty scopeId and nil collectionIds and did a dcp test. The test passed.  http://perf.jenkins.couchbase.com/job/triton/30605/    

            People

            Assignee:
            bo-chun.wang Bo-Chun Wang
            Reporter:
            bo-chun.wang Bo-Chun Wang
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Gerrit Reviews

                There are no open Gerrit changes

                  PagerDuty