Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-56701

[CX] Result tuples unnecessarily serialized twice

    XMLWordPrintable

Details

    • Untriaged
    • 0
    • Unknown
    • Analytics Sprint 19

    Description

      The ResultWriterOperatorDescriptor is the operator that persists the query result to disk. Each partition persists its portion of the result by serializing the tuples (that are in ADM format) as JSON strings into a byte array. This byte array that represents the JSON is added into a frame that is used to write the accumulated tuples to the result file. If the byte array is added to the frame successfully, the byte array is reset and the next tuple is serialized into it. However, if the byte array couldn't be added to the frame because the frame is full, the frame is flushed to disk and is emptied but also the byte array is reset at the same time. This leads to having to re-serialized the tuple again into the byte array and adding it to the frame. This becomes expensive especially for large tuples.

      The byte array should not be reset upon flushing the frame when the frame cannot hold it. Instead, the frame should be flushed without reseting the byte array. Then, adding the byte array to the frame should be attempted again. When adding the byte array is successful, it should be reset.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              ali.alsuliman Ali Alsuliman
              ali.alsuliman Ali Alsuliman
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty