Uploaded image for project: 'Couchbase .NET client library'
  1. Couchbase .NET client library
  2. NCBC-3842

Newtonsoft deserializing UTF8 surrogate pairs can cause exceptions or buffer overflow

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Major
    • 3.6.3
    • 3.6.2
    • library
    • 0
    • Other/not scored

    Description

      The bug https://issues.couchbase.com/browse/NCBC-3543 regarding UTF-8 surrogate pairs was previously addressed to handle cases where reads by Newtonsoft.Json from the UTF-8 byte stream split surrogate pairs across reads. However, there is still a corner case where the second read will fill the destination buffer (rather than leaving extra bytes unfilled).

      In .NET 6 and later this causes an exception trying to slice the destination span. In .NET 4 this is most likely causing a write one character beyond the end of the buffer, which could be a CWE-122 security risk.

      The precise steps to reproduce are unclear, because the JsonTextReader employed by Newtsonsoft reads from our Utf8MemoryReader using buffer lengths which vary situationally. Conceptually, it requires something like this:

      1. Build a long JSON string that contains a surrogate pair in the middle at the location where Newtonsoft buffer lengths will read one half of the pair
      2. Ensure that the JSON string is also long enough that the next read will fill the entire buffer supplied by Newtonsoft
      3. Deserialize from ReadOnlyMemory<byte> using the DefaultSerializer (Newtonsoft.Json)

      Attachments

        Issue Links

          Activity

            People

              btburnett3 Brant Burnett
              btburnett3 Brant Burnett
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:

                PagerDuty