Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-49845

[Investigate] Can only utilise ~70% of scope data size limit.

    XMLWordPrintable

Details

    • Task
    • Status: Resolved
    • Major
    • Resolution: Not a Bug
    • Neo
    • Neo
    • couchbase-bucket
    • None
    • Enterprise Edition 7.1.0 build 1815
    • 1

    Description

      Given a scope limit of 10 million bytes, I was only able to utilise ~70% of the limit using cbworkloadgen.

      Logs: https://cb-engineering.s3.amazonaws.com/MB-49845/collectinfo-2021-12-01T100309-ns_1%40127.0.0.1.zip

      Requirements:
      A single node with a single 'default' bucket.

      Steps to reproduce:
      1. Create scope 'my_scope' with a limit of 10000000 bytes:

      curl -u Administrator:password http://10.112.210.101:8091/pools/default/buckets/default/scopes -d 'name=my_scope&limits={"kv": {"data_size":10000000}}'
      

      2. Create single collection inside the scope using the UI.

      3. Use cbworkloadgen to produce 7000 items of 1000 bytes with the --no-compress flag set.

      /opt/couchbase/bin/cbworkloadgen -b default -u Administrator -p password -s 1000 -i 7000 -c 0x8 --low-compression
      

      What happens?

      Cbworkloadgen fails with a 'MCSink MC error: 52'.

      2021-12-01 09:53:23,586: s0 error: async operation: error: MCSink MC error: 52 on sink: http://127.0.0.1:8091(default@N/A-0)
      error: MCSink MC error: 52
      

      The 'sum(kv_collection_data_size_bytes)' statistic is at 6797930 bytes following this:

      From the user interface I can observe that I have 6.44K items with a total of 6.48MiB disk utilisation for that particular scope:

      What I expected to happen:

      I expected to able to write ~10000000 bytes of documents to the scope.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          jwalker Jim Walker added a comment - - edited

          What happens with the scope data size is that it is enforced against each vbucket.

          So when you set a limit of 10,000,000 each vbucket gets a limit - i.e. 10,000,000 / 1024. This is done so that we can scale up and down without the applied limit changing. I.e. if we did enforce on the total bytes owned by the node, scaling down risks the limit being exceeded.

          So in your test, as the limit is relatively small each vbucket gets a limit of 9765 bytes and cbworkloadgen fails on the first error.

          E.g. if a cluster only had 2 vbuckets and you set the limit to 10 bytes and the workload generator happened to write to vb:0 with 2 x 5 bytes documents before it wrote to vb:1, cbworkloadgen will fail on the 2nd write with "status 0x34 scope data exceeded", and you will only have 50% of you limit used.

          We need a DOC ticket to explain that limits which don't divide by 1024 will also mean you can never get to the limit.

          jwalker Jim Walker added a comment - - edited What happens with the scope data size is that it is enforced against each vbucket. So when you set a limit of 10,000,000 each vbucket gets a limit - i.e. 10,000,000 / 1024. This is done so that we can scale up and down without the applied limit changing. I.e. if we did enforce on the total bytes owned by the node, scaling down risks the limit being exceeded. So in your test, as the limit is relatively small each vbucket gets a limit of 9765 bytes and cbworkloadgen fails on the first error. E.g. if a cluster only had 2 vbuckets and you set the limit to 10 bytes and the workload generator happened to write to vb:0 with 2 x 5 bytes documents before it wrote to vb:1, cbworkloadgen will fail on the 2nd write with "status 0x34 scope data exceeded", and you will only have 50% of you limit used. We need a DOC ticket to explain that limits which don't divide by 1024 will also mean you can never get to the limit.

          People

            asad.zaidi Asad Zaidi (Inactive)
            asad.zaidi Asad Zaidi (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty