Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-51950

[Magma] Magma bucket not honouring RAM quota allocated when analytics is ingesting data from it.

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 7.1.1
    • 7.1.2
    • couchbase-bucket
    • Enterprise Edition 7.1.1 build 3025
    • Untriaged
    • Centos 64-bit
    • 1
    • No

    Description

      Observation -
      Although only 1 GB RAM is allocated to the magma bucket, the RAM usage is reaching around 3.9 GB. This is observed when analytics is trying to ingest data into it's datasets.

      Cluster Info -

      Node Services CPU_utilization Mem_total Mem_free Swap_mem_used Active / Replica Version
      172.23.108.0 cbas 2.46478873239 3.67 GiB 2.50 GiB 0.0 Byte / 3.50 GiB 0 / 0 7.1.1-3025-enterprise
      172.23.108.1 cbas 2.19143576826 3.67 GiB 2.67 GiB 0.0 Byte / 3.50 GiB 0 / 0 7.1.1-3025-enterprise
      172.23.108.102 kv 1.91194968553 3.67 GiB 3.05 GiB 72.00 MiB / 3.50 GiB 0 / 0 7.1.1-3025-enterprise
      172.23.108.100 kv, n1ql 2.16243399547 3.67 GiB 2.94 GiB 3.75 MiB / 3.50 GiB 0 / 0 7.1.1-3025-enterprise

       Steps to reproduce -
      1. Create cluster as mentioned above.
      2. Create a magma bucket with 512 MB RAM (since there are 2 KV nodes, total RAM for bucket becomes 1 GB) allocated to it and replica set as 1.
      3. Create 10 scopes (including the default) and each scope should have 25 collections.
      4. Load 5300 docs in each collection. Each doc size is 1 KB.
      5. Create dataset on each of the collection of the KV bucket created above.
      6. It is observed that the RAM usage of the bucket is exceeding the RAM allocation.

      RAM Quota :

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            umang.agrawal Umang created issue -
            Balakumaran.Gopal Balakumaran Gopal made changes -
            Field Original Value New Value
            Description Observation -
            Although only 1 GB RAM is allocated to the magma bucket, the RAM usage is reaching around 3.9 GB. This is observed when analytics is trying to ingest data into it's datasets.

            Cluster Info -
            |Node|Services|CPU_utilization|Mem_total|Mem_free|Swap_mem_used|Active / Replica|Version|
            |172.23.108.0|cbas|2.46478873239|3.67 GiB|2.50 GiB|0.0 Byte / 3.50 GiB|0 / 0|7.1.1-3025-enterprise|
            |172.23.108.1|cbas|2.19143576826|3.67 GiB|2.67 GiB|0.0 Byte / 3.50 GiB|0 / 0|7.1.1-3025-enterprise|
            |172.23.108.102|kv|1.91194968553|3.67 GiB|3.05 GiB|72.00 MiB / 3.50 GiB|0 / 0|7.1.1-3025-enterprise|
            |172.23.108.100|kv, n1ql|2.16243399547|3.67 GiB|2.94 GiB|3.75 MiB / 3.50 GiB|0 / 0|7.1.1-3025-enterprise|

             Steps to reproduce -
            1. Create cluster as mentioned above.
            2. Create a magma bucket with 512 MB RAM (since there are 2 KV nodes, total RAM for bucket becomes 1 GB) allocated to it and replica set as 1.
            3. Create 10 scopes (including the default) and each scope should have 25 collections.
            4. Load 5300 docs in each collection. Each doc size is 1 KB.
            5. Create dataset on each of the collection of the KV bucket created above.
            6. It is observed that the RAM usage of the bucket is exceeding the RAM allocation.
            Observation -
            Although only 1 GB RAM is allocated to the magma bucket, the RAM usage is reaching around 3.9 GB. This is observed when analytics is trying to ingest data into it's datasets.

            Cluster Info -
            |Node|Services|CPU_utilization|Mem_total|Mem_free|Swap_mem_used|Active / Replica|Version|
            |172.23.108.0|cbas|2.46478873239|3.67 GiB|2.50 GiB|0.0 Byte / 3.50 GiB|0 / 0|7.1.1-3025-enterprise|
            |172.23.108.1|cbas|2.19143576826|3.67 GiB|2.67 GiB|0.0 Byte / 3.50 GiB|0 / 0|7.1.1-3025-enterprise|
            |172.23.108.102|kv|1.91194968553|3.67 GiB|3.05 GiB|72.00 MiB / 3.50 GiB|0 / 0|7.1.1-3025-enterprise|
            |172.23.108.100|kv, n1ql|2.16243399547|3.67 GiB|2.94 GiB|3.75 MiB / 3.50 GiB|0 / 0|7.1.1-3025-enterprise|

             Steps to reproduce -
            1. Create cluster as mentioned above.
            2. Create a magma bucket with 512 MB RAM (since there are 2 KV nodes, total RAM for bucket becomes 1 GB) allocated to it and replica set as 1.
            3. Create 10 scopes (including the default) and each scope should have 25 collections.
            4. Load 5300 docs in each collection. Each doc size is 1 KB.
            5. Create dataset on each of the collection of the KV bucket created above.
            6. It is observed that the RAM usage of the bucket is exceeding the RAM allocation.

            RAM Quota : !Screenshot 2022-04-27 at 2.27.49 PM.png|thumbnail!
            Balakumaran.Gopal Balakumaran Gopal made changes -
            Component/s analytics [ 12717 ]
            umang.agrawal Umang made changes -
            till Till Westmann made changes -
            Assignee Till Westmann [ till ] Sarath Lakshman [ sarath ]
            rohan.suri Rohan Suri made changes -
            Attachment node31_memused.png [ 182544 ]
            rohan.suri Rohan Suri made changes -
            Attachment node31_memused.gif [ 182547 ]
            rohan.suri Rohan Suri made changes -
            Attachment node31_memused.png [ 182544 ]
            sarath Sarath Lakshman made changes -
            Component/s couchbase-bucket [ 10173 ]
            Component/s storage-engine [ 10175 ]
            sarath Sarath Lakshman made changes -
            Assignee Sarath Lakshman [ sarath ] Daniel Owen [ owend ]
            owend Daniel Owen made changes -
            Assignee Daniel Owen [ owend ] Ben Huddleston [ ben.huddleston ]
            ben.huddleston Ben Huddleston made changes -
            Link This issue relates to MB-51968 [ MB-51968 ]
            ben.huddleston Ben Huddleston made changes -
            Component/s analytics [ 12717 ]
            Component/s couchbase-bucket [ 10173 ]
            ben.huddleston Ben Huddleston made changes -
            Assignee Ben Huddleston [ ben.huddleston ] Till Westmann [ till ]
            ben.huddleston Ben Huddleston made changes -
            Component/s couchbase-bucket [ 10173 ]
            Component/s analytics [ 12717 ]
            ben.huddleston Ben Huddleston made changes -
            Assignee Till Westmann [ till ] Ben Huddleston [ ben.huddleston ]
            michael.blow Michael Blow made changes -
            Link This issue relates to MB-45591 [ MB-45591 ]
            ben.huddleston Ben Huddleston made changes -
            Assignee Ben Huddleston [ ben.huddleston ] Umang [ JIRAUSER24787 ]
            lynn.straus Lynn Straus made changes -
            Fix Version/s 7.1.2 [ 18414 ]
            Fix Version/s 7.1.1 [ 18320 ]
            paolo.cocchi Paolo Cocchi made changes -
            Link This issue relates to MB-52490 [ MB-52490 ]
            umang.agrawal Umang made changes -
            Assignee Umang [ JIRAUSER24787 ] Daniel Owen [ owend ]
            owend Daniel Owen made changes -
            Resolution Fixed [ 1 ]
            Status Open [ 1 ] Closed [ 6 ]

            People

              owend Daniel Owen
              umang.agrawal Umang
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty