Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-27425

[Plasma] Indexing crashes on Array indexing of large dataset

    XMLWordPrintable

Details

    • Untriaged
    • Centos 64-bit
    • No
    • CBSS Sprint Feb-16-2018

    Description

       Sample doc: 

      {

        "_sync": {

          "rev": "1-6818bd58742046269be08ffdf0268bb2",

          "role_access": {

             "sg-user-6": {

               "sg-role-0": 142613,

               "sg-role-1": 142613,

               "sg-role-7": 142613,

               "sg-role-9": 142613

       …

      The are 1M such documents in the bucket. The "role_access" field has 1000 elements average.

        

      Index definition:  

      CREATE INDEX `ixRoleAccess` ON `bucket-1`((distinct (array (`op`.`name`) for `op` in object_pairs(_sync.`role_access`) end)),object_pairs(_sync.`role_access`));

       

       

      Index node has 64 GB RAM total.

      35 GB RAM has been allocated to the index.

      Once I run create index query the indexer quickly uses all 64 GB RAM, runs out of memory  and indexer restarts.

      On the time the crash happens the index actual size is only 5GB and its about 40% done.

       

      When using MOI everything works as expected. Total size of the index when completed is about 20 GB.

      It works fine also with lesser arrays (1, 10, 100 elements)

      Attachments

        Issue Links

          For Gerrit Dashboard: MB-27425
          # Subject Branch Project Status CR V

          Activity

            People

              oleksandr.gyryk Alex Gyryk (Inactive)
              oleksandr.gyryk Alex Gyryk (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty