Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-22002

Metadata store that is immediately consistent

    XMLWordPrintable

Details

    • Improvement
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 5.0.0
    • 7.0.0
    • ns_server
    • Security Level: Public
    • Cluster Setup:
      2 nodes

      Node 1: kv index n1ql
      Node 2: index

    Description

      We need a metadata store that is immediately consistent.

      The metadata store should begin with nodes, buckets and collections and should eventually include things such as GSI indexes. An example use case for indexing that fails today that this would fix would be as follows:


      [GSI] Indexes can be created with duplicate index_names when one of the indexer node is in failed over state

      Steps:
      1. Configure cluster. Create bucket default and load the bucket with 10k items.
      Sample document:

      {
        "name": "pymc0",
        "age": 0,
        "index": 0,
        "body": "VTKGNKUHMP"
      }
      

      2. Create index ind_3 on default(name). It gets created on node 2.
      3. Failover Node 2.
      4. Create index ind_3 on default(age). It gets created on node 1.
      5. Rebalance in Node 2 with full recovery.

      Now there are 2 indexes with same name "ind_3" with different index definitions.
      when dropping ind_3, later one gets dropped. There is no way of controlling drop behaviour in such cases.

      Expectation is, indexes with duplicate index_names shouldn't be allowed to be created.

      Snapshot attached.
      Log Location:

      https://s3.amazonaws.com/bugdb/jira/duplicate_indexes/collectinfo-2016-12-19T082442-ns_1%4010.111.170.101.zip
      https://s3.amazonaws.com/bugdb/jira/duplicate_indexes/collectinfo-2016-12-19T082442-ns_1%4010.111.170.102.zip
      

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            prasanna.gholap Prasanna Gholap [X] (Inactive) created issue -
            siri Sriram Melkote (Inactive) made changes -
            Field Original Value New Value
            Fix Version/s Spock [ 12927 ]
            siri Sriram Melkote (Inactive) made changes -
            Priority Major [ 3 ] Critical [ 2 ]
            siri Sriram Melkote (Inactive) made changes -
            Assignee Sriram Melkote [ siri ] Dave Finlay [ dfinlay ]
            siri Sriram Melkote (Inactive) made changes -
            Description Steps:
            1. Configure cluster. Create bucket default and load the bucket with 10k items.
            Sample document:
            {noformat}
            {
              "name": "pymc0",
              "age": 0,
              "index": 0,
              "body": "VTKGNKUHMP"
            }
            {noformat}
            2. Create index ind_3 on default(name). It gets created on node 2.
            3. Failover Node 2.
            4. Create index ind_3 on default(age). It gets created on node 1.
            5. Rebalance in Node 2 with full recovery.

            Now there are 2 indexes with same name "ind_3" with different index definitions.
            when dropping ind_3, later one gets dropped. There is no way of controlling drop behaviour in such cases.

            Expectation is, indexes with duplicate index_names shouldn't be allowed to be created.

            Snapshot attached.
            Log Location:
            {noformat}
            https://s3.amazonaws.com/bugdb/jira/duplicate_indexes/collectinfo-2016-12-19T082442-ns_1%4010.111.170.101.zip
            https://s3.amazonaws.com/bugdb/jira/duplicate_indexes/collectinfo-2016-12-19T082442-ns_1%4010.111.170.102.zip
            {noformat}
            We need a metadata store that has better ACID properties.

            -----

            Original Problem:
            [GSI] Indexes can be created with duplicate index_names when one of the indexer node is in failed over state

            Steps:
            1. Configure cluster. Create bucket default and load the bucket with 10k items.
            Sample document:
            {noformat}
            {
              "name": "pymc0",
              "age": 0,
              "index": 0,
              "body": "VTKGNKUHMP"
            }
            {noformat}
            2. Create index ind_3 on default(name). It gets created on node 2.
            3. Failover Node 2.
            4. Create index ind_3 on default(age). It gets created on node 1.
            5. Rebalance in Node 2 with full recovery.

            Now there are 2 indexes with same name "ind_3" with different index definitions.
            when dropping ind_3, later one gets dropped. There is no way of controlling drop behaviour in such cases.

            Expectation is, indexes with duplicate index_names shouldn't be allowed to be created.

            Snapshot attached.
            Log Location:
            {noformat}
            https://s3.amazonaws.com/bugdb/jira/duplicate_indexes/collectinfo-2016-12-19T082442-ns_1%4010.111.170.101.zip
            https://s3.amazonaws.com/bugdb/jira/duplicate_indexes/collectinfo-2016-12-19T082442-ns_1%4010.111.170.102.zip
            {noformat}
            siri Sriram Melkote (Inactive) made changes -
            Summary [GSI] Indexes can be created with duplicate index_names when one of the indexer node is in failed over state Metadata store that has better ACID properties.
            siri Sriram Melkote (Inactive) made changes -
            Component/s secondary-index [ 11211 ]
            Component/s ns_server [ 10019 ]
            siri Sriram Melkote (Inactive) made changes -
            Fix Version/s Spock [ 12927 ]
            raju Raju Suravarjjala made changes -
            Fix Version/s Spock.Next [ 13624 ]
            siri Sriram Melkote (Inactive) made changes -
            Link This issue blocks MB-24011 [ MB-24011 ]
            siri Sriram Melkote (Inactive) made changes -
            Security Private [ 10010 ]
            siri Sriram Melkote (Inactive) made changes -
            Summary Metadata store that has better ACID properties. Metadata store that is immediately consistent
            siri Sriram Melkote (Inactive) made changes -
            Description We need a metadata store that has better ACID properties.

            -----

            Original Problem:
            [GSI] Indexes can be created with duplicate index_names when one of the indexer node is in failed over state

            Steps:
            1. Configure cluster. Create bucket default and load the bucket with 10k items.
            Sample document:
            {noformat}
            {
              "name": "pymc0",
              "age": 0,
              "index": 0,
              "body": "VTKGNKUHMP"
            }
            {noformat}
            2. Create index ind_3 on default(name). It gets created on node 2.
            3. Failover Node 2.
            4. Create index ind_3 on default(age). It gets created on node 1.
            5. Rebalance in Node 2 with full recovery.

            Now there are 2 indexes with same name "ind_3" with different index definitions.
            when dropping ind_3, later one gets dropped. There is no way of controlling drop behaviour in such cases.

            Expectation is, indexes with duplicate index_names shouldn't be allowed to be created.

            Snapshot attached.
            Log Location:
            {noformat}
            https://s3.amazonaws.com/bugdb/jira/duplicate_indexes/collectinfo-2016-12-19T082442-ns_1%4010.111.170.101.zip
            https://s3.amazonaws.com/bugdb/jira/duplicate_indexes/collectinfo-2016-12-19T082442-ns_1%4010.111.170.102.zip
            {noformat}
            We need a metadata store that is immediately consistent.

            -----

            Original Problem:
            [GSI] Indexes can be created with duplicate index_names when one of the indexer node is in failed over state

            Steps:
            1. Configure cluster. Create bucket default and load the bucket with 10k items.
            Sample document:
            {noformat}
            {
              "name": "pymc0",
              "age": 0,
              "index": 0,
              "body": "VTKGNKUHMP"
            }
            {noformat}
            2. Create index ind_3 on default(name). It gets created on node 2.
            3. Failover Node 2.
            4. Create index ind_3 on default(age). It gets created on node 1.
            5. Rebalance in Node 2 with full recovery.

            Now there are 2 indexes with same name "ind_3" with different index definitions.
            when dropping ind_3, later one gets dropped. There is no way of controlling drop behaviour in such cases.

            Expectation is, indexes with duplicate index_names shouldn't be allowed to be created.

            Snapshot attached.
            Log Location:
            {noformat}
            https://s3.amazonaws.com/bugdb/jira/duplicate_indexes/collectinfo-2016-12-19T082442-ns_1%4010.111.170.101.zip
            https://s3.amazonaws.com/bugdb/jira/duplicate_indexes/collectinfo-2016-12-19T082442-ns_1%4010.111.170.102.zip
            {noformat}
            lynn.straus Lynn Straus made changes -
            Fix Version/s vulcan [ 14610 ]
            dfinlay Dave Finlay made changes -
            Issue Type Bug [ 1 ] Improvement [ 4 ]
            dfinlay Dave Finlay made changes -
            Fix Version/s Mad-Hatter [ 15037 ]
            Fix Version/s Spock.Next [ 13624 ]
            Fix Version/s vulcan [ 14610 ]
            siri Sriram Melkote (Inactive) made changes -
            Link This issue relates to MB-31074 [ MB-31074 ]
            prathibha Prathibha Bisarahalli (Inactive) made changes -
            Link This issue blocks CBSE-5852 [ CBSE-5852 ]
            dfinlay Dave Finlay made changes -
            Fix Version/s Cheshire-Cat [ 15915 ]
            Fix Version/s Mad-Hatter [ 15037 ]
            dfinlay Dave Finlay made changes -
            Security Private [ 10010 ] Public [ 10011 ]
            ajit.yagaty Ajit Yagaty [X] (Inactive) made changes -
            Link This issue relates to CBSE-6979 [ CBSE-6979 ]
            avleen.khanuja Avleen Singh Khanuja (Inactive) made changes -
            Link This issue blocks CBSE-8567 [ CBSE-8567 ]
            dfinlay Dave Finlay made changes -
            Labels functional-test Cheshire-Cat-Committed functional-test
            nirvair.singh Nirvair Singh Bhinder made changes -
            Link This issue relates to CBSE-8879 [ CBSE-8879 ]
            dfinlay Dave Finlay made changes -
            Assignee Dave Finlay [ dfinlay ] Aliaksey Artamonau [ aliaksey artamonau ]
            dfinlay Dave Finlay made changes -
            Link This issue blocks MB-27384 [ MB-27384 ]
            simon.dew Simon Dew made changes -
            Remote Link This issue links to "Page (Couchbase, Inc. Wiki)" [ 20706 ]
            dfinlay Dave Finlay made changes -
            Resolution Fixed [ 1 ]
            Status Open [ 1 ] Resolved [ 5 ]
            Balakumaran.Gopal Balakumaran Gopal made changes -
            Assignee Aliaksey Artamonau [ aliaksey artamonau ] Mihir Kamdar [ mihir.kamdar ]
            mihir.kamdar Mihir Kamdar (Inactive) made changes -
            Assignee Mihir Kamdar [ mihir.kamdar ] Balakumaran Gopal [ balakumaran.gopal ]
            Balakumaran.Gopal Balakumaran Gopal made changes -
            Assignee Balakumaran Gopal [ balakumaran.gopal ] Mihir Kamdar [ mihir.kamdar ]
            mihir.kamdar Mihir Kamdar (Inactive) made changes -
            Assignee Mihir Kamdar [ mihir.kamdar ] Hemant Rajput [ hemant.rajput ]
            hemant.rajput Hemant Rajput made changes -
            Resolution Fixed [ 1 ]
            Status Resolved [ 5 ] Reopened [ 4 ]
            hemant.rajput Hemant Rajput made changes -
            Attachment node2-cb600-centos7.vagrants.zip [ 137965 ]
            Attachment node3-cb600-centos7.vagrants.zip [ 137966 ]
            hemant.rajput Hemant Rajput made changes -
            Attachment node1-cb600-centos7.vagrants.zip [ 137967 ]
            hemant.rajput Hemant Rajput made changes -
            dfinlay Dave Finlay made changes -
            Link This issue is cloned by MB-46007 [ MB-46007 ]
            dfinlay Dave Finlay made changes -
            Description We need a metadata store that is immediately consistent.

            -----

            Original Problem:
            [GSI] Indexes can be created with duplicate index_names when one of the indexer node is in failed over state

            Steps:
            1. Configure cluster. Create bucket default and load the bucket with 10k items.
            Sample document:
            {noformat}
            {
              "name": "pymc0",
              "age": 0,
              "index": 0,
              "body": "VTKGNKUHMP"
            }
            {noformat}
            2. Create index ind_3 on default(name). It gets created on node 2.
            3. Failover Node 2.
            4. Create index ind_3 on default(age). It gets created on node 1.
            5. Rebalance in Node 2 with full recovery.

            Now there are 2 indexes with same name "ind_3" with different index definitions.
            when dropping ind_3, later one gets dropped. There is no way of controlling drop behaviour in such cases.

            Expectation is, indexes with duplicate index_names shouldn't be allowed to be created.

            Snapshot attached.
            Log Location:
            {noformat}
            https://s3.amazonaws.com/bugdb/jira/duplicate_indexes/collectinfo-2016-12-19T082442-ns_1%4010.111.170.101.zip
            https://s3.amazonaws.com/bugdb/jira/duplicate_indexes/collectinfo-2016-12-19T082442-ns_1%4010.111.170.102.zip
            {noformat}
            We need a metadata store that is immediately consistent.

            The metadata store should begin with nodes, buckets and collections and should eventually include things such as GSI indexes. An example use case for indexing that fails today that this would fix would be as follows:

            -----
            [GSI] Indexes can be created with duplicate index_names when one of the indexer node is in failed over state

            Steps:
            1. Configure cluster. Create bucket default and load the bucket with 10k items.
            Sample document:
            {noformat}
            {
              "name": "pymc0",
              "age": 0,
              "index": 0,
              "body": "VTKGNKUHMP"
            }
            {noformat}
            2. Create index ind_3 on default(name). It gets created on node 2.
            3. Failover Node 2.
            4. Create index ind_3 on default(age). It gets created on node 1.
            5. Rebalance in Node 2 with full recovery.

            Now there are 2 indexes with same name "ind_3" with different index definitions.
            when dropping ind_3, later one gets dropped. There is no way of controlling drop behaviour in such cases.

            Expectation is, indexes with duplicate index_names shouldn't be allowed to be created.

            Snapshot attached.
            Log Location:
            {noformat}
            https://s3.amazonaws.com/bugdb/jira/duplicate_indexes/collectinfo-2016-12-19T082442-ns_1%4010.111.170.101.zip
            https://s3.amazonaws.com/bugdb/jira/duplicate_indexes/collectinfo-2016-12-19T082442-ns_1%4010.111.170.102.zip
            {noformat}
            dfinlay Dave Finlay made changes -
            Resolution Fixed [ 1 ]
            Status Reopened [ 4 ] Resolved [ 5 ]
            mihir.kamdar Mihir Kamdar (Inactive) made changes -
            Assignee Hemant Rajput [ hemant.rajput ] Balakumaran Gopal [ balakumaran.gopal ]
            Balakumaran.Gopal Balakumaran Gopal made changes -
            Status Resolved [ 5 ] Closed [ 6 ]
            lynn.straus Lynn Straus made changes -
            Fix Version/s 7.0.0 [ 17233 ]
            lynn.straus Lynn Straus made changes -
            Fix Version/s Cheshire-Cat [ 15915 ]

            People

              Balakumaran.Gopal Balakumaran Gopal
              prasanna.gholap Prasanna Gholap [X] (Inactive)
              Votes:
              1 Vote for this issue
              Watchers:
              25 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty