Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-15784

Deferred index building not idempotent

    XMLWordPrintable

Details

    Description

      I've got some demo scripts that I'm using to create a bunch of deferred indexes and then build them. Running this on a clean system works well, but when I was diagnosing issues and running the script repeatedly, I found that if any indexes already exist in the build index statement, the others are not built.

      i.e.:
      CREATE INDEX def_sourceairport ON `travel-sample`(sourceairport) USING GSI WITH

      {"defer_build":true}

      CREATE INDEX def_name_type ON `travel-sample`(name) WHERE (_type = "User") USING GSI WITH

      {"defer_build":true}

      BUILD INDEX ON `travel-sample`(`def_sourceairport`, `def_name`) USING GSI"

      then
      DROP INDEX `travel-sample`.def_name

      then the same:
      CREATE INDEX def_sourceairport ON `travel-sample`(sourceairport) USING GSI WITH

      {"defer_build":true}

      CREATE INDEX def_name_type ON `travel-sample`(name) WHERE (_type = "User") USING GSI WITH

      {"defer_build":true}

      (fails understandably)
      BUILD INDEX ON `travel-sample`(`def_sourceairport`, `def_name`) USING GSI"
      -this fails saying that def_name is already built, but def_sourceairport does not get built either-

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            perry Perry Krug created issue -

            Perry, GSI commands are not idempotent in general. We can consider making them so, but I'll mark this as an enhancement for this reason.

            siri Sriram Melkote (Inactive) added a comment - Perry, GSI commands are not idempotent in general. We can consider making them so, but I'll mark this as an enhancement for this reason.
            siri Sriram Melkote (Inactive) made changes -
            Field Original Value New Value
            Issue Type Bug [ 1 ] Improvement [ 4 ]
            perry Perry Krug added a comment -

            Thanks Siri, perhaps idempotent is a much bigger ask. The main issue is to be able to have "build index" work even if there are some of those indexes built already. Is that easier to fix?

            perry Perry Krug added a comment - Thanks Siri, perhaps idempotent is a much bigger ask. The main issue is to be able to have "build index" work even if there are some of those indexes built already. Is that easier to fix?
            raju Raju Suravarjjala made changes -
            Fix Version/s watson [ 12540 ]

            I agree that we can queue index build requests and fire them off one at a time instead of returning errors. however this seem expensive so we need to push this to watson.next.

            cihan Cihan Biyikoglu (Inactive) added a comment - I agree that we can queue index build requests and fire them off one at a time instead of returning errors. however this seem expensive so we need to push this to watson.next.
            cihan Cihan Biyikoglu (Inactive) made changes -
            Fix Version/s watson [ 12540 ]
            cihan Cihan Biyikoglu (Inactive) made changes -
            Fix Version/s watson.next [ 12927 ]
            siri Sriram Melkote (Inactive) made changes -
            Fix Version/s feature-backlog [ 10342 ]
            Fix Version/s watson.next [ 12927 ]
            siri Sriram Melkote (Inactive) made changes -
            Assignee Sriram Melkote [ siri ]

            Need to understand this in the context of supporting ALTER INDEX as well.

            venkatraman.subramanian Venkatraman Subramanian (Inactive) added a comment - Need to understand this in the context of supporting ALTER INDEX as well.
            venkatraman.subramanian Venkatraman Subramanian (Inactive) made changes -
            Assignee Deepkaran Salooja [ deepkaran.salooja ]
            venkatraman.subramanian Venkatraman Subramanian (Inactive) made changes -
            Assignee Deepkaran Salooja [ deepkaran.salooja ] Venkatraman Subramanian [ venkatraman.subramanian ]
            venkatraman.subramanian Venkatraman Subramanian (Inactive) made changes -
            Assignee Venkatraman Subramanian [ venkatraman.subramanian ] John Liang [ jliang ]

            John Liang Ideally this should not happen if we are looking at sequencing the index DDLs.

            venkatraman.subramanian Venkatraman Subramanian (Inactive) added a comment - John Liang Ideally this should not happen if we are looking at sequencing the index DDLs.
            deepkaran.salooja Deepkaran Salooja made changes -
            Labels RMS
            jeelan.poola Jeelan Poola made changes -
            Fix Version/s Mad-Hatter [ 15037 ]
            Fix Version/s feature-backlog [ 10342 ]
            jeelan.poola Jeelan Poola made changes -
            Priority Major [ 3 ] Critical [ 2 ]
            jeelan.poola Jeelan Poola made changes -
            Labels RMS RMS mad-hatter-committed

            Build couchbase-server-6.5.0-2250 contains indexing commit 2dbc385 with commit message:
            MB-15784: When executing build index command, skip indexes already built

            build-team Couchbase Build Team added a comment - Build couchbase-server-6.5.0-2250 contains indexing commit 2dbc385 with commit message: MB-15784 : When executing build index command, skip indexes already built
            jliang John Liang made changes -
            Resolution Fixed [ 1 ]
            Status Open [ 1 ] Resolved [ 5 ]
            jliang John Liang made changes -
            Actual End 2019-02-06 11:04 (issue has been resolved)
            ritam.sharma Ritam Sharma made changes -
            Labels RMS mad-hatter-committed RMS mad-hatter-committed needs-qe
            ritam.sharma Ritam Sharma made changes -
            Assignee John Liang [ jliang ] Ramalingam Kalyanasundaram [ r.kalyanasundaram ]
            r.kalyanasundaram Ramalingam Kalyanasundaram [X] (Inactive) made changes -
            VERIFICATION STEPS Reproduced the issue on build #2249 and verified the issue on build #2250
            with following scenarios

            1) Having already built idx and an new index to build in "BUILD INDEX" statement.
            2) Posting "BUILD INDEX" for a single index multiple times.
            Status Resolved [ 5 ] Closed [ 6 ]
            simon.dew Simon Dew made changes -
            Link This issue relates to DOC-6181 [ DOC-6181 ]

            People

              r.kalyanasundaram Ramalingam Kalyanasundaram [X] (Inactive)
              perry Perry Krug
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty