Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-6423

Allow users to set index_path post upgrade to 2.0

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0-beta
    • Fix Version/s: 2.0
    • Component/s: ns_server
    • Security Level: Public
    • Labels:
    • Environment:
      1639

      Description

      Use case:-
      1) Users upgrade to 20 from 181
      2) They decide to separate out partition for indexes. Currently there is no way to configure just index_path without wiping out data_path or reconfiguring the cluster.
      Currently, issuing the data_path command will wipe out the database files.

      One can configure the data path
      curl -d path=/var/tmp/test http://localhost:8091/nodes/self/controller/settings

      This does not seem to work...
      curl -d index_path=/data2 http://localhost:8091/nodes/self/controller/settings

      No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

        karan Karan Kumar (Inactive) created issue -
        Hide
        karan Karan Kumar (Inactive) added a comment -

        I guess right now, we would need separate rest api to configure just indexes.
        Not a bug, but a feature i guess.

        Following seems to work fine.
        curl -d "path=/data&index_path=/indexes" http://localhost:8091/nodes/self/controller/settings

        Show
        karan Karan Kumar (Inactive) added a comment - I guess right now, we would need separate rest api to configure just indexes. Not a bug, but a feature i guess. Following seems to work fine. curl -d "path=/data&index_path=/indexes" http://localhost:8091/nodes/self/controller/settings
        karan Karan Kumar (Inactive) made changes -
        Field Original Value New Value
        Summary Unable to configure index_path using REST No separate api to just configure index_path using REST
        Hide
        farshid Farshid Ghods (Inactive) added a comment -

        yes for our testing we want to try and put each index on a seperate disk. not sure if there is an api that allows us to do so

        Show
        farshid Farshid Ghods (Inactive) added a comment - yes for our testing we want to try and put each index on a seperate disk. not sure if there is an api that allows us to do so
        Hide
        alkondratenko Aleksey Kondratenko (Inactive) added a comment -

        We don't plan to have such API. You can manually replace files with symlinks. Another option is raid 0

        Show
        alkondratenko Aleksey Kondratenko (Inactive) added a comment - We don't plan to have such API. You can manually replace files with symlinks. Another option is raid 0
        Hide
        karan Karan Kumar (Inactive) added a comment -

        From capacity planning point of view,

        if customer upgrade to 20 and later plan to create views,

        by default, views would get created on the same partition as the database files.

        There is no separate rest_api to setup path for just indexes.

        Show
        karan Karan Kumar (Inactive) added a comment - From capacity planning point of view, if customer upgrade to 20 and later plan to create views, by default, views would get created on the same partition as the database files. There is no separate rest_api to setup path for just indexes.
        karan Karan Kumar (Inactive) made changes -
        Assignee Aleksey Kondratenko [ alkondratenko ] Dipti Borkar [ dipti ]
        Hide
        alkondratenko Aleksey Kondratenko (Inactive) added a comment -

        I see no point at all having separate API for setting up just index path.

        We do have API to setup separate index path.

        The only thing we dont have is api to set up per-index path.

        Show
        alkondratenko Aleksey Kondratenko (Inactive) added a comment - I see no point at all having separate API for setting up just index path. We do have API to setup separate index path. The only thing we dont have is api to set up per-index path.
        Hide
        alkondratenko Aleksey Kondratenko (Inactive) added a comment -

        And my performance recommendation would be:

        instead of have disk spindle per index, just raid 0 all spindles you have. That should be as good as or even better than spindle per index.

        Show
        alkondratenko Aleksey Kondratenko (Inactive) added a comment - And my performance recommendation would be: instead of have disk spindle per index, just raid 0 all spindles you have. That should be as good as or even better than spindle per index.
        Hide
        karan Karan Kumar (Inactive) added a comment -

        Sure. I can create a separate ticket to test out raid 0 and see if it provides better performance.

        Show
        karan Karan Kumar (Inactive) added a comment - Sure. I can create a separate ticket to test out raid 0 and see if it provides better performance.
        Hide
        karan Karan Kumar (Inactive) added a comment -

        Will try to convey this in a better way:-

        I can configure the data_path as following
        curl -d path=/data http://localhost:8091/nodes/self/controller/settings

        BUT, I cant configure the index_path as following (not per index, but for all indexes)
        curl -d index_path=/var/tmp/test http://localhost:8091/nodes/self/controller/settings

        curl -vvv -d "index_path=/data" http://Administrator:password@localhost:8091/nodes/self/controller/settings

        • About to connect() to localhost port 8091 (#0)
        • Trying ::1... Connection refused
        • Trying 127.0.0.1... connected
        • Connected to localhost (127.0.0.1) port 8091 (#0)
        • Server auth using Basic with user 'Administrator'
          > POST /nodes/self/controller/settings HTTP/1.1
          > Authorization: Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==
          > User-Agent: curl/7.19.7 (x86_64-unknown-linux-gnu) libcurl/7.19.7 NSS/3.12.7.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2
          > Host: localhost:8091
          > Accept: /
          > Content-Length: 16
          > Content-Type: application/x-www-form-urlencoded
          >
          < HTTP/1.1 400 Bad Request
          < Server: Couchbase Server 2.0.0-1695-rel-enterprise
          < Pragma: no-cache
          < Date: Fri, 07 Sep 2012 19:10:36 GMT
          < Content-Type: application/json
          < Content-Length: 47
          < Cache-Control: no-cache
          <
        • Connection #0 to host localhost left intact
        • Closing connection #0
          ["The database or index path cannot be empty."]

        The only way for me to configure index_path is by specfying both index_path and data_path
        as following:-
        curl -d "path=/data&index_path=/indexes" http://localhost:8091/nodes/self/controller/settings

        The above command nukes the bucket and looses deletes all the data.

        Show
        karan Karan Kumar (Inactive) added a comment - Will try to convey this in a better way:- I can configure the data_path as following curl -d path=/data http://localhost:8091/nodes/self/controller/settings BUT, I cant configure the index_path as following (not per index, but for all indexes) curl -d index_path=/var/tmp/test http://localhost:8091/nodes/self/controller/settings curl -vvv -d "index_path=/data" http://Administrator:password@localhost:8091/nodes/self/controller/settings About to connect() to localhost port 8091 (#0) Trying ::1... Connection refused Trying 127.0.0.1... connected Connected to localhost (127.0.0.1) port 8091 (#0) Server auth using Basic with user 'Administrator' > POST /nodes/self/controller/settings HTTP/1.1 > Authorization: Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA== > User-Agent: curl/7.19.7 (x86_64-unknown-linux-gnu) libcurl/7.19.7 NSS/3.12.7.0 zlib/1.2.3 libidn/1.18 libssh2/1.2.2 > Host: localhost:8091 > Accept: / > Content-Length: 16 > Content-Type: application/x-www-form-urlencoded > < HTTP/1.1 400 Bad Request < Server: Couchbase Server 2.0.0-1695-rel-enterprise < Pragma: no-cache < Date: Fri, 07 Sep 2012 19:10:36 GMT < Content-Type: application/json < Content-Length: 47 < Cache-Control: no-cache < Connection #0 to host localhost left intact Closing connection #0 ["The database or index path cannot be empty."] The only way for me to configure index_path is by specfying both index_path and data_path as following:- curl -d "path=/data&index_path=/indexes" http://localhost:8091/nodes/self/controller/settings The above command nukes the bucket and looses deletes all the data.
        Hide
        karan Karan Kumar (Inactive) added a comment -

        Well here, I am not trying to point out a performance issue. I am trying to point out a functional bug.

        I will take this offline with Alk

        Show
        karan Karan Kumar (Inactive) added a comment - Well here, I am not trying to point out a performance issue. I am trying to point out a functional bug. I will take this offline with Alk
        karan Karan Kumar (Inactive) made changes -
        Labels system-test
        karan Karan Kumar (Inactive) made changes -
        Summary No separate api to just configure index_path using REST Allow users to set index_path post upgrade to 2.0
        Assignee Dipti Borkar [ dipti ] Aleksey Kondratenko [ alkondratenko ]
        Description One can configure the data path
        curl -d path=/var/tmp/test http://localhost:8091/nodes/self/controller/settings

        This does not seem to work...
        curl -d index_path=/data2 http://localhost:8091/nodes/self/controller/settings
        Use case:-
        1) Users upgrade to 20 from 181
        2) They decide to separate out partition for indexes. Currently there is no way to configure just index_path without wiping out data_path or reconfiguring the cluster.
         Currently, issuing the data_path command will wipe out the database files.



        One can configure the data path
        curl -d path=/var/tmp/test http://localhost:8091/nodes/self/controller/settings

        This does not seem to work...
        curl -d index_path=/data2 http://localhost:8091/nodes/self/controller/settings
        Hide
        farshid Farshid Ghods (Inactive) added a comment -

        Karan,
        Any updates ?

        do you need to discuss this with Dipti ?

        Show
        farshid Farshid Ghods (Inactive) added a comment - Karan, Any updates ? do you need to discuss this with Dipti ?
        farshid Farshid Ghods (Inactive) made changes -
        Assignee Aleksey Kondratenko [ alkondratenko ] Karan Kumar [ karan ]
        karan Karan Kumar (Inactive) made changes -
        Assignee Karan Kumar [ karan ] Aleksey Kondratenko [ alkondratenko ]
        Hide
        alkondratenko Aleksey Kondratenko (Inactive) added a comment -

        Looks like simply implementing new API or extending old in some way is the way to go here.

        Show
        alkondratenko Aleksey Kondratenko (Inactive) added a comment - Looks like simply implementing new API or extending old in some way is the way to go here.
        alkondratenko Aleksey Kondratenko (Inactive) made changes -
        Assignee Aleksey Kondratenko [ alkondratenko ] Aliaksey Artamonau [ aliaksey artamonau ]
        Show
        farshid Farshid Ghods (Inactive) added a comment - http://review.membase.org/#/c/21425/
        farshid Farshid Ghods (Inactive) made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        andreibaranouski Andrei Baranouski added a comment -

        build 1820

        verification steps:
        1. 1 node with default bucket and 1 item in it
        2. create 1 view
        3. change index path
        curl --user Administrator:password -d index_path=/tmp/bbbbb/ http://localhost:8091/nodes/self/controller/settings
        observation: view works after indexing, data index location was changed
        4. change index path in second time
        curl --user Administrator:password -d index_path=/tmp/aaaa/ http://localhost:8091/nodes/self/controller/settings
        observation: view works after indexing, data index location was changed on new
        5. change data path
        curl --user Administrator:password -d path=/tmp/data_folder/ http://localhost:8091/nodes/self/controller/settings

        result: view is left on UI for some time but with error on request

        {"error":"not_found","reason":"Design document _design/dev_a not found"}

        after a couple minutes ddoc/view is deleted from ui ( browser cash was expired)

        Show
        andreibaranouski Andrei Baranouski added a comment - build 1820 verification steps: 1. 1 node with default bucket and 1 item in it 2. create 1 view 3. change index path curl --user Administrator:password -d index_path=/tmp/bbbbb/ http://localhost:8091/nodes/self/controller/settings observation: view works after indexing, data index location was changed 4. change index path in second time curl --user Administrator:password -d index_path=/tmp/aaaa/ http://localhost:8091/nodes/self/controller/settings observation: view works after indexing, data index location was changed on new 5. change data path curl --user Administrator:password -d path=/tmp/data_folder/ http://localhost:8091/nodes/self/controller/settings result: view is left on UI for some time but with error on request {"error":"not_found","reason":"Design document _design/dev_a not found"} after a couple minutes ddoc/view is deleted from ui ( browser cash was expired)
        Hide
        alkondratenko Aleksey Kondratenko (Inactive) added a comment -

        Andrei so ?

        BTW changing data path after the fact is a bad idea. We don't necessarily handle it well.

        Show
        alkondratenko Aleksey Kondratenko (Inactive) added a comment - Andrei so ? BTW changing data path after the fact is a bad idea. We don't necessarily handle it well.
        Hide
        kzeller kzeller added a comment -

        In RN: This enables users to change the disk location of an index without destroying
        persisted data. You can now set index_path and it will delete an existing index only
        and create a new disk location for use.

        Show
        kzeller kzeller added a comment - In RN: This enables users to change the disk location of an index without destroying persisted data. You can now set index_path and it will delete an existing index only and create a new disk location for use.
        farshid Farshid Ghods (Inactive) made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Aliaksey Artamonau Aliaksey Artamonau
            Reporter:
            karan Karan Kumar (Inactive)
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Gerrit Reviews

              There are no open Gerrit changes