Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-6972

distribute couchbase-server through yum and ubuntu package repositories

    Details

    • Type: Task
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.1.0, 3.0.3, 4.0.0
    • Fix Version/s: 4.0.0
    • Component/s: build
    • Security Level: Public
    • Labels:
    • Flagged:
      Release Note
    • Sprint:
      Aug 23rd Sprint, Build Sprint 4: 3/31 - 4/18, Build Sprint 5: 4/21 - 5/09, Build Sprint 6: 5/12 - 5/30, Build: Infrastructure Backlog

      Description

      this helps us in handling dependencies that are needed for couchbase server
      sdk team has already implemented this for various sdk packages.

      we might have to make some changes to our packaging metadata to work with this schema

        Issue Links

        # Subject Project Status CR V
        For Gerrit Dashboard: &For+MB-6972=message:MB-6972

          Activity

          Hide
          steve Steve Yen added a comment -

          to 2.0.2 per bug-scrub

          first step is do the repositories?

          Show
          steve Steve Yen added a comment - to 2.0.2 per bug-scrub first step is do the repositories?
          Hide
          steve Steve Yen added a comment -

          back to 2.01, per bug-scrub

          Show
          steve Steve Yen added a comment - back to 2.01, per bug-scrub
          Hide
          steve Steve Yen added a comment -

          back to 2.01, per bug-scrub

          Show
          steve Steve Yen added a comment - back to 2.01, per bug-scrub
          Hide
          farshid Farshid Ghods (Inactive) added a comment -

          Phil,
          please sync up with Farshid and get instructions that Sergey and Pavel sent

          Show
          farshid Farshid Ghods (Inactive) added a comment - Phil, please sync up with Farshid and get instructions that Sergey and Pavel sent
          Hide
          farshid Farshid Ghods (Inactive) added a comment -

          we should resolve this task once 2.0.1 is released .

          Show
          farshid Farshid Ghods (Inactive) added a comment - we should resolve this task once 2.0.1 is released .
          Hide
          dipti Dipti Borkar added a comment -

          Have we figured out the upgrade process moving forward. for example from 2.0.1 to 2.0.2 or 2.0.1 to 2.1 ?

          Show
          dipti Dipti Borkar added a comment - Have we figured out the upgrade process moving forward. for example from 2.0.1 to 2.0.2 or 2.0.1 to 2.1 ?
          Hide
          jin Jin Lim (Inactive) added a comment -

          Please ensure that we also confirm/validate the upgrade process moving from 2.0.1 to 2.0.2. Thanks.

          Show
          jin Jin Lim (Inactive) added a comment - Please ensure that we also confirm/validate the upgrade process moving from 2.0.1 to 2.0.2. Thanks.
          Hide
          plabee Phil Labee (Inactive) added a comment -

          Now have DEB repo working, but another issue has come up: We need to distribute the public key so that users can install the key before running apt-get.

          wiki page has been updated.

          Show
          plabee Phil Labee (Inactive) added a comment - Now have DEB repo working, but another issue has come up: We need to distribute the public key so that users can install the key before running apt-get. wiki page has been updated.
          Hide
          kzeller kzeller added a comment -

          Added to 2.0.1 RN as:

          Fix:

          We now provide Couchbase Server as a yum and Debian package
          repositories.

          Show
          kzeller kzeller added a comment - Added to 2.0.1 RN as: Fix: We now provide Couchbase Server as a yum and Debian package repositories.
          Hide
          ingenthr Matt Ingenthron added a comment -

          What are the public URLs for these repositories? This was mentioned in the release notes here:
          http://www.couchbase.com/docs/couchbase-manual-2.0/couchbase-server-rn_2-0-0l.html

          Show
          ingenthr Matt Ingenthron added a comment - What are the public URLs for these repositories? This was mentioned in the release notes here: http://www.couchbase.com/docs/couchbase-manual-2.0/couchbase-server-rn_2-0-0l.html
          Hide
          ingenthr Matt Ingenthron added a comment -

          Reopening, since this isn't documented that I can find. Apologies if I'm just missing it.

          Show
          ingenthr Matt Ingenthron added a comment - Reopening, since this isn't documented that I can find. Apologies if I'm just missing it.
          Hide
          dipti Dipti Borkar added a comment -

          Anil, can you work with Phil to see what are the next steps here?

          Show
          dipti Dipti Borkar added a comment - Anil, can you work with Phil to see what are the next steps here?
          Hide
          anil Anil Kumar added a comment -

          Yes I'll be having discussion with Phil and will update here with details.

          Show
          anil Anil Kumar added a comment - Yes I'll be having discussion with Phil and will update here with details.
          Hide
          tim.ray Tim Ray added a comment -

          could we either remove the note about yum/deb repo's in the release notes or get those repo locations / sample files / keys added to public pages? The only links that seem that they 'might' contain the info point to internal pages I don't have access to.

          Show
          tim.ray Tim Ray added a comment - could we either remove the note about yum/deb repo's in the release notes or get those repo locations / sample files / keys added to public pages? The only links that seem that they 'might' contain the info point to internal pages I don't have access to.
          Hide
          anil Anil Kumar added a comment -

          thanks Tim, we have removed it from release notes. we will add instructions about yum/deb repo's locations/files/keys to documentation once its available. thanks!

          Show
          anil Anil Kumar added a comment - thanks Tim, we have removed it from release notes. we will add instructions about yum/deb repo's locations/files/keys to documentation once its available. thanks!
          Hide
          kzeller kzeller added a comment -
          Show
          kzeller kzeller added a comment - Removing duplicate ticket: http://www.couchbase.com/issues/browse/MB-7860
          Hide
          h0nIg h0nIg added a comment - - edited

          any update? maybe i created a duplicate issue: http://www.couchbase.com/issues/browse/MB-9409 but it seems that the repositories are outdated on http://hub.internal.couchbase.com/confluence/display/CR/How+to+Use+a+Linux+Repo+--+debian

          Show
          h0nIg h0nIg added a comment - - edited any update? maybe i created a duplicate issue: http://www.couchbase.com/issues/browse/MB-9409 but it seems that the repositories are outdated on http://hub.internal.couchbase.com/confluence/display/CR/How+to+Use+a+Linux+Repo+--+debian
          Hide
          siri Sriram Melkote added a comment -

          I tried to install on Debian today. It failed badly. One .deb package didn't match the libc version of stable. The other didn't match the openssl version. Changing libc or openssl is simply not an option for someone using Debian stable because it messes with the base OS too deeply. So as of 4/23/14, we don't have support for Debian.

          Show
          siri Sriram Melkote added a comment - I tried to install on Debian today. It failed badly. One .deb package didn't match the libc version of stable. The other didn't match the openssl version. Changing libc or openssl is simply not an option for someone using Debian stable because it messes with the base OS too deeply. So as of 4/23/14, we don't have support for Debian.
          Hide
          siri Sriram Melkote added a comment -

          Anil, we have accumulated a lot of input in this bug. I don't think this will realistically go anywhere for 3.0 unless we define specific goals and some considered platform support matrix expansion. Can you please create a goal for 3.0 more precisely?

          Show
          siri Sriram Melkote added a comment - Anil, we have accumulated a lot of input in this bug. I don't think this will realistically go anywhere for 3.0 unless we define specific goals and some considered platform support matrix expansion. Can you please create a goal for 3.0 more precisely?
          Hide
          ingenthr Matt Ingenthron added a comment -

          +1 on Siri's comments. Conversations I had with both Ubuntu (who recommend their PPAs) and Red Hat experts (who recommend setting up a repo or getting into EPEL or the like) indicated that's the best way to ensure coverage of all OSs. Binary packages built on one OS and deployed on another are risky, run into dependency issues.

          Show
          ingenthr Matt Ingenthron added a comment - +1 on Siri's comments. Conversations I had with both Ubuntu (who recommend their PPAs) and Red Hat experts (who recommend setting up a repo or getting into EPEL or the like) indicated that's the best way to ensure coverage of all OSs. Binary packages built on one OS and deployed on another are risky, run into dependency issues.
          Hide
          anil Anil Kumar added a comment -

          This ticket specially for distributing DEB and RPM repositories through YUM and APT repo. We have another ticket for supporting Debian platform MB-10960.

          Show
          anil Anil Kumar added a comment - This ticket specially for distributing DEB and RPM repositories through YUM and APT repo. We have another ticket for supporting Debian platform MB-10960 .
          Hide
          anil Anil Kumar added a comment -

          Assigning ticket to Tony for verification.

          Show
          anil Anil Kumar added a comment - Assigning ticket to Tony for verification.
          Hide
          plabee Phil Labee (Inactive) added a comment -

          Need to do before closing:

          [ ] capture keys and process used for build that is currently posted (3.0.0-628), update tools and keys of record in build repo and wiki page
          [ ] distribute 2.5.1 and 3.0.0-beta1 builds using same process, testing update capability
          [ ] test update from 2.0.0 to 2.5.1 to 3.0.0

          Show
          plabee Phil Labee (Inactive) added a comment - Need to do before closing: [ ] capture keys and process used for build that is currently posted (3.0.0-628), update tools and keys of record in build repo and wiki page [ ] distribute 2.5.1 and 3.0.0-beta1 builds using same process, testing update capability [ ] test update from 2.0.0 to 2.5.1 to 3.0.0
          Hide
          plabee Phil Labee (Inactive) added a comment -

          re-opening to assign to sprint to prepare the distribution repos for testing

          Show
          plabee Phil Labee (Inactive) added a comment - re-opening to assign to sprint to prepare the distribution repos for testing
          Hide
          wayne Wayne Siu added a comment -

          Phil,
          has build 3.0.0-973 be updated in the repos for beta testing?

          Show
          wayne Wayne Siu added a comment - Phil, has build 3.0.0-973 be updated in the repos for beta testing?
          Hide
          wayne Wayne Siu added a comment -

          Phil,
          Please refresh it with build 3.0.0-1205. Thanks.

          Show
          wayne Wayne Siu added a comment - Phil, Please refresh it with build 3.0.0-1205. Thanks.
          Hide
          plabee Phil Labee (Inactive) added a comment - - edited

          Due to loss of private keys used to post 3.0.0-628, created new key pairs. Upgrade testing was never done, so starting with 2.5.1 release version (2.5.1-1100).

          upload and test using location http://packages.couchbase.com/linux-repos/TEST/:

          [X] ubuntu-12.04 x86_64
          [X] ubuntu-10.04 x86_64

          [X] centos-6-x86_64
          [X] centos-5-x86_64

          Show
          plabee Phil Labee (Inactive) added a comment - - edited Due to loss of private keys used to post 3.0.0-628, created new key pairs. Upgrade testing was never done, so starting with 2.5.1 release version (2.5.1-1100). upload and test using location http://packages.couchbase.com/linux-repos/TEST/: [X] ubuntu-12.04 x86_64 [X] ubuntu-10.04 x86_64 [X] centos-6-x86_64 [X] centos-5-x86_64
          Hide
          anil Anil Kumar added a comment -

          Phil / Wayne - Not sure whats happening here please clarify.

          Show
          anil Anil Kumar added a comment - Phil / Wayne - Not sure whats happening here please clarify.
          Hide
          wayne Wayne Siu added a comment -

          Please refresh with the build 3.0.0-1209.

          Show
          wayne Wayne Siu added a comment - Please refresh with the build 3.0.0-1209.
          Hide
          plabee Phil Labee (Inactive) added a comment - - edited

          upgrade to 3.0.0-1209 using test location:

          s3://packages.couchbase.com/linux-repos/TEST/

          [X] ubuntu-12.04 x86_64
          [X] ubuntu-10.04 x86_64

          [X] centos-6-x86_64
          [X] centos-5-x86_64

          Show
          plabee Phil Labee (Inactive) added a comment - - edited upgrade to 3.0.0-1209 using test location: s3://packages.couchbase.com/linux-repos/TEST/ [X] ubuntu-12.04 x86_64 [X] ubuntu-10.04 x86_64 [X] centos-6-x86_64 [X] centos-5-x86_64
          Hide
          plabee Phil Labee (Inactive) added a comment - - edited

          now pushing 3.0.0-1209 to production location:

          s3://packages.couchbase.com/releases/couchbase-server/

          [X] centos-6-x86_64
          [X] centos-5-x86_64

          Please verify with instructions at: http://hub.internal.couchbase.com/confluence/display/CR/How+to+Download+from+a+Linux+Repo+--+RPM

          [X] ubuntu-12.04 x86_64
          [X] ubuntu-10.04 x86_64

          Please verify with instructions at: http://hub.internal.couchbase.com/confluence/display/CR/How+to+Download+from+a+Linux+Repo+--+Ubuntu

          [ ] debian-7-x86_64

          Show
          plabee Phil Labee (Inactive) added a comment - - edited now pushing 3.0.0-1209 to production location: s3://packages.couchbase.com/releases/couchbase-server/ [X] centos-6-x86_64 [X] centos-5-x86_64 Please verify with instructions at: http://hub.internal.couchbase.com/confluence/display/CR/How+to+Download+from+a+Linux+Repo+--+RPM [X] ubuntu-12.04 x86_64 [X] ubuntu-10.04 x86_64 Please verify with instructions at: http://hub.internal.couchbase.com/confluence/display/CR/How+to+Download+from+a+Linux+Repo+--+Ubuntu [ ] debian-7-x86_64
          Hide
          plabee Phil Labee (Inactive) added a comment - - edited

          now pushing 3.0.0-1209 to production location:

          s3://packages.couchbase.com/releases/couchbase-server/

          [X] centos-6-x86_64
          [X] centos-5-x86_64

          Please verify with instructions at: http://hub.internal.couchbase.com/confluence/display/CR/How+to+Download+from+a+Linux+Repo+--+RPM

          [X] ubuntu-12.04 x86_64
          [X] ubuntu-10.04 x86_64

          Please verify with instructions at: http://hub.internal.couchbase.com/confluence/display/CR/How+to+Download+from+a+Linux+Repo+--+Ubuntu

          [X] debian-7-x86_64

          Please verify with instructions at: http://hub.internal.couchbase.com/confluence/display/CR/How+to+Download+from+a+Linux+Repo+--+Debian

          Show
          plabee Phil Labee (Inactive) added a comment - - edited now pushing 3.0.0-1209 to production location: s3://packages.couchbase.com/releases/couchbase-server/ [X] centos-6-x86_64 [X] centos-5-x86_64 Please verify with instructions at: http://hub.internal.couchbase.com/confluence/display/CR/How+to+Download+from+a+Linux+Repo+--+RPM [X] ubuntu-12.04 x86_64 [X] ubuntu-10.04 x86_64 Please verify with instructions at: http://hub.internal.couchbase.com/confluence/display/CR/How+to+Download+from+a+Linux+Repo+--+Ubuntu [X] debian-7-x86_64 Please verify with instructions at: http://hub.internal.couchbase.com/confluence/display/CR/How+to+Download+from+a+Linux+Repo+--+Debian
          Hide
          h0nIg h0nIg added a comment - - edited

          is there a reason that these repositories are not maintained? there is still only 3.0.0 available and 3.0.2 is already released: http://packages.couchbase.com/releases/3.0.2/couchbase-server-enterprise_3.0.2-ubuntu12.04_amd64.deb

          Show
          h0nIg h0nIg added a comment - - edited is there a reason that these repositories are not maintained? there is still only 3.0.0 available and 3.0.2 is already released: http://packages.couchbase.com/releases/3.0.2/couchbase-server-enterprise_3.0.2-ubuntu12.04_amd64.deb
          Hide
          ingenthr Matt Ingenthron added a comment -

          Reopening owing to the comment about updates. Cihan: can you help the poster out?

          Show
          ingenthr Matt Ingenthron added a comment - Reopening owing to the comment about updates. Cihan: can you help the poster out?
          Hide
          cihan Cihan Biyikoglu (Inactive) added a comment -

          This is a mistake on our part. Thanks for the note.

          Show
          cihan Cihan Biyikoglu (Inactive) added a comment - This is a mistake on our part. Thanks for the note.
          Hide
          cihan Cihan Biyikoglu (Inactive) added a comment -

          Anil, could you make sure we have these on the flight checklist for GA for both major and minor releases and fix the existing repos.
          thanks

          Show
          cihan Cihan Biyikoglu (Inactive) added a comment - Anil, could you make sure we have these on the flight checklist for GA for both major and minor releases and fix the existing repos. thanks
          Hide
          anil Anil Kumar added a comment -

          Hari - As discussed lets fix this for Sherlock

          Show
          anil Anil Kumar added a comment - Hari - As discussed lets fix this for Sherlock
          Hide
          dfinlay Dave Finlay added a comment -

          Hari - what's the status on this issue? Are you actively working on it? Thanks!

          Show
          dfinlay Dave Finlay added a comment - Hari - what's the status on this issue? Are you actively working on it? Thanks!
          Hide
          hari.kodungallur hari kodungallur added a comment -

          We have tested out the current process for sherlock. With minor updates to scripts, we tested end to end. (Build -> sign -> create repo -> upload to s3 -> yum install). It works. However, there are a few questions that PM needs to answer. I have talked to Anil and he will answer the following questions.

          1. Platforms to be supported for Beta and GA (centos, debian)
          2. Change of package names (suffixing “community”). Are people ok with it? When should we do it (beta and/or GA)? This is needed because currently both enterprise and community RPMs are named couchbase-server and yum will have no way to distinguish one from the other (hence risky during yum update).
          3. How are we going to provide one user interface for both SDK and Server?

          According to the answers (especially the second question regarding naming), we will need to do more changes in the build/release process and the process will surely change.

          I am going to assign this back to Anil.

          Show
          hari.kodungallur hari kodungallur added a comment - We have tested out the current process for sherlock. With minor updates to scripts, we tested end to end. (Build -> sign -> create repo -> upload to s3 -> yum install). It works. However, there are a few questions that PM needs to answer. I have talked to Anil and he will answer the following questions. Platforms to be supported for Beta and GA (centos, debian) Change of package names (suffixing “community”). Are people ok with it? When should we do it (beta and/or GA)? This is needed because currently both enterprise and community RPMs are named couchbase-server and yum will have no way to distinguish one from the other (hence risky during yum update). How are we going to provide one user interface for both SDK and Server? According to the answers (especially the second question regarding naming), we will need to do more changes in the build/release process and the process will surely change. I am going to assign this back to Anil.
          Hide
          dfinlay Dave Finlay added a comment -

          Anil to sync with Mark N. tomorrow (5/2) and sort this one out.

          Show
          dfinlay Dave Finlay added a comment - Anil to sync with Mark N. tomorrow (5/2) and sort this one out.
          Hide
          dfinlay Dave Finlay added a comment -

          Anil - any update? Thanks.

          Show
          dfinlay Dave Finlay added a comment - Anil - any update? Thanks.
          Hide
          anil Anil Kumar added a comment - - edited

          Here is the summary of what we have decided and assigning this ticket to Hari.

          1. Platforms to be supported for Beta and GA (centos, debian)

          • Aspire to support Yum and Debian for Beta
          • Support Yum and Debian for GA

          2. Change of package names (suffixing “community”). Are people ok with it? When should we do it (beta and/or GA)?

          • Survey done - no concerns. Please go ahead change the RPM name for Community Edition to "Couchbase Community"

          3. How are we going to provide one user interface for both SDK and Server?

          • We will merge the REPO's for Couchbase Server Enterprise Edition and SDK's
          • SDK team will continue to build the RPM and provide the package to Build team to integrate.
          Show
          anil Anil Kumar added a comment - - edited Here is the summary of what we have decided and assigning this ticket to Hari. 1. Platforms to be supported for Beta and GA (centos, debian) Aspire to support Yum and Debian for Beta Support Yum and Debian for GA 2. Change of package names (suffixing “community”). Are people ok with it? When should we do it (beta and/or GA)? Survey done - no concerns. Please go ahead change the RPM name for Community Edition to "Couchbase Community" 3. How are we going to provide one user interface for both SDK and Server? We will merge the REPO's for Couchbase Server Enterprise Edition and SDK's SDK team will continue to build the RPM and provide the package to Build team to integrate.
          Hide
          dfinlay Dave Finlay added a comment -

          Thanks Anil. Hari - are you clear on your action items? Thanks.

          Show
          dfinlay Dave Finlay added a comment - Thanks Anil. Hari - are you clear on your action items? Thanks.
          Hide
          hari.kodungallur hari kodungallur added a comment - - edited

          Yes, I am. I still have to verify how to tackle some open issues that Mark has mentioned in CBD-1545. But for now I am good. I will check back and comment on appropriate issue if I have more questions.

          Show
          hari.kodungallur hari kodungallur added a comment - - edited Yes, I am. I still have to verify how to tackle some open issues that Mark has mentioned in CBD-1545. But for now I am good. I will check back and comment on appropriate issue if I have more questions.
          Hide
          hari.kodungallur hari kodungallur added a comment -

          Anil -

          I am almost done with the work for yum and probably finish it up for deb/ubuntu by EOB today (tuesday 5/12).
          However, I have a few questions that will need resolution and accordingly some minor changes may need to be made the code.

          • Who is going to test this? And what is the test plan? I don't exactly need to know this, but I am curious We will find a place for the entry point rpm/deb on an internal server from which whoever is testing should be able to access it. Will let you know once we decide the location. I will also update the internal wiki page adding instructions to install.
          • GPG Keys. There are 2 keys used by SDK team. We used a different one. Ideally we should just use a single key across the company. I do not know if/when one decides to change these keys. But when a change to the key does happen, all the products will need to be re-signed. In this case, it may be easier to manage the keys separately. But I feel like one doesn't change keys that often. I would like your input on that.
          • For Debian/Ubuntu, there is nothing similar to an epel-release or rpmforge-release package out there. But I created something similar. But it will be a 2 step process – install the entry point deb package, followed by a 'apt-get update' - the latter can't be done in the install scripts. I think this should be ok. But please confirm.
          • I am going to version the couchbase-release rpm and deb files as 1.0. If you think it should be different please let me know.
          Show
          hari.kodungallur hari kodungallur added a comment - Anil - I am almost done with the work for yum and probably finish it up for deb/ubuntu by EOB today (tuesday 5/12). However, I have a few questions that will need resolution and accordingly some minor changes may need to be made the code. Who is going to test this? And what is the test plan? I don't exactly need to know this, but I am curious We will find a place for the entry point rpm/deb on an internal server from which whoever is testing should be able to access it. Will let you know once we decide the location. I will also update the internal wiki page adding instructions to install. GPG Keys. There are 2 keys used by SDK team. We used a different one. Ideally we should just use a single key across the company. I do not know if/when one decides to change these keys. But when a change to the key does happen, all the products will need to be re-signed. In this case, it may be easier to manage the keys separately. But I feel like one doesn't change keys that often. I would like your input on that. For Debian/Ubuntu, there is nothing similar to an epel-release or rpmforge-release package out there. But I created something similar. But it will be a 2 step process – install the entry point deb package, followed by a 'apt-get update' - the latter can't be done in the install scripts. I think this should be ok. But please confirm. I am going to version the couchbase-release rpm and deb files as 1.0. If you think it should be different please let me know.
          Hide
          anil Anil Kumar added a comment -

          Hi Hari, You have my responses for the above questions. Can we resolve the ticket as soon as you complete the last few remaining items. Thank you so much!

          Show
          anil Anil Kumar added a comment - Hi Hari, You have my responses for the above questions. Can we resolve the ticket as soon as you complete the last few remaining items. Thank you so much!
          Hide
          hari.kodungallur hari kodungallur added a comment -

          an "entry point" rpm/deb file is created that will configure the yum or deb repository list etc.
          i also have uploaded rpm and debs to s3 (as pointed to by the repository configuration). i have sent testing instructions out to anil and raju.

          any modifications / bugs – we will track separately.

          Show
          hari.kodungallur hari kodungallur added a comment - an "entry point" rpm/deb file is created that will configure the yum or deb repository list etc. i also have uploaded rpm and debs to s3 (as pointed to by the repository configuration). i have sent testing instructions out to anil and raju. any modifications / bugs – we will track separately.
          Hide
          raju Raju Suravarjjala added a comment -

          Old bugs that are in resolved state for long time

          Show
          raju Raju Suravarjjala added a comment - Old bugs that are in resolved state for long time

            People

            • Assignee:
              hari.kodungallur hari kodungallur
              Reporter:
              anil Anil Kumar
            • Votes:
              3 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Agile

                  Gerrit Reviews

                  There are no open Gerrit changes