Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-48250

[Backport MB-48244 to 7.0.2] Loading travel-sample failed on mixed mode cluster of 7.0.x

    XMLWordPrintable

Details

    • Untriaged
    • 1
    • Unknown

    Description

      Steps to reproduce-

      1. create a mixed mode cluster with 7.0.0-5302 (KV+Query) and 7.0.2-6620 (Indexer) node
      2. Add travel-sample bucket
      3. Check that the bucket is loaded

      Loading sample bucket travel-sample failed. Samples loader exited with status 1. Loader's output was: 2021-08-30T22:55:49.996-07:00 (REST) (Attempt 1) (GET) Dispatching request to 'http://172.23.136.106:8091/pools/default/nodeServices' 2021-08-30T22:55:49.998-07:00 (REST) (Attempt 1) (GET) (200) Received response from 'http://172.23.136.106:8091/pools/default/nodeServices' 2021-08-30T22:55:49.998-07:00 (REST) (Attempt 1) (GET) Dispatching request to 'http://172.23.136.106:8091/pools' 2021-08-30T22:55:49.999-07:00 (REST) (Attempt 1) (GET) (200) Received response from 'http://172.23.136.106:8091/pools' 2021-08-30T22:55:49.999-07:00 (REST) (Attempt 1) (GET) Dispatching request to 'http://172.23.136.106:8091/pools/default' 2021-08-30T22:55:50.003-07:00 (REST) (Attempt 1) (GET) (200) Received response from 'http://172.23.136.106:8091/pools/default' 2021-08-30T22:55:50.003-07:00 (REST) Successfully connected to cluster | {"enterprise":true,"uuid":"a7a18eac5f29fdabd5d0360c600d1619","version":{"min_version":"7.0.0","is_mixed_cluster":true}} 2021-08-30T22:55:50.100-07:00 (REST) (Attempt 1) (GET) Dispatching request to 'http://172.23.136.106:8091/pools/default/buckets/travel-sample' 2021-08-30T22:55:50.102-07:00 (REST) (Attempt 1) (GET) (404) Received response from 'http://172.23.136.106:8091/pools/default/buckets/travel-sample' 2021-08-30T22:55:50.102-07:00 WARN: (REST) (Attempt 1) (GET) Request to endpoint '/pools/default/buckets/travel-sample' failed with status code 404 -- rest.(*Request).Execute() at request.go:158 2021-08-30T22:55:50.102-07:00 (Plan) (Bucket) Creating remote bucket 'travel-sample' if it does not exist 2021-08-30T22:55:50.102-07:00 (REST) (Attempt 1) (GET) Dispatching request to 'http://172.23.136.106:8091/pools/default/buckets/travel-sample' 2021-08-30T22:55:50.103-07:00 (REST) (Attempt 1) (GET) (404) Received response from 'http://172.23.136.106:8091/pools/default/buckets/travel-sample' 2021-08-30T22:55:50.103-07:00 WARN: (REST) (Attempt 1) (GET) Request to endpoint '/pools/default/buckets/travel-sample' failed with status code 404 -- rest.(*Request).Execute() at request.go:158 2021-08-30T22:55:50.103-07:00 Bucket travel-sample does not exist, creating one with the specified settings 2021-08-30T22:55:50.103-07:00 (REST) (Attempt 1) (POST) Dispatching request to 'http://172.23.136.106:8091/pools/default/buckets' 2021-08-30T22:55:50.131-07:00 (REST) (Attempt 1) (POST) (202) Received response from 'http://172.23.136.106:8091/pools/default/buckets' 2021-08-30T22:55:51.132-07:00 (REST) (Attempt 1) (GET) Dispatching request to 'http://172.23.136.106:8091/pools/default/buckets/travel-sample' 2021-08-30T22:55:51.150-07:00 (REST) (Attempt 1) (GET) (200) Received response from 'http://172.23.136.106:8091/pools/default/buckets/travel-sample' 2021-08-30T22:55:52.132-07:00 (REST) (Attempt 1) (GET) Dispatching request to 'http://172.23.136.106:8091/pools/default/buckets/travel-sample' 2021-08-30T22:55:52.137-07:00 (REST) (Attempt 1) (GET) (200) Received response from 'http://172.23.136.106:8091/pools/default/buckets/travel-sample' 2021-08-30T22:55:52.137-07:00 (Plan) (Bucket) Successfully ensured bucket 'travel-sample' exists | {"number":1,"duration":"2.035256874s"} 2021-08-30T22:55:52.137-07:00 (Plan) (Bucket) Automatically creating/editing scopes/collections for bucket 'travel-sample' 2021-08-30T22:55:52.137-07:00 (REST) (Attempt 1) (GET) Dispatching request to 'http://172.23.136.106:8091/pools/default/buckets/travel-sample' 2021-08-30T22:55:52.141-07:00 (REST) (Attempt 1) (GET) (200) Received response from 'http://172.23.136.106:8091/pools/default/buckets/travel-sample' 2021-08-30T22:55:52.141-07:00 (REST) (Attempt 1) (GET) Dispatching request to 'http://172.23.136.106:8091/pools/default/buckets/travel-sample/scopes' 2021-08-30T22:55:52.143-07:00 (REST) (Attempt 1) (GET) (200) Received response from 'http://172.23.136.106:8091/pools/default/buckets/travel-sample/scopes' 2021-08-30T22:55:52.143-07:00 (REST) (Attempt 1) (POST) Dispatching request to 'http://172.23.136.106:8091/pools/default/buckets/travel-sample/scopes/%40ensureManifest/0' 2021-08-30T22:55:52.147-07:00 (REST) (Attempt 1) (POST) (200) Received response from 'http://172.23.136.106:8091/pools/default/buckets/travel-sample/scopes/%40ensureManifest/0' 2021-08-30T22:55:52.147-07:00 (REST) (Attempt 1) (PUT) Dispatching request to 'http://172.23.136.106:8091/pools/default/buckets/travel-sample/scopes?validOnUid=0' 2021-08-30T22:55:52.176-07:00 (REST) (Attempt 1) (PUT) (200) Received response from 'http://172.23.136.106:8091/pools/default/buckets/travel-sample/scopes?validOnUid=0' 2021-08-30T22:55:52.176-07:00 (REST) (Attempt 1) (POST) Dispatching request to 'http://172.23.136.106:8091/pools/default/buckets/travel-sample/scopes/%40ensureManifest/1' 2021-08-30T22:55:52.682-07:00 (REST) (Attempt 1) (POST) (200) Received response from 'http://172.23.136.106:8091/pools/default/buckets/travel-sample/scopes/%40ensureManifest/1' 2021-08-30T22:55:52.682-07:00 (Plan) (Bucket) Successfully created/edited scopes/collections for bucket 'travel-sample' | {"number":2,"duration":"544.899903ms"} 2021-08-30T22:55:52.682-07:00 (Plan) (Bucket) Transferring bucket configuration for bucket 'travel-sample' 2021-08-30T22:55:52.682-07:00 (REST) (Attempt 1) (GET) Dispatching request to 'http://172.23.136.106:8091/pools/default/buckets/travel-sample' 2021-08-30T22:55:52.686-07:00 (REST) (Attempt 1) (GET) (200) Received response from 'http://172.23.136.106:8091/pools/default/buckets/travel-sample' 2021-08-30T22:55:52.687-07:00 Editing bucket to conform with settings provided 2021-08-30T22:55:52.687-07:00 (REST) (Attempt 1) (POST) Dispatching request to 'http://172.23.136.106:8091/pools/default/buckets/travel-sample' 2021-08-30T22:55:52.719-07:00 (REST) (Attempt 1) (POST) (200) Received response from 'http://172.23.136.106:8091/pools/default/buckets/travel-sample' 2021-08-30T22:55:53.720-07:00 (REST) (Attempt 1) (GET) Dispatching request to 'http://172.23.136.106:8091/pools/default/buckets/travel-sample' 2021-08-30T22:55:53.725-07:00 (REST) (Attempt 1) (GET) (200) Received response from 'http://172.23.136.106:8091/pools/default/buckets/travel-sample' 2021-08-30T22:55:53.725-07:00 (Plan) (Bucket) Successfully transferred bucket configuration for bucket 'travel-sample' | {"number":3,"duration":"1.043407578s"} 2021-08-30T22:55:53.726-07:00 (Plan) (Index) Transferring views definitions for bucket 'travel-sample' 2021-08-30T22:55:53.820-07:00 (Plan) (Index) Successfully transferred views definitions for bucket 'travel-sample' | {"number":4,"duration":"94.729289ms"} 2021-08-30T22:55:53.820-07:00 (Plan) (Query) Executing queries for bucket 'travel-sample' 2021-08-30T22:57:39.099-07:00 JSON import failed: failed to execute cluster operations: failed to execute bucket operation for bucket 'travel-sample': failed to execute queries for bucket 'travel-sample': failed to execute queries against sink bucket: failed to execute query, exhausted retry count after 3 retries JSON import failed: failed to execute query, exhausted retry count after 3 retries
      

      Logs are available at - 
      https://cb-engineering.s3.amazonaws.com/MB-48244/48244/collectinfo-2021-08-31T060956-ns_1%40172.23.136.106.zip
      https://cb-engineering.s3.amazonaws.com/MB-48244/48244/collectinfo-2021-08-31T060956-ns_1%40172.23.136.196.zip

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            Keeping it open as I am going to perform further testing on this.

            amit.kulkarni Amit Kulkarni added a comment - Keeping it open as I am going to perform further testing on this.

            Build couchbase-server-7.0.2-6631 contains indexing commit c197e90 with commit message:
            MB-48250: Reintroduce OPCODE_CLEANUP_DEFER_INDEX for mixed mode support

            build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.2-6631 contains indexing commit c197e90 with commit message: MB-48250 : Reintroduce OPCODE_CLEANUP_DEFER_INDEX for mixed mode support

            Wayne Siu I dont seem to find your latest comment on this one. Strange. This is a Critical issue that must be fixed. And Amit has already fixed it. Will resolve it.

            jeelan.poola Jeelan Poola added a comment - Wayne Siu I dont seem to find your latest comment on this one. Strange. This is a Critical issue that must be fixed. And Amit has already fixed it. Will resolve it.

            Validated on 7.0.2-6635.

            hemant.rajput Hemant Rajput added a comment - Validated on 7.0.2-6635.

            People

              amit.kulkarni Amit Kulkarni
              amit.kulkarni Amit Kulkarni
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty