Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-45257

[System Test] [ERRO] rest: error code: 400, msg: rest_create_index: index type is required

    XMLWordPrintable

Details

    • Bug
    • Resolution: Cannot Reproduce
    • Major
    • 7.1.0
    • Cheshire-Cat
    • fts
    • Untriaged
    • 1
    • Unknown

    Description

      Build: 7.0.0-4766
      Componet Clusterops test: -test tests/fts/cheshire-cat/test_fts_clusterops_cheshire_cat_coll_crud.yml -scope tests/fts/cheshire-cat/scope_fts_cheshire_cat.yml
      Test Cycle: 1

      Test with index create/drop loop.

      Test steps:

      1. there are 5 buckets, out of which 20 static fts indexes are created on collections of 3 buckets. Mutations are going on these collections
      2. For the collections on other 2 buckets, we create and drop indexes and no mutations are going on these collections.
      3. Continuously run queries on the indexes of collections of bucket1 and bucket2
      4. wait for 15 mins
      5. kill cbft on 172.23.97.217 and wait for 15 mins
      6. add fts node 172.23.107.4 and start rebalance and wait for 15 mins

       Seeing lot of "[ERRO] rest: error code: 400, msg: rest_create_index: index type is required" thoughout the test.

      172.23.107.4 : fts
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log:2021-03-24T12:26:07.282-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log:2021-03-24T12:28:31.900-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log:2021-03-24T12:28:34.701-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log:2021-03-24T12:29:36.865-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log.1.gz:2021-03-24T12:08:32.935-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log.1.gz:2021-03-24T12:08:41.180-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log.1.gz:2021-03-24T12:20:45.002-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log.1.gz:2021-03-24T12:24:14.873-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log.1.gz:2021-03-24T12:24:27.800-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log.2.gz:2021-03-24T11:38:36.166-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log.2.gz:2021-03-24T11:39:20.952-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log.2.gz:2021-03-24T11:40:13.109-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log.2.gz:2021-03-24T11:42:00.489-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log.2.gz:2021-03-24T11:46:35.995-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log.2.gz:2021-03-24T11:47:08.804-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log.2.gz:2021-03-24T11:47:33.555-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log.2.gz:2021-03-24T11:48:36.951-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log.2.gz:2021-03-24T11:52:08.543-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log.2.gz:2021-03-24T11:53:26.281-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log.2.gz:2021-03-24T12:04:19.192-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log.3.gz:2021-03-24T11:20:18.596-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log.3.gz:2021-03-24T11:20:40.997-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log.3.gz:2021-03-24T11:36:29.614-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log.3.gz:2021-03-24T11:37:00.970-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log.3.gz:2021-03-24T11:37:14.155-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log.4.gz:2021-03-24T11:08:57.192-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log.4.gz:2021-03-24T11:09:14.731-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63
      		 
      /opt/couchbase/var/lib/couchbase/logs/fts.log.4.gz:2021-03-24T11:11:10.767-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required -- rest.ShowErrorBody() at rest.go:63

      As part of the test, we are setting the indexType to scorch in the create index request. From the logs, not able to relate to any create request sent by the test.

      For example:

      172.23.107.4 : fts
      /opt/couchbase/var/lib/couchbase/logs/fts.log:2021-03-24T12:26:07.282-07:00 [ERRO] rest: error code: 400, msg: rest_create_index: index type is required – rest.ShowErrorBody() at rest.go:63

      At this point there were 2 index create threads were running and none of them failed to create index.

      Thread 1:

      {"log":"2021-03-24 05:24:41,421 - ftsindexmanager - INFO - ===== Creating single FTS index on ['scope_2.coll_8'] =====\n","stream":"stderr","time":"2021-03-24T12:24:41.421990568Z"}
      		 
      {"log":"2021-03-24 05:24:53,870 - ftsindexmanager - INFO - FTS indexes in cluster - : \n","stream":"stderr","time":"2021-03-24T12:24:53.871337547Z"}
      		 
      {"log":"['bucket_bucket1_idx_agajc-price', 'bucket_bucket1_idx_evzvn-type-Overall-price-free_parking', 'bucket_bucket1_idx_fhllw-country-price-type-public_likes-Overall-free_parking', 'bucket_bucket1_idx_hcqfg-Overall-country-public_likes-price-type-free_parking', 'bucket_bucket1_idx_hgzkq-country-price-Overall-type-free_parking-public_likes', 'bucket_bucket1_idx_ihjac-type-price', 'bucket_bucket1_idx_ldygi-free_parking-type-country-price-public_likes', 'bucket_bucket1_idx_locux-price-public_likes-country-Overall-free_parking-type', 'bucket_bucket1_idx_qftoa-country-public_likes-Overall', 'bucket_bucket1_idx_snrai-price-country-free_parking-public_likes', 'bucket_bucket1_idx_vurql-Overall-free_parking-country-price-public_likes', 'bucket_bucket1_idx_wmgfk-type', 'bucket_bucket2_idx_cxbkr-type-public_likes-price-Overall-country-free_parking', 'bucket_bucket2_idx_enkit-country-free_parking-public_likes', 'bucket_bucket2_idx_hhetg-country-price-type', 'bucket_bucket2_idx_ivadl-free_parking-price-type-public_likes-country-Overall', 'bucket_bucket2_idx_muevu-type-public_likes', 'bucket_bucket2_idx_ncxws-country-price-public_likes-type-Overall', 'bucket_bucket2_idx_nqizz-Overall-free_parking', 'bucket_bucket2_idx_pqlrt-country-public_likes', 'bucket_bucket2_idx_qrnkb-type-free_parking', 'bucket_bucket2_idx_qrnpf-price-country-free_parking-type-Overall-public_likes', 'bucket_bucket3_idx_dtkot-country-public_likes', 'bucket_bucket3_idx_epkeo-public_likes-type', 'bucket_bucket3_idx_fqfih-price-country-public_likes-type-Overall-free_parking', 'bucket_bucket3_idx_iagxc-type-free_parking-price-public_likes-country-Overall', 'bucket_bucket3_idx_mnyux-public_likes-country', 'bucket_bucket3_idx_mpdlm-country-price-public_likes-type-free_parking-Overall', 'bucket_bucket3_idx_nawrg-type-price', 'bucket_bucket3_idx_nhuht-Overall-price-country', 'bucket_bucket3_idx_ploqv-Overall-free_parking-public_likes-country-price', 'bucket_bucket3_idx_qykji-type', 'bucket_bucket3_idx_rsyse-free_parking-price-public_likes-country-Overall-type', 'bucket_bucket3_idx_tnhec-public_likes-type-free_parking-Overall-price', 'bucket_bucket3_idx_unyut-Overall-price-free_parking-type-country', 'bucket_bucket3_idx_vhboi-public_likes', 'bucket_bucket3_idx_vxxns-price', 'bucket_bucket3_idx_vzbbv-type-country-public_likes', 'bucket_bucket3_idx_xphby-public_likes-type-free_parking-Overall-country-price', 'bucket_bucket3_idx_ygwzr-type-public_likes-price', 'bucket_bucket4_idx_oswmz-Overall-type-country-price-public_likes_33', 'bucket_bucket5_idx_eknqp-price_34']\n","stream":"stderr","time":"2021-03-24T12:24:53.871368814Z"}
      		 
      {"log":"2021-03-24 05:24:53,871 - ftsindexmanager - INFO - Creating FTS index bucket_bucket5_idx_vrzlt-Overall-price-free_parking_36 on ['scope_2.coll_8'] with 1 replicas and 6 partitions\n","stream":"stderr","time":"2021-03-24T12:24:53.872371205Z"}
      		 
      {"log":"2021-03-24 05:25:20,451 - ftsindexmanager - INFO - Index creation on bucket_bucket5_idx_vrzlt-Overall-price-free_parking_36 succeeded. Content = {'status': 'ok', 'uuid': '60566f4891a62c69'}, Response = {'content-type': 'application/json;version=2.0.0', 'date': 'Wed, 24 Mar 2021 12:25:20 GMT', 'content-length': '42', 'status': '200'}\n","stream":"stderr","time":"2021-03-24T12:25:20.452139423Z"}
      		 
      {"log":"2021-03-24 05:29:20,552 - ftsindexmanager - INFO - Deleting index bucket_bucket5_idx_vrzlt-Overall-price-free_parking_36\n","stream":"stderr","time":"2021-03-24T12:29:20.552509058Z"}

      Thread 2:

      {"log":"2021-03-24 05:25:10,326 - ftsindexmanager - INFO - ===== Creating single FTS index on ['scope_0.coll_2'] =====\n","stream":"stderr","time":"2021-03-24T12:25:10.327020604Z"}
      		 
      {"log":"2021-03-24 05:26:09,697 - ftsindexmanager - INFO - FTS indexes in cluster - : \n","stream":"stderr","time":"2021-03-24T12:26:09.698115615Z"}
      		 
      {"log":"['bucket_bucket1_idx_agajc-price', 'bucket_bucket1_idx_evzvn-type-Overall-price-free_parking', 'bucket_bucket1_idx_fhllw-country-price-type-public_likes-Overall-free_parking', 'bucket_bucket1_idx_hcqfg-Overall-country-public_likes-price-type-free_parking', 'bucket_bucket1_idx_hgzkq-country-price-Overall-type-free_parking-public_likes', 'bucket_bucket1_idx_ihjac-type-price', 'bucket_bucket1_idx_ldygi-free_parking-type-country-price-public_likes', 'bucket_bucket1_idx_locux-price-public_likes-country-Overall-free_parking-type', 'bucket_bucket1_idx_qftoa-country-public_likes-Overall', 'bucket_bucket1_idx_snrai-price-country-free_parking-public_likes', 'bucket_bucket1_idx_vurql-Overall-free_parking-country-price-public_likes', 'bucket_bucket1_idx_wmgfk-type', 'bucket_bucket2_idx_cxbkr-type-public_likes-price-Overall-country-free_parking', 'bucket_bucket2_idx_enkit-country-free_parking-public_likes', 'bucket_bucket2_idx_hhetg-country-price-type', 'bucket_bucket2_idx_ivadl-free_parking-price-type-public_likes-country-Overall', 'bucket_bucket2_idx_muevu-type-public_likes', 'bucket_bucket2_idx_ncxws-country-price-public_likes-type-Overall', 'bucket_bucket2_idx_nqizz-Overall-free_parking', 'bucket_bucket2_idx_pqlrt-country-public_likes', 'bucket_bucket2_idx_qrnkb-type-free_parking', 'bucket_bucket2_idx_qrnpf-price-country-free_parking-type-Overall-public_likes', 'bucket_bucket3_idx_dtkot-country-public_likes', 'bucket_bucket3_idx_epkeo-public_likes-type', 'bucket_bucket3_idx_fqfih-price-country-public_likes-type-Overall-free_parking', 'bucket_bucket3_idx_iagxc-type-free_parking-price-public_likes-country-Overall', 'bucket_bucket3_idx_mnyux-public_likes-country', 'bucket_bucket3_idx_mpdlm-country-price-public_likes-type-free_parking-Overall', 'bucket_bucket3_idx_nawrg-type-price', 'bucket_bucket3_idx_nhuht-Overall-price-country', 'bucket_bucket3_idx_ploqv-Overall-free_parking-public_likes-country-price', 'bucket_bucket3_idx_qykji-type', 'bucket_bucket3_idx_rsyse-free_parking-price-public_likes-country-Overall-type', 'bucket_bucket3_idx_tnhec-public_likes-type-free_parking-Overall-price', 'bucket_bucket3_idx_unyut-Overall-price-free_parking-type-country', 'bucket_bucket3_idx_vhboi-public_likes', 'bucket_bucket3_idx_vxxns-price', 'bucket_bucket3_idx_vzbbv-type-country-public_likes', 'bucket_bucket3_idx_xphby-public_likes-type-free_parking-Overall-country-price', 'bucket_bucket3_idx_ygwzr-type-public_likes-price', 'bucket_bucket4_idx_oswmz-Overall-type-country-price-public_likes_33', 'bucket_bucket5_idx_eknqp-price_34']\n","stream":"stderr","time":"2021-03-24T12:26:09.698148959Z"}
      		 
      {"log":"2021-03-24 05:26:09,698 - ftsindexmanager - INFO - Creating FTS index bucket_bucket4_idx_ktyen-country-Overall-public_likes-type-price_37 on ['scope_0.coll_2'] with 0 replicas and 20 partitions\n","stream":"stderr","time":"2021-03-24T12:26:09.6984492Z"}
      		 
      {"log":"2021-03-24 05:26:09,698 - ftsindexmanager - INFO - Field reviews already exists\n","stream":"stderr","time":"2021-03-24T12:26:09.698578476Z"}
      		 
      {"log":"2021-03-24 05:26:29,478 - ftsindexmanager - INFO - Index creation on bucket_bucket4_idx_ktyen-country-Overall-public_likes-type-price_37 succeeded. Content = {'status': 'ok', 'uuid': '2e9f9acff576619a'}, Response = {'content-type': 'application/json;version=2.0.0', 'date': 'Wed, 24 Mar 2021 12:26:29 GMT', 'content-length': '42', 'status': '200'}\n","stream":"stderr","time":"2021-03-24T12:26:29.478725971Z"}
      		 
      {"log":"2021-03-24 05:30:29,484 - ftsindexmanager - INFO - Deleting index bucket_bucket4_idx_ktyen-country-Overall-public_likes-type-price_37\n","stream":"stderr","time":"2021-03-24T12:30:29.484718148Z"}

      cbcollect logs:

      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616615137/collectinfo-2021-03-24T194539-ns_1%40172.23.107.2.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616615137/collectinfo-2021-03-24T194539-ns_1%40172.23.107.3.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616615137/collectinfo-2021-03-24T194539-ns_1%40172.23.107.4.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616615137/collectinfo-2021-03-24T194539-ns_1%40172.23.107.5.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616615137/collectinfo-2021-03-24T194539-ns_1%40172.23.97.215.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616615137/collectinfo-2021-03-24T194539-ns_1%40172.23.97.216.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616615137/collectinfo-2021-03-24T194539-ns_1%40172.23.97.217.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616615137/collectinfo-2021-03-24T194539-ns_1%40172.23.97.227.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616615137/collectinfo-2021-03-24T194539-ns_1%40172.23.97.232.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616615137/collectinfo-2021-03-24T194539-ns_1%40172.23.97.235.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616615137/collectinfo-2021-03-24T194539-ns_1%40172.23.97.236.zip
      url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1616615137/collectinfo-2021-03-24T194539-ns_1%40172.23.97.237.zip

       

      ########## Cluster config ##################
      		 
      ######  fts : 5 ===== > [172.23.107.2:8091 172.23.107.3:8091 172.23.107.5:8091 172.23.97.216:8091 172.23.97.217:8091]  ###########
      		 
      ######  kv : 4 ===== > [172.23.97.215:8091 172.23.97.232:8091 172.23.97.235:8091 172.23.97.237:8091]  ###########
      		 
      ######  n1ql : 2 ===== > [172.23.97.227:8091 172.23.97.236:8091]  ###########

      Attachments

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. MB-44476 [System Test]cannot create index because an index with the same name already exists

          • Major - Major loss of function.
          • Reopened
          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              abhinav Abhi Dangeti
              girish.benakappa Girish Benakappa
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty