Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-48300

[TLS] Strict mode, see errors in query log for 9101 port for indexer client

    XMLWordPrintable

Details

    • Untriaged
    • 1
    • Unknown

    Description

      I see these errors in the query log 

       

      t_02, keyspace: users
      2021-08-18T23:02:18.624-07:00 [Info] GsiClient::UpdateUsecjson: using collatejson as data format between indexer and GsiClient
      2021-08-18T23:02:18.624-07:00 [Info] GSIC[default/travel-sample-tenant_agent_02-bookings-1629352938623032499] started ...
      2021-08-18T23:02:18.624-07:00 [Info] Recieve security change notification. encryption=true
      2021-08-18T23:02:18.625-07:00 [Info] Certificate refreshed successfully
      _time=2021-08-18T23:02:18.625-07:00 _level=INFO _msg=n1fty: NewFTSIndexer2, server: http://127.0.0.1:8091, namespace: default, bucket: travel-sample, scope: tenant_agent_02, keyspace: bookings
      2021-08-18T23:02:21.778-07:00 [Error] transport error between 172.23.99.49:46598->172.23.99.49:9101: write tcp 172.23.99.49:46598->172.23.99.49:9101: write: broken pipe
      2021-08-18T23:02:21.778-07:00 [Error] [GsiScanClient:"172.23.99.49:9101"] retriever request transport failed `write tcp 172.23.99.49:46598->172.23.99.49:9101: write: broken pipe`
      2021-08-18T23:02:21.778-07:00 [Info] [Queryport-connpool:172.23.99.49:9101] closing unhealthy connection "172.23.99.49:46598"
      2021-08-18T23:02:21.778-07:00 [Warn] scan failed: requestId retriever queryport 172.23.99.49:9101 inst 13536734719740518051 partition [0]
      2021-08-18T23:02:21.778-07:00 [Warn] Scan failed with error for index 9705858241300905373.  Trying scan again with replica, reqId:retriever :  write tcp 172.23.99.49:46598->172.23.99.49:9101: write: broken pipe from [172.23.99.49:9101] ...
      2021-08-18T23:02:21.778-07:00 [Error] PickRandom: Fail to find indexer for all index partitions. Num partition 1.  Partition with instances 0
      2021-08-18T23:02:21.778-07:00 [Warn] Fail to find indexers to satisfy query request.  Trying scan again for index 9705858241300905373, reqId:retriever :  write tcp 172.23.99.49:46598->172.23.99.49:9101: write: broken pipe from [172.23.99.49:9101] ...
      

      Steps to recreate 

      2 node cluster with default deployment 

      • enable N2N encryption: /opt/couchbase/bin/couchbase-cli node-to-node-encryption -c http://localhost:8091 -u Administrator -p password --enable
      • set TLS strict mode: /opt/couchbase/bin/couchbase-cli setting-security -c http://localhost:8091 -u Administrator -p password --set --cluster-encryption-level strict

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            Build couchbase-server-7.1.0-1260 contains indexing commit 79356cf with commit message:
            MB-48300 Retry shutdown with local kvaddrs incase of node rename

            build-team Couchbase Build Team added a comment - Build couchbase-server-7.1.0-1260 contains indexing commit 79356cf with commit message: MB-48300 Retry shutdown with local kvaddrs incase of node rename

            Varun Velamuri Can this be resolved now?

            jeelan.poola Jeelan Poola added a comment - Varun Velamuri Can this be resolved now?

            Validated on 7.1.0-1905

             

            [root@node1-cb660-centos7 logs]# less query.log 
            [root@node1-cb660-centos7 logs]# less query.log | grep  'transport error between' 
            [root@node1-cb660-centos7 logs]# less query.log | grep  'retriever request transport failed ' 
            [root@node1-cb660-centos7 logs]# less query.log | grep  'Fail to find indexer for all index partitions'
            

             

            hemant.rajput Hemant Rajput added a comment - Validated on 7.1.0-1905   [root@node1-cb660-centos7 logs]# less query.log  [root@node1-cb660-centos7 logs]# less query.log | grep  'transport error between'  [root@node1-cb660-centos7 logs]# less query.log | grep  'retriever request transport failed '  [root@node1-cb660-centos7 logs]# less query.log | grep  'Fail to find indexer for all index partitions'  

            People

              varun.velamuri Varun Velamuri
              varun.velamuri Varun Velamuri
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty