Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-38221

[BP 6.5.1] - [Regression] Unable to add remote cluster ref w/ full + certificates

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Critical
    • 6.5.1
    • 6.5.1, Cheshire-Cat
    • XDCR
    • Untriaged
    • Yes

    Description

      Attempting to create an XDCR remote cluster reference from 6.5.1 to 6.5.1 is no longer working when using full w/ certificates:

      $ couchbase-cli xdcr-setup -c 192.168.1.210:9000 -u couchbase -p couchbase --create --xdcr-cluster-name cluster_1_tls_certificates --xdcr-hostname 192.168.1.210:9001 --xdcr-secure-connection full --xdcr-certificate /Users/michaelblow/dev/master/target/sslwork/n_1/ca.pem --xdcr-user-certificate /Users/michaelblow/dev/master/target/sslwork/n_1/client.pem --xdcr-user-key /Users/michaelblow/dev/master/target/sslwork/n_1/client.key
      >> ERROR: _ - Invalid remote cluster. Failed to retrieve secruity settings from host=192.168.1.210:9001, err=tls: first record does not look like a TLS handshake, statusCode=0
      

      Retrying the operation repeatedly yields the same exception. Observed the following in the XDCR log on the source cluster:

      2020-03-04T13:34:37.925-05:00 INFO GOXDCR.AdminPort: doCreateRemoteClusterRequest req=&{POST /pools/default/remoteClusters HTTP/1.1 1 1 map[Accept:[*/*] Accept-Encoding:[gzip, deflate] Authorization:[Basic Y291Y2hiYXNlOmNvdWNoYmFzZQ==] Connection:[keep-alive] Content-Length:[4367] Content-Type:[application/x-www-form-urlencoded] Menelaus-Auth-Domain:[admin] Menelaus-Auth-User:[<ud>99</ud> <ud>111</ud> <ud>117</ud> <ud>99</ud> <ud>104</ud> <ud>98</ud> <ud>97</ud> <ud>115</ud> <ud>101</ud>] User-Agent:[couchbase-cli  6.5.1-0000]] 0xc0001c8b80 <nil> 4367 [] false 192.168.1.210:9000 map[] map[] <nil> map[] 127.0.0.1:61827 /pools/default/remoteClusters <nil> <nil> <nil> 0xc0001c8bc0}
      2020-03-04T13:34:37.932-05:00 INFO GOXDCR.AdminPort: Request params: justValidate=false, remoteClusterRef=id:remoteCluster/uZl-4QXnVJnkpMLRIHduSvx48xdMRVSWPYqPewfeqy0=; uuid:; name:cluster_1_tls_certificates; hostName:192.168.1.210:9001; userName:; password:; secureType:full; certificate:[60 117 100 62 45 45 45 45 45 66 69 71 73 78 32 67 69 82 84 73 70 73 67 65 84 69 45 45 45 45 45 10 77 73 73 67 116 68 67 67 65 90 119 67 67 81 68 99 87 118 80 115 98 100 102 101 67 122 65 78 66 103 107 113 104 107 105 71 57 119 48 66 65 81 115 70 65 68 65 99 77 82 111 119 71 65 89 68 86 81 81 68 68 66 70 68 10 98 51 86 106 97 71 74 104 99 50 85 103 85 109 57 118 100 67 66 68 81 84 65 101 70 119 48 121 77 68 65 122 77 68 81 120 79 68 77 48 77 68 90 97 70 119 48 121 77 68 65 122 77 68 85 120 79 68 77 48 77 68 90 97 10 77 66 119 120 71 106 65 89 66 103 78 86 66 65 77 77 69 85 78 118 100 87 78 111 89 109 70 122 90 83 66 83 98 50 57 48 73 69 78 66 77 73 73 66 73 106 65 78 66 103 107 113 104 107 105 71 57 119 48 66 65 81 69 70 10 65 65 79 67 65 81 56 65 77 73 73 66 67 103 75 67 65 81 69 65 116 84 112 85 107 88 113 116 71 72 67 121 53 76 47 113 72 85 81 119 52 110 110 90 109 43 54 78 113 67 107 101 112 81 79 110 75 71 99 120 115 109 73 99 10 54 88 48 107 114 117 77 69 65 72 50 111 99 103 105 78 86 120 77 68 90 55 72 99 101 81 78 66 48 70 111 73 56 121 56 122 109 54 83 122 110 90 119 80 85 114 77 111 75 105 102 47 50 72 76 116 82 50 48 69 115 119 85 116 10 83 65 88 121 49 111 50 118 81 106 102 86 107 104 75 97 80 99 75 68 49 57 122 76 57 84 77 48 119 43 97 90 52 73 48 116 55 78 70 118 120 108 107 74 65 82 85 75 99 120 85 72 51 66 75 122 51 90 80 80 51 70 90 66 10 88 52 110 115 79 65 78 48 82 55 108 72 115 98 52 102 53 87 47 65 47 54 65 113 54 104 110 85 74 76 82 72 113 73 70 75 116 119 66 67 79 54 57 70 78 74 103 55 75 100 78 66 52 98 102 104 97 84 78 100 118 80 74 113 10 55 81 121 55 85 78 103 114 51 115 50 103 109 107 114 53 113 86 108 117 51 48 86 119 111 65 99 70 97 48 74 68 116 71 52 114 65 53 65 84 104 50 88 72 114 90 111 43 80 48 120 108 77 78 87 90 69 98 113 47 101 120 80 48 10 114 97 98 67 54 100 76 110 55 43 53 119 106 113 75 102 110 100 54 48 107 52 66 65 114 107 114 114 67 57 104 48 72 118 69 71 79 113 79 100 111 81 73 68 65 81 65 66 77 65 48 71 67 83 113 71 83 73 98 51 68 81 69 66 10 67 119 85 65 65 52 73 66 65 81 66 88 72 120 112 89 99 67 101 100 65 80 97 79 80 107 85 68 111 79 83 52 109 77 57 78 83 90 79 43 109 110 84 90 85 107 83 52 51 53 49 78 67 56 86 52 50 86 83 50 108 71 112 69 10 66 109 121 108 103 98 107 108 121 107 104 79 117 111 48 82 112 116 121 102 122 83 72 82 103 88 99 119 112 74 75 53 117 114 118 56 115 89 106 118 98 81 103 82 77 82 66 43 100 113 109 80 120 74 49 119 69 84 120 48 86 107 73 101 10 67 75 114 55 74 84 87 47 78 100 104 47 43 43 43 112 85 99 69 55 107 69 119 102 108 83 110 102 112 121 117 74 47 75 107 109 101 83 43 82 55 56 78 66 111 99 67 97 112 116 86 43 109 113 100 86 48 65 106 51 113 83 111 65 10 107 104 101 54 71 85 86 106 49 50 115 88 99 52 73 103 49 115 72 48 119 104 65 74 81 90 78 55 82 84 52 67 105 78 82 48 107 84 78 67 82 97 117 84 109 119 98 112 118 118 77 107 88 103 80 108 100 85 115 102 106 85 109 49 10 81 120 79 77 76 68 79 100 87 105 50 80 100 83 108 122 89 54 54 88 114 118 53 108 107 99 99 69 108 49 56 116 120 108 121 54 100 88 54 43 80 111 100 108 76 121 112 76 51 84 47 103 52 112 79 67 117 53 108 55 77 76 87 118 10 99 117 80 109 110 122 88 51 56 56 88 121 73 104 106 117 121 118 65 43 97 77 70 82 79 120 84 80 87 113 65 53 10 45 45 45 45 45 69 78 68 32 67 69 82 84 73 70 73 67 65 84 69 45 45 45 45 45 10 60 47 117 100 62]; clientCertificate:[60 117 100 62 45 45 45 45 45 66 69 71 73 78 32 67 69 82 84 73 70 73 67 65 84 69 45 45 45 45 45 10 77 73 73 68 86 106 67 67 65 106 54 103 65 119 73 66 65 103 73 74 65 80 100 105 71 88 118 112 100 76 121 80 77 65 48 71 67 83 113 71 83 73 98 51 68 81 69 66 66 81 85 65 77 66 119 120 71 106 65 89 66 103 78 86 10 66 65 77 77 69 85 78 118 100 87 78 111 89 109 70 122 90 83 66 83 98 50 57 48 73 69 78 66 77 66 52 88 68 84 73 119 77 68 77 119 78 68 69 52 77 122 81 119 78 49 111 88 68 84 73 119 77 68 77 119 78 84 69 52 10 77 122 81 119 78 49 111 119 70 68 69 83 77 66 65 71 65 49 85 69 65 119 119 74 89 50 57 49 89 50 104 105 89 88 78 108 77 73 73 66 73 106 65 78 66 103 107 113 104 107 105 71 57 119 48 66 65 81 69 70 65 65 79 67 10 65 81 56 65 77 73 73 66 67 103 75 67 65 81 69 65 56 101 80 73 77 122 118 56 53 71 68 114 76 102 120 70 78 82 88 107 67 97 90 86 77 67 87 98 79 83 83 122 67 82 55 56 68 43 73 84 101 88 78 90 48 109 54 104 10 110 54 48 115 97 118 52 113 87 103 73 99 111 100 102 73 112 49 82 82 50 54 56 90 78 76 87 100 48 109 119 50 110 107 84 104 67 112 52 119 67 84 67 72 89 53 106 115 51 56 113 114 48 71 117 73 106 109 120 118 73 86 43 103 10 56 88 83 85 113 121 117 100 43 43 78 86 43 107 107 103 49 50 112 77 87 116 118 120 113 89 97 57 85 112 112 100 119 117 56 106 73 88 65 53 57 85 78 89 66 88 57 118 116 104 85 52 121 79 102 107 122 105 103 120 114 98 47 89 10 83 48 71 54 82 51 77 85 107 99 120 67 48 97 57 70 51 75 117 100 51 84 104 76 102 67 107 116 109 66 52 107 104 52 88 80 117 118 75 75 113 115 47 67 107 121 65 84 72 115 110 114 77 55 48 113 85 73 75 119 108 66 73 113 10 87 52 98 48 120 113 89 69 89 108 113 50 84 118 105 66 108 77 43 105 99 66 108 74 80 97 110 119 86 81 49 106 49 48 74 111 72 66 116 119 53 106 80 47 49 66 48 90 103 48 79 113 99 50 75 116 80 55 115 71 72 77 43 55 10 84 66 72 90 122 101 52 52 77 103 68 74 117 43 52 77 68 119 56 115 114 102 83 102 56 54 98 69 103 52 70 50 111 102 75 87 109 81 73 68 65 81 65 66 111 52 71 105 77 73 71 102 77 65 107 71 65 49 85 100 69 119 81 67 10 77 65 65 119 72 81 89 68 86 82 48 79 66 66 89 69 70 71 51 54 68 110 78 107 105 47 75 78 76 90 101 76 117 119 74 43 90 52 73 89 80 56 72 107 77 68 89 71 65 49 85 100 73 119 81 118 77 67 50 104 73 75 81 101 10 77 66 119 120 71 106 65 89 66 103 78 86 66 65 77 77 69 85 78 118 100 87 78 111 89 109 70 122 90 83 66 83 98 50 57 48 73 69 78 66 103 103 107 65 51 70 114 122 55 71 51 88 51 103 115 119 72 81 89 68 86 82 48 108 10 66 66 89 119 70 65 89 73 75 119 89 66 66 81 85 72 65 119 69 71 67 67 115 71 65 81 85 70 66 119 77 67 77 65 115 71 65 49 85 100 68 119 81 69 65 119 73 72 103 68 65 80 66 103 78 86 72 82 69 69 67 68 65 71 10 104 119 84 65 113 65 72 83 77 65 48 71 67 83 113 71 83 73 98 51 68 81 69 66 66 81 85 65 65 52 73 66 65 81 65 74 106 70 78 118 57 68 109 67 80 114 112 72 101 108 56 66 47 90 47 54 106 107 52 105 65 84 78 106 10 85 66 107 80 111 111 77 53 83 80 54 50 103 121 110 86 90 75 71 75 83 122 53 43 119 76 83 106 111 117 88 54 79 84 82 118 89 110 66 70 48 119 65 119 54 68 68 99 78 53 50 65 79 119 83 47 86 48 98 103 77 81 108 106 10 87 104 65 108 86 107 67 113 107 113 68 101 76 100 99 111 98 87 87 112 52 72 89 80 70 70 55 103 107 84 108 119 77 111 52 52 86 104 116 121 71 54 86 108 121 79 100 48 67 113 77 109 90 76 83 65 71 54 107 101 78 121 47 102 10 54 82 47 72 53 65 65 121 49 117 97 108 47 98 119 110 81 47 119 75 51 65 97 73 99 110 87 89 86 113 79 106 53 49 50 117 48 122 68 48 107 105 47 112 55 101 119 109 100 80 112 97 48 48 116 101 80 70 117 52 52 97 90 67 10 100 108 81 77 47 71 79 51 111 119 78 74 50 79 47 65 53 102 83 99 97 119 50 56 81 108 114 104 73 54 120 49 118 109 120 114 122 121 80 104 118 79 100 65 51 112 109 82 54 85 108 54 90 88 80 84 48 55 47 105 111 75 67 65 10 56 81 88 56 80 75 112 85 55 72 97 85 113 84 51 80 109 67 90 48 103 79 43 83 87 76 98 43 115 81 69 97 83 66 67 76 66 52 101 56 52 71 51 52 71 90 65 98 56 117 117 104 48 50 74 69 10 45 45 45 45 45 69 78 68 32 67 69 82 84 73 70 73 67 65 84 69 45 45 45 45 45 10 60 47 117 100 62]; clientKey:xxxx; SanInCertificate:false; HttpAuthMech:Plain, revision:<nil>
      2020-03-04T13:34:37.932-05:00 INFO GOXDCR.RemClusterSvc: Adding remote cluster with referenceId remoteCluster/uZl-4QXnVJnkpMLRIHduSvx48xdMRVSWPYqPewfeqy0=
      2020-03-04T13:34:37.941-05:00 ERRO GOXDCR.RemClusterSvc: TLS handshake failed when connecting to 192.168.1.210:9001, err=tls: first record does not look like a TLS handshake
      2020-03-04T13:34:37.941-05:00 ERRO GOXDCR.RemClusterSvc: Failed to get client for request, err=tls: first record does not look like a TLS handshake, req=&{GET https://192.168.1.210:9001/pools/default HTTP/1.1 1 1 map[Content-Type:[application/x-www-form-urlencoded] User-Agent:[couchbase-goxdcr/6.5.1]] {} 0x431acb0 0 [] false 192.168.1.210:9001 map[] map[] <nil> map[]   <nil> <nil> <nil> 0xc0000cc050}
      2020-03-04T13:34:37.941-05:00 INFO GOXDCR.AdminPort: Finished doCreateRemoteClusterRequest
      

      This is reliably failing at current 6.5.1 bits; seems to be a recent regression.

      Attachments

        Issue Links

          Activity

            People

              neil.huang Neil Huang
              neil.huang Neil Huang
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  PagerDuty