Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-25414

[FTS] system testing: cbft exited due to (possibly false positive) SSL-cert change event

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Critical
    • 5.0.0
    • 4.6.3, 5.0.0
    • fts

    Description

      Build

      5.0.0-3358

       

      Mihir spotted this error while running a 2i test with Sarath on the longevity cluster. From the error it appears to come from a n invalid json parse error. Also from the cluster, key 00000000000000000002 contains binary data.

      At this point this all the info I have -

      {code}

      [10:08 AM] Mihir Kamdar: I was helping Sarath to reproduce a 2i issue on the Centos system testing cluster, and I used the longevity test to setup the cluster, but then aborted after the data that I wanted got created. I saw an FTS crash in the logs so thought of letting you know if you want to investigate
      [10:08 AM] Mihir Kamdar: Service 'fts' exited with status 1. Restarting. Messages:
      2017-07-25T09:12:57.110-07:00 [INFO] bleve: json.Unmarshal, partition: 651, key: "00000000000000000002", seq: 152085, err: invalid character '#' looking for beginning of value
      2017-07-25T09:12:57.110-07:00 [INFO] bleve: json.Unmarshal, partition: 591, key: "00000000000000004105", seq: 101776, err: invalid character '#' looking for beginning of value
      2017-07-25T09:12:57.110-07:00 [INFO] main: ------------------------------------------------------
      2017-07-25T09:12:57.110-07:00 [INFO] init_http: web UI / REST API is available: https://:18094
      2017-07-25T09:12:57.110-07:00 [INFO] main: ------------------------------------------------------
      2017-07-25T09:12:57.111-07:00 [INFO] bleve: json.Unmarshal, partition: 153, key: "00000000000000000412", seq: 114444, err: invalid character '#' looking for beginning of value
      2017-07-25T09:12:57.111-07:00 [FATA] init_http: listen, err: listen tcp :18094: bind: address already in use – main.mainServeHTTP() at init_http.go:135
      [goport(/opt/couchbase/bin/cbft)] 2017/07/25 09:13:02 child process exited with status 1
      [10:08 AM] Mihir Kamdar: http://172.23.108.103:8091 is the master node
      [10:09 AM] Mihir Kamdar: pls let the system be in the same state since we are still working on the issue, but you can check this FTS issue
      [10:09 AM] Mihir Kamdar: thanks

      {code}

       

      Logs attached  - .188 and .242 are full text search nodes.

      The restart is on .188. https://s3.amazonaws.com/cb-engineering/Aruna/collectinfo-2017-07-25T180619-ns_1%40172.23.106.188.zip

      The log is filled with json Unmarshall errors that the log doesn't contain the context of the error anymore.

       

      Cbcollect from other nodes are also available.

       

       

       

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            mihir.kamdar Mihir Kamdar (Inactive)
            apiravi Aruna Piravi (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty