Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-42037

Internal error is raised while reconnecting remote link if remote server is shutdown

    XMLWordPrintable

    Details

    • Triage:
      Untriaged
    • Operating System:
      Centos 64-bit
    • Story Points:
      1
    • Is this a Regression?:
      No
    • Sprint:
      CX Sprint 221, CX Sprint 222, CX Sprint 223

      Description

      Steps to reproduce-

      1) Load travel-sample bucket in remote cluster.

      2) Create remote link from webUI.

      3) Create dataset on `travel-sample` bucket in remote cluster.

      4) Connect remote link.

      5) Bring down the remote server.

      6) Disconnect remote link.

      7) Try reconnecting remote link.

      Following error is raised - 

      [
        {
          "code": 22001,
          "msg": "Connect link failed {\"Default.abc.travel-sample\" : \"Internal error\"}",
          "query_from_user": "connect link abc;"
        }
      ]
      

      2020-10-17T12:33:28.479+00:00 WARN CBAS.apache.asterix [Executor-6:ClusterController] CBAS0029: Connect link failed {"Default.abc.travel-sample" : "Internal error"}
      com.couchbase.analytics.common.exceptions.AnalyticsHyracksException: CBAS0029: Connect link failed {"Default.abc.travel-sample" : "Internal error"}
              at com.couchbase.analytics.lang.ConnectLinkStatement.doHandle(ConnectLinkStatement.java:371) ~[cbas-connector.jar:6.6.1-9124]
              at com.couchbase.analytics.lang.ConnectionStatement.handle(ConnectionStatement.java:65) ~[cbas-connector.jar:6.6.1-9124]
              at org.apache.asterix.app.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:430) ~[asterix-app.jar:6.6.1-9124]
              at org.apache.asterix.app.message.ExecuteStatementRequestMessage.handle(ExecuteStatementRequestMessage.java:155) ~[asterix-app.jar:6.6.1-9124]
              at org.apache.asterix.messaging.CCMessageBroker.receivedMessage(CCMessageBroker.java:64) ~[asterix-app.jar:6.6.1-9124]
              at org.apache.hyracks.control.cc.work.ApplicationMessageWork.lambda$notifyMessageBroker$0(ApplicationMessageWork.java:68) ~[hyracks-control-cc.jar:6.6.1-9124]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?]
              at java.lang.Thread.run(Unknown Source) [?:?]
      2020-10-17T12:33:28.482+00:00 INFO CBAS.messaging.NCMessageBroker [Worker:c69e9d555004ff3faec46e4bb11b1d44] Received message: ExecuteStatementResponseMessage(id=134): 0 characters
      2020-10-17T12:33:28.482+00:00 WARN CBAS.server.QueryServiceServlet [HttpExecutor(port:8095)-0] handleException: unexpected exception CBAS0029: Connect link failed {"Default.abc.travel-sample" : "Internal error"}: <ud>{"host":"10.112.205.102:8091","path":"/query/service","statement":"connect link abc;","pretty":false,"mode":"immediate","clientContextID":"54df2870-30c9-4665-9aae-e55895f64376","format":"CLEAN_JSON","timeout":9223372036854775807,"maxResultReads":1,"planFormat":"JSON","expressionTree":false,"rewrittenExpressionTree":false,"logicalPlan":false,"optimizedLogicalPlan":true,"job":false,"profile":"counts","signature":true,"multiStatement":false,"parseOnly":false,"readOnly":false,"maxWarnings":10,"scanConsistency":"not_bounded","scanWait":null}</ud>
      com.couchbase.analytics.common.exceptions.AnalyticsHyracksException: CBAS0029: Connect link failed {"Default.abc.travel-sample" : "Internal error"}
              at com.couchbase.analytics.lang.ConnectLinkStatement.doHandle(ConnectLinkStatement.java:371) ~[cbas-connector.jar:6.6.1-9124]
              at com.couchbase.analytics.lang.ConnectionStatement.handle(ConnectionStatement.java:65) ~[cbas-connector.jar:6.6.1-9124]
              at org.apache.asterix.app.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:430) ~[asterix-app.jar:6.6.1-9124]
              at org.apache.asterix.app.message.ExecuteStatementRequestMessage.handle(ExecuteStatementRequestMessage.java:155) ~[asterix-app.jar:6.6.1-9124]
              at org.apache.asterix.messaging.CCMessageBroker.receivedMessage(CCMessageBroker.java:64) ~[asterix-app.jar:6.6.1-9124]
              at org.apache.hyracks.control.cc.work.ApplicationMessageWork.lambda$notifyMessageBroker$0(ApplicationMessageWork.java:68) ~[hyracks-control-cc.jar:6.6.1-9124]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?]
              at java.lang.Thread.run(Unknown Source) [?:?]
      

        Attachments

        1. internal_err.zip
          10.18 MB
        2. internal_err.zip
          83.64 MB

          Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

            Activity

            umang.agrawal Umang created issue -
            till Till Westmann made changes -
            Field Original Value New Value
            Rank Ranked higher
            till Till Westmann made changes -
            Rank Ranked higher
            till Till Westmann made changes -
            Labels analytics analytics triaged
            till Till Westmann made changes -
            Assignee Till Westmann [ till ] Hussain Towaileb [ hussain.towaileb ]
            Hussain.Towaileb Hussain Towaileb made changes -
            Sprint CX Sprint 221 [ 1271 ]
            Hussain.Towaileb Hussain Towaileb made changes -
            Rank Ranked lower
            Hussain.Towaileb Hussain Towaileb made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            Hussain.Towaileb Hussain Towaileb made changes -
            Rank Ranked higher
            umang.agrawal Umang made changes -
            Description Steps to reproduce-

            1) Create remote link.

            2) Connect remote link.

            3) Bring down the remote server.

            4) Disconnect remote link.

            5) Try reconnecting remote link.

            Following error is raised - 
            {code:java}
            [ { "code": 22001, "msg": "Connect link failed {\"Default.cblink.travel-sample\" : \"Internal error\"}", "query_from_user": "connect link cblink;" } ]
            {code}
            Steps to reproduce-

            1) Load travel-sample bucket in remote cluster.

            2) Create remote link from webUI.

            3) Create dataset on `travel-sample` bucket in remote cluster.

            4) Connect remote link.

            5) Bring down the remote server.

            6) Disconnect remote link.

            7) Try reconnecting remote link.

            Following error is raised - 
            {code:java}[
              {
                "code": 22001,
                "msg": "Connect link failed {\"Default.abc.travel-sample\" : \"Internal error\"}",
                "query_from_user": "connect link abc;"
              }
            ]
            {code}
            {code:java}2020-10-17T12:33:28.479+00:00 WARN CBAS.apache.asterix [Executor-6:ClusterController] CBAS0029: Connect link failed {"Default.abc.travel-sample" : "Internal error"}
            com.couchbase.analytics.common.exceptions.AnalyticsHyracksException: CBAS0029: Connect link failed {"Default.abc.travel-sample" : "Internal error"}
                    at com.couchbase.analytics.lang.ConnectLinkStatement.doHandle(ConnectLinkStatement.java:371) ~[cbas-connector.jar:6.6.1-9124]
                    at com.couchbase.analytics.lang.ConnectionStatement.handle(ConnectionStatement.java:65) ~[cbas-connector.jar:6.6.1-9124]
                    at org.apache.asterix.app.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:430) ~[asterix-app.jar:6.6.1-9124]
                    at org.apache.asterix.app.message.ExecuteStatementRequestMessage.handle(ExecuteStatementRequestMessage.java:155) ~[asterix-app.jar:6.6.1-9124]
                    at org.apache.asterix.messaging.CCMessageBroker.receivedMessage(CCMessageBroker.java:64) ~[asterix-app.jar:6.6.1-9124]
                    at org.apache.hyracks.control.cc.work.ApplicationMessageWork.lambda$notifyMessageBroker$0(ApplicationMessageWork.java:68) ~[hyracks-control-cc.jar:6.6.1-9124]
                    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?]
                    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?]
                    at java.lang.Thread.run(Unknown Source) [?:?]
            2020-10-17T12:33:28.482+00:00 INFO CBAS.messaging.NCMessageBroker [Worker:c69e9d555004ff3faec46e4bb11b1d44] Received message: ExecuteStatementResponseMessage(id=134): 0 characters
            2020-10-17T12:33:28.482+00:00 WARN CBAS.server.QueryServiceServlet [HttpExecutor(port:8095)-0] handleException: unexpected exception CBAS0029: Connect link failed {"Default.abc.travel-sample" : "Internal error"}: <ud>{"host":"10.112.205.102:8091","path":"/query/service","statement":"connect link abc;","pretty":false,"mode":"immediate","clientContextID":"54df2870-30c9-4665-9aae-e55895f64376","format":"CLEAN_JSON","timeout":9223372036854775807,"maxResultReads":1,"planFormat":"JSON","expressionTree":false,"rewrittenExpressionTree":false,"logicalPlan":false,"optimizedLogicalPlan":true,"job":false,"profile":"counts","signature":true,"multiStatement":false,"parseOnly":false,"readOnly":false,"maxWarnings":10,"scanConsistency":"not_bounded","scanWait":null}</ud>
            com.couchbase.analytics.common.exceptions.AnalyticsHyracksException: CBAS0029: Connect link failed {"Default.abc.travel-sample" : "Internal error"}
                    at com.couchbase.analytics.lang.ConnectLinkStatement.doHandle(ConnectLinkStatement.java:371) ~[cbas-connector.jar:6.6.1-9124]
                    at com.couchbase.analytics.lang.ConnectionStatement.handle(ConnectionStatement.java:65) ~[cbas-connector.jar:6.6.1-9124]
                    at org.apache.asterix.app.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:430) ~[asterix-app.jar:6.6.1-9124]
                    at org.apache.asterix.app.message.ExecuteStatementRequestMessage.handle(ExecuteStatementRequestMessage.java:155) ~[asterix-app.jar:6.6.1-9124]
                    at org.apache.asterix.messaging.CCMessageBroker.receivedMessage(CCMessageBroker.java:64) ~[asterix-app.jar:6.6.1-9124]
                    at org.apache.hyracks.control.cc.work.ApplicationMessageWork.lambda$notifyMessageBroker$0(ApplicationMessageWork.java:68) ~[hyracks-control-cc.jar:6.6.1-9124]
                    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?]
                    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?]
                    at java.lang.Thread.run(Unknown Source) [?:?]
            {code}
            umang.agrawal Umang made changes -
            Attachment internal_err.zip [ 111421 ]
            umang.agrawal Umang made changes -
            Attachment internal_err.zip [ 112175 ]
            till Till Westmann made changes -
            Sprint CX Sprint 221 [ 1271 ] CX Sprint 221, CX Sprint 222 [ 1271, 1288 ]
            till Till Westmann made changes -
            Link This issue blocks MB-40528 [ MB-40528 ]
            till Till Westmann made changes -
            Labels analytics triaged analytics approved-for-6.6.1 triaged
            Hussain.Towaileb Hussain Towaileb made changes -
            Resolution Fixed [ 1 ]
            Status In Progress [ 3 ] Resolved [ 5 ]
            umang.agrawal Umang made changes -
            Resolution Fixed [ 1 ]
            Status Resolved [ 5 ] Reopened [ 4 ]
            umang.agrawal Umang made changes -
            Attachment internal_err.zip [ 112971 ]
            Hussain.Towaileb Hussain Towaileb made changes -
            Status Reopened [ 4 ] In Progress [ 3 ]
            Hussain.Towaileb Hussain Towaileb made changes -
            Rank Ranked higher
            till Till Westmann made changes -
            Link This issue relates to MB-42277 [ MB-42277 ]
            Hussain.Towaileb Hussain Towaileb made changes -
            Resolution Fixed [ 1 ]
            Status In Progress [ 3 ] Resolved [ 5 ]
            michael.blow Michael Blow made changes -
            Assignee Hussain Towaileb [ hussain.towaileb ] Michael Blow [ michael.blow ]
            Resolution Fixed [ 1 ]
            Status Resolved [ 5 ] Reopened [ 4 ]
            michael.blow Michael Blow made changes -
            Sprint CX Sprint 221, CX Sprint 222 [ 1271, 1288 ] CX Sprint 221, CX Sprint 222, CX Sprint 223 [ 1271, 1288, 1291 ]
            michael.blow Michael Blow made changes -
            Status Reopened [ 4 ] In Progress [ 3 ]
            michael.blow Michael Blow made changes -
            Resolution Fixed [ 1 ]
            Status In Progress [ 3 ] Resolved [ 5 ]
            umang.agrawal Umang made changes -
            Assignee Michael Blow [ michael.blow ] Umang [ JIRAUSER24787 ]
            Status Resolved [ 5 ] Closed [ 6 ]
            michael.blow Michael Blow made changes -
            Rank Ranked higher

              People

              Assignee:
              umang.agrawal Umang
              Reporter:
              umang.agrawal Umang
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Gerrit Reviews

                  There are no open Gerrit changes

                    PagerDuty