Uploaded image for project: 'Couchbase Spark Connector'
  1. Couchbase Spark Connector
  2. SPARKC-26

Running Spark Streaming Application on spark standalone cluster mode

    XMLWordPrintable

Details

    • Bug
    • Resolution: Incomplete
    • Major
    • None
    • 1.0.0-dp2
    • Spark SQL, Spark Streaming
    • Centos 6.5, Java 1.8, Scala 2.10, spark 1.4.1, Couchbase-Java-Client 2.1.3

    Description

      I am using spark-couchbase connector dp2 in scala. I have spark streaming application which reads file streams from hadoop directory joins with data from another RDD created from textFile. I am using spark SQL over streaming. My application work fine and successfully load data into Couchbase from spark when it runs on standalone spark in local mode. But my requirement is to run my application on standalone spark cluster which currently has two nodes. When I tried to run it on cluster, it does all operations like aggregation and join successfully (I checked it by printing data after aggregation and join) but does not loads into couchbase and hangs instantly. It does not even reads data from next batch. Please have a look into the issue and let me know if this explains my problem or any other information is required from my side.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            daschl Michael Nitschinger
            librar127 librar127
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty