Uploaded image for project: 'Couchbase Spark Connector'
  1. Couchbase Spark Connector
  2. SPARKC-87

add command line config parameter details

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Fixed
    • Critical
    • None
    • 2.2.0
    • Spark SQL
    • None

    Description

      Some spark configuration parameters for couchbase need to be specific during Spark startup and, apparently, not in your custom app code.  I've done this using command line --conf items. 

      t's not clear in our doc code samples how to do this - instead, new users just get a "cannot connect to default bucket" even if they set SparkConf in their code it is ignored.

      This option is really important to note in docs because online systems like databricks require it too because they spin up spark outside of the users code.

      Need to clarify why/where this happens and how to properly set it.

      Here is an example startup command line Sachin Smotra and I were working on that got him over the confusion...

      {{./bin/spark-submit  -class sachin.SimpleTest --master local[4] -conf spark.couchbase.bucket.travel-sample=password --conf com.couchbase.username=travel-sample  /Users/sachin.smotra/Documents/couchbase/dev/eclipse-workspace/sachin/target/sachin-0.0.1-SNAPSHOT.jar }}

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            graham.pople Graham Pople
            tyler.mitchell Tyler Mitchell
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty