Details
-
Improvement
-
Resolution: Fixed
-
Critical
-
None
-
2.2.0
-
None
Description
Some spark configuration parameters for couchbase need to be specific during Spark startup and, apparently, not in your custom app code. I've done this using command line --conf items.
t's not clear in our doc code samples how to do this - instead, new users just get a "cannot connect to default bucket" even if they set SparkConf in their code it is ignored.
This option is really important to note in docs because online systems like databricks require it too because they spin up spark outside of the users code.
Need to clarify why/where this happens and how to properly set it.
Here is an example startup command line Sachin Smotra and I were working on that got him over the confusion...
{{./bin/spark-submit -class sachin.SimpleTest --master local[4] -conf spark.couchbase.bucket.travel-sample=password --conf com.couchbase.username=travel-sample /Users/sachin.smotra/Documents/couchbase/dev/eclipse-workspace/sachin/target/sachin-0.0.1-SNAPSHOT.jar }}