Details
Description
1. Add an analytics nodes to the KV cluster. Set some memory.
2. Setup analytics corresponding to KV buckets and make sure ingestion happens.
3. Now go to setting and increase the analytics memory.
4. Observe that analytics goes down.
Analytics service returned status: 500
Couple of observations from the logs:
1. as soon as the memory is reconfigured there is a restart of cbas
Logs:
2018-05-30T05:23:53.721-07:00 INFO CBAS.dataset.DatasetPartitionWriter [Executor-59:524d0e3c96191f0861b1c5f3ab0d2754] open(0)
|
[goport(/opt/couchbase/bin/cbas)] 2018/05/30 05:23:54 Timeout while flushing stderr
|
2:ok,
|
2018-05-30T05:23:55.160-07:00 INFO CBAS.cbas ++++++++++++ starting cbas main ++++++++++++
|
2018-05-30T05:23:55.160-07:00 INFO CBAS.cbas uuid: 524d0e3c96191f0861b1c5f3ab0d2754
|
2018-05-30T05:23:55.160-07:00 INFO CBAS.cbas A single io device was provided; auto-adjusting...
|
2018-05-30T05:23:55.160-07:00 INFO CBAS.cbas configuring io devices based on logical CPUs: 8
|
2018-05-30T05:23:55.160-07:00 INFO CBAS.cbas io devices were adjusted to a total of: 8
|
2. Incorrect partition mapping on nodes
2018-05-30T05:23:57.755-07:00 ERRO CBAS.control.AnalyticsDriver [main] Exiting AnalyticsDriver due to exception
org.apache.hyracks.api.exceptions.HyracksDataException: org.apache.asterix.common.exceptions.AsterixException: ASX4000: Partition id 2 for node 524d0e3c96191f0861b1c5f3ab0d2754 already in use by node 0cf19b03f954c859d1467a4b54d38e31
at org.apache.hyracks.api.exceptions.HyracksDataException.create(HyracksDataException.java:51) ~[hyracks-api.jar:5.5.0-2814]
at org.apache.asterix.common.config.AsterixProperties.lambda$registerConfigOptions$0(AsterixProperties.java:47) ~[asterix-common.jar:5.5.0-2814]
at org.apache.hyracks.control.common.config.ConfigManager.processConfig(ConfigManager.java:231) ~[hyracks-control-common.jar:5.5.0-2814]
at org.apache.hyracks.control.nc.NodeControllerService.<init>(NodeControllerService.java:200) ~[hyracks-control-nc.jar:5.5.0-2814]
at com.couchbase.analytics.control.AnalyticsDriver.createNcService(AnalyticsDriver.java:189) ~[cbas-server.jar:5.5.0-2814]
at com.couchbase.analytics.control.AnalyticsDriver.initNcService(AnalyticsDriver.java:141) ~[cbas-server.jar:5.5.0-2814]
at com.couchbase.analytics.control.AnalyticsDriver.main(AnalyticsDriver.java:73) [cbas-server.jar:5.5.0-2814]
Caused by: org.apache.asterix.common.exceptions.AsterixException: ASX4000: Partition id 2 for node 524d0e3c96191f0861b1c5f3ab0d2754 already in use by node 0cf19b03f954c859d1467a4b54d38e31
at org.apache.asterix.common.exceptions.AsterixException.create(AsterixException.java:64) ~[asterix-common.jar:5.5.0-2814]
at org.apache.asterix.common.config.PropertiesAccessor.configureNc(PropertiesAccessor.java:128) ~[asterix-common.jar:5.5.0-2814]
at org.apache.asterix.common.config.PropertiesAccessor.<init>(PropertiesAccessor.java:79) ~[asterix-common.jar:5.5.0-2814]
at org.apache.asterix.common.config.PropertiesAccessor.getInstance(PropertiesAccessor.java:202) ~[asterix-common.jar:5.5.0-2814]
at org.apache.asterix.common.config.AsterixProperties.lambda$registerConfigOptions$0(AsterixProperties.java:45) ~[asterix-common.jar:5.5.0-2814]
... 5 more
2018-05-30T05:23:57.826-07:00 INFO CBAS.cbas analytics driver exited; exiting cbas wrapper
2018-05-30T05:23:57.827-07:00 INFO CBAS.cbas ++++++++++++ exiting cbas main ++++++++++++
And after this analytics is restarting and crashing in a infinite loop.