Details
-
Bug
-
Resolution: Fixed
-
Critical
-
7.1.0
-
Untriaged
-
1
-
Yes
Description
Build : 7.1.0-1383
In the GSI tests in build sanity, there is a step to set a config using the 9102/internal/settings endpoint. There is a failure while doing this step.
[2021-09-29 17:15:43,412] - [rest_client:1055] ERROR - POST http://172.23.121.67:9102/internal/settings body: {"indexer.api.enableTestServer": true} headers: {'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': 'Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA==', 'Accept': '*/*'} error: 400 reason: status: 400, content: b'Get "http://127.0.0.1:8091/_metakv/indexing/settings/config": dial tcp 127.0.0.1:8091: connect: connection refused\n' b'Get "http://127.0.0.1:8091/_metakv/indexing/settings/config": dial tcp 127.0.0.1:8091: connect: connection refused\n' auth: xxxx:xxxx
|
The following can be seen in the debug log at the time :
2021-09-29 17:15:29.407467 : test_multi_create_query_explain_drop_index started
|
[ns_server:debug,2021-09-29T17:15:37.380-07:00,ns_1@172.23.121.67:ns_config_log<0.257.0>:ns_config_log:log_common:242]config change:
|
{metakv,<<"/indexing/settings/config">>} ->
|
[{'_vclock',[{<<"05221a7c8a71e76a280898e2111fb747">>,{5,63800180137}}]}|
|
<<"{\"indexer.settings.allow_large_keys\":true,\"indexer.settings.bufferPoolBlockSize\":16384,\"indexer.settings.build.batch_size\":5,\"indexer.settings.compaction.abort_exceed_interval\":false,\"indexer.settings.compaction.check_period\":30,\"indexer.settings.compaction.compaction_mode\":\"circular\",\"indexer.settings.compaction.days_of_week\":\"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday\","...>>]
|
[ns_server:debug,2021-09-29T17:15:37.380-07:00,ns_1@172.23.121.67:ns_config_log<0.257.0>:ns_config_log:log_common:242]config change:
|
{local_changes_count,<<"05221a7c8a71e76a280898e2111fb747">>} ->
|
[{'_vclock',[{<<"05221a7c8a71e76a280898e2111fb747">>,{36,63800180137}}]}]
|
[ns_server:debug,2021-09-29T17:15:37.380-07:00,ns_1@172.23.121.67:ns_config_rep<0.443.0>:ns_config_rep:do_push_keys:346]Replicating some config keys ([{local_changes_count,
|
<<"05221a7c8a71e76a280898e2111fb747">>},
|
{metakv,<<"/indexing/settings/config">>}]..)
|
[error_logger:error,2021-09-29T17:15:37.388-07:00,ns_1@172.23.121.67:index_settings_manager<0.487.0>:ale_error_logger_handler:do_log:101]
|
=========================ERROR REPORT=========================
|
** Generic server index_settings_manager terminating
|
** Last message in was {'$gen_cast',#Fun<json_settings_manager.5.36335668>}
|
** When Server state == []
|
** Reason for termination ==
|
** {badarg,[{dict,fetch,
|
[<<"indexer.settings.enable_page_bloom_filter">>,
|
{dict,66,16,16,8,80,48,
|
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},
|
{{[[<<"projector.settings.log_level">>|<<"info">>]],
|
[[<<"indexer.settings.compaction.days_of_week">>|
|
<<"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday">>],
|
[<<"indexer.settings.eTagPeriod">>|240],
|
[<<"indexer.settings.gc_percent">>|100],
|
[<<"indexer.settings.rebalance.redistribute_indexes">>|
|
false],
|
[<<"indexer.settings.scan_getseqnos_retries">>|30],
|
[<<"indexer.settings.snapshotListeners">>|2],
|
[<<"queryport.client.settings.relConnBatchSize">>|
|
100]],
|
[[<<"indexer.settings.compaction.interval">>|
|
<<"00:00,00:00">>],
|
[<<"indexer.settings.compaction.plasma.optional.min_frag">>|
|
20],
|
[<<"indexer.settings.send_buffer_size">>|1024]],
|
[[<<"indexer.settings.compaction.compaction_mode">>|
|
<<"circular">>],
|
[<<"indexer.settings.log_level">>|<<"info">>],
|
[<<"indexer.settings.persisted_snapshot.interval">>|
|
5000],
|
[<<"indexer.settings.wal_size">>|4096],
|
[<<"queryport.client.settings.minPoolSizeWM">>|
|
1000]],
|
[[<<"indexer.settings.compaction.min_frag">>|30],
|
[<<"indexer.settings.scan_timeout">>|120000],
|
[<<"queryport.client.settings.poolSize">>|5000]],
|
[[<<"indexer.settings.compaction.check_period">>|
|
30],
|
[<<"indexer.settings.fast_flush_mode">>|true],
|
[<<"indexer.settings.inmemory_snapshot.interval">>|
|
200],
|
[<<"indexer.settings.plasma.recovery.max_rollbacks">>|
|
2]],
|
[[<<"indexer.settings.compaction.plasma.optional.quota">>|
|
25],
|
[<<"indexer.settings.memProfDir">>|<<>>],
|
[<<"indexer.settings.moi.debug">>|false],
|
[<<"indexer.settings.moi.persistence_threads">>|
|
2]],
|
[[<<"indexer.settings.allow_large_keys">>|true],
|
[<<"indexer.settings.cpuProfDir">>|<<>>],
|
[<<"indexer.settings.max_array_seckey_size">>|
|
10240],
|
[<<"indexer.settings.max_cpu_percent">>|0],
|
[<<"indexer.settings.max_seckey_size">>|4608],
|
[<<"indexer.settings.sliceBufSize">>|800],
|
[<<"indexer.settings.smallSnapshotThreshold">>|30],
|
[<<"indexer.settings.storage_mode.disable_upgrade">>|
|
false]],
|
[[<<"indexer.settings.persisted_snapshot_init_build.interval">>|
|
5000],
|
[<<"indexer.settings.storage_mode">>|
|
<<"memory_optimized">>]],
|
[[<<"indexer.settings.enable_corrupt_index_backup">>|
|
false],
|
[<<"indexer.settings.max_writer_lock_prob">>|20],
|
[<<"indexer.settings.moi.recovery.max_rollbacks">>|
|
2],
|
[<<"indexer.settings.persisted_snapshot.fdb.interval">>|
|
5000]],
|
[[<<"indexer.settings.bufferPoolBlockSize">>|16384],
|
[<<"indexer.settings.compaction.plasma.manual">>|
|
false],
|
[<<"indexer.settings.persisted_snapshot.moi.interval">>|
|
600000],
|
[<<"indexer.settings.recovery.max_rollbacks">>|2]],
|
[[<<"indexer.settings.build.batch_size">>|5],
|
[<<"indexer.settings.corrupt_index_num_backups">>|
|
1],
|
[<<"indexer.settings.inmemory_snapshot.fdb.interval">>|
|
200],
|
[<<"indexer.settings.memProfile">>|false],
|
[<<"indexer.settings.memory_quota">>|268435456],
|
[<<"indexer.settings.moi.recovery_threads">>|4],
|
[<<"indexer.settings.num_replica">>|0],
|
[<<"indexer.settings.statsLogDumpInterval">>|60],
|
[<<"queryport.client.settings.backfillLimit">>|
|
5120]],
|
[[<<"indexer.settings.cpuProfile">>|false],
|
[<<"indexer.settings.inmemory_snapshot.moi.interval">>|
|
10],
|
[<<"indexer.settings.maxVbQueueLength">>|0]],
|
[[<<"indexer.settings.compaction.abort_exceed_interval">>|
|
false],
|
[<<"indexer.settings.compaction.plasma.optional.decrement">>|
|
5],
|
[<<"indexer.settings.snapshotRequestWorkers">>|2],
|
[<<"queryport.client.settings.poolOverflow">>|30]],
|
[[<<"indexer.settings.minVbQueueLength">>|250],
|
[<<"indexer.settings.persisted_snapshot_init_build.fdb.interval">>|
|
5000]],
|
[[<<"indexer.settings.compaction.min_size">>|
|
524288000],
|
[<<"indexer.settings.largeSnapshotThreshold">>|
|
200],
|
[<<"indexer.settings.persisted_snapshot_init_build.moi.interval">>|
|
600000]]}}}],
|
[{file,"dict.erl"},{line,131}]},
|
{json_settings_manager,'-lens_get_many/2-lc$^0/1-0-',2,
|
[{file,"src/json_settings_manager.erl"},
|
{line,186}]},
|
{json_settings_manager,'-lens_get_many/2-lc$^0/1-0-',2,
|
[{file,"src/json_settings_manager.erl"},
|
{line,186}]},
|
{json_settings_manager,'-lens_get_many/2-lc$^0/1-0-',2,
|
[{file,"src/json_settings_manager.erl"},
|
{line,186}]},
|
{json_settings_manager,'-lens_get_many/2-lc$^0/1-0-',2,
|
[{file,"src/json_settings_manager.erl"},
|
{line,186}]},
|
{json_settings_manager,do_populate_ets_table,3,
|
[{file,"src/json_settings_manager.erl"},
|
{line,169}]},
|
{work_queue,handle_cast,2,[{file,"src/work_queue.erl"},{line,55}]},
|
{gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,637}]}]}
|
|
|
[error_logger:error,2021-09-29T17:15:37.398-07:00,ns_1@172.23.121.67:index_settings_manager<0.487.0>:ale_error_logger_handler:do_log:101]
|
=========================CRASH REPORT=========================
|
crasher:
|
initial call: work_queue:init/1
|
pid: <0.487.0>
|
registered_name: index_settings_manager
|
exception error: bad argument
|
in function dict:fetch/2
|
called as dict:fetch(<<"indexer.settings.enable_page_bloom_filter">>,
|
{dict,66,16,16,8,80,48,
|
{[],[],[],[],[],[],[],[],[],[],[],[],[],
|
[],[],[]},
|
{{[[<<"projector.settings.log_level">>|
|
<<"info">>]],
|
[[<<"indexer.settings.compaction.days_of_week">>|
|
<<"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday">>],
|
[<<"indexer.settings.eTagPeriod">>|240],
|
[<<"indexer.settings.gc_percent">>|100],
|
[<<"indexer.settings.rebalance.redistribute_indexes">>|
|
false],
|
[<<"indexer.settings.scan_getseqnos_retries">>|
|
30],
|
[<<"indexer.settings.snapshotListeners">>|
|
2],
|
[<<"queryport.client.settings.relConnBatchSize">>|
|
100]],
|
[[<<"indexer.settings.compaction.interval">>|
|
<<"00:00,00:00">>],
|
[<<"indexer.settings.compaction.plasma.optional.min_frag">>|
|
20],
|
[<<"indexer.settings.send_buffer_size">>|
|
1024]],
|
[[<<"indexer.settings.compaction.compaction_mode">>|
|
<<"circular">>],
|
[<<"indexer.settings.log_level">>|
|
<<"info">>],
|
[<<"indexer.settings.persisted_snapshot.interval">>|
|
5000],
|
[<<"indexer.settings.wal_size">>|4096],
|
[<<"queryport.client.settings.minPoolSizeWM">>|
|
1000]],
|
[[<<"indexer.settings.compaction.min_frag">>|
|
30],
|
[<<"indexer.settings.scan_timeout">>|
|
120000],
|
[<<"queryport.client.settings.poolSize">>|
|
5000]],
|
[[<<"indexer.settings.compaction.check_period">>|
|
30],
|
[<<"indexer.settings.fast_flush_mode">>|
|
true],
|
[<<"indexer.settings.inmemory_snapshot.interval">>|
|
200],
|
[<<"indexer.settings.plasma.recovery.max_rollbacks">>|
|
2]],
|
[[<<"indexer.settings.compaction.plasma.optional.quota">>|
|
25],
|
[<<"indexer.settings.memProfDir">>|
|
<<>>],
|
[<<"indexer.settings.moi.debug">>|
|
false],
|
[<<"indexer.settings.moi.persistence_threads">>|
|
2]],
|
[[<<"indexer.settings.allow_large_keys">>|
|
true],
|
[<<"indexer.settings.cpuProfDir">>|
|
<<>>],
|
[<<"indexer.settings.max_array_seckey_size">>|
|
10240],
|
[<<"indexer.settings.max_cpu_percent">>|
|
0],
|
[<<"indexer.settings.max_seckey_size">>|
|
4608],
|
[<<"indexer.settings.sliceBufSize">>|
|
800],
|
[<<"indexer.settings.smallSnapshotThreshold">>|
|
30],
|
[<<"indexer.settings.storage_mode.disable_upgrade">>|
|
false]],
|
[[<<"indexer.settings.persisted_snapshot_init_build.interval">>|
|
5000],
|
[<<"indexer.settings.storage_mode">>|
|
<<"memory_optimized">>]],
|
[[<<"indexer.settings.enable_corrupt_index_backup">>|
|
false],
|
[<<"indexer.settings.max_writer_lock_prob">>|
|
20],
|
[<<"indexer.settings.moi.recovery.max_rollbacks">>|
|
2],
|
[<<"indexer.settings.persisted_snapshot.fdb.interval">>|
|
5000]],
|
[[<<"indexer.settings.bufferPoolBlockSize">>|
|
16384],
|
[<<"indexer.settings.compaction.plasma.manual">>|
|
false],
|
[<<"indexer.settings.persisted_snapshot.moi.interval">>|
|
600000],
|
[<<"indexer.settings.recovery.max_rollbacks">>|
|
2]],
|
[[<<"indexer.settings.build.batch_size">>|
|
5],
|
[<<"indexer.settings.corrupt_index_num_backups">>|
|
1],
|
[<<"indexer.settings.inmemory_snapshot.fdb.interval">>|
|
200],
|
[<<"indexer.settings.memProfile">>|
|
false],
|
[<<"indexer.settings.memory_quota">>|
|
268435456],
|
[<<"indexer.settings.moi.recovery_threads">>|
|
4],
|
[<<"indexer.settings.num_replica">>|0],
|
[<<"indexer.settings.statsLogDumpInterval">>|
|
60],
|
[<<"queryport.client.settings.backfillLimit">>|
|
5120]],
|
[[<<"indexer.settings.cpuProfile">>|
|
false],
|
[<<"indexer.settings.inmemory_snapshot.moi.interval">>|
|
10],
|
[<<"indexer.settings.maxVbQueueLength">>|
|
0]],
|
[[<<"indexer.settings.compaction.abort_exceed_interval">>|
|
false],
|
[<<"indexer.settings.compaction.plasma.optional.decrement">>|
|
5],
|
[<<"indexer.settings.snapshotRequestWorkers">>|
|
2],
|
[<<"queryport.client.settings.poolOverflow">>|
|
30]],
|
[[<<"indexer.settings.minVbQueueLength">>|
|
250],
|
[<<"indexer.settings.persisted_snapshot_init_build.fdb.interval">>|
|
5000]],
|
[[<<"indexer.settings.compaction.min_size">>|
|
524288000],
|
[<<"indexer.settings.largeSnapshotThreshold">>|
|
200],
|
[<<"indexer.settings.persisted_snapshot_init_build.moi.interval">>|
|
600000]]}}})
|
in call from json_settings_manager:'-lens_get_many/2-lc$^0/1-0-'/2 (src/json_settings_manager.erl, line 186)
|
in call from json_settings_manager:'-lens_get_many/2-lc$^0/1-0-'/2 (src/json_settings_manager.erl, line 186)
|
in call from json_settings_manager:'-lens_get_many/2-lc$^0/1-0-'/2 (src/json_settings_manager.erl, line 186)
|
in call from json_settings_manager:'-lens_get_many/2-lc$^0/1-0-'/2 (src/json_settings_manager.erl, line 186)
|
in call from json_settings_manager:do_populate_ets_table/3 (src/json_settings_manager.erl, line 169)
|
in call from work_queue:handle_cast/2 (src/work_queue.erl, line 55)
|
in call from gen_server:try_dispatch/4 (gen_server.erl, line 637)
|
ancestors: [ns_server_sup,ns_server_nodes_sup,<0.263.0>,
|
ns_server_cluster_sup,root_sup,<0.141.0>]
|
message_queue_len: 0
|
messages: []
|
links: [<0.401.0>,<0.488.0>]
|
dictionary: [{prev_json,<<"{\"indexer.settings.compaction.days_of_week\":\"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday\",\"indexer.settings.rebalance.redistribute_indexes\":false,\"indexer.settings.compact"...>>}]
|
trap_exit: false
|
status: running
|
heap_size: 4185
|
stack_size: 27
|
reductions: 165724
|
neighbours:
|
|
This is a regression since 7.1.0-1376, where these tests worked fine. This is due to https://issues.couchbase.com/browse/MB-47195.
Attachments
| For Gerrit Dashboard: MB-48679 | ||||||
|---|---|---|---|---|---|---|
| # | Subject | Branch | Project | Status | CR | V |
| 162711,3 | Revert "MB-47195 Add enablePageBloomFilter to index settings" | master | ns_server | Status: MERGED | +2 | +1 |
| 162712,2 | Revert "MB-47195 Add enablePageBloomFilter during upgrade" | master | ns_server | Status: MERGED | +2 | +1 |