Details
-
Bug
-
Resolution: Fixed
-
Critical
-
6.0.0
-
centos longevity 1
-
Untriaged
-
-
No
Description
Build : 6.0.0-1564
Test : -test tests/integration/test_allFeatures_alice.yml -scope tests/integration/scope_Xattrs_Alice.yml
Scale : 3
Iteration : 1
Around the time of rebalance operations on KV nodes, the projector logs are showing some fatal errors. This does not impact the rebalance operation, which completes successful, it would be useful to see why these fatal errors are logged and is there any impact on indexer operations.
Logs from 172.23.108.104
2018-08-24T22:04:12.506-07:00 [Info] FEED[<=>MAINT_STREAM_TOPIC_55fd2942aee98ae8e79656ed033e2314(127.0.0.1:8091)] ##6 bucket default removed ...
|
2018-08-24T22:04:12.506-07:00 [Warn] FEED[<=>MAINT_STREAM_TOPIC_55fd2942aee98ae8e79656ed033e2314(127.0.0.1:8091)] ##6 control channel has 136 messages
|
2018-08-24T22:04:12.506-07:00 [Fatal] FEED[<=>MAINT_STREAM_TOPIC_55fd2942aee98ae8e79656ed033e2314(127.0.0.1:8091)] ##169 backch flush &{default 361 0 186 122230002950088 196356}: timestamp empty
|
2018-08-24T22:04:12.506-07:00 [Info] FEED[<=>MAINT_STREAM_TOPIC_55fd2942aee98ae8e79656ed033e2314(127.0.0.1:8091)] ##&{64656661756c74 169 0 ba 6f2ae3e81fc8 2ff04} backch flush success uint16: {controlStreamRequest, SUCCESS, default, 186, 6f2ae3e81fc8, 196356, ##169}
|
2018-08-24T22:04:12.506-07:00 [Fatal] FEED[<=>MAINT_STREAM_TOPIC_55fd2942aee98ae8e79656ed033e2314(127.0.0.1:8091)] ##169 backch flush &{default 361 0 187 240241410912602 139883}: timestamp empty
|
2018-08-24T22:04:12.506-07:00 [Info] FEED[<=>MAINT_STREAM_TOPIC_55fd2942aee98ae8e79656ed033e2314(127.0.0.1:8091)] ##&{64656661756c74 169 0 bb da7f8ff5315a 2226b} backch flush success uint16: {controlStreamRequest, SUCCESS, default, 187, da7f8ff5315a, 139883, ##169}
|
2018-08-24T22:04:12.506-07:00 [Fatal] FEED[<=>MAINT_STREAM_TOPIC_55fd2942aee98ae8e79656ed033e2314(127.0.0.1:8091)] ##169 backch flush &{default 361 0 188 36227555102096 139857}: timestamp empty
|
2018-08-24T22:04:12.506-07:00 [Info] FEED[<=>MAINT_STREAM_TOPIC_55fd2942aee98ae8e79656ed033e2314(127.0.0.1:8091)] ##&{64656661756c74 169 0 bc 20f2e28d8d90 22251} backch flush success uint16: {controlStreamRequest, SUCCESS, default, 188, 20f2e28d8d90, 139857, ##169}
|
2018-08-24T22:04:12.506-07:00 [Fatal] FEED[<=>MAINT_STREAM_TOPIC_55fd2942aee98ae8e79656ed033e2314(127.0.0.1:8091)] ##169 backch flush &{default 361 0 189 273194831544992 168365}: timestamp empty
|
2018-08-24T22:04:12.506-07:00 [Info] FEED[<=>MAINT_STREAM_TOPIC_55fd2942aee98ae8e79656ed033e2314(127.0.0.1:8091)] ##&{64656661756c74 169 0 bd f87820e382a0 291ad} backch flush success uint16: {controlStreamRequest, SUCCESS, default, 189, f87820e382a0, 168365, ##169}
|
2018-08-24T22:04:12.506-07:00 [Fatal] FEED[<=>MAINT_STREAM_TOPIC_55fd2942aee98ae8e79656ed033e2314(127.0.0.1:8091)] ##169 backch flush &{default 361 0 190 179610774497894 111534}: timestamp empty
|
2018-08-24T22:04:12.506-07:00 [Info] FEED[<=>MAINT_STREAM_TOPIC_55fd2942aee98ae8e79656ed033e2314(127.0.0.1:8091)] ##&{64656661756c74 169 0 be a35ae4696a66 1b3ae} backch flush success uint16: {controlStreamRequest, SUCCESS, default, 190, a35ae4696a66, 111534, ##169}
|
2018-08-24T22:04:12.506-07:00 [Fatal] FEED[<=>MAINT_STREAM_TOPIC_55fd2942aee98ae8e79656ed033e2314(127.0.0.1:8091)] ##169 backch flush &{default 361 0 191 31726640014927 139685}: timestamp empty
|
2018-08-24T22:04:12.506-07:00 [Info] FEED[<=>MAINT_STREAM_TOPIC_55fd2942aee98ae8e79656ed033e2314(127.0.0.1:8091)] ##&{64656661756c74 169 0 bf 1cdaef1ba64f 221a5} backch flush success uint16: {controlStreamRequest, SUCCESS, default, 191, 1cdaef1ba64f, 139685, ##169}
|
2018-08-24T22:04:12.506-07:00 [Fatal] FEED[<=>MAINT_STREAM_TOPIC_55fd2942aee98ae8e79656ed033e2314(127.0.0.1:8091)] ##169 backch flush &{default 361 0 192 47483853661319 83016}: timestamp empty
|
2018-08-24T22:04:12.506-07:00 [Info] FEED[<=>MAINT_STREAM_TOPIC_55fd2942aee98ae8e79656ed033e2314(127.0.0.1:8091)] ##&{64656661756c74 169 0 c0 2b2fb23ae487 14448} backch flush success uint16: {controlStreamRequest, SUCCESS, default, 192, 2b2fb23ae487, 83016, ##169}
|
Similar errors are seen in the projector logs on other KV nodes as well.