Details
-
Bug
-
Resolution: Duplicate
-
Critical
-
Cheshire-Cat
-
Untriaged
-
1
-
Unknown
Description
Build : 7.0.0-3273
Test : ./testrunner -i node_conf.ini -p get-cbcollect-info=True,get-couch-dbinfo=True,skip_cleanup=False -t tuqquery.tuq_index.QueriesViewsTests.test_primary_create_delete_index,doc-per-day=3,force_clean=True,reload_data=False,nodes_init=1
Job : 1-node sanity (Test_14)
This looks like an intermittent issue when a new bucket is created and then an index is created on this bucket, the create primary index query times out. This is a single node cluster test.
[2020-10-01 15:52:17,252] - [tuq:1408] INFO - RUN QUERY CREATE PRIMARY INDEX ON `default` USING gsi
|
[2020-10-01 15:52:17,253] - [rest_client:3738] INFO - query params : statement=CREATE+PRIMARY+INDEX+ON+%60default%60+USING+gsi&scan_consistency=REQUEST_PLUS
|
[2020-10-01 16:13:57,288] - [rest_client:991] ERROR - socket error while connecting to http://172.23.104.78:8093/query?statement=CREATE+PRIMARY+INDEX+ON+%60default%60+USING+gsi&scan_consistency=REQUEST_PLUS error timed out
|
Indexer receives the Create Index notification
2020-10-01T15:52:17.546-07:00 [Info] clustMgrAgent::OnIndexCreate Notification Received for Create Index DefnId: 16646363056869198685 Name: #primary Using: plasma Bucket: default Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 0 InstVersion: 0
|
But then there are errors like these -
2020-10-01T15:52:18.492-07:00 [Info] Fetch new stats upon request by life cycle manager
|
2020-10-01T15:52:18.976-07:00 [Error] KVSender::openMutationStream INIT_STREAM default Error from Projector feed.clusterInfo
|
2020-10-01T15:52:18.976-07:00 [Error] Indexer::sendStreamUpdateForBuildIndex Stream INIT_STREAM KeyspaceId default SessionId 1. Error from Projector feed.clusterInfo. Retrying 1.
|
2020-10-01T15:52:24.066-07:00 [Info] KVSender::sendMutationTopicRequest Projector 127.0.0.1:9999 Topic INIT_STREAM_TOPIC_8a4915e3926bef1276ac982136ae1945 default
|
Instances [indexInstance:<instId:16248317553658887296 state:IndexPending definition:<defnID:16646363056869198685 bucket:default scope:_default collection:_default isPrimary:true name:#primary using:0 exprType:N1QL secExpressions:<ud>([])</ud> partitionScheme:SINGLE whereExpression:<ud>()</ud> > singlePartn:endpoints:"127.0.0.1:9103" > ]
|
2020-10-01T15:52:24.068-07:00 [Error] KVSender::sendMutationTopicRequest Projector 127.0.0.1:9999 Topic INIT_STREAM_TOPIC_8a4915e3926bef1276ac982136ae1945 default
|
Unexpected Error feed.clusterInfo
|
2020-10-01T15:52:24.068-07:00 [Error] KVSender::openMutationStream INIT_STREAM default Error Received feed.clusterInfo from 127.0.0.1:9999
|
2020-10-01T15:52:25.110-07:00 [Error] KVSender::openMutationStream INIT_STREAM default Error from Projector feed.clusterInfo
|
2020-10-01T15:52:25.110-07:00 [Error] Indexer::sendStreamUpdateForBuildIndex Stream INIT_STREAM KeyspaceId default SessionId 1. Error from Projector feed.clusterInfo. Retrying 2.
|
2020-10-01T15:52:30.206-07:00 [Info] KVSender::sendMutationTopicRequest Projector 127.0.0.1:9999 Topic INIT_STREAM_TOPIC_8a4915e3926bef1276ac982136ae1945 default
|
Instances [indexInstance:<instId:16248317553658887296 state:IndexPending definition:<defnID:16646363056869198685 bucket:default scope:_default collection:_default isPrimary:true name:#primary using:0 exprType:N1QL secExpressions:<ud>([])</ud> partitionScheme:SINGLE whereExpression:<ud>()</ud> > singlePartn:endpoints:"127.0.0.1:9103" > ]
|
2020-10-01T15:52:30.209-07:00 [Error] KVSender::sendMutationTopicRequest Projector 127.0.0.1:9999 Topic INIT_STREAM_TOPIC_8a4915e3926bef1276ac982136ae1945 default
|
Unexpected Error feed.clusterInfo
|
The following errors are seen in the projector logs :
2020-10-01T15:52:24.067-07:00 [Info] pram[:9999] Request "/adminport/mutationTopicRequest"
|
2020-10-01T15:52:24.067-07:00 [Info] PROJ[:9999] ##4 doMutationTopic() "INIT_STREAM_TOPIC_8a4915e3926bef1276ac982136ae1945"
|
2020-10-01T15:52:24.067-07:00 [Info] FEED[<=>INIT_STREAM_TOPIC_8a4915e3926bef1276ac982136ae1945(127.0.0.1:8091)] ##4 endpoint "127.0.0.1:9103" active ...
|
2020-10-01T15:52:24.068-07:00 [Error] FEED[<=>INIT_STREAM_TOPIC_8a4915e3926bef1276ac982136ae1945(127.0.0.1:8091)] ##4 cinfo.GetVBuckets(0, `default`): No bucket named default
|
2020-10-01T15:52:24.068-07:00 [Info] FEED[<=>INIT_STREAM_TOPIC_8a4915e3926bef1276ac982136ae1945(127.0.0.1:8091)] ##3 keyspace default removed ...
|
2020-10-01T15:52:24.068-07:00 [Info] PROJ[:9999] ##4 doMutationTopic() returns ...
|
2020-10-01T15:52:30.207-07:00 [Info] pram[:9999] Request "/adminport/mutationTopicRequest"
|
2020-10-01T15:52:30.208-07:00 [Info] PROJ[:9999] ##5 doMutationTopic() "INIT_STREAM_TOPIC_8a4915e3926bef1276ac982136ae1945"
|
2020-10-01T15:52:30.209-07:00 [Info] FEED[<=>INIT_STREAM_TOPIC_8a4915e3926bef1276ac982136ae1945(127.0.0.1:8091)] ##5 endpoint "127.0.0.1:9103" active ...
|
2020-10-01T15:52:30.209-07:00 [Error] FEED[<=>INIT_STREAM_TOPIC_8a4915e3926bef1276ac982136ae1945(127.0.0.1:8091)] ##5 cinfo.GetVBuckets(0, `default`): No bucket named default
|
2020-10-01T15:52:30.209-07:00 [Info] FEED[<=>INIT_STREAM_TOPIC_8a4915e3926bef1276ac982136ae1945(127.0.0.1:8091)] ##3 keyspace default removed ...
|
2020-10-01T15:52:30.209-07:00 [Info] PROJ[:9999] ##5 doMutationTopic() returns ...
|
Not sure if this is a regression, because this issue can be intermittently seen in the sanity jobs on a couple of platforms atleast since 7.0.0-3237.