Details
-
Bug
-
Resolution: Fixed
-
Critical
-
4.5.0
-
Untriaged
-
Unknown
Description
Build
4.5.0-2478
Testcase
./testrunner -i INI_FILE.ini -p get-cbcollect-info=True,get-coredumps=True,get-logs=False,stop-on-failure=False,GROUP=P1 -t fts.stable_topology_fts.StableTopFTS.index_query_in_parallel,items=1000000,num_queries=200,query_max_matches=10000,GROUP=P1
Testlog - https://gist.github.com/arunapiravi/43cb9e413aafde826adb5cfc19b5de03
[2016-05-09 00:55:08,030] - [task:1099] INFO - ------------------------------------------------------------------ Query # 90 -----------------------------------------------------------------
|
[2016-05-09 00:55:08,045] - [fts_base:1132] INFO - Running query {"from": 0, "indexName": "default_index", "fields": [], "explain": false, "ctl": {"timeout": 60000, "consistency": {"vectors": {}, "level": ""}}, "query": {"field": "email", "match": "gallia@mcdiabetes.com"}, "size": 10000} on node: 172.23.107.234:
|
[2016-05-09 00:56:08,365] - [rest_client:775] ERROR - http://172.23.107.234:8094/api/index/default_index/query error 400 reason: status: 400, content: rest_index: Query, indexName: default_index, requestBody: {"from": 0, "indexName": "default_index", "fields": [], "explain": false, "ctl": {"timeout": 60000, "consistency": {"vectors": {}, "level": ""}}, "query": {"field": "email", "match": "gallia@mcdiabetes.com"}, "size": 10000}, req: &http.Request{Method:"POST", URL:(*url.URL)(0xc8209c4280), Proto:"HTTP/1.1", ProtoMajor:1, ProtoMinor:1, Header:http.Header{"Content-Type":[]string{"application/json"}, "Authorization":[]string{"Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA=="}, "Accept":[]string{"*/*"}, "User-Agent":[]string{"Python-httplib2/$Rev: 259 $"}, "Accept-Encoding":[]string{"identity"}, "Content-Length":[]string{"223"}}, Body:(*http.body)(0xc82c346340), ContentLength:223, TransferEncoding:[]string(nil), Close:false, Host:"172.23.107.234:8094", Form:url.Values{}, PostForm:url.Values{}, MultipartForm:(*multipart.Form)(nil), Trailer:http.Header(nil), RemoteAddr:"172.23.106.205:42895", RequestURI:"/api/index/default_index/query", TLS:(*tls.ConnectionState)(nil), Cancel:(<-chan struct {})(nil)}, err: bleve: QueryBlevePIndexImpl remote client returned status: 403 body: rest_auth: preparePerm, err: pindex not found
|
|
rest_index: Query, indexName: default_index, requestBody: {"from": 0, "indexName": "default_index", "fields": [], "explain": false, "ctl": {"timeout": 60000, "consistency": {"vectors": {}, "level": ""}}, "query": {"field": "email", "match": "gallia@mcdiabetes.com"}, "size": 10000}, req: &http.Request{Method:"POST", URL:(*url.URL)(0xc8209c4280), Proto:"HTTP/1.1", ProtoMajor:1, ProtoMinor:1, Header:http.Header{"Content-Type":[]string{"application/json"}, "Authorization":[]string{"Basic QWRtaW5pc3RyYXRvcjpwYXNzd29yZA=="}, "Accept":[]string{"*/*"}, "User-Agent":[]string{"Python-httplib2/$Rev: 259 $"}, "Accept-Encoding":[]string{"identity"}, "Content-Length":[]string{"223"}}, Body:(*http.body)(0xc82c346340), ContentLength:223, TransferEncoding:[]string(nil), Close:false, Host:"172.23.107.234:8094", Form:url.Values{}, PostForm:url.Values{}, MultipartForm:(*multipart.Form)(nil), Trailer:http.Header(nil), RemoteAddr:"172.23.106.205:42895", RequestURI:"/api/index/default_index/query", TLS:(*tls.ConnectionState)(nil), Cancel:(<-chan struct {})(nil)}, err: bleve: QueryBlevePIndexImpl remote client returned status: 403 body: rest_auth: preparePerm, err: pindex not found
|
[2016-05-09 00:56:08,365] - [fts_base:689] ERROR - Error running query: 'NoneType' object is not iterable
|
[2016-05-09 00:56:08,365] - [task:1103] INFO - Status: {}
|
[2016-05-09 00:56:08,365] - [task:1127] INFO - FTS hits for query: {"field": "email", "match": "gallia@mcdiabetes.com"} is -1 (took 0.0ms)
|
On .145's syslog, I see 2 OOMs-
May 9 00:24:46 localhost kernel: Out of memory: Kill process 14040 (cbft) score 331 or sacrifice child
|
:
|
:
|
May 9 00:55:29 localhost kernel: Out of memory: Kill process 14772 (cbft) score 288 or sacrifice child
|
Attaching cbcollect info.