Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-52765

Analytics external storage runs failed while running query BF11

    XMLWordPrintable

Details

    • 1
    • CX Sprint 286

    Description

      Job: http://perf.jenkins.couchbase.com/job/Cloud-Analytics-Tester/87/

      Build: 7.1.0-2556

      Logs: 

      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-Cloud-Analytics-Tester-87/ec2-3-237-83-74.compute-1.amazonaws.com.zip

      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-Cloud-Analytics-Tester-87/ec2-44-197-119-181.compute-1.amazonaws.com.zip

      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-Cloud-Analytics-Tester-87/ec2-44-198-159-199.compute-1.amazonaws.com.zip

      https://s3-us-west-2.amazonaws.com/perf-artifacts/jenkins-Cloud-Analytics-Tester-87/ec2-44-199-218-45.compute-1.amazonaws.com.zip

       

      I can create external datasets.

      2022-06-24T12:54:34 [INFO] Creating external datasets

      2022-06-24T12:54:34 [INFO] statement: CREATE EXTERNAL DATASET `GleambookUsers-1` ON `analytics-bigfun20m-json` AT `external_link` Using 'GleambookUsers-1' WITH { 'format': 'json', 'include': '*.json' };

      2022-06-24T12:54:35 [INFO] statement: CREATE EXTERNAL DATASET `GleambookMessages-1` ON `analytics-bigfun20m-json` AT `external_link` Using 'GleambookMessages-1' WITH { 'format': 'json', 'include': '*.json' };

      2022-06-24T12:54:35 [INFO] statement: CREATE EXTERNAL DATASET `ChirpMessages-1` ON `analytics-bigfun20m-json` AT `external_link` Using 'ChirpMessages-1' WITH { 'format': 'json', 'include': '*.json' };

      However, when I ran the queries, I hit this error.

      WARNING

      "requestID": "d345fe58-bbfd-4fa1-9a92-98243c15ac9f",

      "errors": "code": 24086, "msg": "External source error. org.apache.http.ConnectionClosedException: Premature end of Content-Length delimited message body (expected: 384,044,886; received: 352,304,588)",

      "status": "fatal",

      "metrics":

      "elapsedTime": "16.899668786s",

      "executionTime": "16.896774092s",

      "resultCount": 0,

      "resultSize": 0,

      "processedObjects": 0,

      "errorCount": 1

      The queries failed after 20 attempts.

      2022-06-24T12:58:15 [WARNING] Retrying http://ec2-3-237-83-74.compute-1.amazonaws.com:8095/analytics/service
      2022-06-24T13:16:58 [ERROR] Request failed after 20 attempts
      2022-06-24T13:23:40 [ERROR] Request failed after 20 attempts
      2022-06-24T13:30:23 [ERROR] Request failed after 20 attempts

      The queries are the same queries we run in our weekly tests, BF03, 08, 10, 11, and 14
      https://github.com/couchbase/perfrunner/blob/master/perfrunner/workloads/bigfun/query_gen.py

      From the bigfun log, it looks like BF03, BF08 and BF10 worked, but the run failed while running BF11.
      http://perf.jenkins.couchbase.com/job/Cloud-Analytics-Tester/87/artifact/bigfun.log

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          Bo-Chun Wang,

          The external collections do not have meta(). You need to replace the meta() usage on that query by the actual field in the generated json.

          murtadha.hubail Murtadha Hubail added a comment - Bo-Chun Wang , The external collections do not have meta(). You need to replace the meta() usage on that query by the actual field in the generated json.
          bo-chun.wang Bo-Chun Wang added a comment -

          In the GleambookUsers-1 json file, there is a field called "id". I change meta(u).id to u.id and get a good run.

          http://perf.jenkins.couchbase.com/job/Cloud-Analytics-Tester/93/ 

              'BF16': 'SELECT u.id AS id, COUNT(* ) AS count '
                      'FROM `GleambookUsers-{}` u, `GleambookMessages-{}` m '
                      'WHERE u.id = m.author_id '
                      'AND u.user_since >= "{}" AND u.user_since < "{}" '
                      'AND m.send_time >= "{}" AND m.send_time < "{}" '
                      'GROUP BY u.id;',

          Doc example in GleambookUsers-1: 

          {
          "alias":"Augusta000000019742029",
          "collection":"GleambookUsers-1",
          "employment":[{"end_date":"2012-04-22","organization":"Rungozoom","start_date":"2010-04-14"},{"organization":"Hexsanhex","start_date":"2003-12-28"},{"organization":"Sublamdox","start_date":"2010-11-30"}],
          "friend_ids":["000000012071576","000000019882318","000000004483171","000000016808978","000000011557364","000000012462191"],
          "id":"000000019742029",
          "key":"000000019742029",
          "name":"Augusta Emrick",
          "scope":"scope-1",
          "type":"GleambookUsers",
          "user_since":"2014-06-12T06:51:09"
          }
          

           

          Murtadha Hubail 

          Is the new query correct? Does it match the original query? If yes, I will close this ticket.

          bo-chun.wang Bo-Chun Wang added a comment - In the GleambookUsers-1 json file, there is a field called "id". I change meta(u).id to u.id and get a good run. http://perf.jenkins.couchbase.com/job/Cloud-Analytics-Tester/93/       'BF16': 'SELECT u.id AS id, COUNT(* ) AS count '             'FROM `GleambookUsers-{}` u, `GleambookMessages-{}` m '             'WHERE u.id = m.author_id '             'AND u.user_since >= "{}" AND u.user_since < "{}" '             'AND m.send_time >= "{}" AND m.send_time < "{}" '             'GROUP BY u.id;', Doc example in GleambookUsers-1:  { "alias":"Augusta000000019742029", "collection":"GleambookUsers-1", "employment":[{"end_date":"2012-04-22","organization":"Rungozoom","start_date":"2010-04-14"},{"organization":"Hexsanhex","start_date":"2003-12-28"},{"organization":"Sublamdox","start_date":"2010-11-30"}], "friend_ids":["000000012071576","000000019882318","000000004483171","000000016808978","000000011557364","000000012462191"], "id":"000000019742029", "key":"000000019742029", "name":"Augusta Emrick", "scope":"scope-1", "type":"GleambookUsers", "user_since":"2014-06-12T06:51:09" }   Murtadha Hubail   Is the new query correct? Does it match the original query? If yes, I will close this ticket.

          Bo-Chun Wang,

          Yes, that should match the original query.

          murtadha.hubail Murtadha Hubail added a comment - Bo-Chun Wang , Yes, that should match the original query.

          Build couchbase-server-7.2.0-1648 contains kv_engine commit 22285c5 with commit message:
          MB-52765: Remove pager_vb_active_pct

          build-team Couchbase Build Team added a comment - Build couchbase-server-7.2.0-1648 contains kv_engine commit 22285c5 with commit message: MB-52765 : Remove pager_vb_active_pct

          Build couchbase-server-8.0.0-1058 contains kv_engine commit 22285c5 with commit message:
          MB-52765: Remove pager_vb_active_pct

          build-team Couchbase Build Team added a comment - Build couchbase-server-8.0.0-1058 contains kv_engine commit 22285c5 with commit message: MB-52765 : Remove pager_vb_active_pct

          People

            bo-chun.wang Bo-Chun Wang
            bo-chun.wang Bo-Chun Wang
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty