Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-61473

95% of the analytics queries couldn't complete in 75s SDK default timeout when ran on external collections pointing to s3 bucket with 1M items in 1000 files each(1B total)

    XMLWordPrintable

Details

    Description

      1. Create a 1 compute unit columnar cluster
      2. Create an s3 link and 10 external collections pointing to a s3 bucket having 1B docs. 1M items in 1000 files approx.
      3. Start a query workload on external collections.

        CBAS Query Statistics
        +--------+---------------+----------------+-----------------+------------------+-------------------+-----------------+-----------------+
        | Bucket | Total Queries | Failed Queries | Success Queries | Rejected Queries | Cancelled Queries | Timeout Queries | Errored Queries |
        +--------+---------------+----------------+-----------------+------------------+-------------------+-----------------+-----------------+
        | s3     | count(37)     | count(0)       | count(3)        | count(0)         | count(0)          | count(27)       | count(0)        |
        +--------+---------------+----------------+-----------------+------------------+-------------------+-----------------+-----------------+
        

      4. Scale Out from 1->2

        CBAS Query Statistics
        +--------+---------------+----------------+-----------------+------------------+-------------------+-----------------+-----------------+
        | Bucket | Total Queries | Failed Queries | Success Queries | Rejected Queries | Cancelled Queries | Timeout Queries | Errored Queries |
        +--------+---------------+----------------+-----------------+------------------+-------------------+-----------------+-----------------+
        | s3     | count(78)     | count(0)       | count(7)        | count(0)         | count(0)          | count(65)       | count(0)        |
        +--------+---------------+----------------+-----------------+------------------+-------------------+-----------------+-----------------+
        

      5. Scale Out from 2->4

        CBAS Query Statistics
        +--------+---------------+----------------+-----------------+------------------+-------------------+-----------------+-----------------+
        | Bucket | Total Queries | Failed Queries | Success Queries | Rejected Queries | Cancelled Queries | Timeout Queries | Errored Queries |
        +--------+---------------+----------------+-----------------+------------------+-------------------+-----------------+-----------------+
        | s3     | count(126)    | count(0)       | count(9)        | count(0)         | count(0)          | count(112)      | count(0)        |
        +--------+---------------+----------------+-----------------+------------------+-------------------+-----------------+-----------------+
        

      6. Scale IN from 4->2

        CBAS Query Statistics
        +--------+---------------+----------------+-----------------+------------------+-------------------+-----------------+-----------------+
        | Bucket | Total Queries | Failed Queries | Success Queries | Rejected Queries | Cancelled Queries | Timeout Queries | Errored Queries |
        +--------+---------------+----------------+-----------------+------------------+-------------------+-----------------+-----------------+
        | s3     | count(275)    | count(0)       | count(10)       | count(0)         | count(14)         | count(242)      | count(0)        |
        +--------+---------------+----------------+-----------------+------------------+-------------------+-----------------+-----------------+
        

      7. Scale IN from 2->1

        CBAS Query Statistics
        +--------+---------------+----------------+-----------------+------------------+-------------------+-----------------+-----------------+
        | Bucket | Total Queries | Failed Queries | Success Queries | Rejected Queries | Cancelled Queries | Timeout Queries | Errored Queries |
        +--------+---------------+----------------+-----------------+------------------+-------------------+-----------------+-----------------+
        | s3     | count(1121)   | count(0)       | count(19)       | count(0)         | count(27)         | count(1063)     | count(0)        |
        +--------+---------------+----------------+-----------------+------------------+-------------------+-----------------+-----------------+
        

      Queries

      SELECT * from s3_xSuIj_volCollection_0 where phone like "4%" limit 100
       
      SELECT * from s3_xSuIj_volCollection_1 where `type` is not null limit 100
       
      SELECT COUNT(CASE WHEN free_breakfast THEN 1 ELSE NULL END) AS count_free_breakfast, COUNT(CASE WHEN free_parking THEN 1 ELSE NULL END) AS count_free_parking, COUNT(CASE WHEN free_parking AND free_breakfast THEN 1 ELSE NULL END) AS count_free_parking_and_breakfast FROM s3_xSuIj_volCollection_2 WHERE city LIKE 'North%' ORDER BY count_free_parking_and_breakfast DESC  limit 100
       
      WITH city_avg AS (SELECT city, AVG(price) AS avgprice FROM s3_xSuIj_volCollection_3 WHERE price IS NOT NULL GROUP BY city) SELECT h.name, h.price FROM s3_xSuIj_volCollection_3 h JOIN city_avg ON h.city = city_avg.city WHERE h.price < city_avg.avgprice AND h.price IS NOT NULL limit 100
       
      select city,country,count(*) from s3_xSuIj_volCollection_4 where free_breakfast=True and free_parking=True group by country,city order by country,city limit 100
       
      select avg(price) as AvgPrice, min(price) as MinPrice, max(price) as MaxPrice from s3_xSuIj_volCollection_5 where free_breakfast=True and free_parking=True and price is not null and array_count(public_likes)>5 and `type`='Hotel' group by country limit 100
      

      Attachments

        Activity

          People

            ritesh.agarwal Ritesh Agarwal
            ritesh.agarwal Ritesh Agarwal
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              PagerDuty