Details
-
Bug
-
Resolution: Done
-
Blocker
-
None
-
6.5.0
-
Untriaged
-
Centos 64-bit
-
Unknown
-
CX Sprint 177, CX Sprint 178
Description
Running the tpcds volume test with 1.3 TB of data... the following query took around 4 hours then analytics service crashed:
WITH customer_total_return AS (SELECT sr.sr_customer_sk AS ctr_customer_sk, sr.sr_store_sk AS ctr_store_sk, SUM(sr.sr_return_amt) AS ctr_total_return FROM store_returns sr, date_dim dd WHERE sr.sr_returned_date_sk = dd.d_date_sk AND dd.d_year = 2000 GROUP BY sr.sr_customer_sk, sr.sr_store_sk) SELECT c.c_customer_id FROM customer_total_return ctr1, store s, customer c WHERE ctr1.ctr_total_return > (SELECT VALUE AVG(ctr2.ctr_total_return) * 1.2 FROM customer_total_return ctr2 WHERE ctr1.ctr_store_sk = ctr2.ctr_store_sk)[0] AND s.s_store_sk = ctr1.ctr_store_sk AND s.s_state = "TN" AND ctr1.ctr_customer_sk = c.c_customer_sk ORDER BY c.c_customer_id LIMIT 100;
Here are the logs:
https://s3.amazonaws.com/bugdb/jira/qe/collectinfo-2019-11-22T180830-ns_1%40172.23.96.205.zip
https://s3.amazonaws.com/bugdb/jira/qe/collectinfo-2019-11-22T180830-ns_1%40172.23.96.5.zip
https://s3.amazonaws.com/bugdb/jira/qe/collectinfo-2019-11-22T180830-ns_1%40172.23.96.57.zip
https://s3.amazonaws.com/bugdb/jira/qe/collectinfo-2019-11-22T180830-ns_1%40172.23.96.7.zip
https://s3.amazonaws.com/bugdb/jira/qe/collectinfo-2019-11-22T180830-ns_1%40172.23.96.8.zip
https://s3.amazonaws.com/bugdb/jira/qe/collectinfo-2019-11-22T180830-ns_1%40172.23.96.9.zip
The job: http://perf.jenkins.couchbase.com/job/oceanus/2511/
Node 172.23.96.5 complained about compaction being unable to proceed due to 0 bytes available on disk. However the disk on that node has at least 80GB of available space:
[root@oceanus-kv1 ~]# df -h |
Filesystem Size Used Avail Use% Mounted on
|
/dev/sda2 477G 5.3G 471G 2% / |
devtmpfs 16G 0 16G 0% /dev |
tmpfs 16G 0 16G 0% /dev/shm |
tmpfs 16G 931M 15G 6% /run |
tmpfs 16G 0 16G 0% /sys/fs/cgroup |
/dev/sda1 504M 145M 334M 31% /boot |
/dev/sdb1 440G 317G 101G 76% /backups |
/dev/mapper/vg_data-data 880G 747G 89G 90% /data |
tmpfs 3.2G 0 3.2G 0% /run/user/0 |
Attachments
Issue Links
- relates to
-
MB-37208 [CX] Run files are not removed on time and filling up disk space
- Closed