Details
-
Bug
-
Resolution: Not a Bug
-
Major
-
None
-
6.0.0
-
Docker container couchbase/server:community-6.0.0
-
Untriaged
-
Unknown
Description
All of a sudden couchbase starts to produce 100% disk usage and eat all available free space on 2 of 3 nodes.
Probably all eaten disk space located inside this files:
/lib/couchbase/data/@indexes/babycare/main_{hash}.view.{id} |
|
|
on first node:
|
126.2 GiB [##########] main_72cc6e6eba2986295f83acae24e19759.view.31 |
110.3 GiB [######## ] main_28b5fcaa1f54e64ef868700af2adfb0f.view.35 |
100.5 GiB [####### ] main_6d2d09ed191879579a4230b19c17cc29.view.44 |
|
|
on second node:
|
135.0 GiB [##########] main_72cc6e6eba2986295f83acae24e19759.view.31 |
114.3 GiB [######## ] main_28b5fcaa1f54e64ef868700af2adfb0f.view.35 |
99.3 GiB [####### ] main_6d2d09ed191879579a4230b19c17cc29.view.44 |
All logs at the moment of cluster crash: https://yadi.sk/d/LAd-pF1typLO_A
Our configuration:
- 3 couchbase nodes
- 500Gb volume on each node (before crash data occupy around 150Gb on each node)
- 60Gb ram for each couchbase node
- 8 core CPU on each node
- this cluster has XDCR replication enabled (data was replicated from crashed cluster to another one with similar configuration).