Details
-
Bug
-
Resolution: Not a Bug
-
Major
-
7.0.0
-
Enterprise Edition 7.0.0 build 13157
-
Untriaged
-
Centos 64-bit
-
-
1
-
No
Description
Test Scenario –
Ingest approximately 30TB (approx) of data from Remote KV cluster to Local CBAS cluster.
Infra –
| Remote KV cluster: |
| Number of nodes – 20 |
| RAM allocated to KV per Node – 22.2 GB |
| Disk allocated per node – 25 GB |
| Number of buckets – 1 |
| Number of scopes/bucket – 1 |
| Number of collections/scope – 1 |
| Number of items – 130,259,631 |
| Total size of the data – 77 GB |
| Local CBAS cluster: |
| Number of nodes – 4 |
| RAM allocated to CBAS per Node – 200 GB |
| Disk allocated per node – 10 TB |
| RAM left for system - 50GB |
Stage 1 Test Setup-
| Number of dataverses – 10 |
| Number of Remote links – 10 |
| Number of datasets – 200 (20 dataset per link) |
Stage 2 Test Setup-
| Additional Number of datasets – 200 (20 dataset per link) |
Expected result -
Ingest approximately 30TB of data (77GB * 400)
Observations –
With stage 1 setup, it took almost 2.5 days to ingest around 12.5 TB of data. Once the stage 1 data was completely ingested, stage 2 was started by increasing the number of datasets by 200, so that we had a total of 400 datasets in CBAS now. The ingestion for the additional datasets in still going on even after 5 days now.
|