Description
slow data load on newly created bucket if cbimport is run immediately after the bucket is created.
This is a bad developer experience, one would assume providing the --wait flag that the bucket would be ready.
The below cbimport takes 53.3 sec.
couchbase-cli bucket-create -c localhost:8091 \
|
-u $CB_USERNAME -p $CB_PASSWORD \
|
--bucket=vb${inst} \
|
--bucket-type=couchbase \
|
--bucket-ramsize=100 \
|
--bucket-replica=1 \
|
--enable-flush=1 \
|
--wait
|
|
# no sleep run cbimport as soon as possible
|
|
cbimport json -c couchbase://127.0.0.1 -u $CB_USERNAME -p $CB_PASSWORD -b vb01 -d file:///tmp/data.json -f lines -t 2 -g evtest0::%id%
|
JSON `file:///tmp/data.json` imported to `couchbase://127.0.0.1` successfully
|
Documents imported: 3000 Documents failed: 0
|
|
real 0m53.357s
|
user 0m0.306s
|
sys 0m0.180s
|
adding just a 10 second sleep before the cbimport it runs 10X faster taking only 5.1 sec.
couchbase-cli bucket-create -c localhost:8091 \
|
-u $CB_USERNAME -p $CB_PASSWORD \
|
--bucket=vb01 \
|
--bucket-type=couchbase \
|
--bucket-ramsize=100 \
|
--bucket-replica=1 \
|
--enable-flush=1 \
|
--wait
|
|
sleep 10
|
|
time cbimport json -c couchbase://127.0.0.1 \
|
-u $CB_USERNAME -p $CB_PASSWORD -b vb01 \
|
-d file:///tmp/data.json -f lines -t 2 -g evtest0::%id%
|
JSON `file:///tmp/data.json` imported to `couchbase://127.0.0.1` successfully
|
Documents imported: 3000 Documents failed: 0
|
|
real 0m5.152s
|
user 0m0.146s
|
sys 0m0.052s
|
SUCCESS: Bucket created
add 3000 docs to vb01
gen/cbimport block: 1 of size: 3000 start at Sat Mar 27 21:57:30 UTC 2021
3000 /tmp/data.json
JSON `file:///tmp/data.json` imported to `couchbase://127.0.0.1` successfully
Documents imported: 3000 Documents failed: 0
real 0m53.500s
user 0m0.345s
sys 0m0.219s
[ec2-user@ip-10-0-0-187 scripts]$ ./test_del_bucket_gen.sh admin jtester 1
SUCCESS: Bucket deleted
[ec2-user@ip-10-0-0-187 scripts]$ !vi
vi test_add_bucket_gen.sh
[ec2-user@ip-10-0-0-187 scripts]$
[ec2-user@ip-10-0-0-187 scripts]$
[ec2-user@ip-10-0-0-187 scripts]$
[ec2-user@ip-10-0-0-187 scripts]$ ./test_add_bucket_gen.sh admin jtester 1 3000
SUCCESS: Bucket created
add 3000 docs to vb01
gen/cbimport block: 1 of size: 3000 start at Sat Mar 27 22:00:00 UTC 2021
3000 /tmp/data.json
JSON `file:///tmp/data.json` imported to `couchbase://127.0.0.1` successfully
Documents imported: 3000 Documents failed: 0
real 0m5.119s
user 0m0.143s
sys 0m0.048s
[ec2-user@ip-10-0-0-187 scripts]$ !vi
Attachments
Issue Links
- relates to
-
MB-45333 cbbackupmgr restore: fails with --enable-bucket-config
- Open