Details
-
Bug
-
Resolution: Fixed
-
Major
-
Cheshire-Cat
-
Untriaged
-
No
Description
What's the issue?
When creating a large number of collections during a restore (for me, over 500 collections) it's highly likely that you will eventually get a connection failure (which isn't explicitly handled in our retry logic meaning we exit early).
What's the solution?
There are multiple possible solutions to the problem:
1) We explicitly handle the connection failure and retry to create the scope/collection
2) We wait until there is a bulk creation endpoint which we can use to create the collections using a single request
The preferred solution here is the second, not only being it solves this problem but because it will also alleviate the severe performance issue I encountered whilst attempting to reproduce this issue. Below are some basic benchmarks which display how long is takes to create 'n' collections.
1) Creating 10 collections takes 6.03 seconds
2) Creating 100 collections takes 90.04 seconds
3) Creating 500 collections takes 506.10 seconds
4) Creating more than 500 generally results in a connection failure.
Attachments
Issue Links
- relates to
-
MB-37999 Collections: Bulk collection creation
- Closed
For Gerrit Dashboard: MB-38316 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
137043,2 | MB-38316 Tidy up error definitions in the collections module | master | backup | Status: MERGED | +2 | +1 |
137044,2 | MB-38316 Change source/sink bucket type to string | master | backup | Status: MERGED | +2 | +1 |
137045,4 | MB-38316 Add a 'BulkUpdateManifest' data structure | master | backup | Status: MERGED | +2 | +1 |
137046,5 | MB-38316 Add collection manifest bulk update API function | master | backup | Status: MERGED | +2 | +1 |
137047,5 | MB-38316 Use the bulk update API to auto create/delete collections | master | backup | Status: MERGED | +2 | +1 |
137055,6 | MB-38316 Refactor how we wait for manifest changes to propagate | master | backup | Status: MERGED | +2 | +1 |
137074,6 | MB-38316 Blackhole sink will now bulk update the manifest | master | backup | Status: MERGED | +2 | +1 |
137076,3 | MB-38316 Remove old collection create/delete functionality | master | backup | Status: MERGED | +2 | +1 |