Details
-
Bug
-
Resolution: Fixed
-
Major
-
Cheshire-Cat
-
Untriaged
-
No
Description
What's the issue?
When creating a large number of collections during a restore (for me, over 500 collections) it's highly likely that you will eventually get a connection failure (which isn't explicitly handled in our retry logic meaning we exit early).
What's the solution?
There are multiple possible solutions to the problem:
1) We explicitly handle the connection failure and retry to create the scope/collection
2) We wait until there is a bulk creation endpoint which we can use to create the collections using a single request
The preferred solution here is the second, not only being it solves this problem but because it will also alleviate the severe performance issue I encountered whilst attempting to reproduce this issue. Below are some basic benchmarks which display how long is takes to create 'n' collections.
1) Creating 10 collections takes 6.03 seconds
2) Creating 100 collections takes 90.04 seconds
3) Creating 500 collections takes 506.10 seconds
4) Creating more than 500 generally results in a connection failure.
Attachments
Issue Links
- relates to
-
MB-37999 Collections: Bulk collection creation
- Closed