Details
Critical Description
This is an issue introduced by the fix for MB-34892
I added 64 near identical Eventing handlers (one per minute) named verifier_12w_inst## starting with verifier_12w_inst01 and ending with verifier_12w_inst64 (added in order).
After 9 items (in order) had been added e.g. only verifier_12w_inst10 on are left, I was surprised to see an end user popup message form the UI as follows:
"verifier_12w_inst06 was created remotely added!"
Especially because that function was created more than 4 minutes prior [as per my logging messages for this test]. I am thus positive sure that the above handler existed completely stable at the time and we had just added "verifier_12w_inst10" when this message occurred.
time=16:36:57 cur_inst=6 numOnXX=2 bucket=meta "diskUsed":16804222, "dataUsed":15064064,
|
time=16:36:57 cur_inst=6 numOnXX=2 bucket=register "diskUsed":62256717, "dataUsed":4336415,
|
time=16:36:58 cur_inst=6 numOnXX=2 bucket=review "diskUsed":8517993, "dataUsed":4340882,
|
time=16:38:11 cur_inst=7 numOnXX=2 bucket=meta "diskUsed":19212701, "dataUsed":17812480,
|
time=16:38:11 cur_inst=7 numOnXX=2 bucket=register "diskUsed":68219513, "dataUsed":4336415,
|
time=16:38:11 cur_inst=7 numOnXX=2 bucket=review "diskUsed":8505705, "dataUsed":4316306,
|
time=16:39:23 cur_inst=8 numOnXX=2 bucket=meta "diskUsed":18946484, "dataUsed":17288192,
|
time=16:39:23 cur_inst=8 numOnXX=2 bucket=register "diskUsed":74759198, "dataUsed":4336415,
|
time=16:39:24 cur_inst=8 numOnXX=2 bucket=review "diskUsed":8505707, "dataUsed":4349074,
|
time=16:40:32 cur_inst=9 numOnXX=2 bucket=meta "diskUsed":20687280, "dataUsed":19278848,
|
time=16:40:32 cur_inst=9 numOnXX=2 bucket=register "diskUsed":91892380, "dataUsed":4336415,
|
time=16:40:32 cur_inst=9 numOnXX=2 bucket=review "diskUsed":8513901, "dataUsed":4332690,
|
time=16:41:46 cur_inst=10 numOnXX=2 bucket=meta "diskUsed":20990377, "dataUsed":21163008,
|
time=16:41:46 cur_inst=10 numOnXX=2 bucket=register "diskUsed":100754658, "dataUsed":4336415,
|
time=16:41:46 cur_inst=10 numOnXX=2 bucket=review "diskUsed":8493425, "dataUsed":4291730,
|
See image of bottom of UI "odd_wrong_message_for_added_function_05.png" this message is completely wrong.
The various images I captured all show issues for example "odd_wrong_message_for_added_function_00.png" shows a message claiming a "deletion" where none occurs this is obviously an issue in the new MH "eventing.js" as of Nov 5, 2019
At the completion of the test we do indeed have all 64 of the expected handlers e.g.
# curl -X GET -s 'http://Administrator:password@localhost:8096/api/v1/status' | jq ".apps[] | { name: .name }" | grep name | sort | wc -l
|
64
|
Problems like the above can be reproduced on a small single node system - setup according to https://eventing-workshop.s3.amazonaws.com/Instructions.pdf run once and remove all functions- make sure you have jq (https://stedolan.github.io/jq/download) then untar the attached file test_many_handlers.tar.gz
tar xvzf test_many_handlers.tar.gz
|
cd test_many_handlers
|
edit the file "test_runner.sh" and correct any PATH issues and star a UI and goto the Eventing tab.
vi test_runner.sh
|
sh test_runner.sh
|
The test will tack about 1:20 minutes on a 12 core 2GHz Xeon (pretty slow due to a 20 second sleep and because the handlers will only run with 3 cores see MB-36662)
Now watch what happens toward the end (e.g. as you get close to 60 deployed functionsthe pop-up will show the incorrect function name that was just added (remotely via the CLI called in test_runner.sh)
Once done (or if you kill the test_runner.sh script) you can start from scratch via "./cbeventing -flush" the drop and recreate both meta and result buckets.
Then rerun test_runner.sh as this is reproducible.
A possible fix for eventing.js at about lines 36-47 (works for me but could still have a async issue or race) is as follows
// BEG JAS issue app.name is not stable in this loop across the callbacks // it changes I'm not too good at promises so I don't know how to pass things // down thus I set this variable moreStableApp (it works better but it still might // be volatile and stepped on - I haven't reproduced the issue with this update ) |
|
|
var moreStableApp = app; |
ApplicationService.public.importFunctionFromServer(moreStableApp.name)
|
.then(result => {
|
ApplicationService.local.createApp(result.data);
|
})
|
.then(result => {
|
// Inform user on why it appeard |
ApplicationService.server.showSuccessAlert(`${moreStableApp.name} was created remotely added!`);
|
})
|
.then(result => {
|
appList.add(moreStableApp.name);
|
self.appList[moreStableApp.name].status = moreStableApp.composite_status;
|
});
|
// END JAS issue app.name is not stable in this loop across the callbacks |
