Pinned fields
Click on the next to a field label to start pinning.
Details
Assignee
Kevin CherkauerKevin Cherkauer(Deactivated)Reporter
Kevin CherkauerKevin Cherkauer(Deactivated)Story Points
1Priority
MajorInstabug
Open Instabug
Details
Details
Assignee
Kevin Cherkauer
Kevin Cherkauer(Deactivated)Reporter
Kevin Cherkauer
Kevin Cherkauer(Deactivated)Story Points
1
Priority
Instabug
Open Instabug
PagerDuty
PagerDuty
PagerDuty
Sentry
Sentry
Sentry
Zendesk Support
Zendesk Support
Zendesk Support
Created May 27, 2021 at 10:00 PM
Updated February 4, 2022 at 7:12 AM
Planner currently stops iterating when one of the following is achieved:
Estimated cost falls below a fixed threshold t, or
A fixed maximum number of iterationsi is reached
There are some problems with this approach. Let c be the (unknown) optimal cost.
If c << t, Planner will stop too early based on reaching t, even though much improvement is still possible.
If c > t, Planner will iterate the full i iterations even if it is no longer making progress.
A better stopping criterion is to calculate a trailing average proportional cost improvement per iteration p over the last 1,000 iterations (= sample size for "how much improvement is still being achieved") and replacing t with an "epsilon" value e based on this metric, so Planner would instead stop when one of the following is achieved:
p for the last 1,000 iterations falls below e
A fixed maximum number of iterations i is reached
Using this approach, Planner would not stop too early if it was still making good progress, nor would it continue to iterate futilely if only minimal or no progress was still being made.