Lack of incremental progress updates while partitioned indexes are building during rebalance

Balakumaran Gopal There could be a bug in the progress updating for partitioned indexes such that as these build their progress may be staying at 0% until the build is done and they move to TransferTokenMerge state, at which point their progress will instantly change to 100%. The progress computation code is pretty complex, especially for partitioned indexes. I looked through it and could not find a bug but also could not convince myself it will get non-zero progress for partitioned indexes while they are in TransferTokenInProgress state. In the current MB a rebalance with 6 transfer tokens stayed at progress 0.16666666666666669 for a long time (this is exactly 1/6th except last digit is 9 instead of 7 which might just be due to floating point precision limitations) and in repro MB-46552 another rebalance progress stayed at 0.5555555555555556 for a long time (this is exactly 5/9ths). This implies it is not getting incremental progress updates for some transfer tokens but instead they are reporting either 0% or 100%.

You may wish to open an MB for the apparent lack of incremental progress updates while partitioned indexes are building during rebalance. Presumably we can defer that new MB to CCNext

Tue May 25 20:34:47 PDT 2021

{

  "status": "running",

  "msg": "Rebalance is running",

  "details": {

    "progress": 55.55555555555556,

    "refresh": 0.25,

    "totalBuckets": 10,

    "curBucket": 10,

    "curBucketName": "default",

    "docsRemaining": 0

  }

}

Rebalance completed successfully.

Rebalance Operation Id = ce74d9abdb4cfff2d661b7ff0a20a220