[system test] create gsi index stuck for 10M items

0. Create primary gsi index on the bucket
1. Load data on the bucket
2. During data loading, issue "Create index city on default(city) using gsi;"
3. About at 5M index building, the resident ratio is at 80% for the bucket, so I changed the eviction policy to full eviction and modified the bucket size.

Post this, the indexes appear to be stuck for gsi at 8.5M, the remaining 1.4M items are not being indexed.

The DCP streams show little/no activity
The getIndex Status - shows BUild state
curl -u Administrator:password "http://10.6.2.164:9102/getIndexStatus"
{"code":"success","status":[

{"defnId":12498268352876551419,"name":"city","bucket":"default","secExprs":["`city`"],"status":"Build","definition":"CREATE INDEX city ON default(`city`) USING GSI","hosts":["127.0.0.1:8091"]}

,

{"defnId":7071704309248423639,"name":"#primary","bucket":"default","isPrimary":true,"status":"Ready","definition":"CREATE PRIMARY INDEX #primary ON default USING GSI","hosts":["127.0.0.1:8091"]}

,

{"defnId":9406413160319752558,"name":"name","bucket":"default","secExprs":["`name`"],"where":"(`name` = \"mary\")","status":"Ready","definition":"CREATE INDEX name ON default(`name`) WHERE (`name` = \"mary\") USING GSI","hosts":["127.0.0.1:8091"]}

]

attaching logs.