I put in a lot of debug in various files ./consumer/exported_functions.go ./producer/producer.go ./v8_consumer/src/client.cc To see hwy the p.app.FunctionID was getting reset when I do a pause/resume with no code change I think we are fine. But when I do a pause/edit/resuem the p.app.FunctionID becomes '0' and then we no longer reference our 1024 docs in the vBuckets (checkpints right?) and we start from Everything. I realize on an edit (when paused) we do need to compile, but for some reason the p.app.FunctionID is set to zeror and not kept from the time of 'pause'. deploy fasScan 2021-05-05T12:45:58.923-07:00 [Info] Consumer::SpawnCompilationWorker [::0] (exported_functions.go) JAS A Compilation ... uuid srcCid 0 2021-05-05T12:45:58.943-07:00 [Info] Consumer::SpawnCompilationWorker (client.cc) JAS main starting appname: fastScan function_id function_id ipc_type: af_inet feedback_port: 26999 port: 26999 2021-05-05T12:45:59.946-07:00 [Info] Consumer::SpawnCompilationWorker [:26999:17588] JAS B compilation status &common.CompileStatus{Area:"", Column:0, CompileSuccess:true, Description:"Compilation success", Index:0, Language:"Javascript", Line:0} ... uuid srcCid 0 2021-05-05T12:46:05.050-07:00 [Info] Producer::Serve [fastScan:0] Bootstrapping status: true JAS p.app.FunctionID 3903977903 2021-05-05T12:46:10.166-07:00 [Info] Producer::startBucket [fastScan:0] Connecting with bucket: "source" JAS p.app.FunctionID%!(EXTRA uint32=3903977903) 2021-05-05T12:46:10.399-07:00 [Info] eventing-consumer [worker_fastScan_0:/tmp/127.0.0.1:8091_0_3903977903.sock:17604] (client.cc) JAS main starting appname: fastScan function_id 3903977903 ipc_type: af_unix feedback_port: /tmp/f_127.0.0.1:8091_0_3903977903.sock port: /tmp/127.0.0.1:8091_0_3903977903.sock import test_adv_insert3 from saved file 2021-05-05T12:46:42.951-07:00 [Info] Consumer::SpawnCompilationWorker [::0] (exported_functions.go) JAS A Compilation ... uuid srcCid 0 2021-05-05T12:46:42.964-07:00 [Info] Consumer::SpawnCompilationWorker (client.cc) JAS main starting appname: test_adv_insert3 function_id function_id ipc_type: af_inet feedback_port: 21765 port: 21765 2021-05-05T12:46:43.967-07:00 [Info] Consumer::SpawnCompilationWorker [:21765:17637] JAS B compilation status &common.CompileStatus{Area:"", Column:0, CompileSuccess:true, Description:"Compilation success", Index:0, Language:"Javascript", Line:0} ... uuid srcCid 0 deploy test_adv_insert3 2021-05-05T12:47:16.277-07:00 [Info] Consumer::SpawnCompilationWorker [::0] (exported_functions.go) JAS A Compilation ... uuid srcCid 0 2021-05-05T12:47:16.287-07:00 [Info] Consumer::SpawnCompilationWorker (client.cc) JAS main starting appname: test_adv_insert3 function_id function_id ipc_type: af_inet feedback_port: 20959 port: 20959 2021-05-05T12:47:17.290-07:00 [Info] Consumer::SpawnCompilationWorker [:20959:17659] JAS B compilation status &common.CompileStatus{Area:"", Column:0, CompileSuccess:true, Description:"Compilation success", Index:0, Language:"Javascript", Line:0} ... uuid srcCid 0 2021-05-05T12:47:22.329-07:00 [Info] Producer::Serve [test_adv_insert3:0] Bootstrapping status: true JAS p.app.FunctionID 1953710818 2021-05-05T12:47:27.399-07:00 [Info] Producer::startBucket [test_adv_insert3:0] Connecting with bucket: "source" JAS p.app.FunctionID%!(EXTRA uint32=1953710818) 2021-05-05T12:47:27.614-07:00 [Info] eventing-consumer [worker_test_adv_insert3_0:/tmp/127.0.0.1:8091_0_1953710818.sock:17673] (client.cc) JAS main starting appname: test_adv_insert3 function_id 1953710818 ipc_type: af_unix feedback_port: /tmp/f_127.0.0.1:8091_0_1953710818.sock port: /tmp/127.0.0.1:8091_0_1953710818.sock test_adv_insert3 was "From now" first pause (no edit) first resume 2021-05-05T12:48:25.033-07:00 [Info] Consumer::SpawnCompilationWorker [::0] (exported_functions.go) JAS A Compilation ... uuid srcCid 0 2021-05-05T12:48:25.051-07:00 [Info] Consumer::SpawnCompilationWorker (client.cc) JAS main starting appname: test_adv_insert3 function_id function_id ipc_type: af_inet feedback_port: 37341 port: 37341 2021-05-05T12:48:26.054-07:00 [Info] Consumer::SpawnCompilationWorker [:37341:17693] JAS B compilation status &common.CompileStatus{Area:"", Column:0, CompileSuccess:true, Description:"Compilation success", Index:0, Language:"Javascript", Line:0} ... uuid srcCid 0 2021-05-05T12:48:31.095-07:00 [Info] Producer::resumeProducer [test_adv_insert3:0] calling p.startBucket() JAS p.app.FunctionID%!(EXTRA uint32=1953710818) 2021-05-05T12:48:31.095-07:00 [Info] Producer::startBucket [test_adv_insert3:0] Connecting with bucket: "source" JAS p.app.FunctionID%!(EXTRA uint32=1953710818) 2021-05-05T12:48:31.314-07:00 [Info] eventing-consumer [worker_test_adv_insert3_0:/tmp/127.0.0.1:8091_0_1953710818.sock:17707] (client.cc) JAS main starting appname: test_adv_insert3 function_id 1953710818 ipc_type: af_unix feedback_port: /tmp/f_127.0.0.1:8091_0_1953710818.sock port: /tmp/127.0.0.1:8091_0_1953710818.sock second pause (with edit add a space) second resume will fail (difference was an edit) 2021-05-05T12:49:55.046-07:00 [Info] Consumer::SpawnCompilationWorker [::0] (exported_functions.go) JAS A Compilation ... uuid srcCid 0 2021-05-05T12:49:55.064-07:00 [Info] Consumer::SpawnCompilationWorker (client.cc) JAS main starting appname: test_adv_insert3 function_id function_id ipc_type: af_inet feedback_port: 24017 port: 24017 2021-05-05T12:49:56.067-07:00 [Info] Consumer::SpawnCompilationWorker [:24017:17730] JAS B compilation status &common.CompileStatus{Area:"", Column:0, CompileSuccess:true, Description:"Compilation success", Index:0, Language:"Javascript", Line:0} ... uuid srcCid 0 2021-05-05T12:50:01.112-07:00 [Info] Producer::resumeProducer [test_adv_insert3:0] calling p.startBucket() JAS p.app.FunctionID%!(EXTRA uint32=0) 2021-05-05T12:50:01.112-07:00 [Info] Producer::startBucket [test_adv_insert3:0] Connecting with bucket: "source" JAS p.app.FunctionID%!(EXTRA uint32=0) 2021-05-05T12:50:01.112-07:00 [Error] Producer::handleV8Consumer JAS this is BAD p.app.FunctionID is 0 why? 2021-05-05T12:50:01.335-07:00 [Info] eventing-consumer [worker_test_adv_insert3_0:/tmp/127.0.0.1:8091_0_0.sock:17744] (client.cc) JAS main starting appname: test_adv_insert3 function_id 0 ipc_type: af_unix feedback_port: /tmp/f_127.0.0.1:8091_0_0.sock port: /tmp/127.0.0.1:8091_0_0.sock 2021-05-05T12:50:01.338-07:00 [Info] eventing-consumer [worker_test_adv_insert3_0:/tmp/127.0.0.1:8091_0_0.sock:17744] ERROR JAS well this looks bad appname: test_adv_insert3 worker id: worker_test_adv_insert3_0 uds_sock_path:/tmp/127.0.0.1:8091_0_0.sock we are now processing from "Everything" note the /tmp/127.0.0.1:8091_0_0.sock (the second "0" should have been "1953710818") undeploy both functons This explains the cruft 1024 documents left in metadata see 1024 docs or cruft in the bucket "meta" (Eventing storage) eventing::1953710818::test_adv_insert3::vb::0 * * * eventing::1953710818::test_adv_insert3::vb::1023 The issue is in FunctionID gettgin cloberred on a resume where there was either a code change or a settings which seems to be set in just three files linuxbrew@couch01:~/couch/goproj/src/github.com/couchbase/eventing$ find . -type f -exec grep -l 'FunctionID =' {} \; ./util/util.go ./service_manager/http_handlers.go ./producer/depcfg_parser.go linuxbrew@couch01:~/couch/goproj/src/github.com/couchbase/eventing$ find . -type f -exec grep 'FunctionID =' {} \; app.FunctionID = uint32(config.HandlerUUID()) app.FunctionID = uint32(config.HandlerUUID()) app.FunctionID = tApp.FunctionID stats.FunctionID = app.FunctionID p.app.FunctionID = uint32(config.HandlerUUID()) Adding a bit more debug 2021-05-05T13:12:01.481-07:00 [Info] Producer::parseDepcfg [test_adv_insert3] JAS AA p.app.FunctionID is 0 2021-05-05T13:12:01.481-07:00 [Info] Producer::parseDepcfg [test_adv_insert3] JAS AB p.app.FunctionID is 0 set to config.HandlerUUID() Seems like line 42 of ./producer/depcfg_parser.go (no debug actual line number) is the culprit p.app.FunctionID = uint32(config.HandlerUUID()) Why is this set to 0 when we edit either the JavaScript or Settings when the Function is paused? seems like we need something like cfg.ConfigAddHandlerUUID(builder, app.FunctionID) if we are resumming.