FILE collectinfo-2021-10-19t180046-ns_1@172.23.120.170.zip =======================
|
heap profile: 2967: 944190816 [407706: 25814793088] @ heap/1048576
|
1471: 879681536 [21745: 13003857920] @ 0x411f79 0x413c29 0x41720a 0x806ced 0xa9affa 0xa975a7 0xac535e 0x473ee1
|
# 0x806cec github.com/couchbase/goxdcr/base.DcpStatsMapType.Clone+0xec /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/base/types.go:2523
|
FILE collectinfo-2021-10-19t180046-ns_1@172.23.121.116.zip =======================
|
heap profile: 2967: 520960568 [616067: 28055843800] @ heap/1048576
|
758: 453296128 [20648: 12347834368] @ 0x411f79 0x413c29 0x41720a 0x806ced 0xa9affa 0xa975a7 0xac535e 0x473ee1
|
# 0x806cec github.com/couchbase/goxdcr/base.DcpStatsMapType.Clone+0xec /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/base/types.go:2523
|
FILE collectinfo-2021-10-19t180046-ns_1@172.23.121.124.zip =======================
|
heap profile: 4994: 1684465296 [590896: 29239874112] @ heap/1048576
|
2484: 1485471744 [20913: 12506308608] @ 0x411f79 0x413c29 0x41720a 0x806ced 0xa9affa 0xa975a7 0xac535e 0x473ee1
|
# 0x806cec github.com/couchbase/goxdcr/base.DcpStatsMapType.Clone+0xec /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/base/types.go:2523
|
FILE collectinfo-2021-10-19t180046-ns_1@172.23.121.126.zip =======================
|
heap profile: 2628: 789272976 [328979: 23836952672] @ heap/1048576
|
1231: 736157696 [21617: 12927311872] @ 0x411f79 0x413c29 0x41720a 0x806ced 0xa9affa 0xa975a7 0xac535e 0x473ee1
|
# 0x806cec github.com/couchbase/goxdcr/base.DcpStatsMapType.Clone+0xec /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/base/types.go:2523
|
FILE collectinfo-2021-10-19t180046-ns_1@172.23.121.127.zip =======================
|
heap profile: 4932: 1697172864 [512581: 29020569448] @ heap/1048576
|
2523: 1508794368 [21320: 12749701120] @ 0x411f79 0x413c29 0x41720a 0x806ced 0xa9affa 0xa975a7 0xac535e 0x473ee1
|
# 0x806cec github.com/couchbase/goxdcr/base.DcpStatsMapType.Clone+0xec /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/base/types.go:2523
|
FILE collectinfo-2021-10-19t180046-ns_1@172.23.121.128.zip =======================
|
heap profile: 3203: 938742712 [427921: 25978389472] @ heap/1048576
|
1434: 857554944 [21751: 13007446016] @ 0x411f79 0x413c29 0x41720a 0x806ced 0xa9affa 0xa975a7 0xac535e 0x473ee1
|
# 0x806cec github.com/couchbase/goxdcr/base.DcpStatsMapType.Clone+0xec /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/base/types.go:2523
|
FILE collectinfo-2021-10-19t180046-ns_1@172.23.121.129.zip =======================
|
heap profile: 4975: 1863978192 [427660: 25546669032] @ heap/1048576
|
2676: 1600290816 [21343: 12763455488] @ 0x411f79 0x413c29 0x41720a 0x806ced 0xa9affa 0xa975a7 0xac535e 0x473ee1
|
# 0x806cec github.com/couchbase/goxdcr/base.DcpStatsMapType.Clone+0xec /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/base/types.go:2523
|
FILE collectinfo-2021-10-19t180046-ns_1@172.23.121.130.zip =======================
|
heap profile: 2789: 811993104 [503844: 27821964288] @ heap/1048576
|
1240: 741539840 [21826: 13052297216] @ 0x411f79 0x413c29 0x41720a 0x806ced 0xa9affa 0xa975a7 0xac535e 0x473ee1
|
# 0x806cec github.com/couchbase/goxdcr/base.DcpStatsMapType.Clone+0xec /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/base/types.go:2523
|
FILE collectinfo-2021-10-19t180046-ns_1@172.23.121.48.zip =======================
|
heap profile: 4736: 1441074616 [525424: 28881340904] @ heap/1048576
|
1970: 1178091520 [21161: 12654616576] @ 0x411f79 0x413c29 0x41720a 0x806ced 0xa9affa 0xa975a7 0xac535e 0x473ee1
|
# 0x806cec github.com/couchbase/goxdcr/base.DcpStatsMapType.Clone+0xec /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/base/types.go:2523
|
Main pipeline seems to be fine:
FILE collectinfo-2021-10-19t180046-ns_1@172.23.120.170.zip =======================
cbcollect_info_ns_1@172.23.120.170_20211019-180049/ns_server.goxdcr.log:2021-10-19T02:08:55.832-07:00 INFO GOXDCR.StatsMgr: e36f78aae37740197f0fd990ee484227/GleamBookUsers0/GleamBookUsers0 total_docs=0, docs_processed=0, changes_left=0
cbcollect_info_ns_1@172.23.120.170_20211019-180049/ns_server.goxdcr.log:2021-10-19T02:08:57.213-07:00 INFO GOXDCR.StatsMgr: e36f78aae37740197f0fd990ee484227/GleamBookUsers0/GleamBookUsers0 total_docs=8926006, docs_processed=0, changes_left=8926006
cbcollect_info_ns_1@172.23.120.170_20211019-180049/ns_server.goxdcr.log:2021-10-19T02:08:58.095-07:00 INFO GOXDCR.StatsMgr: e36f78aae37740197f0fd990ee484227/GleamBookUsers0/GleamBookUsers0 total_docs=8927222, docs_processed=0, changes_left=8927222
cbcollect_info_ns_1@172.23.120.170_20211019-180049/ns_server.goxdcr.log:2021-10-19T02:08:59.089-07:00 INFO GOXDCR.StatsMgr: e36f78aae37740197f0fd990ee484227/GleamBookUsers0/GleamBookUsers0 total_docs=8927225, docs_processed=0, changes_left=8927225
cbcollect_info_ns_1@172.23.120.170_20211019-180049/ns_server.goxdcr.log:2021-10-19T02:09:00.091-07:00 INFO GOXDCR.StatsMgr: e36f78aae37740197f0fd990ee484227/GleamBookUsers0/GleamBookUsers0 total_docs=8930428, docs_processed=0, changes_left=8930428
…
cbcollect_info_ns_1@172.23.120.170_20211019-180049/ns_server.goxdcr.log:2021-10-19T11:09:36.113-07:00 INFO GOXDCR.StatsMgr: e36f78aae37740197f0fd990ee484227/GleamBookUsers0/GleamBookUsers0 total_docs=14846146, docs_processed=14846146, changes_left=0
FILE collectinfo-2021-10-19t180046-ns_1@172.23.121.116.zip =======================
cbcollect_info_ns_1@172.23.121.116_20211019-180047/ns_server.goxdcr.log:2021-10-19T02:09:19.273-07:00 INFO GOXDCR.StatsMgr: e36f78aae37740197f0fd990ee484227/GleamBookUsers0/GleamBookUsers0 total_docs=0, docs_processed=0, changes_left=0
cbcollect_info_ns_1@172.23.121.116_20211019-180047/ns_server.goxdcr.log:2021-10-19T02:09:20.925-07:00 INFO GOXDCR.StatsMgr: e36f78aae37740197f0fd990ee484227/GleamBookUsers0/GleamBookUsers0 total_docs=77994, docs_processed=21511, changes_left=56483
cbcollect_info_ns_1@172.23.121.116_20211019-180047/ns_server.goxdcr.log:2021-10-19T02:09:21.924-07:00 INFO GOXDCR.StatsMgr: e36f78aae37740197f0fd990ee484227/GleamBookUsers0/GleamBookUsers0 total_docs=78010, docs_processed=25512, changes_left=52498
…
cbcollect_info_ns_1@172.23.121.116_20211019-180047/ns_server.goxdcr.log:2021-10-19T11:05:10.953-07:00 INFO GOXDCR.StatsMgr: e36f78aae37740197f0fd990ee484227/GleamBookUsers0/GleamBookUsers0 total_docs=14977946, docs_processed=14977946, changes_left=0
cbcollect_info_ns_1@172.23.121.116_20211019-180047/ns_server.goxdcr.log:2021-10-19T11:05:11.953-07:00 INFO GOXDCR.StatsMgr: e36f78aae37740197f0fd990ee484227/GleamBookUsers0/GleamBookUsers0 total_docs=14977946, docs_processed=14977946, changes_left=0
FILE collectinfo-2021-10-19t180046-ns_1@172.23.121.124.zip =======================
cbcollect_info_ns_1@172.23.121.124_20211019-180050/ns_server.goxdcr.log:2021-10-19T02:08:56.087-07:00 INFO GOXDCR.StatsMgr: e36f78aae37740197f0fd990ee484227/GleamBookUsers0/GleamBookUsers0 total_docs=0, docs_processed=0, changes_left=0
cbcollect_info_ns_1@172.23.121.124_20211019-180050/ns_server.goxdcr.log:2021-10-19T02:08:57.261-07:00 INFO GOXDCR.StatsMgr: e36f78aae37740197f0fd990ee484227/GleamBookUsers0/GleamBookUsers0 total_docs=8922547, docs_processed=0, changes_left=8922547
cbcollect_info_ns_1@172.23.121.124_20211019-180050/ns_server.goxdcr.log:2021-10-19T02:08:59.027-07:00 INFO GOXDCR.StatsMgr: e36f78aae37740197f0fd990ee484227/GleamBookUsers0/GleamBookUsers0 total_docs=8923428, docs_processed=0, changes_left=8923428
cbcollect_info_ns_1@172.23.121.124_20211019-180050/ns_server.goxdcr.log:2021-10-19T02:09:00.033-07:00 INFO GOXDCR.StatsMgr: e36f78aae37740197f0fd990ee484227/GleamBookUsers0/GleamBookUsers0 total_docs=8926181, docs_processed=0, changes_left=8926181
…
cbcollect_info_ns_1@172.23.121.126_20211019-180050/ns_server.goxdcr.log:2021-10-19T11:08:29.661-07:00 INFO GOXDCR.StatsMgr: e36f78aae37740197f0fd990ee484227/GleamBookUsers0/GleamBookUsers0 total_docs=14896920, docs_processed=14896920, changes_left=0
I would say that the number of mutations you're seeing is probably due to the inability to start backfill pipelines due to a few issues. I'll link them in this MB.