Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0-beta
    • Component/s: UI, XDCR
    • Security Level: Public
    • Labels:
    No reviews matched the request. Check your Options in the drop-down menu of this sections header.

      Activity

      Hide
      peter peter added a comment -

      I believe some of the stats will be there for beta.

      Show
      peter peter added a comment - I believe some of the stats will be there for beta.
      Hide
      thuan Thuan Nguyen added a comment -

      Integrated in github-ns-server-2-0 #437 (See http://qa.hq.northscale.net/job/github-ns-server-2-0/437/)
      MB-5943: Restructure XDCR for stats, performance and reliability (Revision a8fad0e67d70d73d570cfd25e46d0167958e3c2c)

      Result = SUCCESS
      Damien Katz :
      Files :

      • src/xdc_vbucket_rep.erl
      • src/concurrency_throttle.erl
      • src/ns_server_sup.erl
      • src/xdc_rep_manager.erl
      • src/xdc_replication_sup.erl
      • src/xdc_vbucket_rep_sup.erl
      • src/xdc_rep_utils.erl
      • src/xdc_rep_notifier.erl
      • src/xdc_vbucket_rep_worker.erl
      • src/xdc_vbucket_rep_ckpt.erl
      • src/xdc_rep_manager_helper.erl
      • src/xdc_replication.erl
      • include/xdc_replicator.hrl
      Show
      thuan Thuan Nguyen added a comment - Integrated in github-ns-server-2-0 #437 (See http://qa.hq.northscale.net/job/github-ns-server-2-0/437/ ) MB-5943 : Restructure XDCR for stats, performance and reliability (Revision a8fad0e67d70d73d570cfd25e46d0167958e3c2c) Result = SUCCESS Damien Katz : Files : src/xdc_vbucket_rep.erl src/concurrency_throttle.erl src/ns_server_sup.erl src/xdc_rep_manager.erl src/xdc_replication_sup.erl src/xdc_vbucket_rep_sup.erl src/xdc_rep_utils.erl src/xdc_rep_notifier.erl src/xdc_vbucket_rep_worker.erl src/xdc_vbucket_rep_ckpt.erl src/xdc_rep_manager_helper.erl src/xdc_replication.erl include/xdc_replicator.hrl
      Hide
      thuan Thuan Nguyen added a comment -

      Integrated in github-ns-server-2-0 #441 (See http://qa.hq.northscale.net/job/github-ns-server-2-0/441/)
      MB-5943: Maintain and collect XDCR vb stats (Revision c0e0791c1433eb5d97c4f36704a8159b47b061e1)

      Result = SUCCESS
      Damien Katz :
      Files :

      • src/xdc_replication_sup.erl
      • src/xdc_vbucket_rep_worker.erl
      • src/xdc_vbucket_rep_sup.erl
      • src/xdc_vbucket_rep.erl
      • src/xdc_replication.erl
      • include/xdc_replicator.hrl
      • src/xdc_rep_manager.erl
      • src/xdc_vbucket_rep_ckpt.erl
      • src/xdc_rep_utils.erl
      Show
      thuan Thuan Nguyen added a comment - Integrated in github-ns-server-2-0 #441 (See http://qa.hq.northscale.net/job/github-ns-server-2-0/441/ ) MB-5943 : Maintain and collect XDCR vb stats (Revision c0e0791c1433eb5d97c4f36704a8159b47b061e1) Result = SUCCESS Damien Katz : Files : src/xdc_replication_sup.erl src/xdc_vbucket_rep_worker.erl src/xdc_vbucket_rep_sup.erl src/xdc_vbucket_rep.erl src/xdc_replication.erl include/xdc_replicator.hrl src/xdc_rep_manager.erl src/xdc_vbucket_rep_ckpt.erl src/xdc_rep_utils.erl
      Hide
      steve Steve Yen added a comment -

      Reopening to get a publically available REST API on this.

      Show
      steve Steve Yen added a comment - Reopening to get a publically available REST API on this.
      Hide
      alkondratenko Aleksey Kondratenko (Inactive) added a comment -

      Steve, what is proposed here http://review.couchbase.org/#/c/19993/1/src/menelaus_web.erl,unified is not acceptable for me. My comments there may look strange, but fact is I don't need random API that doesn't take into account what UI needs. Or at least current UI needs.

      I understand you need this stats badly. So here's proposal.

      POST this:

      BucketName = "default", {json, {struct, [{RId, {struct, KVs}} ||

      {RId, KVs} <- xdc_rep_manager:stats(BucketName)]}}

      to /diag/eval and parse resultant JSON.

      E.g.

      wget O --user=Administrator --password=asdasd --post-data="BucketName = \"default\", {json, {struct, [{RId, {struct, KVs}} || {RId, KVs}

      <- xdc_rep_manager:stats(BucketName)]}}" http://lh:9000/diag/eval

      Show
      alkondratenko Aleksey Kondratenko (Inactive) added a comment - Steve, what is proposed here http://review.couchbase.org/#/c/19993/1/src/menelaus_web.erl,unified is not acceptable for me. My comments there may look strange, but fact is I don't need random API that doesn't take into account what UI needs. Or at least current UI needs. I understand you need this stats badly. So here's proposal. POST this: BucketName = "default", {json, {struct, [{RId, {struct, KVs}} || {RId, KVs} <- xdc_rep_manager:stats(BucketName)]}} to /diag/eval and parse resultant JSON. E.g. wget O --user=Administrator --password=asdasd --post-data="BucketName = \"default\", {json, {struct, [{RId, {struct, KVs}} || {RId, KVs} <- xdc_rep_manager:stats(BucketName)]}}" http://lh:9000/diag/eval
      Hide
      junyi Junyi Xie (Inactive) added a comment -

      Thanks Aaron for taking care of this.

      Show
      junyi Junyi Xie (Inactive) added a comment - Thanks Aaron for taking care of this.
      Hide
      thuan Thuan Nguyen added a comment -

      Integrated in github-ns-server-2-0 #450 (See http://qa.hq.northscale.net/job/github-ns-server-2-0/450/)
      MB-5943 Replication stats on UI (Revision 4d4ed0a4bd497aeb66fc36f6b20b9cacb208c10b)

      Result = SUCCESS
      Aliaksey Kandratsenka :
      Files :

      • src/menelaus_stats.erl
      • src/xdc_replication.erl
      • src/stats_collector.erl
      Show
      thuan Thuan Nguyen added a comment - Integrated in github-ns-server-2-0 #450 (See http://qa.hq.northscale.net/job/github-ns-server-2-0/450/ ) MB-5943 Replication stats on UI (Revision 4d4ed0a4bd497aeb66fc36f6b20b9cacb208c10b) Result = SUCCESS Aliaksey Kandratsenka : Files : src/menelaus_stats.erl src/xdc_replication.erl src/stats_collector.erl
      Hide
      ketaki Ketaki Gangal added a comment -

      Not seeing "Deletes" on the destination cluster UI stats. on build 1697

      Show
      ketaki Ketaki Gangal added a comment - Not seeing "Deletes" on the destination cluster UI stats. on build 1697
      Hide
      farshid Farshid Ghods (Inactive) added a comment -

      is this a UI only issue ?

      Show
      farshid Farshid Ghods (Inactive) added a comment - is this a UI only issue ?
      Hide
      junyi Junyi Xie (Inactive) added a comment -

      Ketaki,

      In "XDCR Destination" section, there is a delete stat figure, can you verify that we do not see XDCR delete stat in that section?

      Show
      junyi Junyi Xie (Inactive) added a comment - Ketaki, In "XDCR Destination" section, there is a delete stat figure, can you verify that we do not see XDCR delete stat in that section?
      Hide
      junyi Junyi Xie (Inactive) added a comment -

      The deletion ops from XDCR is shown at XDCR Destination section but not summary section.

      Show
      junyi Junyi Xie (Inactive) added a comment - The deletion ops from XDCR is shown at XDCR Destination section but not summary section.
      Hide
      ketaki Ketaki Gangal added a comment -

      On xdc stats on destination today,show -
      1. XDC ops/sec- seen as expected
      2. XDC sets/sec- seen as expected
      3. XDC deletes/sec - seen as expected

      The summary UI
      1. Creates - show xdc creates
      2. Updates - show xdc updates
      3. Deletes - does not show any xdc deletes.

      IMO this is very confusing for the users to understand that data is being deleted, but not displayed.

      For the destination if the incoming data via xdc is same as any other data from any other clients, then all creates/updates/deletes should be consistently displayed.

      Here, the xdc deletes/sec does show deletes, but this needs to also be reflected on the aggregate stats.

      Show
      ketaki Ketaki Gangal added a comment - On xdc stats on destination today,show - 1. XDC ops/sec- seen as expected 2. XDC sets/sec- seen as expected 3. XDC deletes/sec - seen as expected The summary UI 1. Creates - show xdc creates 2. Updates - show xdc updates 3. Deletes - does not show any xdc deletes. IMO this is very confusing for the users to understand that data is being deleted, but not displayed. For the destination if the incoming data via xdc is same as any other data from any other clients, then all creates/updates/deletes should be consistently displayed. Here, the xdc deletes/sec does show deletes, but this needs to also be reflected on the aggregate stats.
      Hide
      aaron Aaron Miller (Inactive) added a comment -

      ep_num_ops_del_meta, which is the stat used to show "XDC deletes/sec" is included in the sum stat that is used in the summary UI, so the code as I read it should be doing this. Not sure what's going on here...

      https://github.com/couchbase/ns_server/blob/master/src/stats_collector.erl#L286
      https://github.com/couchbase/ns_server/blob/master/src/menelaus_stats.erl#L1103

      Show
      aaron Aaron Miller (Inactive) added a comment - ep_num_ops_del_meta, which is the stat used to show "XDC deletes/sec" is included in the sum stat that is used in the summary UI, so the code as I read it should be doing this. Not sure what's going on here... https://github.com/couchbase/ns_server/blob/master/src/stats_collector.erl#L286 https://github.com/couchbase/ns_server/blob/master/src/menelaus_stats.erl#L1103
      Hide
      junyi Junyi Xie (Inactive) added a comment -

      Stats in XDC sections are all correct. In Summary UI, however, the Delete stat in Summary UI comes from stat delete_hits, which seems to me maintained in bucket_engine, instead of ep_engine (not sure why it is this way, since update and create on Summary UI are from ep_engine) .

      Ketaki,

      You may want to assign to bucket engine owner to take a look.

      Show
      junyi Junyi Xie (Inactive) added a comment - Stats in XDC sections are all correct. In Summary UI, however, the Delete stat in Summary UI comes from stat delete_hits, which seems to me maintained in bucket_engine, instead of ep_engine (not sure why it is this way, since update and create on Summary UI are from ep_engine) . Ketaki, You may want to assign to bucket engine owner to take a look.
      Hide
      peter peter added a comment -

      We want to pick up the delete stats from ep_engine in the summary like we do for the other XDC stats.

      Show
      peter peter added a comment - We want to pick up the delete stats from ep_engine in the summary like we do for the other XDC stats.
      Hide
      dipti Dipti Borkar added a comment -

      UI part for XDCR stats is done. cleanup of text in the works as well. ep_engine issue should be tracked separately

      Show
      dipti Dipti Borkar added a comment - UI part for XDCR stats is done. cleanup of text in the works as well. ep_engine issue should be tracked separately
      Hide
      thuan Thuan Nguyen added a comment -

      Integrated in github-ns-server-2-0 #468 (See http://qa.hq.northscale.net/job/github-ns-server-2-0/468/)
      MB-5943: Add xdcr stats for work and commit times (Revision 7ae1c343c58e31ddf39d61e6c844b50ccb4bfc36)

      Result = SUCCESS
      pwansch :
      Files :

      • src/xdc_replication.erl
      • src/ns_doctor.erl
      • src/xdc_vbucket_rep.erl
      • src/stats_collector.erl
      • include/xdc_replicator.hrl
      • src/menelaus_stats.erl
      Show
      thuan Thuan Nguyen added a comment - Integrated in github-ns-server-2-0 #468 (See http://qa.hq.northscale.net/job/github-ns-server-2-0/468/ ) MB-5943 : Add xdcr stats for work and commit times (Revision 7ae1c343c58e31ddf39d61e6c844b50ccb4bfc36) Result = SUCCESS pwansch : Files : src/xdc_replication.erl src/ns_doctor.erl src/xdc_vbucket_rep.erl src/stats_collector.erl include/xdc_replicator.hrl src/menelaus_stats.erl

        People

        • Assignee:
          peter peter
          Reporter:
          junyi Junyi Xie (Inactive)
        • Votes:
          0 Vote for this issue
          Watchers:
          2 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Gerrit Reviews

            There are no open Gerrit changes