Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-46351

[Upgrade] dcp_stream_boundary remains as "from_prior" after upgrade

    XMLWordPrintable

Details

    • Untriaged
    • Centos 64-bit
    • 1
    • Unknown

    Description

      The UI and REST endpoints will give mixed messages until the first lifecycle operation

      CB server - 6.6.1 - 9213
      create and deploy handler having "dcp_stream_boundary": "everything".
      pause and resume handler.
      "dcp_stream_boundary" will become "from_prior".
      Upgrade cluster to 7.0.0 - 5177
      "dcp_stream_boundary" remains as "from_prior"

      curl -XGET http://Administrator:password@10.112.190.102:8096/api/v1/functions/f1
      {
       "appcode": "function OnUpdate(doc, meta) {\n    log('docId', meta.id);\n}\nfunction OnDelete(meta, options) {\n}",
       "depcfg": {
        "source_bucket": "src_bucket",
        "source_scope": "_default",
        "source_collection": "_default",
        "metadata_bucket": "metadata",
        "metadata_scope": "_default",
        "metadata_collection": "_default"
       },
       "version": "evt-6.6.1-9213-ee",
       "enforce_schema": false,
       "handleruuid": 3666138572,
       "function_instance_id": "vHsTc1",
       "appname": "f1",
       "settings": {
        "dcp_stream_boundary": "from_prior",
        "deadline_timeout": 62,
        "deployment_status": true,
        "description": "",
        "execution_timeout": 60,
        "language_compatibility": "6.5.0",
        "log_level": "INFO",
        "n1ql_consistency": "none",
        "num_timer_partitions": 128,
        "processing_status": true,
        "user_prefix": "eventing",
        "using_timer": false,
        "worker_count": 3
       }
      }
      

       

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          ritam.sharma Ritam Sharma added a comment -

          Ankit Prabhu - Can you take a look at this, might be similar to the one where we don't save the state in 6.6.x and the state resets.

          ritam.sharma Ritam Sharma added a comment - Ankit Prabhu - Can you take a look at this, might be similar to the one where we don't save the state in 6.6.x and the state resets.

          This is expected behavior, as we do not want to make any changes to the handler created by the user unless the user explicitly changes it. The cluster will treat it as "everything". (Other similar examples are language_compatibility and version)

          Might be worth including this in the release notes.

          vinayaka.kamath Vinayaka Kamath (Inactive) added a comment - This is expected behavior, as we do not want to make any changes to the handler created by the user unless the user explicitly changes it. The cluster will treat it as "everything". (Other similar examples are language_compatibility and version) Might be worth including this in the release notes.
          ritam.sharma Ritam Sharma added a comment - - edited

          After speaking with Sujay, the issue might be similar to MB-45973 and what Vinayak mentioned comment as well. After the upgrade there is no functional loss, but on UI it will be confusing.
          We should release note this issues and fix this for 7.0.1.

          ritam.sharma Ritam Sharma added a comment - - edited After speaking with Sujay, the issue might be similar to MB-45973 and what Vinayak mentioned comment as well. After the upgrade there is no functional loss, but on UI it will be confusing. We should release note this issues and fix this for 7.0.1.

          Ritam SharmaChanabasappa Ghali This needs to be fixed in 7.0.1 for better user experience post upgrade. Request inclusion in 7.0.1. Thank you!

          jeelan.poola Jeelan Poola added a comment - Ritam Sharma Chanabasappa Ghali This needs to be fixed in 7.0.1 for better user experience post upgrade. Request inclusion in 7.0.1. Thank you!
          ritam.sharma Ritam Sharma added a comment -

          Jeelan Poola - This definitely needs to be taken care for upgrade. Thank you, please add it to 7.0.1

          ritam.sharma Ritam Sharma added a comment - Jeelan Poola - This definitely needs to be taken care for upgrade. Thank you, please add it to 7.0.1

          Build couchbase-server-7.0.2-6503 contains eventing commit ccf27f9 with commit message:
          MB-46351: Replace 'from_prior' while writing out from store

          build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.2-6503 contains eventing commit ccf27f9 with commit message: MB-46351 : Replace 'from_prior' while writing out from store

          Build couchbase-server-7.1.0-1140 contains eventing commit 144c54b with commit message:
          MB-46351: Replace 'from_prior' while writing out from store

          build-team Couchbase Build Team added a comment - Build couchbase-server-7.1.0-1140 contains eventing commit 144c54b with commit message: MB-46351 : Replace 'from_prior' while writing out from store
          sujay.gad Sujay Gad added a comment -

          Verified using 7.0.2 - 6547.

          Stats before upgrade -

          curl -XGET http://Administrator:password@10.112.190.102:8096/api/v1/stats
          [
           {
            "dcp_feed_boundary": "from_prior",
            "event_processing_stats": {
             "agg_messages_sent_to_worker": 1390,
             "agg_queue_memory": 0,
             "agg_queue_memory_cap": 132120576,
             "agg_queue_size": 0,
             "agg_queue_size_cap": 300000,
             "agg_timer_feedback_queue_cap": 1500,
             "dcp_stream_req_counter": 1024,
             "dcp_streamreq": 1024,
             "execution_stats": 69,
             "failure_stats": 69,
             "latency_stats": 69,
             "lcb_exception_stats": 69,
             "log_level": 3,
             "num_processed_events": 0,
             "processed_events_size": 0,
             "thr_count": 3,
             "thr_map": 3,
             "v8_init": 3,
             "v8_load": 3
            },
            "events_remaining": {
             "dcp_backlog": 0
            },
            "execution_stats": {
             "agg_queue_memory": 0,
             "agg_queue_size": 0,
             "curl": {
              "delete": 0,
              "get": 0,
              "head": 0,
              "post": 0,
              "put": 0
             },
             "dcp_delete_msg_counter": 0,
             "dcp_delete_parse_failure": 0,
             "dcp_mutation_msg_counter": 0,
             "dcp_mutation_parse_failure": 0,
             "enqueued_dcp_delete_msg_counter": 0,
             "enqueued_dcp_mutation_msg_counter": 0,
             "enqueued_timer_msg_counter": 0,
             "feedback_queue_size": 0,
             "filtered_dcp_delete_counter": 0,
             "filtered_dcp_mutation_counter": 0,
             "lcb_retry_failure": 0,
             "messages_parsed": 1378,
             "num_processed_events": 0,
             "on_delete_failure": 0,
             "on_delete_success": 0,
             "on_update_failure": 0,
             "on_update_success": 0,
             "processed_events_size": 0,
             "timer_cancel_counter": 0,
             "timer_create_counter": 0,
             "timer_create_failure": 0,
             "timer_msg_counter": 0,
             "timer_responses_sent": 0,
             "timestamp": {
              "21549": "2021-08-20T11:49:38Z",
              "21559": "2021-08-20T11:49:38Z",
              "21569": "2021-08-20T11:49:38Z"
             },
             "uv_msg_parse_failure": 0,
             "uv_try_write_failure_counter": 0
            },
            "failure_stats": {
             "app_worker_setting_events_lost": 0,
             "bkt_ops_cas_mismatch_count": 0,
             "bucket_op_exception_count": 0,
             "checkpoint_failure_count": 0,
             "curl_non_200_response": 0,
             "dcp_events_lost": 0,
             "debugger_events_lost": 0,
             "delete_events_lost": 0,
             "mutation_events_lost": 0,
             "n1ql_op_exception_count": 0,
             "timeout_count": 0,
             "timer_callback_missing_counter": 0,
             "timer_context_size_exceeded_counter": 0,
             "timer_events_lost": 0,
             "timestamp": {
              "21549": "2021-08-20T11:49:38Z",
              "21559": "2021-08-20T11:49:38Z",
              "21569": "2021-08-20T11:49:38Z"
             },
             "v8worker_events_lost": 0
            },
            "function_name": "test",
            "gocb_creds_request_counter": 14,
            "function_id": 3142852543,
            "internal_vb_distribution_stats": {
             "worker_test_0": "[0-341]",
             "worker_test_1": "[342-682]",
             "worker_test_2": "[683-1023]"
            },
            "latency_percentile_stats": {
             "100": 0,
             "50": 0,
             "80": 0,
             "90": 0,
             "95": 0,
             "99": 0
            },
            "lcb_creds_request_counter": 0,
            "lcb_exception_stats": {},
            "planner_stats": [
             {
              "host_name": "10.112.190.102:8096",
              "start_vb": 0,
              "vb_count": 1024
             }
            ],
            "metastore_stats": {},
            "vb_distribution_stats_from_metadata": {},
            "worker_pids": {
             "worker_test_0": 21549,
             "worker_test_1": 21569,
             "worker_test_2": 21559
            }
           }
          ]
          

          Stats after upgrade -

          curl -XGET http://Administrator:password@10.112.190.102:8096/api/v1/stats
          [
           {
            "dcp_feed_boundary": "everything",
            "event_processing_stats": {
             "agg_messages_sent_to_worker": 1345,
             "agg_queue_memory": 0,
             "agg_queue_memory_cap": 53477376,
             "agg_queue_size": 0,
             "agg_queue_size_cap": 300000,
             "agg_timer_feedback_queue_cap": 1500,
             "dcp_stream_req_counter": 1024,
             "dcp_streamreq": 1024,
             "execution_stats": 60,
             "failure_stats": 60,
             "latency_stats": 60,
             "lcb_exception_stats": 60,
             "log_level": 3,
             "num_processed_events": 0,
             "processed_events_size": 0,
             "thr_count": 3,
             "thr_map": 3,
             "v8_init": 3,
             "v8_load": 3
            },
            "events_remaining": {
             "dcp_backlog": 0
            },
            "execution_stats": {
             "agg_queue_memory": 0,
             "agg_queue_size": 0,
             "curl": {
              "delete": 0,
              "get": 0,
              "head": 0,
              "post": 0,
              "put": 0
             },
             "curl_success_count": 0,
             "dcp_delete_msg_counter": 0,
             "dcp_delete_parse_failure": 0,
             "dcp_mutation_msg_counter": 0,
             "dcp_mutation_parse_failure": 0,
             "enqueued_dcp_delete_msg_counter": 0,
             "enqueued_dcp_mutation_msg_counter": 0,
             "enqueued_timer_msg_counter": 0,
             "feedback_queue_size": 0,
             "filtered_dcp_delete_counter": 0,
             "filtered_dcp_mutation_counter": 0,
             "lcb_retry_failure": 0,
             "messages_parsed": 1333,
             "no_op_counter": 0,
             "num_processed_events": 0,
             "on_delete_failure": 0,
             "on_delete_success": 0,
             "on_update_failure": 0,
             "on_update_success": 0,
             "processed_events_size": 0,
             "timer_callback_failure": 0,
             "timer_callback_success": 0,
             "timer_cancel_counter": 0,
             "timer_create_counter": 0,
             "timer_create_failure": 0,
             "timer_msg_counter": 0,
             "timer_responses_sent": 0,
             "timestamp": {
              "22082": "2021-08-20T11:53:15Z",
              "22088": "2021-08-20T11:53:15Z",
              "22093": "2021-08-20T11:53:15Z"
             },
             "uv_msg_parse_failure": 0,
             "uv_try_write_failure_counter": 0
            },
            "failure_stats": {
             "app_worker_setting_events_lost": 0,
             "bkt_ops_cas_mismatch_count": 0,
             "bucket_cache_overflow_count": 0,
             "bucket_op_cache_miss_count": 0,
             "bucket_op_exception_count": 0,
             "checkpoint_failure_count": 0,
             "curl_failure_count": 0,
             "curl_max_resp_size_exceeded": 0,
             "curl_non_200_response": 0,
             "curl_timeout_count": 0,
             "dcp_events_lost": 0,
             "debugger_events_lost": 0,
             "delete_events_lost": 0,
             "mutation_events_lost": 0,
             "n1ql_op_exception_count": 0,
             "timeout_count": 0,
             "timer_callback_missing_counter": 0,
             "timer_context_size_exceeded_counter": 0,
             "timer_events_lost": 0,
             "timestamp": {
              "22082": "2021-08-20T11:53:15Z",
              "22088": "2021-08-20T11:53:15Z",
              "22093": "2021-08-20T11:53:15Z"
             },
             "v8worker_events_lost": 0
            },
            "function_name": "test",
            "gocb_creds_request_counter": 2,
            "function_id": 3142852543,
            "internal_vb_distribution_stats": {
             "worker_test_0": "[0-341]",
             "worker_test_1": "[342-682]",
             "worker_test_2": "[683-1023]"
            },
            "latency_percentile_stats": {
             "100": 0,
             "50": 0,
             "80": 0,
             "90": 0,
             "95": 0,
             "99": 0
            },
            "lcb_creds_request_counter": 0,
            "lcb_exception_stats": {},
            "planner_stats": [
             {
              "host_name": "10.112.190.102:8096",
              "start_vb": 0,
              "vb_count": 1024
             }
            ],
            "metastore_stats": {},
            "vb_distribution_stats_from_metadata": {},
            "worker_pids": {
             "worker_test_0": 22088,
             "worker_test_1": 22093,
             "worker_test_2": 22082
            }
           }
          ]
          

          sujay.gad Sujay Gad added a comment - Verified using 7.0.2 - 6547. Stats before upgrade - curl -XGET http: //Administrator:password@10.112.190.102:8096/api/v1/stats [ { "dcp_feed_boundary" : "from_prior" , "event_processing_stats" : { "agg_messages_sent_to_worker" : 1390 , "agg_queue_memory" : 0 , "agg_queue_memory_cap" : 132120576 , "agg_queue_size" : 0 , "agg_queue_size_cap" : 300000 , "agg_timer_feedback_queue_cap" : 1500 , "dcp_stream_req_counter" : 1024 , "dcp_streamreq" : 1024 , "execution_stats" : 69 , "failure_stats" : 69 , "latency_stats" : 69 , "lcb_exception_stats" : 69 , "log_level" : 3 , "num_processed_events" : 0 , "processed_events_size" : 0 , "thr_count" : 3 , "thr_map" : 3 , "v8_init" : 3 , "v8_load" : 3 }, "events_remaining" : { "dcp_backlog" : 0 }, "execution_stats" : { "agg_queue_memory" : 0 , "agg_queue_size" : 0 , "curl" : { "delete" : 0 , "get" : 0 , "head" : 0 , "post" : 0 , "put" : 0 }, "dcp_delete_msg_counter" : 0 , "dcp_delete_parse_failure" : 0 , "dcp_mutation_msg_counter" : 0 , "dcp_mutation_parse_failure" : 0 , "enqueued_dcp_delete_msg_counter" : 0 , "enqueued_dcp_mutation_msg_counter" : 0 , "enqueued_timer_msg_counter" : 0 , "feedback_queue_size" : 0 , "filtered_dcp_delete_counter" : 0 , "filtered_dcp_mutation_counter" : 0 , "lcb_retry_failure" : 0 , "messages_parsed" : 1378 , "num_processed_events" : 0 , "on_delete_failure" : 0 , "on_delete_success" : 0 , "on_update_failure" : 0 , "on_update_success" : 0 , "processed_events_size" : 0 , "timer_cancel_counter" : 0 , "timer_create_counter" : 0 , "timer_create_failure" : 0 , "timer_msg_counter" : 0 , "timer_responses_sent" : 0 , "timestamp" : { "21549" : "2021-08-20T11:49:38Z" , "21559" : "2021-08-20T11:49:38Z" , "21569" : "2021-08-20T11:49:38Z" }, "uv_msg_parse_failure" : 0 , "uv_try_write_failure_counter" : 0 }, "failure_stats" : { "app_worker_setting_events_lost" : 0 , "bkt_ops_cas_mismatch_count" : 0 , "bucket_op_exception_count" : 0 , "checkpoint_failure_count" : 0 , "curl_non_200_response" : 0 , "dcp_events_lost" : 0 , "debugger_events_lost" : 0 , "delete_events_lost" : 0 , "mutation_events_lost" : 0 , "n1ql_op_exception_count" : 0 , "timeout_count" : 0 , "timer_callback_missing_counter" : 0 , "timer_context_size_exceeded_counter" : 0 , "timer_events_lost" : 0 , "timestamp" : { "21549" : "2021-08-20T11:49:38Z" , "21559" : "2021-08-20T11:49:38Z" , "21569" : "2021-08-20T11:49:38Z" }, "v8worker_events_lost" : 0 }, "function_name" : "test" , "gocb_creds_request_counter" : 14 , "function_id" : 3142852543 , "internal_vb_distribution_stats" : { "worker_test_0" : "[0-341]" , "worker_test_1" : "[342-682]" , "worker_test_2" : "[683-1023]" }, "latency_percentile_stats" : { "100" : 0 , "50" : 0 , "80" : 0 , "90" : 0 , "95" : 0 , "99" : 0 }, "lcb_creds_request_counter" : 0 , "lcb_exception_stats" : {}, "planner_stats" : [ { "host_name" : "10.112.190.102:8096" , "start_vb" : 0 , "vb_count" : 1024 } ], "metastore_stats" : {}, "vb_distribution_stats_from_metadata" : {}, "worker_pids" : { "worker_test_0" : 21549 , "worker_test_1" : 21569 , "worker_test_2" : 21559 } } ] Stats after upgrade - curl -XGET http: //Administrator:password@10.112.190.102:8096/api/v1/stats [ { "dcp_feed_boundary" : "everything" , "event_processing_stats" : { "agg_messages_sent_to_worker" : 1345 , "agg_queue_memory" : 0 , "agg_queue_memory_cap" : 53477376 , "agg_queue_size" : 0 , "agg_queue_size_cap" : 300000 , "agg_timer_feedback_queue_cap" : 1500 , "dcp_stream_req_counter" : 1024 , "dcp_streamreq" : 1024 , "execution_stats" : 60 , "failure_stats" : 60 , "latency_stats" : 60 , "lcb_exception_stats" : 60 , "log_level" : 3 , "num_processed_events" : 0 , "processed_events_size" : 0 , "thr_count" : 3 , "thr_map" : 3 , "v8_init" : 3 , "v8_load" : 3 }, "events_remaining" : { "dcp_backlog" : 0 }, "execution_stats" : { "agg_queue_memory" : 0 , "agg_queue_size" : 0 , "curl" : { "delete" : 0 , "get" : 0 , "head" : 0 , "post" : 0 , "put" : 0 }, "curl_success_count" : 0 , "dcp_delete_msg_counter" : 0 , "dcp_delete_parse_failure" : 0 , "dcp_mutation_msg_counter" : 0 , "dcp_mutation_parse_failure" : 0 , "enqueued_dcp_delete_msg_counter" : 0 , "enqueued_dcp_mutation_msg_counter" : 0 , "enqueued_timer_msg_counter" : 0 , "feedback_queue_size" : 0 , "filtered_dcp_delete_counter" : 0 , "filtered_dcp_mutation_counter" : 0 , "lcb_retry_failure" : 0 , "messages_parsed" : 1333 , "no_op_counter" : 0 , "num_processed_events" : 0 , "on_delete_failure" : 0 , "on_delete_success" : 0 , "on_update_failure" : 0 , "on_update_success" : 0 , "processed_events_size" : 0 , "timer_callback_failure" : 0 , "timer_callback_success" : 0 , "timer_cancel_counter" : 0 , "timer_create_counter" : 0 , "timer_create_failure" : 0 , "timer_msg_counter" : 0 , "timer_responses_sent" : 0 , "timestamp" : { "22082" : "2021-08-20T11:53:15Z" , "22088" : "2021-08-20T11:53:15Z" , "22093" : "2021-08-20T11:53:15Z" }, "uv_msg_parse_failure" : 0 , "uv_try_write_failure_counter" : 0 }, "failure_stats" : { "app_worker_setting_events_lost" : 0 , "bkt_ops_cas_mismatch_count" : 0 , "bucket_cache_overflow_count" : 0 , "bucket_op_cache_miss_count" : 0 , "bucket_op_exception_count" : 0 , "checkpoint_failure_count" : 0 , "curl_failure_count" : 0 , "curl_max_resp_size_exceeded" : 0 , "curl_non_200_response" : 0 , "curl_timeout_count" : 0 , "dcp_events_lost" : 0 , "debugger_events_lost" : 0 , "delete_events_lost" : 0 , "mutation_events_lost" : 0 , "n1ql_op_exception_count" : 0 , "timeout_count" : 0 , "timer_callback_missing_counter" : 0 , "timer_context_size_exceeded_counter" : 0 , "timer_events_lost" : 0 , "timestamp" : { "22082" : "2021-08-20T11:53:15Z" , "22088" : "2021-08-20T11:53:15Z" , "22093" : "2021-08-20T11:53:15Z" }, "v8worker_events_lost" : 0 }, "function_name" : "test" , "gocb_creds_request_counter" : 2 , "function_id" : 3142852543 , "internal_vb_distribution_stats" : { "worker_test_0" : "[0-341]" , "worker_test_1" : "[342-682]" , "worker_test_2" : "[683-1023]" }, "latency_percentile_stats" : { "100" : 0 , "50" : 0 , "80" : 0 , "90" : 0 , "95" : 0 , "99" : 0 }, "lcb_creds_request_counter" : 0 , "lcb_exception_stats" : {}, "planner_stats" : [ { "host_name" : "10.112.190.102:8096" , "start_vb" : 0 , "vb_count" : 1024 } ], "metastore_stats" : {}, "vb_distribution_stats_from_metadata" : {}, "worker_pids" : { "worker_test_0" : 22088 , "worker_test_1" : 22093 , "worker_test_2" : 22082 } } ]

          People

            abhishek.jindal Abhishek Jindal
            sujay.gad Sujay Gad
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty