Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-21973

[EntBackupRestore] restore hangs when erlang is crashed - does not come back with error message after 180s - panic found in logs

    XMLWordPrintable

Details

    • Untriaged
    • Yes

    Description

      Steps:

      1. Create default bucket in cluster host and load 500k items
      2. Configure backup and take backup of the cluster host
      3. Create empty default bucket in restore host
      4. Trigger restore from backup host
      5. Simultaneously kill erlang process on restore host
      6. Expected behavior: restore stops and times out after 180s with an error message
      Current behavior: restore just hangs forever - no error message thrown - panic detected in logs:

      2016-12-13T22:40:00.451-08:00 ^[[31mERROR: Queue overflow.^[[0m^[[2m -- couchbase.(*Callbacks).handleError() at callbacks.go:117^[[0m
      		 
      2016-12-13T22:40:00.451-08:00 ^[[31mERROR: Queue overflow.^[[0m^[[2m -- couchbase.(*Callbacks).handleError() at callbacks.go:117^[[0m
      		 
      2016-12-13T22:40:00.451-08:00 ^[[31mERROR: Queue overflow.^[[0m^[[2m -- couchbase.(*Callbacks).handleError() at callbacks.go:117^[[0m
      		 
      2016-12-13T22:40:00.451-08:00 ^[[31mERROR: Queue overflow.^[[0m^[[2m -- couchbase.(*Callbacks).handleError() at callbacks.go:117^[[0m
      		 
      2016-12-13T22:40:00.451-08:00 ^[[31mERROR: Queue overflow.^[[0m^[[2m -- couchbase.(*Callbacks).handleError() at callbacks.go:117^[[0m
      		 
      2016-12-13T22:40:00.451-08:00 ^[[31mERROR: Queue overflow.^[[0m^[[2m -- couchbase.(*Callbacks).handleError() at callbacks.go:117^[[0m
      		 
      2016-12-13T22:40:00.451-08:00 ^[[31mERROR: Queue overflow.^[[0m^[[2m -- couchbase.(*Callbacks).handleError() at callbacks.go:117^[[0m
      		 
      2016-12-13T22:40:00.451-08:00 ^[[31mERROR: Queue overflow.^[[0m^[[2m -- couchbase.(*Callbacks).handleError() at callbacks.go:117^[[0m
      		 
      2016-12-13T22:40:00.451-08:00 ^[[31mERROR: Queue overflow.^[[0m^[[2m -- couchbase.(*Callbacks).handleError() at callbacks.go:117^[[0m
      		 
      2016-12-13T22:40:00.451-08:00 ^[[31mERROR: Queue overflow.^[[0m^[[2m -- couchbase.(*Callbacks).handleError() at callbacks.go:117^[[0m
      		 
      2016-12-13T22:40:00.451-08:00 ^[[31mERROR: Queue overflow.^[[0m^[[2m -- couchbase.(*Callbacks).handleError() at callbacks.go:117^[[0m
      		 
      2016-12-13T22:40:00.451-08:00 ^[[31mERROR: Queue overflow.^[[0m^[[2m -- couchbase.(*Callbacks).handleError() at callbacks.go:117^[[0m
      		 
      2016-12-13T22:40:00.451-08:00 ^[[31mERROR: Queue overflow.^[[0m^[[2m -- couchbase.(*Callbacks).handleError() at callbacks.go:117^[[0m
      		 
      2016-12-13T22:40:00.451-08:00 ^[[31mERROR: Queue overflow.^[[0m^[[2m -- couchbase.(*Callbacks).handleError() at callbacks.go:117^[[0m

      same behavior is observed with cb server crash and stopping the cb server as well - 3 test cases failing due to this - behavior observed in both spock and 4.6 - attaching logs and screenshots

      Attachments

        Issue Links

          blocks

          Task - A task that needs to be done. MB-20290 4.6.0 Point Release

          • Major - Major loss of function.
          • Closed

          Activity

            People

              mikew Mike Wiederhold [X] (Inactive)
              arunkumar Arunkumar Senthilnathan (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                PagerDuty