Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-24734

cbbackupmgr : no error when archive location is out of disk space

    XMLWordPrintable

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Done
    • 4.6.3, 5.0.0, 5.1.0, 5.0.1, 5.1.1, 6.0.0, 5.5.0, 5.5.1, 5.1.2, 5.5.2, 5.1.3, 5.5.3, 6.0.1, 6.0.2, 5.5.4, 6.0.3, 5.5.5, 5.5.6, 5.5.5-MP2
    • 6.5.0
    • tools

    Description

      Build : 5.0.0-3064

      The disk where the archive dir was located went out of space while a backup was being taken. The backup logs show the relevant error :

      2017-06-08T21:15:03.965+05:30 (DCP) (vb 437) Creating dcp stream with start seqno 0, end seqno 149654, vbuuid 0, snap start seqno 0, and snap end seqno 0
      2017-06-08T21:15:03.965+05:30 (DCP) (vb 802) Creating dcp stream with start seqno 0, end seqno 149832, vbuuid 0, snap start seqno 0, and snap end seqno 0
      [FDB ERR] Error in FLUSH on a database file '/tmp/backup/backup/2017-06-08T21_10_23.739528753+05_30/other-1-98ba038193f683dae6168f87ad2a1277/data/shard_0.fdb', errno = 28: 'No space left on device'
      2017-06-08T21:39:02.683+05:30 ERRO: unknown forestdb error: -65 – couchbase.dcpClient.Mutation() at dcp.go:314
      [FDB ERR] Error in FLUSH on a database file '/tmp/backup/backup/2017-06-08T21_10_23.739528753+05_30/other-1-98ba038193f683dae6168f87ad2a1277/data/shard_0.fdb', errno = 28: 'No space left on device'
      2017-06-08T21:39:04.064+05:30 ERRO: unknown forestdb error: -65 – couchbase.dcpClient.Mutation() at dcp.go:314
      [FDB ERR] Error in FLUSH on a database file '/tmp/backup/backup/2017-06-08T21_10_23.739528753+05_30/other-1-98ba038193f683dae6168f87ad2a1277/data/shard_0.fdb', errno = 28: 'No space left on device'
      2017-06-08T21:39:04.667+05:30 ERRO: unknown forestdb error: -65 – couchbase.dcpClient.Mutation() at dcp.go:314
      [FDB ERR] Error in FLUSH on a database file '/tmp/backup/backup/2017-06-08T21_10_23.739528753+05_30/other-1-98ba038193f683dae6168f87ad2a1277/data/shard_0.fdb', errno = 28: 'No space left on device'
      2017-06-08T21:39:05.156+05:30 ERRO: unknown forestdb error: -65 – couchbase.dcpCli

      But the CLI is hung for quite some time. There is no error on the CLI.

      root@espresso:/opt/couchbase/bin# ./cbbackupmgr backup -c 172.16.12.9 -u Administrator -p password -a /tmp/backup -r backup

      Backing up to 2017-06-08T21_10_23.739528753+05_30
      Copying at 0B (about 5h 17m 14s remaining) 19397899 items / 29.67GB
      ephemeral [==========================================================================================================================================================================] 100.00%
      other-1 [================= ] 10.44%

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            Build couchbase-server-6.5.0-1525 contains backup commit 6f4175d with commit message:
            MB-24734 Error message when archive out of space

            build-team Couchbase Build Team added a comment - Build couchbase-server-6.5.0-1525 contains backup commit 6f4175d with commit message: MB-24734 Error message when archive out of space

            automated and verified

            arunkumar Arunkumar Senthilnathan (Inactive) added a comment - automated and verified
            owend Daniel Owen added a comment -

            Not yet closed as still to be decided if should backport to 6.0.5

            owend Daniel Owen added a comment - Not yet closed as still to be decided if should backport to 6.0.5

            People

              carlos.gonzalez Carlos Gonzalez Betancort (Inactive)
              mihir.kamdar Mihir Kamdar (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty