Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-40765

cbbackupmgr shouldn't log/collect sensitive information

    XMLWordPrintable

    Details

    • Triage:
      Untriaged
    • Story Points:
      1
    • Is this a Regression?:
      No

      Description

      What's the issue?
      There is a couple of places where 'cbbackupmgr' may implicitly scrape sensitive information where it shouldn't. For example:
      1) 'cbbackupmgr' logs the S3 access keys at the beginning of running a sub-command (6.6.x+ only)
      2) Some of the (platform specific) commands run when collecting system information may collect the command line arguments of other processes on the system.

      Steps to reproduce #1
      1) Install Couchbase server 6.6.0-7897 on a windows server 2016
      2) Run backup to S3.
      3) Run collect-logs in S3 using cbbackupmgr, logs collect ok but it display all S3 credentials in raw text.

       
      		 
       -c localhost -u <ud>Administrator</ud> -p ******** -r backup -a s3://bkrepo --obj-access-key-id AKIAJP --obj-secret-access-key xzsNfaTXZWBf --obj-staging-dir /root/bk-staging --obj-region us-west-2 
      2020-08-04T22:48:59.413+00:00 (Cmd) mounted archive with id: 2b4c1837-86c1-4275-8934-9d138b2f7709
      		 
      2020-08-04T22:48:59.415+00:00 (Rest) GET http://localhost:8091/pools 200
      		 
      2020-08-04T22:48:59.419+00:00 (Rest) GET http://localhost:8091/pools/default 200
      		 
      2020-08-04T22:48:59.424+00:00 (Rest) GET http://localhost:8091/pools/default/buckets 200
      		 
      2020-08-04T22:48:59.468+00:00 (Rest) GET http://localhost:8091/pools/default/buckets 200
      		 
      2020-08-04T22:48:59.473+00:00 (Rest) GET http://localhost:8091/pools/default/buckets/travel-sample 200
      		 
      2020-08-04T22:48:59.475+00:00 (Rest) GET http://localhost:8091/pools 200
      		 
      2020-08-04T22:48:59.475+00:00 (Cmd) Backing up cluster 759547ebd21e733e4173ad953bb0b196
      		 
      2020-08-04T22:48:59.476+00:00 (Stats) Starting stat gathering - stat timestamp: 1596581339
      		 
      2020-08-04T22:48:59.476+00:00 (Plan) Transferring cluster configuration
      		 
      2020-08-04T22:48:59.477+00:00 (Rest)

      We need to hide it as in password
      This issue prevent upload logs to MB-40764

        Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

          Ascending order - Click to sort in descending order
          Hide
          Permalink
          thuan Thuan Nguyen added a comment -

          I tried again with --redact flag, collect-logs still has raw access key and secret key in log.

          Show
          thuan Thuan Nguyen added a comment - I tried again with --redact flag, collect-logs still has raw access key and secret key in log.
          Hide
          Permalink
          james.lee James Lee added a comment - - edited

          Hi Thuan Nguyen,

          This is a simple logical mistake that happened during the object store related argument renaming e.g. '--s3-access-key' -> '--obj-access-key'. The fix is simple, and is something that is unit tested in master, however, it's currently testing the wrong behavior.

          Show
          james.lee James Lee added a comment - - edited Hi Thuan Nguyen , This is a simple logical mistake that happened during the object store related argument renaming e.g. '--s3-access-key' -> '--obj-access-key' . The fix is simple, and is something that is unit tested in master , however, it's currently testing the wrong behavior.
          Hide
          Permalink
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-6.6.0-7903 contains backup commit 1b96331 with commit message:
          MB-40765 Correctly mask sensitive object store flags

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.0-7903 contains backup commit 1b96331 with commit message: MB-40765 Correctly mask sensitive object store flags
          Hide
          Permalink
          thuan Thuan Nguyen added a comment -

          When run collect-logs in build 6.6.0-7905,
          in system_info file, it still shows plain text access and secret key

          terSystem  EC2AMAZ-A513PGD  NisSrv.exe                                                                                                                                                             6788    171                       312500                                                        NisSrv.exe               Win32_OperatingSystem  Microsoft Windows Server 2016 Datacenter|C:\Windows|\Device\Harddisk0\Partition1  2473                 1411293             4723        4328           596              5752               2199108046848    10800               8         4431872           6788       10                      128                  40                          128                      0                   0                  0                                   8            625000        2199098613760  10.0.14393      10366976        0                    0                   
           
          		 
          cbbackupmgr.exe          .\cbbackupmgr  collect-logs -o /root  -a s3://bkrepo --obj-access-key-id AKIAJPG7 --obj-secret-access-key xzsNfaTXZWBfB

          Show
          thuan Thuan Nguyen added a comment - When run collect-logs in build 6.6.0-7905, in system_info file, it still shows plain text access and secret key terSystem EC2AMAZ-A513PGD NisSrv.exe 6788 171 312500 NisSrv.exe Win32_OperatingSystem Microsoft Windows Server 2016 Datacenter|C:\Windows|\Device\Harddisk0\Partition1 2473 1411293 4723 4328 596 5752 2199108046848 10800 8 4431872 6788 10 128 40 128 0 0 0 8 625000 2199098613760 10.0.14393 10366976 0 0   cbbackupmgr.exe .\cbbackupmgr collect-logs -o /root -a s3://bkrepo --obj-access-key-id AKIAJPG7 --obj-secret-access-key xzsNfaTXZWBfB
          Hide
          Permalink
          thuan Thuan Nguyen added a comment -

          This system_info is generated in both linux and windows when collect-logs using cloud bucket.

          Show
          thuan Thuan Nguyen added a comment - This system_info is generated in both linux and windows when collect-logs using cloud bucket.
          Hide
          Permalink
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-7.0.0-2772 contains backup commit 1b96331 with commit message:
          MB-40765 Correctly mask sensitive object store flags

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-2772 contains backup commit 1b96331 with commit message: MB-40765 Correctly mask sensitive object store flags
          Hide
          Permalink
          james.lee James Lee added a comment - - edited

          Hi Thuan Nguyen,

          This is a slightly different issue to the one described initially in this MB, however, it ultimately results in the same issue (leaking credentials). When running 'collect-logs' we will collect some information about the host system (this is platform depended) on Linux we will scrape the output of 'ps' (including the command used to execute) and on Windows we'll scrape the output of 'wmic process' (which includes the command used to execute). When providing credentials with the static credentials flags, they will be implicitly scraped and placed into the 'system_info' file.

          We could perform a trivial string replace on the output to remove the credentials, however, there are some additional security concerns e.g. we could be implicitly scraping then storing passwords/credentials provided as flags for commands other than our own. This is a slightly more difficult problem to solve, the easiest immediate fix for this issue to change the commands that we run slightly so that they don't include the command.

          For Linux this will be be a case of changing the 'ps' command from

          ps -Aww -o user,pid,lwp,ppid,nlwp,pcpu,pri,nice,vsize,rss,tty,stat,wchan:12,start,bsdtime,command

          into

          ps -Aww -o user,pid,lwp,ppid,nlwp,pcpu,pri,nice,vsize,rss,tty,stat,wchan:12,start,bsdtime

          For windows, the 'wmic process' command offers a 'brief' flag which won't include the extra information (including the command), however, it will still show all the relevant information that we would like to collect.

          Show
          james.lee James Lee added a comment - - edited Hi Thuan Nguyen , This is a slightly different issue to the one described initially in this MB, however, it ultimately results in the same issue (leaking credentials). When running 'collect-logs' we will collect some information about the host system (this is platform depended) on Linux we will scrape the output of ' ps ' (including the command used to execute) and on Windows we'll scrape the output of ' wmic process ' (which includes the command used to execute). When providing credentials with the static credentials flags, they will be implicitly scraped and placed into the ' system_info ' file. We could perform a trivial string replace on the output to remove the credentials, however, there are some additional security concerns e.g. we could be implicitly scraping then storing passwords/credentials provided as flags for commands other than our own. This is a slightly more difficult problem to solve, the easiest immediate fix for this issue to change the commands that we run slightly so that they don't include the command. For Linux this will be be a case of changing the ' ps ' command from ps -Aww -o user,pid,lwp,ppid,nlwp,pcpu,pri,nice,vsize,rss,tty,stat,wchan:12,start,bsdtime,command into ps -Aww -o user,pid,lwp,ppid,nlwp,pcpu,pri,nice,vsize,rss,tty,stat,wchan:12,start,bsdtime For windows, the ' wmic process ' command offers a ' brief ' flag which won't include the extra information (including the command), however, it will still show all the relevant information that we would like to collect.
          Hide
          Permalink
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-6.6.0-7906 contains backup commit f97e82b with commit message:
          MB-40765 Don't collect sensitive information when collecting logs

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.0-7906 contains backup commit f97e82b with commit message: MB-40765 Don't collect sensitive information when collecting logs
          Hide
          Permalink
          thuan Thuan Nguyen added a comment - - edited

          I will verify it on build 6.6.0-7907

          Show
          thuan Thuan Nguyen added a comment - - edited I will verify it on build 6.6.0-7907
          Hide
          Permalink
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-6.6.0-7907 contains backup commit 269d6ec with commit message:
          MB-40765 Add 'comm' to 'ps' call to 'exec.Command'

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.0-7907 contains backup commit 269d6ec with commit message: MB-40765 Add 'comm' to 'ps' call to 'exec.Command'
          Hide
          Permalink
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-6.6.0-7907 contains backup commit 13925be with commit message:
          MB-40765 Display the command without arguments in 'ps' output

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.0-7907 contains backup commit 13925be with commit message: MB-40765 Display the command without arguments in 'ps' output
          Hide
          Permalink
          james.lee James Lee added a comment - - edited

          'cbbackupmgr' should no longer leak S3 credentials in the logs or when running collect-logs. I have double checked on Linux/Mac and will resolve the issue once I've checked on Windows (waiting on the build).

          Steps taken to prove this is the case:
          1) Configure an archive in S3 using static credentials
          2) Run 'collect-logs' on the newly configured archive
          3) Examine the logs; we see that the values for '-obj-access-key-id' and '-obj-secret-access-key' are both obscured.
          4) Examine the 'system_info' file; we see that the output from 'ps' no longer contains the running commands arguments meaning we don't implicitly collect the S3 credentials.

          I've attached both the Linux/Mac OS collected logs used in this verification. I will update with the Windows logs once I've got them.

          Show
          james.lee James Lee added a comment - - edited ' cbbackupmgr ' should no longer leak S3 credentials in the logs or when running collect-logs. I have double checked on Linux/Mac and will resolve the issue once I've checked on Windows (waiting on the build). Steps taken to prove this is the case: 1) Configure an archive in S3 using static credentials 2) Run ' collect-logs ' on the newly configured archive 3) Examine the logs; we see that the values for ' - obj-access-key-id ' and ' -obj-secret-access-key ' are both obscured. 4) Examine the ' system_info ' file; we see that the output from ' ps ' no longer contains the running commands arguments meaning we don't implicitly collect the S3 credentials. I've attached both the Linux/Mac OS collected logs used in this verification. I will update with the Windows logs once I've got them.
          Hide
          Permalink
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-6.6.0-7908 contains backup commit 5e545ae with commit message:
          MB-40765 Don't show verbose task information using 'tasklist'

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.0-7908 contains backup commit 5e545ae with commit message: MB-40765 Don't show verbose task information using 'tasklist'
          Hide
          Permalink
          thuan Thuan Nguyen added a comment - - edited

          Verified on Windows server 2016 with build 6.6.0-7908

          PS C:\root\cbbackupmgr-collectinfo-bk-staging-2020-08-06T215908\cbbackupmgr-collectinfo-bk-staging-2020-08-06T215908> cat .\logs\backup-0.* | Select-String 'access-key'
          		 
           
          		 
          2020-08-06T21:50:59.022+00:00 (Cmd) config -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/bk-staging --obj-region us-west-2
          		 
          2020-08-06T21:51:14.736+00:00 (Cmd) backup -c localhost -u <ud>Administrator</ud> -p ***** -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/bk-staging --obj-region us-west-2
          		 
          2020-08-06T21:59:08.952+00:00 (Cmd) collect-logs -o /root -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/bk-staging --obj-region us-west-2
          		 
          2020-08-06T21:50:59.022+00:00 (Cmd) config -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/bk-staging --obj-region us-west-2
          		 
          2020-08-06T21:51:14.736+00:00 (Cmd) backup -c localhost -u <ud>Administrator</ud> -p ***** -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/bk-staging --obj-region us-west-2
          		 
           
          		 
           
          		 
          PS C:\root\cbbackupmgr-collectinfo-bk-staging-2020-08-06T215908\cbbackupmgr-collectinfo-bk-staging-2020-08-06T215908> cat .\system_info.log | Select-String 'access-key'
          		 
           
          		 
          PS C:\root\cbbackupmgr-collectinfo-bk-staging-2020-08-06T215908\cbbackupmgr-collectinfo-bk-staging-2020-08-06T215908>

          Show
          thuan Thuan Nguyen added a comment - - edited Verified on Windows server 2016 with build 6.6.0-7908 PS C:\root\cbbackupmgr-collectinfo-bk-staging-2020-08-06T215908\cbbackupmgr-collectinfo-bk-staging-2020-08-06T215908> cat .\logs\backup-0.* | Select-String 'access-key'   2020-08-06T21:50:59.022+00:00 (Cmd) config -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/bk-staging --obj-region us-west-2 2020-08-06T21:51:14.736+00:00 (Cmd) backup -c localhost -u <ud>Administrator</ud> -p ***** -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/bk-staging --obj-region us-west-2 2020-08-06T21:59:08.952+00:00 (Cmd) collect-logs -o /root -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/bk-staging --obj-region us-west-2 2020-08-06T21:50:59.022+00:00 (Cmd) config -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/bk-staging --obj-region us-west-2 2020-08-06T21:51:14.736+00:00 (Cmd) backup -c localhost -u <ud>Administrator</ud> -p ***** -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/bk-staging --obj-region us-west-2     PS C:\root\cbbackupmgr-collectinfo-bk-staging-2020-08-06T215908\cbbackupmgr-collectinfo-bk-staging-2020-08-06T215908> cat .\system_info.log | Select-String 'access-key'   PS C:\root\cbbackupmgr-collectinfo-bk-staging-2020-08-06T215908\cbbackupmgr-collectinfo-bk-staging-2020-08-06T215908>
          Hide
          Permalink
          thuan Thuan Nguyen added a comment -

          Verified on Windows server2016 with build 6.6.0-7908 with --redact flag

           
          		 
          PS C:\root\redacted-cbbackupmgr-collectinfo-bk-staging-2020-08-06T221129\redacted-cbbackupmgr-collectinfo-bk-staging-2020-08-06T221129\cbbackupmgr-collectinfo-bk-staging-2020-08
          		 
          -06T221129> cat .\system_info.log | Select-String 'access-key'
          		 
          PS C:\root\redacted-cbbackupmgr-collectinfo-bk-staging-2020-08-06T221129\redacted-cbbackupmgr-collectinfo-bk-staging-2020-08-06T221129\cbbackupmgr-collectinfo-bk-staging-2020-08
          		 
          -06T221129> cat .\logs\backup-0.* | Select-String 'access-key'
          		 
           
          		 
          2020-08-06T21:50:59.022+00:00 (Cmd) config -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/bk-staging --obj-region
          		 
          us-west-2
          		 
          2020-08-06T21:51:14.736+00:00 (Cmd) backup -c localhost -u <ud>22fde3ef0e935f43371673ce0a44869798d6b152</ud> -p ***** -r backup -a s3://bkrepo --obj-access-key-id *****
          		 
          --obj-secret-access-key ***** --obj-staging-dir /root/bk-staging --obj-region us-west-2
          		 
          2020-08-06T21:59:08.952+00:00 (Cmd) collect-logs -o /root -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/bk-staging
          		 
          --obj-region us-west-2
          		 
          2020-08-06T22:11:29.744+00:00 (Cmd) collect-logs -o /root -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/bk-staging
          		 
          --obj-region us-west-2 --redact
          		 
          2020-08-06T21:50:59.022+00:00 (Cmd) config -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/bk-staging --obj-region
          		 
          us-west-2
          		 
          2020-08-06T21:51:14.736+00:00 (Cmd) backup -c localhost -u <ud>22fde3ef0e935f43371673ce0a44869798d6b152</ud> -p ***** -r backup -a s3://bkrepo --obj-access-key-id *****
          		 
          --obj-secret-access-key ***** --obj-staging-dir /root/bk-staging --obj-region us-west-2
          		 
          2020-08-06T21:59:08.952+00:00 (Cmd) collect-logs -o /root -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/bk-staging
          		 
          --obj-region us-west-2
          		 
           
          		 
           
          		 
          PS C:\root\redacted-cbbackupmgr-collectinfo-bk-staging-2020-08-06T221129\redacted-cbbackupmgr-collectinfo-bk-staging-2020-08-06T221129\cbbackupmgr-collectinfo-bk-staging-2020-08
          		 
          -06T221129>

          Show
          thuan Thuan Nguyen added a comment - Verified on Windows server2016 with build 6.6.0-7908 with --redact flag   PS C:\root\redacted-cbbackupmgr-collectinfo-bk-staging-2020-08-06T221129\redacted-cbbackupmgr-collectinfo-bk-staging-2020-08-06T221129\cbbackupmgr-collectinfo-bk-staging-2020-08 -06T221129> cat .\system_info.log | Select-String 'access-key' PS C:\root\redacted-cbbackupmgr-collectinfo-bk-staging-2020-08-06T221129\redacted-cbbackupmgr-collectinfo-bk-staging-2020-08-06T221129\cbbackupmgr-collectinfo-bk-staging-2020-08 -06T221129> cat .\logs\backup-0.* | Select-String 'access-key'   2020-08-06T21:50:59.022+00:00 (Cmd) config -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/bk-staging --obj-region us-west-2 2020-08-06T21:51:14.736+00:00 (Cmd) backup -c localhost -u <ud>22fde3ef0e935f43371673ce0a44869798d6b152</ud> -p ***** -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/bk-staging --obj-region us-west-2 2020-08-06T21:59:08.952+00:00 (Cmd) collect-logs -o /root -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/bk-staging --obj-region us-west-2 2020-08-06T22:11:29.744+00:00 (Cmd) collect-logs -o /root -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/bk-staging --obj-region us-west-2 --redact 2020-08-06T21:50:59.022+00:00 (Cmd) config -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/bk-staging --obj-region us-west-2 2020-08-06T21:51:14.736+00:00 (Cmd) backup -c localhost -u <ud>22fde3ef0e935f43371673ce0a44869798d6b152</ud> -p ***** -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/bk-staging --obj-region us-west-2 2020-08-06T21:59:08.952+00:00 (Cmd) collect-logs -o /root -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/bk-staging --obj-region us-west-2     PS C:\root\redacted-cbbackupmgr-collectinfo-bk-staging-2020-08-06T221129\redacted-cbbackupmgr-collectinfo-bk-staging-2020-08-06T221129\cbbackupmgr-collectinfo-bk-staging-2020-08 -06T221129>
          Hide
          Permalink
          thuan Thuan Nguyen added a comment -

          Verified on mac Mojave 10.14.6 with build 6.6.0-7908

          :cbbackupmgr-collectinfo-tmp-2020-08-06T163248 thuan$ 
          Saigon:cbbackupmgr-collectinfo-tmp-2020-08-06T163248 thuan$ grep access-key system_info.log 
          Saigon:cbbackupmgr-collectinfo-tmp-2020-08-06T163248 thuan$ grep access-key ./*
          		 
          grep: ./backup: Is a directory
          		 
          grep: ./hello: Is a directory
          		 
          grep: ./logs: Is a directory
          		 
          Saigon:cbbackupmgr-collectinfo-tmp-2020-08-06T163248 thuan$ grep access-key ./*/*
          		 
          grep: ./backup/2020-08-06T16_28_35.073324-07_00: Is a directory
          		 
          ./logs/backup-0.log:2020-08-06T16:23:22.824-07:00 (Cmd) config -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /trtmp/tmp/ --obj-region us-west-2
          		 
          ./logs/backup-0.log:2020-08-06T16:28:35.041-07:00 (Cmd) backup -c localhost -u <ud>Administrator</ud> -p ***** -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /trtmp/tmp/ --obj-region us-west-2
          		 
          ./logs/backup-0.log:2020-08-06T16:32:48.296-07:00 (Cmd) collect-logs -o /trtmp/tmp/ -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /trtmp/tmp/ --obj-region us-west-2
          		 
          ./logs/backup-0.staged.log:2020-08-06T16:23:22.824-07:00 (Cmd) config -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /trtmp/tmp/ --obj-region us-west-2
          		 
          ./logs/backup-0.staged.log:2020-08-06T16:28:35.041-07:00 (Cmd) backup -c localhost -u <ud>Administrator</ud> -p ***** -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /trtmp/tmp/ --obj-region us-west-2
          		 
          ./logs/backup-0.staged.log:2020-08-06T16:29:57.515-07:00 (Cmd) info --all -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /trtmp/tmp/ --obj-region us-west-2 -r backup
          		 
          grep: ./logs/stats: Is a directory
          		 
          Saigon:cbbackupmgr-collectinfo-tmp-2020-08-06T163248 thuan$

          Show
          thuan Thuan Nguyen added a comment - Verified on mac Mojave 10.14.6 with build 6.6.0-7908 :cbbackupmgr-collectinfo-tmp-2020-08-06T163248 thuan$ Saigon:cbbackupmgr-collectinfo-tmp-2020-08-06T163248 thuan$ grep access-key system_info.log Saigon:cbbackupmgr-collectinfo-tmp-2020-08-06T163248 thuan$ grep access-key ./* grep: ./backup: Is a directory grep: ./hello: Is a directory grep: ./logs: Is a directory Saigon:cbbackupmgr-collectinfo-tmp-2020-08-06T163248 thuan$ grep access-key ./*/* grep: ./backup/2020-08-06T16_28_35.073324-07_00: Is a directory ./logs/backup-0.log:2020-08-06T16:23:22.824-07:00 (Cmd) config -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /trtmp/tmp/ --obj-region us-west-2 ./logs/backup-0.log:2020-08-06T16:28:35.041-07:00 (Cmd) backup -c localhost -u <ud>Administrator</ud> -p ***** -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /trtmp/tmp/ --obj-region us-west-2 ./logs/backup-0.log:2020-08-06T16:32:48.296-07:00 (Cmd) collect-logs -o /trtmp/tmp/ -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /trtmp/tmp/ --obj-region us-west-2 ./logs/backup-0.staged.log:2020-08-06T16:23:22.824-07:00 (Cmd) config -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /trtmp/tmp/ --obj-region us-west-2 ./logs/backup-0.staged.log:2020-08-06T16:28:35.041-07:00 (Cmd) backup -c localhost -u <ud>Administrator</ud> -p ***** -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /trtmp/tmp/ --obj-region us-west-2 ./logs/backup-0.staged.log:2020-08-06T16:29:57.515-07:00 (Cmd) info --all -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /trtmp/tmp/ --obj-region us-west-2 -r backup grep: ./logs/stats: Is a directory Saigon:cbbackupmgr-collectinfo-tmp-2020-08-06T163248 thuan$
          Hide
          Permalink
          thuan Thuan Nguyen added a comment -

          Verified on mac Mojave 10.14.6 with build 6.6.0-7908 with --redact flag

           thuan$ grep access-key ./*
          		 
          grep: ./backup: Is a directory
          		 
          grep: ./logs: Is a directory
          		 
          Saigon:cbbackupmgr-collectinfo-tmp-2020-08-06T164431 thuan$ grep access-key ./*/*
          		 
          grep: ./backup/2020-08-06T16_28_35.073324-07_00: Is a directory
          		 
          ./logs/backup-0.log:2020-08-06T16:23:22.824-07:00 (Cmd) config -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /trtmp/tmp/ --obj-region us-west-2
          		 
          ./logs/backup-0.log:2020-08-06T16:28:35.041-07:00 (Cmd) backup -c localhost -u <ud>22fde3ef0e935f43371673ce0a44869798d6b152</ud> -p ***** -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /trtmp/tmp/ --obj-region us-west-2
          		 
          ./logs/backup-0.log:2020-08-06T16:32:48.296-07:00 (Cmd) collect-logs -o /trtmp/tmp/ -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /trtmp/tmp/ --obj-region us-west-2
          		 
          ./logs/backup-0.log:2020-08-06T16:44:31.496-07:00 (Cmd) collect-logs -o /trtmp/tmp/ -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /trtmp/tmp/ --obj-region us-west-2 --redact
          		 
          ./logs/backup-0.staged.log:2020-08-06T16:23:22.824-07:00 (Cmd) config -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /trtmp/tmp/ --obj-region us-west-2
          		 
          ./logs/backup-0.staged.log:2020-08-06T16:28:35.041-07:00 (Cmd) backup -c localhost -u <ud>22fde3ef0e935f43371673ce0a44869798d6b152</ud> -p ***** -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /trtmp/tmp/ --obj-region us-west-2
          		 
          ./logs/backup-0.staged.log:2020-08-06T16:32:48.296-07:00 (Cmd) collect-logs -o /trtmp/tmp/ -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /trtmp/tmp/ --obj-region us-west-2
          		 
          grep: ./logs/stats: Is a directory
          		 
          Saigon:cbbackupmgr-collectinfo-tmp-2020-08-06T164431 thuan$

          Show
          thuan Thuan Nguyen added a comment - Verified on mac Mojave 10.14.6 with build 6.6.0-7908 with --redact flag thuan$ grep access-key ./* grep: ./backup: Is a directory grep: ./logs: Is a directory Saigon:cbbackupmgr-collectinfo-tmp-2020-08-06T164431 thuan$ grep access-key ./*/* grep: ./backup/2020-08-06T16_28_35.073324-07_00: Is a directory ./logs/backup-0.log:2020-08-06T16:23:22.824-07:00 (Cmd) config -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /trtmp/tmp/ --obj-region us-west-2 ./logs/backup-0.log:2020-08-06T16:28:35.041-07:00 (Cmd) backup -c localhost -u <ud>22fde3ef0e935f43371673ce0a44869798d6b152</ud> -p ***** -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /trtmp/tmp/ --obj-region us-west-2 ./logs/backup-0.log:2020-08-06T16:32:48.296-07:00 (Cmd) collect-logs -o /trtmp/tmp/ -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /trtmp/tmp/ --obj-region us-west-2 ./logs/backup-0.log:2020-08-06T16:44:31.496-07:00 (Cmd) collect-logs -o /trtmp/tmp/ -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /trtmp/tmp/ --obj-region us-west-2 --redact ./logs/backup-0.staged.log:2020-08-06T16:23:22.824-07:00 (Cmd) config -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /trtmp/tmp/ --obj-region us-west-2 ./logs/backup-0.staged.log:2020-08-06T16:28:35.041-07:00 (Cmd) backup -c localhost -u <ud>22fde3ef0e935f43371673ce0a44869798d6b152</ud> -p ***** -r backup -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /trtmp/tmp/ --obj-region us-west-2 ./logs/backup-0.staged.log:2020-08-06T16:32:48.296-07:00 (Cmd) collect-logs -o /trtmp/tmp/ -a s3://bkrepo --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /trtmp/tmp/ --obj-region us-west-2 grep: ./logs/stats: Is a directory Saigon:cbbackupmgr-collectinfo-tmp-2020-08-06T164431 thuan$
          Hide
          Permalink
          arunkumar Arunkumar Senthilnathan added a comment -

          Verified in centos7:

          [root@node1-mad-hatter-testing-centos7 logs]# cd cbbackupmgr-collectinfo-tests3-2020-08-06T172446
          		 
          [root@node1-mad-hatter-testing-centos7 cbbackupmgr-collectinfo-tests3-2020-08-06T172446]# grep access-key system_info.log
          		 
          [root@node1-mad-hatter-testing-centos7 cbbackupmgr-collectinfo-tests3-2020-08-06T172446]# grep access-key ./*
          		 
          grep: ./backup: Is a directory
          		 
          grep: ./logs: Is a directory
          		 
          [root@node1-mad-hatter-testing-centos7 cbbackupmgr-collectinfo-tests3-2020-08-06T172446]# grep access-key ./*/*
          		 
          grep: ./backup/2020-08-06T17_18_09.165566263-07_00: Is a directory
          		 
          ./logs/backup-0.log:2020-08-06T17:17:44.326-07:00 (Cmd) config -r backup -a s3://mb40765 --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/tests3 --obj-region us-west-1
          		 
          ./logs/backup-0.log:2020-08-06T17:18:09.156-07:00 (Cmd) backup -a s3://mb40765 -r backup --obj-access-key-id ***** --obj-secret-access-key ***** -c 10.112.194.101 -u <ud>Administrator</ud> -p ***** --obj-staging-dir /root/tests3 -obj-region us-west-1
          		 
          ./logs/backup-0.log:2020-08-06T17:24:46.411-07:00 (Cmd) collect-logs -a s3://mb40765 --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/tests3 -obj-region us-west-1
          		 
          ./logs/backup-0.staged.log:2020-08-06T17:17:44.326-07:00 (Cmd) config -r backup -a s3://mb40765 --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/tests3 --obj-region us-west-1
          		 
          ./logs/backup-0.staged.log:2020-08-06T17:18:09.156-07:00 (Cmd) backup -a s3://mb40765 -r backup --obj-access-key-id ***** --obj-secret-access-key ***** -c 10.112.194.101 -u <ud>Administrator</ud> -p ***** --obj-staging-dir /root/tests3 -obj-region us-west-1
          		 
          grep: ./logs/stats: Is a directory
          		 
          [root@node1-mad-hatter-testing-centos7 cbbackupmgr-collectinfo-tests3-2020-08-06T172446]# cd ../
          		 
          [root@node1-mad-hatter-testing-centos7 logs]# ls
          		 
          backup-0.log  backup-0.staged.log  cbbackupmgr-collectinfo-tests3-2020-08-06T172446  cbbackupmgr-collectinfo-tests3-2020-08-06T172446.zip  cbbackupmgr-collectinfo-tests3-2020-08-06T172503.zip  redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503.zip  stats
          		 
          [root@node1-mad-hatter-testing-centos7 logs]# unzip redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503.zip
          		 
          Archive:  redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503.zip
          		 
            inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/system_info.log
          		 
            inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/info.json
          		 
            inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/logs/backup-0.log
          		 
            inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/logs/backup-0.staged.log
          		 
            inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/logs/stats/cpu/backup-backup-1596759489
          		 
            inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/logs/stats/disk/backup-backup-1596759489
          		 
            inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/logs/stats/net/backup-backup-1596759489
          		 
            inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/backup/backup-meta.json
          		 
            inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/backup/2020-08-06T17_18_09.165566263-07_00/plan.json
          		 
            inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/backup/2020-08-06T17_18_09.165566263-07_00/info.json
          		 
            inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/backup/2020-08-06T17_18_09.165566263-07_00/travel-sample-f286a0b41467f3132ebb57db77a44db3/range.json
          		 
            inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/backup/2020-08-06T17_18_09.165566263-07_00/travel-sample-f286a0b41467f3132ebb57db77a44db3/restrictions.json
          		 
            inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/backup/2020-08-06T17_18_09.165566263-07_00/travel-sample-f286a0b41467f3132ebb57db77a44db3/data.json
          		 
            inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/backup/2020-08-06T17_18_09.165566263-07_00/travel-sample-f286a0b41467f3132ebb57db77a44db3/data/snapshots.zip
          		 
            inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/backup/2020-08-06T17_18_09.165566263-07_00/travel-sample-f286a0b41467f3132ebb57db77a44db3/data/failoverlogs.zip
          		 
            inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/backup/2020-08-06T17_18_09.165566263-07_00/travel-sample-f286a0b41467f3132ebb57db77a44db3/data/stats.zip
          		 
          [root@node1-mad-hatter-testing-centos7 logs]# cd redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503
          		 
          [root@node1-mad-hatter-testing-centos7 redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503]# grep access-key system_info.log
          		 
          grep: system_info.log: No such file or directory
          		 
          [root@node1-mad-hatter-testing-centos7 redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503]# grep access-key ./*
          		 
          grep: ./cbbackupmgr-collectinfo-tests3-2020-08-06T172503: Is a directory
          		 
          [root@node1-mad-hatter-testing-centos7 redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503]# grep access-key ./*/*
          		 
          grep: ./cbbackupmgr-collectinfo-tests3-2020-08-06T172503/backup: Is a directory
          		 
          grep: ./cbbackupmgr-collectinfo-tests3-2020-08-06T172503/logs: Is a directory
          		 
          [root@node1-mad-hatter-testing-centos7 redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503]# ls
          		 
          cbbackupmgr-collectinfo-tests3-2020-08-06T172503
          		 
          [root@node1-mad-hatter-testing-centos7 redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503]# cd cbbackupmgr-collectinfo-tests3-2020-08-06T172503/
          		 
          [root@node1-mad-hatter-testing-centos7 cbbackupmgr-collectinfo-tests3-2020-08-06T172503]# grep access-key system_info.log
          		 
          [root@node1-mad-hatter-testing-centos7 cbbackupmgr-collectinfo-tests3-2020-08-06T172503]# grep access-key ./*
          		 
          grep: ./backup: Is a directory
          		 
          grep: ./logs: Is a directory
          		 
          [root@node1-mad-hatter-testing-centos7 cbbackupmgr-collectinfo-tests3-2020-08-06T172503]# grep access-key ./*/*
          		 
          grep: ./backup/2020-08-06T17_18_09.165566263-07_00: Is a directory
          		 
          ./logs/backup-0.log:2020-08-06T17:17:44.326-07:00 (Cmd) config -r backup -a s3://mb40765 --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/tests3 --obj-region us-west-1
          		 
          ./logs/backup-0.log:2020-08-06T17:18:09.156-07:00 (Cmd) backup -a s3://mb40765 -r backup --obj-access-key-id ***** --obj-secret-access-key ***** -c 10.112.194.101 -u <ud>5c0bf01eafe80aa7b699e71e097488b25190d8ad</ud> -p ***** --obj-staging-dir /root/tests3 -obj-region us-west-1
          		 
          ./logs/backup-0.log:2020-08-06T17:24:46.411-07:00 (Cmd) collect-logs -a s3://mb40765 --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/tests3 -obj-region us-west-1
          		 
          ./logs/backup-0.log:2020-08-06T17:25:03.465-07:00 (Cmd) collect-logs -a s3://mb40765 --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/tests3 -obj-region us-west-1 --redact
          		 
          ./logs/backup-0.staged.log:2020-08-06T17:17:44.326-07:00 (Cmd) config -r backup -a s3://mb40765 --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/tests3 --obj-region us-west-1
          		 
          ./logs/backup-0.staged.log:2020-08-06T17:18:09.156-07:00 (Cmd) backup -a s3://mb40765 -r backup --obj-access-key-id ***** --obj-secret-access-key ***** -c 10.112.194.101 -u <ud>5c0bf01eafe80aa7b699e71e097488b25190d8ad</ud> -p ***** --obj-staging-dir /root/tests3 -obj-region us-west-1
          		 
          ./logs/backup-0.staged.log:2020-08-06T17:24:46.411-07:00 (Cmd) collect-logs -a s3://mb40765 --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/tests3 -obj-region us-west-1
          		 
          grep: ./logs/stats: Is a directory
          		 
          [root@node1-mad-hatter-testing-centos7 cbbackupmgr-collectinfo-tests3-2020-08-06T172503]#

          Show
          arunkumar Arunkumar Senthilnathan added a comment - Verified in centos7: [root@node1-mad-hatter-testing-centos7 logs]# cd cbbackupmgr-collectinfo-tests3-2020-08-06T172446 [root@node1-mad-hatter-testing-centos7 cbbackupmgr-collectinfo-tests3-2020-08-06T172446]# grep access-key system_info.log [root@node1-mad-hatter-testing-centos7 cbbackupmgr-collectinfo-tests3-2020-08-06T172446]# grep access-key ./* grep: ./backup: Is a directory grep: ./logs: Is a directory [root@node1-mad-hatter-testing-centos7 cbbackupmgr-collectinfo-tests3-2020-08-06T172446]# grep access-key ./*/* grep: ./backup/2020-08-06T17_18_09.165566263-07_00: Is a directory ./logs/backup-0.log:2020-08-06T17:17:44.326-07:00 (Cmd) config -r backup -a s3://mb40765 --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/tests3 --obj-region us-west-1 ./logs/backup-0.log:2020-08-06T17:18:09.156-07:00 (Cmd) backup -a s3://mb40765 -r backup --obj-access-key-id ***** --obj-secret-access-key ***** -c 10.112.194.101 -u <ud>Administrator</ud> -p ***** --obj-staging-dir /root/tests3 -obj-region us-west-1 ./logs/backup-0.log:2020-08-06T17:24:46.411-07:00 (Cmd) collect-logs -a s3://mb40765 --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/tests3 -obj-region us-west-1 ./logs/backup-0.staged.log:2020-08-06T17:17:44.326-07:00 (Cmd) config -r backup -a s3://mb40765 --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/tests3 --obj-region us-west-1 ./logs/backup-0.staged.log:2020-08-06T17:18:09.156-07:00 (Cmd) backup -a s3://mb40765 -r backup --obj-access-key-id ***** --obj-secret-access-key ***** -c 10.112.194.101 -u <ud>Administrator</ud> -p ***** --obj-staging-dir /root/tests3 -obj-region us-west-1 grep: ./logs/stats: Is a directory [root@node1-mad-hatter-testing-centos7 cbbackupmgr-collectinfo-tests3-2020-08-06T172446]# cd ../ [root@node1-mad-hatter-testing-centos7 logs]# ls backup-0.log backup-0.staged.log cbbackupmgr-collectinfo-tests3-2020-08-06T172446 cbbackupmgr-collectinfo-tests3-2020-08-06T172446.zip cbbackupmgr-collectinfo-tests3-2020-08-06T172503.zip redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503.zip stats [root@node1-mad-hatter-testing-centos7 logs]# unzip redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503.zip Archive: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503.zip inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/system_info.log inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/info.json inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/logs/backup-0.log inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/logs/backup-0.staged.log inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/logs/stats/cpu/backup-backup-1596759489 inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/logs/stats/disk/backup-backup-1596759489 inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/logs/stats/net/backup-backup-1596759489 inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/backup/backup-meta.json inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/backup/2020-08-06T17_18_09.165566263-07_00/plan.json inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/backup/2020-08-06T17_18_09.165566263-07_00/info.json inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/backup/2020-08-06T17_18_09.165566263-07_00/travel-sample-f286a0b41467f3132ebb57db77a44db3/range.json inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/backup/2020-08-06T17_18_09.165566263-07_00/travel-sample-f286a0b41467f3132ebb57db77a44db3/restrictions.json inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/backup/2020-08-06T17_18_09.165566263-07_00/travel-sample-f286a0b41467f3132ebb57db77a44db3/data.json inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/backup/2020-08-06T17_18_09.165566263-07_00/travel-sample-f286a0b41467f3132ebb57db77a44db3/data/snapshots.zip inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/backup/2020-08-06T17_18_09.165566263-07_00/travel-sample-f286a0b41467f3132ebb57db77a44db3/data/failoverlogs.zip inflating: redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503/cbbackupmgr-collectinfo-tests3-2020-08-06T172503/backup/2020-08-06T17_18_09.165566263-07_00/travel-sample-f286a0b41467f3132ebb57db77a44db3/data/stats.zip [root@node1-mad-hatter-testing-centos7 logs]# cd redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503 [root@node1-mad-hatter-testing-centos7 redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503]# grep access-key system_info.log grep: system_info.log: No such file or directory [root@node1-mad-hatter-testing-centos7 redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503]# grep access-key ./* grep: ./cbbackupmgr-collectinfo-tests3-2020-08-06T172503: Is a directory [root@node1-mad-hatter-testing-centos7 redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503]# grep access-key ./*/* grep: ./cbbackupmgr-collectinfo-tests3-2020-08-06T172503/backup: Is a directory grep: ./cbbackupmgr-collectinfo-tests3-2020-08-06T172503/logs: Is a directory [root@node1-mad-hatter-testing-centos7 redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503]# ls cbbackupmgr-collectinfo-tests3-2020-08-06T172503 [root@node1-mad-hatter-testing-centos7 redacted-cbbackupmgr-collectinfo-tests3-2020-08-06T172503]# cd cbbackupmgr-collectinfo-tests3-2020-08-06T172503/ [root@node1-mad-hatter-testing-centos7 cbbackupmgr-collectinfo-tests3-2020-08-06T172503]# grep access-key system_info.log [root@node1-mad-hatter-testing-centos7 cbbackupmgr-collectinfo-tests3-2020-08-06T172503]# grep access-key ./* grep: ./backup: Is a directory grep: ./logs: Is a directory [root@node1-mad-hatter-testing-centos7 cbbackupmgr-collectinfo-tests3-2020-08-06T172503]# grep access-key ./*/* grep: ./backup/2020-08-06T17_18_09.165566263-07_00: Is a directory ./logs/backup-0.log:2020-08-06T17:17:44.326-07:00 (Cmd) config -r backup -a s3://mb40765 --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/tests3 --obj-region us-west-1 ./logs/backup-0.log:2020-08-06T17:18:09.156-07:00 (Cmd) backup -a s3://mb40765 -r backup --obj-access-key-id ***** --obj-secret-access-key ***** -c 10.112.194.101 -u <ud>5c0bf01eafe80aa7b699e71e097488b25190d8ad</ud> -p ***** --obj-staging-dir /root/tests3 -obj-region us-west-1 ./logs/backup-0.log:2020-08-06T17:24:46.411-07:00 (Cmd) collect-logs -a s3://mb40765 --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/tests3 -obj-region us-west-1 ./logs/backup-0.log:2020-08-06T17:25:03.465-07:00 (Cmd) collect-logs -a s3://mb40765 --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/tests3 -obj-region us-west-1 --redact ./logs/backup-0.staged.log:2020-08-06T17:17:44.326-07:00 (Cmd) config -r backup -a s3://mb40765 --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/tests3 --obj-region us-west-1 ./logs/backup-0.staged.log:2020-08-06T17:18:09.156-07:00 (Cmd) backup -a s3://mb40765 -r backup --obj-access-key-id ***** --obj-secret-access-key ***** -c 10.112.194.101 -u <ud>5c0bf01eafe80aa7b699e71e097488b25190d8ad</ud> -p ***** --obj-staging-dir /root/tests3 -obj-region us-west-1 ./logs/backup-0.staged.log:2020-08-06T17:24:46.411-07:00 (Cmd) collect-logs -a s3://mb40765 --obj-access-key-id ***** --obj-secret-access-key ***** --obj-staging-dir /root/tests3 -obj-region us-west-1 grep: ./logs/stats: Is a directory [root@node1-mad-hatter-testing-centos7 cbbackupmgr-collectinfo-tests3-2020-08-06T172503]#
          Hide
          Permalink
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-7.0.0-2787 contains backup commit 5e545ae with commit message:
          MB-40765 Don't show verbose task information using 'tasklist'

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-2787 contains backup commit 5e545ae with commit message: MB-40765 Don't show verbose task information using 'tasklist'
          Hide
          Permalink
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-7.0.0-2787 contains backup commit 269d6ec with commit message:
          MB-40765 Add 'comm' to 'ps' call to 'exec.Command'

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-2787 contains backup commit 269d6ec with commit message: MB-40765 Add 'comm' to 'ps' call to 'exec.Command'
          Hide
          Permalink
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-7.0.0-2787 contains backup commit 13925be with commit message:
          MB-40765 Display the command without arguments in 'ps' output

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-2787 contains backup commit 13925be with commit message: MB-40765 Display the command without arguments in 'ps' output
          Hide
          Permalink
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-7.0.0-2787 contains backup commit f97e82b with commit message:
          MB-40765 Don't collect sensitive information when collecting logs

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-2787 contains backup commit f97e82b with commit message: MB-40765 Don't collect sensitive information when collecting logs
          Hide
          Permalink
          james.lee James Lee added a comment -

          Reopening because patches under this MB need to be backported to the 6.5.2. release

          Show
          james.lee James Lee added a comment - Reopening because patches under this MB need to be backported to the 6.5.2. release
          Hide
          Permalink
          wayne Wayne Siu added a comment - - edited

          James Lee
          Added to 6.5.2. Please merge the changes as soon as you can. Thanks.

          Show
          wayne Wayne Siu added a comment - - edited James Lee Added to 6.5.2. Please merge the changes as soon as you can. Thanks.
          Hide
          Permalink
          arunkumar Arunkumar Senthilnathan added a comment -

          James Lee did you mean 6.6.2? this is only for backup to s3 right? which is not there in 6.5.x

          Show
          arunkumar Arunkumar Senthilnathan added a comment - James Lee did you mean 6.6.2? this is only for backup to s3 right? which is not there in 6.5.x
          Hide
          Permalink
          james.lee James Lee added a comment -

          Arunkumar Senthilnathan, whilst fixing the 'cmdLineArgsToString' function, some other related security issues were fixed; for example some of the 'collect-logs' related 'ps' and 'wmic' invocations which would implicitly scrape the cloud credentials. It's those fixes which need to be backported into 6.5.2 since it's also possible to scrape other sensitive information; I'll update the MB to be slightly more clear.

          Show
          james.lee James Lee added a comment - Arunkumar Senthilnathan , whilst fixing the ' cmdLineArgsToString ' function, some other related security issues were fixed; for example some of the ' collect-logs ' related ' ps ' and ' wmic ' invocations which would implicitly scrape the cloud credentials. It's those fixes which need to be backported into 6.5.2 since it's also possible to scrape other sensitive information; I'll update the MB to be slightly more clear.
          Hide
          Permalink
          james.lee James Lee added a comment -

          An update on progress: security fixes have been merged into 6.5.2, the issue remains open whilst we merge the changes forward into mad-hatter and cheshire-cat (note that theses are noop merges i.e. these issue have already been fixed in MH and CC).

          Show
          james.lee James Lee added a comment - An update on progress: security fixes have been merged into 6.5.2, the issue remains open whilst we merge the changes forward into mad-hatter and cheshire-cat (note that theses are noop merges i.e. these issue have already been fixed in MH and CC).
          Hide
          Permalink
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-6.6.2-9452 contains backup commit dd571dc with commit message:
          MB-40765 Merge branch '6.5.2' into mad-hatter

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.2-9452 contains backup commit dd571dc with commit message: MB-40765 Merge branch '6.5.2' into mad-hatter
          Hide
          Permalink
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-6.6.2-9452 contains backup commit 09258c5 with commit message:
          MB-40765 [BP] Don't show verbose task information using 'tasklist'

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.2-9452 contains backup commit 09258c5 with commit message: MB-40765 [BP] Don't show verbose task information using 'tasklist'
          Hide
          Permalink
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-6.6.2-9452 contains backup commit f7a2eaa with commit message:
          MB-40765 [BP] Add 'comm' to 'ps' call to 'exec.Command'

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.2-9452 contains backup commit f7a2eaa with commit message: MB-40765 [BP] Add 'comm' to 'ps' call to 'exec.Command'
          Hide
          Permalink
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-6.6.2-9452 contains backup commit 123066a with commit message:
          MB-40765 [BP] Display the command without arguments in 'ps' output

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.2-9452 contains backup commit 123066a with commit message: MB-40765 [BP] Display the command without arguments in 'ps' output
          Hide
          Permalink
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-6.6.2-9452 contains backup commit db275f5 with commit message:
          MB-40765 [BP] Don't collect sensitive information when collecting logs

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-6.6.2-9452 contains backup commit db275f5 with commit message: MB-40765 [BP] Don't collect sensitive information when collecting logs
          Hide
          Permalink
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-7.0.0-4300 contains backup commit af8df59 with commit message:
          MB-40765 Merge branch 'mad-hatter' into master

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-4300 contains backup commit af8df59 with commit message: MB-40765 Merge branch 'mad-hatter' into master
          Hide
          Permalink
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-7.0.0-4300 contains backup commit dd571dc with commit message:
          MB-40765 Merge branch '6.5.2' into mad-hatter

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-4300 contains backup commit dd571dc with commit message: MB-40765 Merge branch '6.5.2' into mad-hatter
          Hide
          Permalink
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-7.0.0-4300 contains backup commit 09258c5 with commit message:
          MB-40765 [BP] Don't show verbose task information using 'tasklist'

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-4300 contains backup commit 09258c5 with commit message: MB-40765 [BP] Don't show verbose task information using 'tasklist'
          Hide
          Permalink
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-7.0.0-4300 contains backup commit f7a2eaa with commit message:
          MB-40765 [BP] Add 'comm' to 'ps' call to 'exec.Command'

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-4300 contains backup commit f7a2eaa with commit message: MB-40765 [BP] Add 'comm' to 'ps' call to 'exec.Command'
          Hide
          Permalink
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-7.0.0-4300 contains backup commit 123066a with commit message:
          MB-40765 [BP] Display the command without arguments in 'ps' output

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-4300 contains backup commit 123066a with commit message: MB-40765 [BP] Display the command without arguments in 'ps' output
          Hide
          Permalink
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-7.0.0-4300 contains backup commit db275f5 with commit message:
          MB-40765 [BP] Don't collect sensitive information when collecting logs

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-7.0.0-4300 contains backup commit db275f5 with commit message: MB-40765 [BP] Don't collect sensitive information when collecting logs
          Hide
          Permalink
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-6.5.2-6622 contains backup commit 09258c5 with commit message:
          MB-40765 [BP] Don't show verbose task information using 'tasklist'

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-6.5.2-6622 contains backup commit 09258c5 with commit message: MB-40765 [BP] Don't show verbose task information using 'tasklist'
          Hide
          Permalink
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-6.5.2-6622 contains backup commit f7a2eaa with commit message:
          MB-40765 [BP] Add 'comm' to 'ps' call to 'exec.Command'

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-6.5.2-6622 contains backup commit f7a2eaa with commit message: MB-40765 [BP] Add 'comm' to 'ps' call to 'exec.Command'
          Hide
          Permalink
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-6.5.2-6622 contains backup commit 123066a with commit message:
          MB-40765 [BP] Display the command without arguments in 'ps' output

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-6.5.2-6622 contains backup commit 123066a with commit message: MB-40765 [BP] Display the command without arguments in 'ps' output
          Hide
          Permalink
          build-team Couchbase Build Team added a comment -

          Build couchbase-server-6.5.2-6622 contains backup commit db275f5 with commit message:
          MB-40765 [BP] Don't collect sensitive information when collecting logs

          Show
          build-team Couchbase Build Team added a comment - Build couchbase-server-6.5.2-6622 contains backup commit db275f5 with commit message: MB-40765 [BP] Don't collect sensitive information when collecting logs
          Hide
          Permalink
          asad.zaidi Asad Zaidi added a comment - - edited

          Verification for 6.5.2-6624 on Windows:

           

          This ticket seems to discuss two related issues.

          1. The first issue is that cbbackupmgr logged AWS credentials, this only affects versions 6.6.x and above. 

          Cbbackupmgr in version 6.5.2 cannot backup to s3 so it is not affected by this issue.

          2. The second issue is that process arguments were being logged revealing sensitive information.

          We can check that this is fixed by running various cbbackupmgr sub-commands and checking if process arguments are leaked and if any sensitive information is revealed.

          I created a backup archive, performed 3 backups and used the info command to check which backups exist.

          I then used the collect-log sub command to produce the following logs:

          cbbackupmgr-collectinfo-my_archive-2021-01-28T182737.zip

          I performed a case-case-insensistive grep for 'administrator' and 'password'. In addition, I do not seem to see any process arguments.

          (It's definitely worth if someone could take a second look at these).

          An additional collection where I also performed the restore command.

            cbbackupmgr-collectinfo-my_archive-2021-01-28T185045.zip
           

           

          The grep results:

          sadzaidi@EM224 cbbackupmgr-collectinfo-my_archive-2021-01-28T185045 % grep -ri 'administrator' ./
          		 
          asadzaidi@EM224 cbbackupmgr-collectinfo-my_archive-2021-01-28T185045 % grep -ri 'password' ./     
          .//system_info.log:AdminPasswordStatus  AutomaticManagedPagefile  AutomaticResetBootOption  AutomaticResetCapability  BootOptionOnLimit  BootOptionOnWatchDog  BootROMSupported  BootStatus  BootupState  Caption          ChassisBootupState  ChassisSKUNumber  CreationClassName     CurrentTimeZone  DaylightInEffect  Description       DNSHostName      Domain     DomainRole  EnableDaylightSavingsTime  FrontPanelResetStatus  HypervisorPresent  InfraredSupported  InitialLoadInfo  InstallDate  KeyboardPasswordStatus  LastLoadInfo  Manufacturer  Model       Name             NameFormat  NetworkServerModeEnabled  NumberOfLogicalProcessors  NumberOfProcessors  OEMLogoBitmap  OEMStringArray  PartOfDomain  PauseAfterReset  PCSystemType  PCSystemTypeEx  PowerManagementCapabilities  PowerManagementSupported  PowerOnPasswordStatus  PowerState  PowerSupplyState  PrimaryOwnerContact  PrimaryOwnerName  ResetCapability  ResetCount  ResetLimit  Roles                                               Status  SupportContactDescription  SystemFamily  SystemSKUNumber  SystemStartupDelay  SystemStartupOptions  SystemStartupSetting  SystemType    ThermalState  TotalPhysicalMemory  UserName  WakeUpType  Workgroup  
          asadzaidi@EM224 cbbackupmgr-collectinfo-my_archive-2021-01-28T185045 %

           

          Show
          asad.zaidi Asad Zaidi added a comment - - edited Verification for 6.5.2-6624 on Windows:   This ticket seems to discuss two related issues. 1. The first issue is that cbbackupmgr logged AWS credentials, this only affects versions 6.6.x and above.  Cbbackupmgr in version 6.5.2 cannot backup to s3 so it is not affected by this issue. 2. The second issue is that process arguments were being logged revealing sensitive information. We can check that this is fixed by running various cbbackupmgr sub-commands and checking if process arguments are leaked and if any sensitive information is revealed. I created a backup archive, performed 3 backups and used the info command to check which backups exist. I then used the collect-log sub command to produce the following logs: cbbackupmgr-collectinfo-my_archive-2021-01-28T182737.zip I performed a case-case-insensistive grep for 'administrator' and 'password'. In addition, I do not seem to see any process arguments. (It's definitely worth if someone could take a second look at these). An additional collection where I also performed the restore command.   cbbackupmgr-collectinfo-my_archive-2021-01-28T185045.zip     The grep results: sadzaidi@EM224 cbbackupmgr-collectinfo-my_archive-2021-01-28T185045 % grep -ri 'administrator' ./ asadzaidi@EM224 cbbackupmgr-collectinfo-my_archive-2021-01-28T185045 % grep -ri 'password' ./ .//system_info.log:AdminPasswordStatus AutomaticManagedPagefile AutomaticResetBootOption AutomaticResetCapability BootOptionOnLimit BootOptionOnWatchDog BootROMSupported BootStatus BootupState Caption ChassisBootupState ChassisSKUNumber CreationClassName CurrentTimeZone DaylightInEffect Description DNSHostName Domain DomainRole EnableDaylightSavingsTime FrontPanelResetStatus HypervisorPresent InfraredSupported InitialLoadInfo InstallDate KeyboardPasswordStatus LastLoadInfo Manufacturer Model Name NameFormat NetworkServerModeEnabled NumberOfLogicalProcessors NumberOfProcessors OEMLogoBitmap OEMStringArray PartOfDomain PauseAfterReset PCSystemType PCSystemTypeEx PowerManagementCapabilities PowerManagementSupported PowerOnPasswordStatus PowerState PowerSupplyState PrimaryOwnerContact PrimaryOwnerName ResetCapability ResetCount ResetLimit Roles Status SupportContactDescription SystemFamily SystemSKUNumber SystemStartupDelay SystemStartupOptions SystemStartupSetting SystemType ThermalState TotalPhysicalMemory UserName WakeUpType Workgroup asadzaidi@EM224 cbbackupmgr-collectinfo-my_archive-2021-01-28T185045 %  

            People

            Assignee:
            thuan Thuan Nguyen
            Reporter:
            thuan Thuan Nguyen
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Dates

              Due:
              Created:
              Updated:
              Resolved:

                PagerDuty