Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-4376

setup get HDD and memory totale and free space wrong (0Mb)

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 1.7.1.1, 1.7.2, 2.0.1
    • Fix Version/s: 2.0-beta
    • Component/s: ns_server
    • Security Level: Public
    • Labels:
      None
    • Environment:

      Description

      when connecting for the first time to the HTTP setup GUI (http://195.154.218.88:8091/index.html)

      I got for the HDD :
      Configure Disk Storage
      Path: /opt/membase/var/lib/membase/data Free: 0 GB

      And for the memory (if creating new cluster):
      RAM Available: 0 MB
      Per Server RAM Quota: MB (256 MB — 0 MB)

      Or for the memory (if joining existing cluster):
      Attention - Prepare join failed. This server does not have sufficient memory to support the cluster quota (Cluster Quota is 14968MB per node, this server only has 0MB)!

      Joining the cluster with the CLI do the same :
      /opt/membase/bin/membase rebalance -c X.X.X.1:8091 -uLOGIN -pPASSWD --server-add=X.X.X.2:8091
      ERROR: unable to server-add X.X.X.2:8091 (400) Bad Request
      [u'Prepare join failed. This server does not have sufficient memory to support the cluster quota (Cluster Quota is 14968MB per node, this server only has 0MB)!']

      On those 2 identical virtual server (Openvz virtualisation) i have :
      #free -m
      total used free shared buffers cached
      Mem: 15992 1054 14937 0 0 0
      -/+ buffers/cache: 1054 14937
      Swap: 0 0 0
      #df -h
      Filesystem Size Used Avail Use% Mounted on
      /dev/simfs 98G 70G 23G 76% /
      tmpfs 7.9G 0 7.9G 0% /lib/init/rw
      tmpfs 7.9G 0 7.9G 0% /dev/shm

      As you see : The code in charge of detecting hdd and memory space fails and reports 0Mb , thus prevents me from creatingor joining a cluster.

      I happy to give you any info you need : just ask.

      Kinds regards, Aurelien

        Issue Links

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

          Hide
          orellem orel lem added a comment -
          Show
          orellem orel lem added a comment - Related bug report : http://www.couchbase.org/issues/browse/FORUM-516
          Hide
          orellem orel lem added a comment -

          info ++ :
          When i install membase on the parent server (not the Guest server) : it works.
          Obviously this is not a solution (install on the parent) but a good info .

          Regards, Aurélien lemaire

          Show
          orellem orel lem added a comment - info ++ : When i install membase on the parent server (not the Guest server) : it works. Obviously this is not a solution (install on the parent) but a good info . Regards, Aurélien lemaire
          Hide
          farshid Farshid Ghods (Inactive) added a comment -

          Can you please also attach the diagnostics from that node
          You can grab the digs by going to node:8091/diag

          Show
          farshid Farshid Ghods (Inactive) added a comment - Can you please also attach the diagnostics from that node You can grab the digs by going to node:8091/diag
          Hide
          orellem orel lem added a comment -

          diagnostic file as requested

          Show
          orellem orel lem added a comment - diagnostic file as requested
          Hide
          vorobev Anton Vorobev added a comment - - edited

          Hi, guys.
          I've got the same problem. And it seems, that I've found the reason.
          OpenVZ kernel does not provide information about buffers and/or cache (depends on version) via /proc/meminfo. On the other side, Erlang's memsup parses /proc/meminfo in not completely safe way (it calls sscanf() on char* which is NULL in this case) and crashes with segmentation fault. I've fixed it and now everything works for me. I've already sent this patch to Erlang team via github.

          Orel, could you please provide content of your /proc/meminfo from OpenVZ container just to check if it is the same problem.

          Thanks.

          Show
          vorobev Anton Vorobev added a comment - - edited Hi, guys. I've got the same problem. And it seems, that I've found the reason. OpenVZ kernel does not provide information about buffers and/or cache (depends on version) via /proc/meminfo. On the other side, Erlang's memsup parses /proc/meminfo in not completely safe way (it calls sscanf() on char* which is NULL in this case) and crashes with segmentation fault. I've fixed it and now everything works for me. I've already sent this patch to Erlang team via github. Orel, could you please provide content of your /proc/meminfo from OpenVZ container just to check if it is the same problem. Thanks.
          Hide
          orellem orel lem added a comment -

          Hi folks,

          Here my /proc/meminfo on the guest OS (where i try to run membase):
          cat /proc/meminfo
          MemTotal: 16376004 kB
          MemFree: 15134748 kB
          SwapTotal: 0 kB
          SwapFree: 0 kB

          And here my proc/meminfo on the parent OS (for comparaison):
          MemTotal: 16376004 kB
          MemFree: 886768 kB
          Buffers: 1440120 kB
          Cached: 12620992 kB
          SwapCached: 1536 kB
          Active: 4361308 kB
          Inactive: 9971924 kB
          Active(anon): 241416 kB
          Inactive(anon): 31000 kB
          Active(file): 4119892 kB
          Inactive(file): 9940924 kB
          Unevictable: 0 kB
          Mlocked: 0 kB
          SwapTotal: 16022520 kB
          SwapFree: 16015832 kB
          Dirty: 76 kB
          Writeback: 0 kB
          AnonPages: 270996 kB
          Mapped: 33196 kB
          Shmem: 300 kB
          Slab: 1042376 kB
          SReclaimable: 1000140 kB
          SUnreclaim: 42236 kB
          KernelStack: 1544 kB
          PageTables: 8972 kB
          NFS_Unstable: 0 kB
          Bounce: 0 kB
          WritebackTmp: 0 kB
          CommitLimit: 24210520 kB
          Committed_AS: 2215672 kB
          VmallocTotal: 34359738367 kB
          VmallocUsed: 98108 kB
          VmallocChunk: 34351264004 kB
          HardwareCorrupted: 0 kB
          HugePages_Total: 0
          HugePages_Free: 0
          HugePages_Rsvd: 0
          HugePages_Surp: 0
          Hugepagesize: 2048 kB
          DirectMap4k: 9664 kB
          DirectMap2M: 16701440 kB

          Any chance to have this erlang patch applied anytime soon ?
          @Anton : Can you refer to you erlang bug report and github patch url ?
          @anton : Thanks for the info and your troubleshooting , though.

          Regards, Aurélien

          Show
          orellem orel lem added a comment - Hi folks, Here my /proc/meminfo on the guest OS (where i try to run membase): cat /proc/meminfo MemTotal: 16376004 kB MemFree: 15134748 kB SwapTotal: 0 kB SwapFree: 0 kB And here my proc/meminfo on the parent OS (for comparaison): MemTotal: 16376004 kB MemFree: 886768 kB Buffers: 1440120 kB Cached: 12620992 kB SwapCached: 1536 kB Active: 4361308 kB Inactive: 9971924 kB Active(anon): 241416 kB Inactive(anon): 31000 kB Active(file): 4119892 kB Inactive(file): 9940924 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 16022520 kB SwapFree: 16015832 kB Dirty: 76 kB Writeback: 0 kB AnonPages: 270996 kB Mapped: 33196 kB Shmem: 300 kB Slab: 1042376 kB SReclaimable: 1000140 kB SUnreclaim: 42236 kB KernelStack: 1544 kB PageTables: 8972 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 24210520 kB Committed_AS: 2215672 kB VmallocTotal: 34359738367 kB VmallocUsed: 98108 kB VmallocChunk: 34351264004 kB HardwareCorrupted: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 9664 kB DirectMap2M: 16701440 kB Any chance to have this erlang patch applied anytime soon ? @Anton : Can you refer to you erlang bug report and github patch url ? @anton : Thanks for the info and your troubleshooting , though. Regards, Aurélien
          Hide
          vorobev Anton Vorobev added a comment -

          So, it's the same trouble. That is why it works on VE0 (master) and does not work in container.

          Patch is here: https://github.com/vorobev/otp/compare/maint-memsup.patch
          I've sent report via email according to their rules: https://github.com/erlang/otp/wiki/submitting-patches

          If you want, you can download two files:

          https://raw.github.com/vorobev/otp/maint-memsup/lib/os_mon/c_src/memsup.c
          and
          https://raw.github.com/vorobev/otp/maint-memsup/lib/os_mon/c_src/memsup.h

          and compile memsup by yourself: gcc memsup.c -o memsup

          And put it to /opt/membase/lib/erlang/lib/os_mon-2.2.5/priv/bin/ , but make a backup copy of original memsup, just for sure.

          Everything should work.

          Regards,
          Anton.

          Show
          vorobev Anton Vorobev added a comment - So, it's the same trouble. That is why it works on VE0 (master) and does not work in container. Patch is here: https://github.com/vorobev/otp/compare/maint-memsup.patch I've sent report via email according to their rules: https://github.com/erlang/otp/wiki/submitting-patches If you want, you can download two files: https://raw.github.com/vorobev/otp/maint-memsup/lib/os_mon/c_src/memsup.c and https://raw.github.com/vorobev/otp/maint-memsup/lib/os_mon/c_src/memsup.h and compile memsup by yourself: gcc memsup.c -o memsup And put it to /opt/membase/lib/erlang/lib/os_mon-2.2.5/priv/bin/ , but make a backup copy of original memsup, just for sure. Everything should work. Regards, Anton.
          Hide
          vorobev Anton Vorobev added a comment -

          Fix has been accepted : https://github.com/erlang/otp/commit/a95ebdd1232122d168931f55c9863960316576e2

          Suppose, ticket can be closed. Anyway, it is not a membase issue.

          regards,
          Anton.

          Show
          vorobev Anton Vorobev added a comment - Fix has been accepted : https://github.com/erlang/otp/commit/a95ebdd1232122d168931f55c9863960316576e2 Suppose, ticket can be closed. Anyway, it is not a membase issue. regards, Anton.
          Hide
          orellem orel lem added a comment -

          Cheers Anton,

          Long time no update ... sorry about that.

          As it has been accepted and committed to erlang github.

          Is it possible to get that new erlang code version into a new membase version : let's say the 1.7.2 any time soon

          Any info on the ETA would be much appreciated.

          regards, Aurélien

          Show
          orellem orel lem added a comment - Cheers Anton, Long time no update ... sorry about that. As it has been accepted and committed to erlang github. Is it possible to get that new erlang code version into a new membase version : let's say the 1.7.2 any time soon Any info on the ETA would be much appreciated. regards, Aurélien
          Hide
          farshid Farshid Ghods (Inactive) added a comment -

          Steve,

          is this change picked up by the erlang version used for building 1.8 or 2.0 ?

          if not we can take this into account and have it fix by 1.8.x ?

          -Farshid

          Show
          farshid Farshid Ghods (Inactive) added a comment - Steve, is this change picked up by the erlang version used for building 1.8 or 2.0 ? if not we can take this into account and have it fix by 1.8.x ? -Farshid
          Hide
          alkondratenko Aleksey Kondratenko (Inactive) added a comment -

          Anton's fix is not yet part of any formal erlang release. http://i.imgur.com/UH47B.png

          Show
          alkondratenko Aleksey Kondratenko (Inactive) added a comment - Anton's fix is not yet part of any formal erlang release. http://i.imgur.com/UH47B.png
          Hide
          steve Steve Yen added a comment -

          From alk...

          I believe it was assigned to linux_installer because we will eventually need to ship erlang vm version with openvz fix.

          Show
          steve Steve Yen added a comment - From alk... I believe it was assigned to linux_installer because we will eventually need to ship erlang vm version with openvz fix.
          Hide
          rrrb Ricardo added a comment -

          Just a quick word to thank Anton Vorobev !
          Your quick fix (recompiling memsup with the fixed version) did the trick.

          Once Couchbase restarted, the avalaible space was visible.

          Tested with Couchbase 1.8 on a OpenVZ instance of a centos-5-standard_5.2-1 server

          Show
          rrrb Ricardo added a comment - Just a quick word to thank Anton Vorobev ! Your quick fix (recompiling memsup with the fixed version) did the trick. Once Couchbase restarted, the avalaible space was visible. Tested with Couchbase 1.8 on a OpenVZ instance of a centos-5-standard_5.2-1 server
          Hide
          perry Perry Krug added a comment -

          Apparently this issue has resurfaced for OpenVZ on 2.0.1

          Reopening to make sure it gets fixed

          Show
          perry Perry Krug added a comment - Apparently this issue has resurfaced for OpenVZ on 2.0.1 Reopening to make sure it gets fixed
          Hide
          maria Maria McDuff (Inactive) added a comment -

          closing as dupes.

          Show
          maria Maria McDuff (Inactive) added a comment - closing as dupes.

            People

            • Assignee:
              steve Steve Yen
              Reporter:
              orellem orel lem
            • Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Gerrit Reviews

                There are no open Gerrit changes