Uploaded image for project: 'Couchbase Python Client Library'
  1. Couchbase Python Client Library
  2. PYCBC-1029

Memory leak on get_multi() and other ..._multi functions

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • 3.2.4
    • 3.0.5
    • library
    • None
    • CentOS 7 x86_64, Python 3.6 (IUS)
    • 1

    Description

      Memory usage quickly builds up on querying many batches of keys, with bucket.get_multi() or by using any other ..._multi() functions, eg. upsert_mult().

      You may test it with the following script:

      from couchbase.bucket import Bucket
      import resource
       
      keys = [line.rstrip() for line in open('1000_keys.txt', 'rt')]
      bucket = Bucket('couchbase://hostname/a_bucket', username=..., password=...)
       
      for i in range(100):
          bucket.get_multi(keys)
          mem_rss = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
          print('Memory usage: ', mem_rss)

      Libcouchbase 2.10.7 + couchbase 2.5.12 Python package does not have this issue, but 3.0.5 does have.

      Attachments

        1. asan_get_multi_1000_fix.txt
          571 kB
        2. asan_get_multi_1000.txt
          573 kB
        3. cb_leak_get.py
          0.7 kB
        4. cb_leaks_2.py
          1 kB
        5. cb_leaks.py
          0.7 kB
        6. cb-leaks-2.xml.gz
          8 kB
        7. leak.xml.gz
          120 kB
        8. leak.xml-1.gz
          7 kB
        9. leak-2.xml.gz
          9 kB
        10. leak-3.8.xml.gz
          91 kB
        11. leak-get.xml.gz
          8 kB
        12. san-leaks.txt
          1 kB
        13. valgrind_multi_upsert.zip
          1.68 MB
        For Gerrit Dashboard: PYCBC-1029
        # Subject Branch Project Status CR V

        Activity

          People

            jared.casey Jared Casey
            andor_toth Andor Tóth
            Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty