Leak with tracing enabled

We have a service written in Python3, connecting to one or more Couchbase clusters.

During normal operations hundreds operations per second are being processed and each of them translates to one or more Couchbase operations.

The service is deployed on Ubuntu 16.04 amd64 instances, under uwsgi.

We ran the same application using:

libcouchbase 2.10.3-1

python3-couchbase 2.5.0

and with 2.3.3:

libcouchbase 2.8.4-1

python3-couchbase 2.3.3

With the old versions (2.3.3 and 2.8.4-1 for libcouchbase) the memory allocation is stable over 24h.

With the new versions (2.5.0 and 2.10.3-1 for libcouchbase) the memory grows steadily over 10-12 hours until the application needs to be restarted.

0

We used also tried python3-couchbase 2.5.5 and still, the memory leak is there.

==32187== 66 bytes in 11 blocks are possibly lost in loss record 701 of 2,641

==32187== at 0x4C2FFAC: calloc (vg_replace_malloc.c:762)

==32187== by 0x9D88AF0: lcb::trace::Span::add_tag(char const*, int, char const*, unsigned long) (span.cc:375)

==32187== by 0x9D88A6D: lcb::trace::Span::add_tag(char const*, int, char const*) (span.cc:362)

==32187== by 0x9D8885B: lcb::trace::Span::Span(lcbtrace_TRACER*, char const*, unsigned long, lcbtrace_REF_TYPE, lcb::trace::Span*) (span.cc:326)

==32187== by 0x9D87B49: lcbtrace_span_start (tracer.cc:38)

==32187== by 0x9A6954B: pycbc_Context_init (ext.c:1386)

==32187== by 0x9A698BF: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A78E13: pycbc_Bucket_lookup_in (get.c:408)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53D352: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x5401EE: ??? (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53BC92: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187==

==32187== 72 bytes in 12 blocks are possibly lost in loss record 710 of 2,641

==32187== at 0x4C2FFAC: calloc (vg_replace_malloc.c:762)

==32187== by 0x9D88AF0: lcb::trace::Span::add_tag(char const*, int, char const*, unsigned long) (span.cc:375)

==32187== by 0x9D88A6D: lcb::trace::Span::add_tag(char const*, int, char const*) (span.cc:362)

==32187== by 0x9D8885B: lcb::trace::Span::Span(lcbtrace_TRACER*, char const*, unsigned long, lcbtrace_REF_TYPE, lcb::trace::Span*) (span.cc:326)

==32187== by 0x9D87B49: lcbtrace_span_start (tracer.cc:38)

==32187== by 0x9A69467: pycbc_Context_init (ext.c:1386)

==32187== by 0x9A69840: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A69948: pycbc_explicit_named_setup (ext.c:1276)

==32187== by 0x9A77033: pycbc_oputil_iter_multi (oputil.c:401)

==32187== by 0x9A78D04: sdlookup_common (get.c:388)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53D352: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187==

==32187== 90 bytes in 10 blocks are possibly lost in loss record 715 of 2,641

==32187== at 0x4C2FFAC: calloc (vg_replace_malloc.c:762)

==32187== by 0x9D88AF0: lcb::trace::Span::add_tag(char const*, int, char const*, unsigned long) (span.cc:375)

==32187== by 0x9D88A6D: lcb::trace::Span::add_tag(char const*, int, char const*) (span.cc:362)

==32187== by 0x9D8883C: lcb::trace::Span::Span(lcbtrace_TRACER*, char const*, unsigned long, lcbtrace_REF_TYPE, lcb::trace::Span*) (span.cc:325)

==32187== by 0x9D87B49: lcbtrace_span_start (tracer.cc:38)

==32187== by 0x9A6954B: pycbc_Context_init (ext.c:1386)

==32187== by 0x9A698BF: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A78E13: pycbc_Bucket_lookup_in (get.c:408)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53D352: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x5401EE: ??? (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53BC92: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187==

==32187== 90 bytes in 15 blocks are possibly lost in loss record 716 of 2,641

==32187== at 0x4C2FFAC: calloc (vg_replace_malloc.c:762)

==32187== by 0x9D88AF0: lcb::trace::Span::add_tag(char const*, int, char const*, unsigned long) (span.cc:375)

==32187== by 0x9D88A6D: lcb::trace::Span::add_tag(char const*, int, char const*) (span.cc:362)

==32187== by 0x9D8885B: lcb::trace::Span::Span(lcbtrace_TRACER*, char const*, unsigned long, lcbtrace_REF_TYPE, lcb::trace::Span*) (span.cc:326)

==32187== by 0x9D87B49: lcbtrace_span_start (tracer.cc:38)

==32187== by 0x9A69467: pycbc_Context_init (ext.c:1386)

==32187== by 0x9A69840: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A69948: pycbc_explicit_named_setup (ext.c:1276)

==32187== by 0x9A7809B: handle_single_lookup (get.c:359)

==32187== by 0x9A77059: pycbc_oputil_iter_multi (oputil.c:401)

==32187== by 0x9A78D04: sdlookup_common (get.c:388)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187==

==32187== 99 bytes in 11 blocks are possibly lost in loss record 734 of 2,641

==32187== at 0x4C2FFAC: calloc (vg_replace_malloc.c:762)

==32187== by 0x9D88AF0: lcb::trace::Span::add_tag(char const*, int, char const*, unsigned long) (span.cc:375)

==32187== by 0x9D88A6D: lcb::trace::Span::add_tag(char const*, int, char const*) (span.cc:362)

==32187== by 0x9D8883C: lcb::trace::Span::Span(lcbtrace_TRACER*, char const*, unsigned long, lcbtrace_REF_TYPE, lcb::trace::Span*) (span.cc:325)

==32187== by 0x9D87B49: lcbtrace_span_start (tracer.cc:38)

==32187== by 0x9A69467: pycbc_Context_init (ext.c:1386)

==32187== by 0x9A69840: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A69948: pycbc_explicit_named_setup (ext.c:1276)

==32187== by 0x9A77033: pycbc_oputil_iter_multi (oputil.c:401)

==32187== by 0x9A78D04: sdlookup_common (get.c:388)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53D352: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187==

==32187== 110 bytes in 11 blocks are possibly lost in loss record 740 of 2,641

==32187== at 0x4C2DE96: malloc (vg_replace_malloc.c:309)

==32187== by 0x50E3489: strdup (strdup.c:42)

==32187== by 0x9D88AC6: lcb::trace::Span::add_tag(char const*, int, char const*, unsigned long) (span.cc:371)

==32187== by 0x9D88A6D: lcb::trace::Span::add_tag(char const*, int, char const*) (span.cc:362)

==32187== by 0x9D87F54: lcbtrace_span_add_tag_str (span.cc:80)

==32187== by 0x9A694CB: pycbc_Context_init (ext.c:1420)

==32187== by 0x9A698BF: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A78E13: pycbc_Bucket_lookup_in (get.c:408)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53D352: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x5401EE: ??? (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53BC92: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187==

==32187== 117 bytes in 13 blocks are possibly lost in loss record 745 of 2,641

==32187== at 0x4C2FFAC: calloc (vg_replace_malloc.c:762)

==32187== by 0x9D88AF0: lcb::trace::Span::add_tag(char const*, int, char const*, unsigned long) (span.cc:375)

==32187== by 0x9D88A6D: lcb::trace::Span::add_tag(char const*, int, char const*) (span.cc:362)

==32187== by 0x9D8883C: lcb::trace::Span::Span(lcbtrace_TRACER*, char const*, unsigned long, lcbtrace_REF_TYPE, lcb::trace::Span*) (span.cc:325)

==32187== by 0x9D87B49: lcbtrace_span_start (tracer.cc:38)

==32187== by 0x9A69467: pycbc_Context_init (ext.c:1386)

==32187== by 0x9A69840: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A69948: pycbc_explicit_named_setup (ext.c:1276)

==32187== by 0x9A7809B: handle_single_lookup (get.c:359)

==32187== by 0x9A77059: pycbc_oputil_iter_multi (oputil.c:401)

==32187== by 0x9A78D04: sdlookup_common (get.c:388)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187==

==32187== 130 bytes in 13 blocks are possibly lost in loss record 931 of 2,641

==32187== at 0x4C2DE96: malloc (vg_replace_malloc.c:309)

==32187== by 0x50E3489: strdup (strdup.c:42)

==32187== by 0x9D88AC6: lcb::trace::Span::add_tag(char const*, int, char const*, unsigned long) (span.cc:371)

==32187== by 0x9D88A6D: lcb::trace::Span::add_tag(char const*, int, char const*) (span.cc:362)

==32187== by 0x9D87F54: lcbtrace_span_add_tag_str (span.cc:80)

==32187== by 0x9A694CB: pycbc_Context_init (ext.c:1420)

==32187== by 0x9A69840: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A69948: pycbc_explicit_named_setup (ext.c:1276)

==32187== by 0x9A77033: pycbc_oputil_iter_multi (oputil.c:401)

==32187== by 0x9A78D04: sdlookup_common (get.c:388)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53D352: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187==

==32187== 136 bytes in 8 blocks are possibly lost in loss record 935 of 2,641

==32187== at 0x4C2E4B6: operator new(unsigned long) (vg_replace_malloc.c:344)

==32187== by 0x9D2C124: void std::__cxx11::basic_string<char, std::char_traits, std::allocator >::_M_construct<char const*>(char const*, char const*, std::forward_iterator_tag) (basic_string.tcc:223)

==32187== by 0xAC46C4B: std::__cxx11::basic_string<char, std::char_traits, std::allocator >::basic_string(char const*, std::allocator const&) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21)

==32187== by 0x9D887C5: lcb::trace::Span::Span(lcbtrace_TRACER*, char const*, unsigned long, lcbtrace_REF_TYPE, lcb::trace::Span*) (span.cc:319)

==32187== by 0x9D87B49: lcbtrace_span_start (tracer.cc:38)

==32187== by 0x9A69467: pycbc_Context_init (ext.c:1386)

==32187== by 0x9A69840: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A69948: pycbc_explicit_named_setup (ext.c:1276)

==32187== by 0x9A7809B: handle_single_lookup (get.c:359)

==32187== by 0x9A77059: pycbc_oputil_iter_multi (oputil.c:401)

==32187== by 0x9A78D04: sdlookup_common (get.c:388)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187==

==32187== 153 bytes in 9 blocks are possibly lost in loss record 940 of 2,641

==32187== at 0x4C2E4B6: operator new(unsigned long) (vg_replace_malloc.c:344)

==32187== by 0x9D2C124: void std::__cxx11::basic_string<char, std::char_traits, std::allocator >::_M_construct<char const*>(char const*, char const*, std::forward_iterator_tag) (basic_string.tcc:223)

==32187== by 0xAC46C4B: std::__cxx11::basic_string<char, std::char_traits, std::allocator >::basic_string(char const*, std::allocator const&) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21)

==32187== by 0x9D887C5: lcb::trace::Span::Span(lcbtrace_TRACER*, char const*, unsigned long, lcbtrace_REF_TYPE, lcb::trace::Span*) (span.cc:319)

==32187== by 0x9D87B49: lcbtrace_span_start (tracer.cc:38)

==32187== by 0x9A69467: pycbc_Context_init (ext.c:1386)

==32187== by 0x9A69840: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A69948: pycbc_explicit_named_setup (ext.c:1276)

==32187== by 0x9A77033: pycbc_oputil_iter_multi (oputil.c:401)

==32187== by 0x9A78D04: sdlookup_common (get.c:388)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53D352: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187==

==32187== 170 bytes in 10 blocks are possibly lost in loss record 956 of 2,641

==32187== at 0x4C2E4B6: operator new(unsigned long) (vg_replace_malloc.c:344)

==32187== by 0x9D2C124: void std::__cxx11::basic_string<char, std::char_traits, std::allocator >::_M_construct<char const*>(char const*, char const*, std::forward_iterator_tag) (basic_string.tcc:223)

==32187== by 0xAC46C4B: std::__cxx11::basic_string<char, std::char_traits, std::allocator >::basic_string(char const*, std::allocator const&) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21)

==32187== by 0x9D887C5: lcb::trace::Span::Span(lcbtrace_TRACER*, char const*, unsigned long, lcbtrace_REF_TYPE, lcb::trace::Span*) (span.cc:319)

==32187== by 0x9D87B49: lcbtrace_span_start (tracer.cc:38)

==32187== by 0x9A6954B: pycbc_Context_init (ext.c:1386)

==32187== by 0x9A698BF: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A78E13: pycbc_Bucket_lookup_in (get.c:408)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53D352: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x5401EE: ??? (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53BC92: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187==

==32187== 180 bytes in 18 blocks are possibly lost in loss record 960 of 2,641

==32187== at 0x4C2DE96: malloc (vg_replace_malloc.c:309)

==32187== by 0x50E3489: strdup (strdup.c:42)

==32187== by 0x9D88AC6: lcb::trace::Span::add_tag(char const*, int, char const*, unsigned long) (span.cc:371)

==32187== by 0x9D88A6D: lcb::trace::Span::add_tag(char const*, int, char const*) (span.cc:362)

==32187== by 0x9D87F54: lcbtrace_span_add_tag_str (span.cc:80)

==32187== by 0x9A694CB: pycbc_Context_init (ext.c:1420)

==32187== by 0x9A69840: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A69948: pycbc_explicit_named_setup (ext.c:1276)

==32187== by 0x9A7809B: handle_single_lookup (get.c:359)

==32187== by 0x9A77059: pycbc_oputil_iter_multi (oputil.c:401)

==32187== by 0x9A78D04: sdlookup_common (get.c:388)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187==

==32187== 192 bytes in 12 blocks are possibly lost in loss record 971 of 2,641

==32187== at 0x4C2FFAC: calloc (vg_replace_malloc.c:762)

==32187== by 0x9D88AF0: lcb::trace::Span::add_tag(char const*, int, char const*, unsigned long) (span.cc:375)

==32187== by 0x9D88A6D: lcb::trace::Span::add_tag(char const*, int, char const*) (span.cc:362)

==32187== by 0x9D87F54: lcbtrace_span_add_tag_str (span.cc:80)

==32187== by 0x9A694CB: pycbc_Context_init (ext.c:1420)

==32187== by 0x9A698BF: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A78E13: pycbc_Bucket_lookup_in (get.c:408)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53D352: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x5401EE: ??? (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53BC92: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x540B0A: PyEval_EvalCodeEx (in /home/tl/memleak/venv/bin/python3)

==32187==

==32187== 260 bytes in 13 blocks are possibly lost in loss record 1,334 of 2,641

==32187== at 0x4C2FFAC: calloc (vg_replace_malloc.c:762)

==32187== by 0x9D88AF0: lcb::trace::Span::add_tag(char const*, int, char const*, unsigned long) (span.cc:375)

==32187== by 0x9D88A6D: lcb::trace::Span::add_tag(char const*, int, char const*) (span.cc:362)

==32187== by 0x9D87F54: lcbtrace_span_add_tag_str (span.cc:80)

==32187== by 0x9A694CB: pycbc_Context_init (ext.c:1420)

==32187== by 0x9A69840: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A69948: pycbc_explicit_named_setup (ext.c:1276)

==32187== by 0x9A77033: pycbc_oputil_iter_multi (oputil.c:401)

==32187== by 0x9A78D04: sdlookup_common (get.c:388)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53D352: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x5401EE: ??? (in /home/tl/memleak/venv/bin/python3)

==32187==

==32187== 408 bytes in 17 blocks are possibly lost in loss record 1,379 of 2,641

==32187== at 0x4C2FFAC: calloc (vg_replace_malloc.c:762)

==32187== by 0x9D88AF0: lcb::trace::Span::add_tag(char const*, int, char const*, unsigned long) (span.cc:375)

==32187== by 0x9D88A6D: lcb::trace::Span::add_tag(char const*, int, char const*) (span.cc:362)

==32187== by 0x9D87F54: lcbtrace_span_add_tag_str (span.cc:80)

==32187== by 0x9A694CB: pycbc_Context_init (ext.c:1420)

==32187== by 0x9A69840: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A69948: pycbc_explicit_named_setup (ext.c:1276)

==32187== by 0x9A7809B: handle_single_lookup (get.c:359)

==32187== by 0x9A77059: pycbc_oputil_iter_multi (oputil.c:401)

==32187== by 0x9A78D04: sdlookup_common (get.c:388)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53D352: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187==

==32187== 480 bytes in 10 blocks are possibly lost in loss record 1,541 of 2,641

==32187== at 0x4C2FFAC: calloc (vg_replace_malloc.c:762)

==32187== by 0x9D88A9B: lcb::trace::Span::add_tag(char const*, int, char const*, unsigned long) (span.cc:367)

==32187== by 0x9D88A6D: lcb::trace::Span::add_tag(char const*, int, char const*) (span.cc:362)

==32187== by 0x9D8883C: lcb::trace::Span::Span(lcbtrace_TRACER*, char const*, unsigned long, lcbtrace_REF_TYPE, lcb::trace::Span*) (span.cc:325)

==32187== by 0x9D87B49: lcbtrace_span_start (tracer.cc:38)

==32187== by 0x9A6954B: pycbc_Context_init (ext.c:1386)

==32187== by 0x9A698BF: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A78E13: pycbc_Bucket_lookup_in (get.c:408)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53D352: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x5401EE: ??? (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53BC92: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187==

==32187==

==32187== 528 bytes in 11 blocks are possibly lost in loss record 1,624 of 2,641

==32187== at 0x4C2FFAC: calloc (vg_replace_malloc.c:762)

==32187== by 0x9D88A9B: lcb::trace::Span::add_tag(char const*, int, char const*, unsigned long) (span.cc:367)

==32187== by 0x9D88A6D: lcb::trace::Span::add_tag(char const*, int, char const*) (span.cc:362)

==32187== by 0x9D8885B: lcb::trace::Span::Span(lcbtrace_TRACER*, char const*, unsigned long, lcbtrace_REF_TYPE, lcb::trace::Span*) (span.cc:326)

==32187== by 0x9D87B49: lcbtrace_span_start (tracer.cc:38)

==32187== by 0x9A6954B: pycbc_Context_init (ext.c:1386)

==32187== by 0x9A698BF: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A78E13: pycbc_Bucket_lookup_in (get.c:408)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53D352: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x5401EE: ??? (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53BC92: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187==

==32187== 528 bytes in 11 blocks are possibly lost in loss record 1,625 of 2,641

==32187== at 0x4C2FFAC: calloc (vg_replace_malloc.c:762)

==32187== by 0x9D88A9B: lcb::trace::Span::add_tag(char const*, int, char const*, unsigned long) (span.cc:367)

==32187== by 0x9D88A6D: lcb::trace::Span::add_tag(char const*, int, char const*) (span.cc:362)

==32187== by 0x9D8883C: lcb::trace::Span::Span(lcbtrace_TRACER*, char const*, unsigned long, lcbtrace_REF_TYPE, lcb::trace::Span*) (span.cc:325)

==32187== by 0x9D87B49: lcbtrace_span_start (tracer.cc:38)

==32187== by 0x9A69467: pycbc_Context_init (ext.c:1386)

==32187== by 0x9A69840: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A69948: pycbc_explicit_named_setup (ext.c:1276)

==32187== by 0x9A77033: pycbc_oputil_iter_multi (oputil.c:401)

==32187== by 0x9A78D04: sdlookup_common (get.c:388)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53D352: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187==

==32187==

==32187== 576 bytes in 12 blocks are possibly lost in loss record 1,710 of 2,641

==32187== at 0x4C2FFAC: calloc (vg_replace_malloc.c:762)

==32187== by 0x9D88A9B: lcb::trace::Span::add_tag(char const*, int, char const*, unsigned long) (span.cc:367)

==32187== by 0x9D88A6D: lcb::trace::Span::add_tag(char const*, int, char const*) (span.cc:362)

==32187== by 0x9D87F54: lcbtrace_span_add_tag_str (span.cc:80)

==32187== by 0x9A694CB: pycbc_Context_init (ext.c:1420)

==32187== by 0x9A698BF: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A78E13: pycbc_Bucket_lookup_in (get.c:408)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53D352: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x5401EE: ??? (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53BC92: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x540B0A: PyEval_EvalCodeEx (in /home/tl/memleak/venv/bin/python3)

==32187==

==32187== 576 bytes in 12 blocks are possibly lost in loss record 1,711 of 2,641

==32187== at 0x4C2FFAC: calloc (vg_replace_malloc.c:762)

==32187== by 0x9D88A9B: lcb::trace::Span::add_tag(char const*, int, char const*, unsigned long) (span.cc:367)

==32187== by 0x9D88A6D: lcb::trace::Span::add_tag(char const*, int, char const*) (span.cc:362)

==32187== by 0x9D8885B: lcb::trace::Span::Span(lcbtrace_TRACER*, char const*, unsigned long, lcbtrace_REF_TYPE, lcb::trace::Span*) (span.cc:326)

==32187== by 0x9D87B49: lcbtrace_span_start (tracer.cc:38)

==32187== by 0x9A69467: pycbc_Context_init (ext.c:1386)

==32187== by 0x9A69840: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A69948: pycbc_explicit_named_setup (ext.c:1276)

==32187== by 0x9A77033: pycbc_oputil_iter_multi (oputil.c:401)

==32187== by 0x9A78D04: sdlookup_common (get.c:388)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53D352: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187==

==32187== 576 bytes in 12 blocks are possibly lost in loss record 1,712 of 2,641

==32187== at 0x4C2FFAC: calloc (vg_replace_malloc.c:762)

==32187== by 0x9D88A9B: lcb::trace::Span::add_tag(char const*, int, char const*, unsigned long) (span.cc:367)

==32187== by 0x9D88A6D: lcb::trace::Span::add_tag(char const*, int, char const*) (span.cc:362)

==32187== by 0x9D8883C: lcb::trace::Span::Span(lcbtrace_TRACER*, char const*, unsigned long, lcbtrace_REF_TYPE, lcb::trace::Span*) (span.cc:325)

==32187== by 0x9D87B49: lcbtrace_span_start (tracer.cc:38)

==32187== by 0x9A69467: pycbc_Context_init (ext.c:1386)

==32187== by 0x9A69840: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A69948: pycbc_explicit_named_setup (ext.c:1276)

==32187== by 0x9A7809B: handle_single_lookup (get.c:359)

==32187== by 0x9A77059: pycbc_oputil_iter_multi (oputil.c:401)

==32187== by 0x9A78D04: sdlookup_common (get.c:388)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187==

==32187==

==32187== 624 bytes in 13 blocks are possibly lost in loss record 1,749 of 2,641

==32187== at 0x4C2FFAC: calloc (vg_replace_malloc.c:762)

==32187== by 0x9D88A9B: lcb::trace::Span::add_tag(char const*, int, char const*, unsigned long) (span.cc:367)

==32187== by 0x9D88A6D: lcb::trace::Span::add_tag(char const*, int, char const*) (span.cc:362)

==32187== by 0x9D87F54: lcbtrace_span_add_tag_str (span.cc:80)

==32187== by 0x9A694CB: pycbc_Context_init (ext.c:1420)

==32187== by 0x9A69840: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A69948: pycbc_explicit_named_setup (ext.c:1276)

==32187== by 0x9A77033: pycbc_oputil_iter_multi (oputil.c:401)

==32187== by 0x9A78D04: sdlookup_common (get.c:388)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53D352: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x5401EE: ??? (in /home/tl/memleak/venv/bin/python3)

==32187==

==32187==

==32187== 672 bytes in 7 blocks are possibly lost in loss record 1,795 of 2,641

==32187== at 0x4C2E4B6: operator new(unsigned long) (vg_replace_malloc.c:344)

==32187== by 0x9D87B22: lcbtrace_span_start (tracer.cc:38)

==32187== by 0x9A69467: pycbc_Context_init (ext.c:1386)

==32187== by 0x9A69840: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A69948: pycbc_explicit_named_setup (ext.c:1276)

==32187== by 0x9A7809B: handle_single_lookup (get.c:359)

==32187== by 0x9A77059: pycbc_oputil_iter_multi (oputil.c:401)

==32187== by 0x9A78D04: sdlookup_common (get.c:388)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53D352: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x5401EE: ??? (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53BC92: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187==

==32187== 720 bytes in 15 blocks are possibly lost in loss record 1,814 of 2,641

==32187== at 0x4C2FFAC: calloc (vg_replace_malloc.c:762)

==32187== by 0x9D88A9B: lcb::trace::Span::add_tag(char const*, int, char const*, unsigned long) (span.cc:367)

==32187== by 0x9D88A6D: lcb::trace::Span::add_tag(char const*, int, char const*) (span.cc:362)

==32187== by 0x9D8885B: lcb::trace::Span::Span(lcbtrace_TRACER*, char const*, unsigned long, lcbtrace_REF_TYPE, lcb::trace::Span*) (span.cc:326)

==32187== by 0x9D87B49: lcbtrace_span_start (tracer.cc:38)

==32187== by 0x9A69467: pycbc_Context_init (ext.c:1386)

==32187== by 0x9A69840: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A69948: pycbc_explicit_named_setup (ext.c:1276)

==32187== by 0x9A7809B: handle_single_lookup (get.c:359)

==32187== by 0x9A77059: pycbc_oputil_iter_multi (oputil.c:401)

==32187== by 0x9A78D04: sdlookup_common (get.c:388)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187==

==32187==

==32187== 816 bytes in 17 blocks are possibly lost in loss record 1,910 of 2,641

==32187== at 0x4C2FFAC: calloc (vg_replace_malloc.c:762)

==32187== by 0x9D88A9B: lcb::trace::Span::add_tag(char const*, int, char const*, unsigned long) (span.cc:367)

==32187== by 0x9D88A6D: lcb::trace::Span::add_tag(char const*, int, char const*) (span.cc:362)

==32187== by 0x9D87F54: lcbtrace_span_add_tag_str (span.cc:80)

==32187== by 0x9A694CB: pycbc_Context_init (ext.c:1420)

==32187== by 0x9A69840: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A69948: pycbc_explicit_named_setup (ext.c:1276)

==32187== by 0x9A7809B: handle_single_lookup (get.c:359)

==32187== by 0x9A77059: pycbc_oputil_iter_multi (oputil.c:401)

==32187== by 0x9A78D04: sdlookup_common (get.c:388)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53D352: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187==

==32187== 864 bytes in 9 blocks are possibly lost in loss record 1,925 of 2,641

==32187== at 0x4C2E4B6: operator new(unsigned long) (vg_replace_malloc.c:344)

==32187== by 0x9D87B22: lcbtrace_span_start (tracer.cc:38)

==32187== by 0x9A69467: pycbc_Context_init (ext.c:1386)

==32187== by 0x9A69840: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A69948: pycbc_explicit_named_setup (ext.c:1276)

==32187== by 0x9A77033: pycbc_oputil_iter_multi (oputil.c:401)

==32187== by 0x9A78D04: sdlookup_common (get.c:388)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53D352: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x5401EE: ??? (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53BC92: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x540B0A: PyEval_EvalCodeEx (in /home/tl/memleak/venv/bin/python3)

==32187==

==32187==

==32187== 960 bytes in 10 blocks are possibly lost in loss record 2,084 of 2,641

==32187== at 0x4C2E4B6: operator new(unsigned long) (vg_replace_malloc.c:344)

==32187== by 0x9D87B22: lcbtrace_span_start (tracer.cc:38)

==32187== by 0x9A6954B: pycbc_Context_init (ext.c:1386)

==32187== by 0x9A698BF: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A78E13: pycbc_Bucket_lookup_in (get.c:408)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53D352: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x5401EE: ??? (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53BC92: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x540B0A: PyEval_EvalCodeEx (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x4EC3F6: ??? (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x5C20E6: PyObject_Call (in /home/tl/memleak/venv/bin/python3)

==32187==

==32187==

==32187== 1,037,727 (47,952 direct, 989,775 indirect) bytes in 999 blocks are definitely lost in loss record 2,640 of 2,641

==32187== at 0x4C2DE96: malloc (vg_replace_malloc.c:309)

==32187== by 0x9A6942B: pycbc_Context_init (ext.c:1378)

==32187== by 0x9A69840: pycbc_Tracer_start_span (ext.c:1518)

==32187== by 0x9A69948: pycbc_explicit_named_setup (ext.c:1276)

==32187== by 0x9A7809B: handle_single_lookup (get.c:359)

==32187== by 0x9A77059: pycbc_oputil_iter_multi (oputil.c:401)

==32187== by 0x9A78D04: sdlookup_common (get.c:388)

==32187== by 0x4EA136: PyCFunction_Call (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53D352: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x5401EE: ??? (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x53BC92: PyEval_EvalFrameEx (in /home/tl/memleak/venv/bin/python3)

==32187== by 0x540B0A: PyEval_EvalCodeEx (in /home/tl/memleak/venv/bin/python3)