Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-30022

testapp: Don't create a new connection for every testcase

    XMLWordPrintable

Details

    Description

      The memcached_testapp tests create a new TCP/IP connection for every testcase - see for example GetSetTest::doTestAppend()

      void GetSetTest::doTestAppend(bool compressedSource, bool compressedData) {
          MemcachedConnection& conn = getConnection();
          ...
          conn.mutate(document, 0, MutationType::Append);
      

      Given we have many thousands of tests; this results in thousands of TCP/IP connections being created and disconnected; when these are disconnected they must go into TIME_WAIT. On all platforms (but particularly Windows), this can cause random, intermittent test failures if we run out of ephemeral ports.

      I would speculate that the majority of intermittent CV failures we see on Windows are due to this TIME_WAIT issue.

      Originally (before the introduction of MemcachedConnection object) we simply connected once at the start of the test, and used the same connection for all testcases. This had the disadvantage that if one test failed then it could "pollute" the connection for future tests if it say left partially read/written data on the socket. It was for this reason that we moved to the current model. However, I feat we've gone too far the other way and now have a worse problem.

      One possible solution would be to create a connection per test suite - i.e. use a single connection (of each protocol) for all tests which share a given test fixture.

      For example, the TransportProtocols/GetSetSnappyOnOffTest suite contains 112 individual tests (which will currently each create their own connection); but there's only 4 different types of connection used for the suite:

      $ ./memcached_testapp --gtest_list_tests --gtest_filter=TransportProtocols/GetSetSnappyOnOffTest*|grep TestInvalidCompressedData|cut -d _ -f 1|uniq 
        TestInvalidCompressedData/Mcbp
        TestInvalidCompressedData/McbpIpv6
        TestInvalidCompressedData/McbpSsl
        TestInvalidCompressedData/McbpIpv6Ssl
      

      If we changed to one connection of each type per suite; that would reduce the connection count for the 38 suites which exist to ~152; instead of the current ~2000 connections created.

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            trond Trond Norbye
            drigby Dave Rigby (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty