Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-36956

Migrate to Facebook Folly executors for CPU & IO background tasks

    XMLWordPrintable

Details

    Description

      Investigate the feasibility of migrating from our own executor to Folly::ThreadPoolExecutor (CPU and IO) instead.

      Notes:

      What:
      1. Add a folly::IOThreadPoolExecutor + HHWheelTimer (does it already have one by default?) for scheduled tasks
      2. Add micro-benchmark to test basic functionality
      2a. Simple ping-pong (schedule then execute) latency for single task
      2b. Expand to larger number of outstanding scheduled events (10,000s - see MB-39815 which requires up to 1024 tasks per Bucket).

      2. Add stats similar to current scheduler / runtime stats - maybe viaTaskStatsCallback? Should give us functionally same as current ExecutorPool (NonIO?)
      3. Migrate Tasks over - ideally allowing to run on either thread pools (at least compile-time, preferably runtime at startup; ideally on-the-fly but maybe not feasible).
      3a) DurabilityTimeoutTask is good first candidate - already a problem - see MB-39815.

      Attachments

        Issue Links

          blocks

          Improvement - An improvement or enhancement to an existing feature or task. MB-39815 Avoid busy-polling for SyncWrite timeout checking

          • Major - Major loss of function.
          • Closed
          causes

          Bug - A problem which impairs or prevents the functions of the product. MB-43015 Observing 50% drop xdcr throughput in 7.0.0-3870 build

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          depends on

          Bug - A problem which impairs or prevents the functions of the product. MB-42027 Data race on ThreadLocal<EventuallyPersistentEngine*> when using FollyExecutorPool

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. MB-41805 False positive TSan race under FollyExecutorPool due to mixed GCC / Clang built libraries

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MB-41844 Data race in FollyExecutorPool::doTaskQStat

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MB-41896 ThreadSanitizer CHECK failed: sanitizer_deadlock_detector1.cc:166 "((!dd.isHeld(&lt->dd, m->id))) != (0)" (0x0, 0x0)

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MB-42029 Data race during EventuallyPersistentEngine dtor with FollyExecutorPool

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. MB-42204 FollyExecutorPool: Intermittent hang during unregisterTaskable() + concurrent schedule()

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MB-43156 FollyExecutorPool does not update wakeTime when wake() called

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. MB-32204 Investigate safer GlobalTask scheduling API

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. MB-26832 Bring the many 'copied' task trigger mechanisms into the task API

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. MB-26831 ActiveStreamCheckpointProcessorTask::run wakeUp logic could be simplified

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. MB-36610 Investigate using eventfd() for inter-thread notification

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. MB-38426 Reduce SyncWrite latency

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. MB-43240 Unify memcached and ep-engine thread pools

          • Major - Major loss of function.
          • Closed
          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              drigby Dave Rigby (Inactive)
              trond Trond Norbye
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty