Q2, Q3, CI2, CI3 stale=false latency increased by ~100% in Spock (fdb only)

CHANGELOG for forestdb

 * Commit: <a href='https://github.com/couchbase/forestdb/commit/7bc7cfc7728442ec211031f88b6fb1c674223ba7'>7bc7cfc7728442ec211031f88b6fb1c674223ba7</a> (in build: 876)

   Author: Sundar Sridharan

   <a href="https://issues.couchbase.com/browse/MB-18747">MB-18747</a>: Maintain snapshots in lists instead of avl trees

   Lists allow for lock free traversal in one part while

   concurrent modifications are happening to another part of the list.

   With avl trees this is not possible since some modifications can

   incur tree rotations.

   Change-Id: I7b2e1fb8f55a09fe63b2da936f86c91577047255

   Reviewed-on: <a href="http://review.couchbase.org/63762">http://review.couchbase.org/63762</a>

   Tested-by: buildbot <build@couchbase.com>

   Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>

 * Commit: <a href='https://github.com/couchbase/forestdb/commit/2b543a8dbecbb3c75ad3b5eff54fd1528e832b00'>2b543a8dbecbb3c75ad3b5eff54fd1528e832b00</a> (in build: 876)

   Author: Sundar Sridharan

   <a href="https://issues.couchbase.com/browse/MB-18747">MB-18747</a>: Enable MVCC WAL

   Change-Id: I9ccbd3580fb4371f3dc5c0ee376be146e463a2c6

   Reviewed-on: <a href="http://review.couchbase.org/65511">http://review.couchbase.org/65511</a>

   Tested-by: buildbot <build@couchbase.com>

   Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>

 * Commit: <a href='https://github.com/couchbase/forestdb/commit/a54d2f706b2fa6f93c6b006e4278dd2046a8a25c'>a54d2f706b2fa6f93c6b006e4278dd2046a8a25c</a> (in build: 876)

   Author: Sundar Sridharan

   <a href="https://issues.couchbase.com/browse/MB-18747">MB-18747</a>: Lock-Free MVCC WAL Snapshots

   + Insert each version of key (wal_item) into the latest mutable

     snapshot key and sequence trees of its KV Store

   + Snapshot open marks latest mutable snapshot as immutable

   + New inserts do not overwrite immutable snapshot items

     but will always go into their own mutable snapshot tree

   + Snapshot iterators do overlapping merge-sort over all

     past snapshots lock-free unaffected by concurrent mutations

   + Sequence iterators do simple merge-sort over all past

     snapshots lock-free

   + Snapshot close decrements ref_counts on current and past

     snapshot handles

   + WAL flush removes all items and snapshots which are closed.

   This avoids the concurrency issues seen in golang when

   very large number of iterator_init calls are made.

   Change-Id: I9ad0ad9e30923b152fed0db1d9bee6a927a0aabf

   Reviewed-on: <a href="http://review.couchbase.org/65198">http://review.couchbase.org/65198</a>

   Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>

   Tested-by: buildbot <build@couchbase.com>

 * Commit: <a href='https://github.com/couchbase/forestdb/commit/0348c70b70180ebd082da95329b4df60539f69e4'>0348c70b70180ebd082da95329b4df60539f69e4</a> (in build: 876)

   Author: Sundar Sridharan

   <a href="https://issues.couchbase.com/browse/MB-18747">MB-18747</a>: Ref count past snapshots to simplify garbage collection

   Each new snapshot simply increments ref count on prior snapshots

   until wal flush. When snapshot is closed, ref counts of past

   snapshots are decremented.

   This allows for simple and O(1) checks during garbage collection

   and eliminates the _wal_can_discard() complexity completely.

   Change-Id: Ic9ac858275c7c0cd5a7b6e25a2224a63bd9aff64

   Reviewed-on: <a href="http://review.couchbase.org/63763">http://review.couchbase.org/63763</a>

   Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>

   Tested-by: Chiyoung Seo <chiyoung@couchbase.com>

 * Commit: <a href='https://github.com/couchbase/forestdb/commit/5d0563776d50618e39fa342f9c352d884211480d'>5d0563776d50618e39fa342f9c352d884211480d</a> (in build: 876)

   Author: abhinavdangeti

   Address leak found with prev_filename through address sanitizer

   14:27:10 Direct leak of 16 byte(s) in 1 object(s) allocated from:

   14:27:10     #0 0x4cde12 in __interceptor_malloc (/home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/build/forestdb/tests/functional/multi_kv_functional_test+0x4cde12)

   14:27:10     #1 0x56cc2c in fdb_fetch_header /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/forestdb.cc:249:34

   14:27:10     #2 0x56f38a in _fdb_open /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/forestdb.cc:1737:13

   14:27:10     #3 0x57484b in fdb_open_custom_cmp /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/src/forestdb.cc:898:10

   14:27:10     #4 0x4f1ae0 in multi_kv_test(unsigned char, unsigned long) /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/tests/functional/multi_kv_functional_test.cc:438:13

   14:27:10     #5 0x50e2b3 in main /home/couchbase/jenkins/workspace/forestdb-addresssanitizer-master/forestdb/tests/functional/multi_kv_functional_test.cc:2527:13

   14:27:10     #6 0x2b8e2aa0e76c in __libc_start_main /build/buildd/eglibc-2.15/csu/libc-start.c:226

   Change-Id: Ib923774ff6508dd296590bf910375a1090a7dcde

   Reviewed-on: <a href="http://review.couchbase.org/65698">http://review.couchbase.org/65698</a>

   Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>

   Tested-by: Chiyoung Seo <chiyoung@couchbase.com>