Details
-
Task
-
Resolution: Fixed
-
Major
-
Cheshire-Cat
-
1
-
KV-Engine Sprint 2021 June
Description
Looking at the compile-time report from clang-build-analyser (http://cv.jenkins.couchbase.com/view/Nightly/job/kv_engine-nightly-clang-build-analyzer/lastSuccessfulBuild/artifact/clang_build_analyzer.log) when building all of KV-Engine and it's dependancies, we can see that of the 10 most costly headers to include, 8 are from Magma
*** Expensive headers:
|
214999 ms: ../magma/tests/magma_gtest.h (included 43 times, avg 4999 ms)
|
174591 ms: ../third_party/googletest/googletest/include/gtest/gtest.h (included 269 times, avg 649 ms)
|
164202 ms: ../magma/include/libmagma/magma.h (included 139 times, avg 1181 ms)
|
137946 ms: ../magma/lsm/lsm_tree.h (included 79 times, avg 1746 ms)
|
134045 ms: tlm/deps/json.exploded/include/nlohmann/json.hpp (included 241 times, avg 556 ms)
|
131177 ms: ../magma/lsm/mvcc.h (included 87 times, avg 1507 ms)
|
125446 ms: ../magma/lsm/config.h (included 80 times, avg 1568 ms)
|
109811 ms: ../magma/include/libmagma/stats.h (included 142 times, avg 773 ms)
|
105467 ms: tlm/deps/folly.exploded/include/folly/portability/GTest.h (included 161 times, avg 655 ms)
|
97648 ms: ../magma/lsm/sstable/sstable.h (included 94 times, avg 1038 ms)
|
Given the entire build is around 47 minutes user+sys time, Just including those Magma headers is around 14 minutes :
**** Time summary:
|
Compilation (695 times):
|
Parsing (frontend): 2419.7 s
|
Codegen & opts (backend): 443.3 s
|
We should try to optimise these headers - remove unnecessary includes; de-inline code which doesn't need to be etc.
Attachments
For Gerrit Dashboard: MB-46770 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
155221,10 | MB-46770 [2/8]: Move magma_gtest.h implementation to magma_test.cc | master | magma | Status: MERGED | +2 | +1 |
155317,5 | MB-46770 [1/8]: Avoid compiling magma source files twice | master | magma | Status: MERGED | +2 | +1 |
155318,7 | MB-46770 [3/8]: Move macros and random from util/common.h to own headers | master | magma | Status: MERGED | +2 | +1 |
155458,5 | MB-46770 [4/8]: De-inline util/file/file_system.h | master | magma | Status: MERGED | +2 | +1 |
155473,7 | MB-46770 [5/8]: Minimise includes in lsm/config.h | master | magma | Status: MERGED | +2 | +1 |
155474,10 | MB-46770 [6/8]: Minimise includes in magma_gtest.h | master | magma | Status: MERGED | +2 | +1 |
155475,7 | MB-46770 [7/8]: Minimise includes in wal.h / wal_segment.h | master | magma | Status: MERGED | +2 | +1 |
155476,7 | MB-46770 [8/8]: Minimise includes in lsm_tree.h | master | magma | Status: MERGED | +2 | +1 |
155734,5 | MB-46770 [9/8]: Minimise includes of lsm_tree.h | master | magma | Status: MERGED | +2 | +1 |