The runtime exception of C + + is not extensible

Https://groups.google.com/forum/#!topic/seastar-dev/RuK-OajeqHk

Https://www.google.com/search?ei=gTH-Wtr5O4WQsAW-io7wCQ&q=dl_iterate_phdr+gcc+exception&oq=dl_iterate_ Phdr+gcc+exception&gs_l=psy-ab.3 ... 5055.7797.0.8041.10.8.0.0.0.0.382.997.3-3.3.0....0...1.1.64.psy-ab. 7.1.306...33i160k1.0.ddzskq-paa4

G++/GLIBC Combination provides a C + + runtime with non scaleable
C + + exception. This is dues-to-global locks in stack unwinding code.

To summarize all the locks we have now and their purpose:
1. There is a lock in _unwind_find_fde (LIBGCC) that protects
List of fdes registered with __register_frame* functions.
The catch is this dynamically linked binary do
So all of it protects is checking a certain list is empty.
This lock no longer relevant in gcc7 since there is a path there
Checks that list is empty outside of the lock.
2. The lock in Dl_iterate_phdr (GLIBC), that protects loaded object
List against runtime object loading/unloading.

To get rid of the first lock one have to use GCC7.

To get rid of the second one, we can use the fact, we do not
Load/unload objects dynamically (at least for now). To does that we
Can mirror all ELF headers information in Seastar and provide our
own DL_ITERATE_PHDR symbol which uses this mirror without locking.

Unfortunately there is another gotcha in this APPROACH:DL_ITERATE_PHDR
Supplied by glibc never calls more then one callback simultaneously as an
Unintended consequences of the lock there, but unfortunately LIBGCC relies
On this to maintain small cache of translations. The access to the cache is
Not protected by any lock since-until now only one callback could has
Run at a time. But luckily LIBGCC cannot use the cache if older version
Of Dl_phdr_info is provided to the callback because the older version
Do not has a indication that loaded object list may has changed,
So LIBGCC does is know when the cache should be invalidated and disables it
Entirely. By calling the callback with old version of Dl_phdr_info from
Our DL_ITERATE_PHDR we can effectively make LIBGCC callback thread safe.

Diff--git A/configure.py b/configure.py
Index facdd8f: 33B310B 100755
---a/configure.py
+ + b/configure.py
@@ -297,6 +297,8 @@ -297,6 ('--static-boost ', dest = ' staticboost ', action = ' store_
Add_tristate (arg_parser, name = ' Hwloc ', dest = ' Hwloc ', help = ' hwloc ')
Arg_parser.add_argument ('--enable-gcc6-concepts ', dest= ' gcc6_concepts ', action= ' store_true ', Default=False,
Help= ' Enable experimental support for C + + concepts as implemented in GCC 6 ')
+arg_parser.add_argument ('--disable-exception-scalability-workaround ', dest= ' exception_workaround ', action= ' Store_true ', Default=false,
+ help= ' disabling override of Dl_iterate_phdr symbol to workaround C + + exception scalability ')
args = Arg_parser.parse_args ()

Libnet = [
@@ -337,6 +339,7 @@ -337,6 = [
' net/inet_address.cc ',
' rpc/rpc.cc ',
' rpc/lz4_compressor.cc ',
+ ' core/exception_hacks.cc ',
]

Protobuf = [
@@ -392,6 +395,9 @@ -392,6 args.gcc6_concepts:
Defines.append (' have_gcc6_concepts ')
Args.user_cflags + = '-fconcepts '

+if Args.exception_workaround:
+ defines.append (' No_exception_hack ')
+
If Args.staticcxx:
Libs = libs.replace ('-lstdc++ ', ')
Libs + = '-static-libgcc-static-libstdc++ '
Diff--git A/core/exception_hacks.cc b/core/exception_hacks.cc
New File Mode 100644
Index 0000000..6f04630
---/dev/null
+ + b/core/exception_hacks.cc
@@ -0,0 +1,108 @@
+/*
+ * This file was open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * Distributed with the additional information regarding copyright
+ * ownership. You are not a use of this file except in compliance with the License.
+ *
+ * Obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * unless required by applicable or agreed to in writing,
+ * Software distributed under the License is distributed in an
+ * ' As is ' BASIS, without warranties OR CONDITIONS of any
+ * KIND, either express or implied. See the License for the
+ * Specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Copyright (C) scylladb
+ */
+
+//The purpose of the hacks here are to workaround C + + Exception scalability problem
+//with GCC and glibc. For the best result gcc-7 is required.
+//
+//to summarize all the locks we had now and their purpose:
+//1. There is a lock in _unwind_find_fde (LIBGCC) that protects
+//List of fdes registered with __register_frame* functions.
+//The catch is that dynamically linked binary does not does that,
+//so all, it protects is checking, a certain list is empty.
+//this lock no longer relevant in gcc-7 since there is a path there
+//that checks this list is empty outside of the lock and it'll be
+//always true for us.
+//2. The lock in Dl_iterate_phdr (GLIBC), that protects loaded object
+//list against runtime object loading/unloading.
+//
+//to get rid of the first lock using gcc-7 is required.
+//
+//to get rid of the second one we can use the fact that we do not
+//Load/unload objects dynamically (at least for now). To does that we
+//can mirror all elf headers information in Seastar and provide our
+//own DL_ITERATE_PHDR symbol which uses this mirror without locking.
+//
+//Unfortunately there is another gotcha in this APPROACH:DL_ITERATE_PHDR
+//supplied by glibc never calls and then one callback simultaneously as an
+//unintended consequences of the lock there, but unfortunately LIBGCC relies
+//on this to maintain small cache of translations. The access to the cache is
+//not protected by no lock since up until now only one callback could has
+//run at a time. But luckily LIBGCC cannot use the cache if older version
+//of Dl_phdr_info is provided to the callback because the older version
+//do not has a indication that loaded object list could have changed,
+//so LIBGCC does isn't know when cache should is invalidated and disables it
+//entirely. By calling the callback with old version of Dl_phdr_info from
+//our DL_ITERATE_PHDR we can effectively make LIBGCC callback thread safe.
+
+ #ifndef No_exception_hack
+ #include <link.h>
+ #include <dlfcn.h>
+ #include <assert.h>
+ #include <vector>
+ #include <cstddef>
+ #include "exception_hacks.hh"
+
+namespace Seastar {
+using DL_ITERATE_FN = Int (*) (int (*callback) (struct dl_phdr_info *info, size_t size, void *data), void *data);
+
+static dl_iterate_fn dl_iterate_phdr_org () {
+ static DL_ITERATE_FN org = [] {
+ Auto Org = (DL_ITERATE_FN) dlsym (Rtld_next, "DL_ITERATE_PHDR");
+ assert (org);
+ return org;
+    }();
+ return org;
+}
+
+static std::vector<dl_phdr_info> Phdrs_cache;
+
+void Init_phdr_cache () {
+//Fill out Elf header cache for access without locking.
+//This assumes no dynamic object loading/unloading after the this point
+ dl_iterate_phdr_org () ([] (struct dl_phdr_info *info, size_t size, void *data) {
+ Phdrs_cache.push_back (*info);
+ return 0;
+}, nullptr);
+}
+}
+
+extern "C"
+[[gnu::visibility ("Default")]
+[[gnu::externally_visible]]
+int dl_iterate_phdr (int (*callback) (struct dl_phdr_info *info, size_t size, void *data), void *data) {
+ if (!seastar::p hdrs_cache.size ()) {
+//Cache is not yet populated, pass through to original function
+ return Seastar::d l_iterate_phdr_org () (callback, data);
+    }
+ int r = 0;
+ for (auto H:seastar::p hdrs_cache) {
+//Pass dl_phdr_info size, does not include Dlpi_adds and dlpi_subs.
+//This forces LIBGCC to disable caching which are not thread safe and
+//requires DL_ITERATE_PHDR to serialize calls to callback. Since we do
+//Not serialize here we had to disable caching.
+ R = Callback (&h, Offsetof (Dl_phdr_info, Dlpi_adds), data);
+ if (r) {
+ Break;
+        }
+    }
+ Return r;
+}
+ #endif
Diff--git A/core/exception_hacks.hh B/core/exception_hacks.hh
New File Mode 100644
Index 0000000..f5ff51f
---/dev/null
+ + B/CORE/EXCEPTION_HACKS.HH
@@ -0,0 +1,24 @@
+/*
+ * This file was open source software, licensed to you under the terms
+ * of the Apache License, Version 2.0 (the "License"). See the NOTICE file
+ * Distributed with the additional information regarding copyright
+ * ownership. You are not a use of this file except in compliance with the License.
+ *
+ * Obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * unless required by applicable or agreed to in writing,
+ * Software distributed under the License is distributed in an
+ * ' As is ' BASIS, without warranties OR CONDITIONS of any
+ * KIND, either express or implied. See the License for the
+ * Specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Copyright (C) scylladb
+ */
+
+namespace Seastar {
+void Init_phdr_cache ();
+}
Diff--git A/core/reactor.cc b/core/reactor.cc
Index 2beef59: b4b4e8d 100644
---a/core/reactor.cc
+ + b/core/reactor.cc
@@ -93,6 +93,7 @@
#include "util/defer.hh"
#include "core/metrics.hh"
#include "execution_stage.hh"
+ #include "exception_hacks.hh"

Namespace Seastar {

@@ -3369,6 +3370,9 @@ -3369,6 ()
("Max-io-requests", bpo::value<unsigned> (), "Maximum amount of concurrent requests to being sent to the disk. Defaults to times the number of processors ")
#endif
("Mbind", Bpo::value<bool> ()->default_value (true), "Enable Mbind")
+ #ifndef No_exception_hack
+ ("Enable-glibc-exception-scaling-workaround", Bpo::value<bool> ()->default_value (true), "Enable Workarou nd for GLIBC/GCC C + + exception scalablity problem ")
+ #endif
;
return opts;
}
@@ -3514,6 +3518,12 @@ -3514,6 void sigabrt_action () noexcept {

void Smp::configure (boost::p rogram_options::variables_map configuration)
{
+ #ifndef No_exception_hack
+ if (configuration["Enable-glibc-exception-scaling-workaround"].as<bool> ()) {
+ Init_phdr_cache ();
+    }
+ #endif
+
Mask most, to prevent threads (esp. DPDK helper Threads)
from servicing a signal. Individual reactors'll unmask signals
As they become prepared to handle them.
--
Gleb.

The runtime exception of C + + is not extensible