Coroutines with reentrant scheduling

Short post for today, with some thoughts on one of the gotchas you can see with C++ coroutines that is sometimes easy to overlook. My prior post on Threading Models should be useful background for today.

Common example

This isn't the simplest case necessarily, but it's often very close to the "shape" of design/code that will give rise to some problems.

Let's say you have a struct C that is single-threaded.

struct C {
 std::string val;
 my_task foo();
};


And now let's say foo is implemented like so:

my_task C::foo() {
 do_something(val); // first use of val
 co_await something_else();
 do_something_else(val); // second use
 co_return;
}


The problem here is that depending on how coroutines are dispatched (and it might be hard to see purely from local inspection), the second use of val might be different from the first one even if the function doesn't modify it, and there are no concurrent accesses, and there are no side-effects from something_else one if something else is allowed to run.

That can lead to semantically incorrect results if the value is presumed to be consistent, or in outright errors if you did something like this. We don't even need anything particularly fancy.

my_task C::foo() {
 const char* c = val.c_str();
 printf("value is %s\n", c);
 co_await something_else();
 printf("value is still %s\n", c);
 val = "a different value that could reallocate "
       "the string and invalidate c";
 co_return;
}


Two calls to foo that interleave, even without races, can leave one of them with an invalid c pointer such that it will crash when dereferenced. Less obvious bugs can of course occur.

How to solve for this

Classic things to do are:

• Disallow reentrancy if you can.
• Handle ownernship with a coroutine-friendly lock (something that won't actually block as such but that you can co_await to enter) to avoid reentrancy on specific areas of code.
• Copy the values you need once into the coroutine.
• Reacquire values that might have mutated after the co_await, check consistency if needed.

A lot of the approaches really are similar to what you would do you this was a function that could run concurrently where the coroutine straight-execution blocks have some sort of lock on state. All the solutions enumerated above also apply to multithreading, where you could disallowed multithreaded access to state by design, use locks to control access, copy values once out of the multithreaded state and use standalone, or 'snap' some values under a lock, use them, then reacquire the lock and see if the state is still consistent.

Happy concurrent / reentrant programming!

Tags:  design

Home