Mircea Baja - 6 Aug 2023 # Coroutines problem domain
--- # Concurrent vs. parallel - intuitively illustrated using abstract pictures - for intro into C++ standardese doublespeak see ["Forward Progress Guarantees in C++ - Olivier Giroux - CppNow 2023"](https://www.youtube.com/watch?v=g9Rgu6YEuqY&t=978s) --- # Sequential
# Concurrent
- purple and yellow end later - there is additional overhead (not illustrated) --- # Sequential
# Parallel
- there is additional overhead (not illustrated) --- # Sequential
# Concurrent and parallel
- purple ends later - there is additional overhead (not illustrated) --- # Use cases for concurrency --- # Compilers (maybe)
source
lexical
analysis
syntactical
analysis
data
analysis
instruction
generator
code
- Melvin Conway: Design of a separable transition-diagram compiler - of Conway law fame: product design mirrors organisation structure - motivation: single pass compiler in memory constraints environments, each "coroutine" can output zero or more than one results each time it's invoked - in the meantime: better ways of designing a compiler --- # Simulations - Bjarne Stroustrup: [A Set of C++ Classes for Co−routine Style Programming.](https://www.softwarepreservation.org/projects/c_plus_plus/cfront/release_e/doc/ClassesForCoroutines.pdf) Bell Laboratories Computer Science Technical Report CSTR−90. November 1980. - motivation: event driven simulations - `task` base class used to represent a independent activity which: - suspends voluntarily - can be resumed, canceled - can wait, sleep - provides an `int` result and communicates with other `task`s (e.g. producer and consumer, server reading from a queue) - work is done in the derived class constructor - one of the earliest libraries in C++ (think `complex` and `string` without templates), yet did not make it to the standard - some of the problems it tries to solve turn out to be recurring: cancellation, time management, forking and joining, queues, etc. --- # GUI ```cpp BOOL bRet; while( (bRet = GetMessage( &msg, hWnd, 0, 0 )) != 0) { if (bRet == -1) { // handle the error and possibly exit } else { TranslateMessage(&msg); DispatchMessage(&msg); } } ``` --- # GUI - usually single threaded: a single dedicated UI thread - the thread has a queue - the thread is blocked waiting for a message from the queue - when a message is returned, the thread processes it via a `switch` statements - messages are added to the queue via `PostMessage` (from the same thread) or `PostThreadMessage` (from a different thread) - there is also a special "quit" message - GUIs show the usage of queue of work to manage concurrency, of a blocking function to get next piece of work, not necessarily in need of coroutines (often long background activities are scheduled on additional threads) --- # Async IO - motivation: C10K (1999) - networking is often IO bound (wait for data to be sent or received) - what's wrong with `WaitForMultipleObjects` (more later) - `O(N)` time complexity with the number of handles provided - similar `select` for Linux - IO completion ports: it's a queue again (more later) - similar `kevent` for Linux - (boost) ASIO - file IO - timers/events/registry monitoring - one problem: thread(s) blocked using API to get data from the queue, can't have a thread check for work in multiple incompatible queue types - Windows thread pools to unify work covered by IO completion ports e.g. networking, with work not covered via IO completion ports - similar evolution on other OSes --- # WaitForMultipleObjects - when a thread calls `WaitForMultipleObjects`, the caller provides a set of objects it waits for - for each object in the set, the OS has to atomically (thread safe) add this thread ID to a list associated to the object - the thread is suspended - when an object in the set is signaled, the OS finds the thread ID in the object's list - the thread is scheduled to resume - when the thread resumes - for each object in the set, the OS has to atomically remove this thread ID from the list associated to the object - `WaitForMultipleObjects` returns - The steps above lead to `O(N)` time complexity with the number of objects provided, which is why `WaitForMultipleObjects` limits it to 64 --- # Completion port - when a thread calls `GetQueuedCompletionStatus` - it checks if work is queued (requires a lock) - if not, it suspends, a single object needs to have this thread ID stored - when an object associated with the completion port is signaled - it's added to the queue - the thread is scheduled to resume (if needed) - when the thread resumes - it picks work from the queue - `GetQueuedCompletionStatus` return - it solves the `O(N)` problem, work to queue/dequeue one item does not depend on the number of items in the queue - the disadvantage is that most such queueing systems are incompatible, a thread is blocked on reading from a particular queue system --- # Async IO - network echo work with coroutine support ```cpp task echo(socket s) { buffer buff; while (true) { std::size_t length = co_await read_some(s, buff); co_await write(s, buff, length); } } ``` --- # Async IO - without coroutines it's a lot of callbacks and manual memory management ```cpp class session : public std::enable_shared_from_this
{ // ... void do_read() { auto self(shared_from_this()); socket_.async_read_some(buff_/*...*/, [this, self](boost::system::error_code ec, std::size_t length) { do_write(length); }); } void do_write(std::size_t length) { auto self(shared_from_this()); boost::asio::async_write(buff_/*...*/, [this, self](boost::system::error_code ec, std::size_t /*length*/) { do_read(); }); } ``` --- # Nanocoroutines - motivation: is handle memory latencies for database JOINs - CppCon 2018: G. Nishanov [“Nano-coroutines to the Rescue! (Using Coroutines TS, of Course)”](https://www.youtube.com/watch?v=j9tlJAqMV7U) --- # Generator ```cpp struct fib_state { int x_1 = 1; int x_2 = 0; }; int next_fib(fib_state & state) { int val = state.x_1 + state.x_2; state.x_2 = state.x_1; state.x_1 = val; return val; } void foo() { fib_state state; while (true) { int x = next_fib(state); // use x } } ``` --- # Generator ```cpp generator fib() { int x_1 = 1; int x_2 = 0; while (true) { int val = x_1 + x_2; yield val; x_2 = x_1; x_1 = val; } } void foo() { for (int x: fib()) { // use x } } ``` - `val` is `yield`ed before `x_2` and `x_1` are updated --- # Generator ```cpp std::generator
fib() { int x_1 = 1; int x_2 = 0; while (true) { int val = x_1 + x_2; co_yield val; x_2 = x_1; x_1 = val; } } void foo() { for (int x: fib()) { // use x } } ``` --- # Exception-less mode - e.g. for using in kernel code without using C++ exceptions - this was important for Microsoft (that significantly drove towards C++ coroutine adoptions) --- # GPU work - there is a lot of effort (e.g. by Nvidia) to make calculations work from C++ (algorithms, coroutines, sender/receives etc.) compatible with "normal" CPU usage - beware that choices made for a use case (e.g. GPU) might be not the right ones for another use case (e.g. async networking IO) --- # Interaction with threads - single thread simplifies a lot of things, but has limits of how much computation is available - explicit multithreading - can a coroutine resume in a different thread? - `scoped_lock|lock_guard
` potential for misuse --- # Timing matters/choices - read operations might get more than what was requested (e.g. when request is less than data in the IP package). Subsequent reads: - could always queue the continuation (to allow fairness, at cost of additional work) - avoid queueing if data already available (maximise speed by avoiding work in this case) - wait when_all(op1, op2, op3) can mean different things for different environments: - initiate all operations, the progress in parallel, return when the last one (by time) completes (e.g. op2) - initiate sequentially, progress is not parallel, return when the last one (in sequence) completes (i.e. op3) --- # What are coroutines then? --- # Function vs. coroutine
--- # Coroutines
--- # Stack
--- # Stackful
--- # Stackless
--- # Duality
--- # Questions?