A memo on `Speeding up a distributed computation in Haskell’
A story on Linux system calls: select, poll, epoll
1980年代にはあったLindaのmaster-workers modelの話の延長みたいなもの。master workersにしたら解決するのかと思ったらなんとscale-upしないとか。粒度を下げてもダメなんてことがあるとは想像しなかった(基本のタスクは0.03秒。これが結構微妙なところなのかもしれない)。
Another suspicion is that the multi-threadedness of the first version played at our disadvantage since there would be a lot of pointless context-switches while one thread was already modifying the
MVar MasterState
. In other words, any context switch between slave threads while one slave thread is already holding theMVar MasterState
is (almost) wasted, since it'll be blocked on theMVar MasterState
right after receiving a slave response and will yield, delaying the completion of the loop body in the thread that was already processing theMasterState
.
割り込み禁止みたいなことか。
The main change we wanted was to turn the server from a multi-threaded server multiplexing through the event manager to a single-threaded application multiplexing the sockets directly.
マルチスレッドプログラミングのわかりやすさの代償が隠れていた。これって(IO マネージャーのレベルで)型で解決できないのだろうか。。。
In Linux, the
epoll
set of syscalls exist for this exact reason: you can register multiple sockets to wait on withepoll_ctl
, and then wait for any of them to be ready usingepoll_wait
.
select
や、そのオーダーを下げるために導入された poll
までは知っていたけど epoll
は知らなかった。10月までに調べておこう。
多分今年一杯くらいまでは並列や分散にてを出すことはないと思うのだけど、これは読んでおいてよかったかもしれない。
補足
Reading from the
Socket
directly into aByteBuffer
and deserializing directly from there rather than copying into intermediateByteString
s, reducing allocations drastically to perform deserialization, since we allocate the buffer where the socket data is read into upfront.
やはりこういうことも重要なのね。