A memo on `Speeding up a distributed computation in Haskell’

A story on Linux system calls: select, poll, epoll

Speeding up a distributed computation in Haskell

1980年代にはあったLindaのmaster-workers modelの話の延長みたいなもの。master workersにしたら解決するのかと思ったらなんとscale-upしないとか。粒度を下げてもダメなんてことがあるとは想像しなかった（基本のタスクは0.03秒。これが結構微妙なところなのかもしれない）。

Another suspicion is that the multi-threadedness of the first version played at our disadvantage since there would be a lot of pointless context-switches while one thread was already modifying the MVar MasterState. In other words, any context switch between slave threads while one slave thread is already holding the MVar MasterState is (almost) wasted, since it'll be blocked on the MVar MasterState right after receiving a slave response and will yield, delaying the completion of the loop body in the thread that was already processing the MasterState.

割り込み禁止みたいなことか。

The main change we wanted was to turn the server from a multi-threaded server multiplexing through the event manager to a single-threaded application multiplexing the sockets directly.

マルチスレッドプログラミングのわかりやすさの代償が隠れていた。これって（IO マネージャーのレベルで）型で解決できないのだろうか。。。

In Linux, the epoll set of syscalls exist for this exact reason: you can register multiple sockets to wait on with epoll_ctl, and then wait for any of them to be ready using epoll_wait.

select や、そのオーダーを下げるために導入された poll までは知っていたけど epoll は知らなかった。10月までに調べておこう。

多分今年一杯くらいまでは並列や分散にてを出すことはないと思うのだけど、これは読んでおいてよかったかもしれない。

補足

Reading from the Socket directly into a ByteBuffer and deserializing directly from there rather than copying into intermediate ByteStrings, reducing allocations drastically to perform deserialization, since we allocate the buffer where the socket data is read into upfront.

やはりこういうことも重要なのね。