Being under constant overload may require a new solution. Whereas both queues and buffers will be great for cases where overload happens from time to time (even if it’s a rather prolonged period of time), they both work more reliably when you expect the input rate to eventually drop, letting you catch up.
You’ll mostly get problems when trying to send so many messages they can’t make it all to one process without overloading it.
Two approaches are generally good for this case:
• Have many processes that act as buffers and load-balance through them (scale horizontally)
• use ETS tables as locks and counters (reduce the input)
ETS tables are generally able to handle a ton more requests per second than a process, but the operations they support are a lot more basic. A single read, or adding or removing from a counter atomically is as fancy as you should expect things to get for the general case.
ETS tables will be required for both approaches. Generally speaking, the first approach could work well with the regular process registry:
you take N processes to divide up the load, give them all a known name, and pick one of them to send the message to. Given you’re pretty much going to assume you’ll be overloaded, randomly picking a process with an even distribution tends to be reliable: no state communication is required, work will be shared in a roughly equal manner, and it’s rather insensitive to failure.
In practice, though, we want to avoid atoms generated dynamically, so I tend to prefer to register workers in an ETS table with read_concurrency set to true.
It’s a bit more work, but it gives more flexibility when it comes to updating the number of workers later on.
An approach similar to this one is used in the lhttpc 22 library mentioned earlier, to split load balancers on a per-domain basis.
[22] The lhttpc_lb module in this library implements it.
[23] By using ets:update_counter/3.
[24] https://github.com/ferd/dispcount