• (nodebb)

    One thread per connected socket??? Remy, you (and anyone else who thinks this is a good idea) are TRWTF. It scales up to this much ==> and then completely stops scaling because you run out of memory for all the thread stacks.

    One thread per N connected sockets, where N is chosen carefully, but definitely bigger than just one, and wait using select, poll, or whatever your preferred mechanism is, then dispatch the new data onto the relevant handler. For bonus points, in small enough systems, have the thread watch for new incoming connections as well.

  • Sole Purpose Of Visit (unregistered) in reply to Steve_The_Cynic

    Sounds like Remy has never heard of select or poll. (Poll is to be preferred, not least because it scales up better than select.)

    Also, adding a randomly-parameterised sleep is an utterly terrible idea. At some point in the future, the lumpiness of response times is really going to bite you in the butt.

  • Anonymous') OR 1=1; DROP TABLE wtf; -- (unregistered)

    One thread per socket isn't the worst idea for simpler, lower-scale applications. It's a heck of a lot easier to code an application that uses blocking sends/recvs than to deal with the complexity of non-blocking sockets and select/poll/epoll if you don't need to scale to 1000's of sockets.

    If you do need to scale up that high, then yes you absolutely need to be using select/poll/epoll, and preferably poll/epoll at that, since select() scales as O(largest fd) instead of O(number of fds).

  • Sauron (unregistered)

    Some wisdom about threads:

    http://thecodelesscode.com/case/121

    http://thecodelesscode.com/case/126

  • (author) in reply to Steve_The_Cynic

    That was just bad writing on my part- I didn't mean to imply one thread per socket, that'd be ridiculous. As far as "sleeping", I didn't mean sleep, I meant "go to sleep", which yeah, frequently done via select.

  • (nodebb) in reply to Steve_The_Cynic

    Not even one thread per N sockets. Have a pool that is sized for the system they are running on and allocate tasks to any free thread in the pool.

    Of course, this is hard to implement correctly from the ground up, so use the resources the OS provides. For example, macOS has dispatch queues. Just put a closure on a concurrent queue and it will run asynchronously as soon as there is a thread available to run it. Of course, you have design the code slightly differently because you don't want the threads to block, ever. But the same applies to a system where you have one thread per N sockets.

  • The Beast in Black (unregistered)

    No WTF here, it's just a thread-implementation variant of The Speed-up Loop.

  • (nodebb)

    Historical note:

    The one thread per socket model is an evolution of the old Unix model of one process per socket. This also has scaling problems - but different ones.

    Also - and I may be remembering this incorrectly - the first implementation of select I came across had a hard upper limit on the highest file descriptor you could have. I think IIRC if you had a file descriptor greater than 256, you were screwed.

  • Barry Margolin (github)

    A programmer who can't even get one-thread-per-connection right is never going to be able to do thread pools or select/epoll.

  • MaxiTB (unregistered)

    Everything is wrong with this solutions from the design up.

    First, if we are taking asynchronous operations we need exactly one thread, because the request will be asynchronously done on a basis of callbacks in combination with whatever asynchronous state machine your are making. That is the power of async implementations, a good example is WinSocket which needed to be a async version because of older Windows versions.

    Secondly, if you want to use the power of your machine the fullest, you need exactly one thread on every processor core to get the maximum out of your machine, nothing more. Combine that with an async model and you end up with 100% usage of your hardware and the least amount of wasted resources.

    Thirdly and finally, threads are not free, threads are extremely expensive actually, especially creating and destroying them. So the anti-pattern of creating threads based on unpredictable events is pretty much as bad as writing logs without quotas - it's a machine killer anti-pattern. These days nobody should even touch threads but use parallel frameworks which manage all the nuts and bolts that come with the proper life-cycle of concurrent objects. In .net the parallel async engine is a good example, under C++ there are many options like Charm++ and as always when it comes to Java the solution is obviously ditching Java.

  • Sole Purpose Of Visit (unregistered) in reply to Anonymous') OR 1=1; DROP TABLE wtf; --

    No, sorry, there is no complexity at all. Specifically not if you are using a non-platformed language like C++.

    Use one thread per ethernet interface. That thread will deal with socket state, via select(). If you want, add a thread pool to deal with the side-effects of reading and writing. Under no circumstances whatsoever would you use some cretin call to sleep() or the equivalent.

    This is not remotely difficult. I'm certain that there are dozens or even thousands of open source offerings.

    It isn't even premature optimisation. It's just understanding state machines and doing it right in the first place. Do it right and you will never have to worry about it again.*

    Now, Java or C# or whatever -- you'll need to rely on the (correct) assumption that the platform does that stuff for you.

    • Caveat: you will need to deal with ddos and the less evil customer-based version, where a gazillion customers try to access your site at the same time. (See, for example, the initial version of the Obamacare website, which was truly a WTF.) There are ways to deal with this, but massive over-threading coupled with a ten millisecond sleep are ... not terrifically useful.
  • Sole Purpose Of Visit (unregistered) in reply to MaxiTB

    I guess I should have simplified my previous comment.

    Yes. What you said.

    A distinctly large number of programmers should never be allowed anywhere threads that are not managed by the platform.

Leave a comment on “Hanging By a Thread”

Log In or post as a guest

Replying to comment #:

Log Out

« Return to Article