More Stephen Toub: Array Pools
Published by marco on
In this otherwise excellent video, I found myself very much wishing that Toub had written at least a single test for the ArrayPool implementation that he built in this video. Still, check out the selected citation below to get a feeling for how they consider performance implications—there are no easy answers, there is only testing and benchmarking.
Deep .NET: Let's Build Our Own ArrayPool with Stephen Toub and Scott Hanselman by dotnet (YouTube)
At 34:45,
“Hanselmann: For folks that may not know what NUMA is: so NUMA is this non-uniform memory access that the computer knows that, like, this CPU is near this memory and…that memory over there, we’re going to consider that remote memory. And it’s all meant to reduce latency. Is that right?
“Toub: Yeah and this definitely factors into things like the GC, right? And even with threadpool scheduling: you want to put the work where the data is.
“Hanselmann: Put the work where the data is. Yes.
“Toub: Otherwise, you spend all your time moving stuff around and thrashing your cache.
“Hanselmann: …and moving things around at different layers of abstraction. Because you would not want to move between NUMA nodes. You don’t want CPU zero to be looking at memory one over there. But then there’s the higher-level question of ‘is the adding of an array pool to my application going to cause memory fragmentation or do I just trust the GC to handle that?’ It’s a constant series of trade-offs. Like, did the complexity I added give me the performance that I wanted or did I just make things more complicated.
“Toub: Well, it’s a great example of where you know I mentioned at the beginning there’s a lot of complexity with pools. And this is a great example of it, If you’re running on a core over here and you use an object that was last used over here, right? Are you better off using that object? Or are you better off just asking the GC to give you a new one? That’s going to be allocated in memory that’s very closely associated with where you are. So these pools aren’t always wins.”