tl;dr the saying is that "premature optimisation is the root of all evil", and Casey burrows into contemporary data to show that really although the claim was 3% of the code takes up 90% of the runtime even then it was more likely 4% takes 50%.
The best thing you could take away from this lecture is something a reasonable person should take away from the original "root of all evil" saying anyway. Measure. Measure. Measure. If you aren't measuring that's not optimization it's masturbation.
Ironically in the process of measuring for a third time yesterday I tripped a bug in Bill's language for which I opened an issue. This is not the goal of measuring three times but merely a happy accident.
Along the way Casey discovers (?) that Structured Programming means just what we today call programming†, that software was a lot smaller in the days when 4096 bytes of RAM was a good entry level option and that loads of these famous people from 1970s computer science knew each other.
† And knowing about this is one reason the Structured Concurrency people want that everywhere. Very possibly there's a future where it seems silly that people once wrote programs which did not use structured concurrency.
The problem is knowing what to measure. There's another saying
"When a measure becomes a target, it ceases to be a good measure."
As an example from memory, there was a game dev company that celebrated they had maxed out the cores on the PS3. That didn't mean anything though, anyone can max out the cores by filing them with bad code. But hey, their "measurement" told them they had maxed out the machine
This can be a problem, but much less so because so often we're doing "easy mode" where we don't need a proxy. The "it ceases to be a good measure" is because you're measuring a proxy. You wanted to deliver happiness, you measured wealth because it was easier to measure but seemed correlated and now you've got rich miserable people, oops. But software engineers can often measure the actual thing they want to improve directly, not a proxy and so it cannot cease to be a good measure.
With a few caveats, though they do tend to get fixed over time. For example, frame rate. Higher frame rate is better, the end, right? Wellll...
Latency and hitching are annoying to measure, so for a long time, they were pretty much ignored. That has improved and hopefully will continue to improve. There is the bufferbloat initiative, gaming magazines take frame time histograms now, input devices and screens commonly have their latency measured. But latency is still under-measured - for GPUs / GPU drivers, for all software, for games and particularly for websites.
Casey generally, across all his material, advocates for non-pessimisation. Measurement takes too long to apply it to your entire program. He advocates for thinking about how much work the computer should actually have to do, then not making it do much more than that, at all times. This means avoiding serial dependency chains on the network, and huge towers of abstractions, and redundant work. He allows for writing lazy slow code as an intentional tradeoff that you may revise later if it becomes a bottleneck. He does not allow for inherently slow architecture.
Sure, but there are performance issues no profiler will catch in a straightforward flamegraph reading.
Some examples:
1. Your hottest loop is spilling registers which only shows up as non-local cache thrashing (i.e. some other random code becomes slow) or randomly slow instructions i.e. "why is this xorps to initialise this int suddenly slow" due to pipeline stalls
2. Your code stops fitting into cache due to the code bloat, there's no one method which is slow, everything is slowed down by a percentage factor
3. A lot of useless work being done like temporary strings being copied everywhere
4. Your code is "I/O bound" because all the data you're accessing is scattered all across memory, leading to completely predictable TLB stalls
It's very easy to make a large program, quite a bit harder to make a small one...
In other videos Casey argues against the profile–fix–repeat workflow (I’m not saying that you necessarily meant this by measuring), instead arguing for estimating the theoretical maximum, then trying to get close enough to it. His argument is that the former might push you towards a local minimum without realising that you could do much better
When "Estimating the theoretical maximum" you're going to bring your assumptions to that estimate and so you're equally likely to get stuck in some local optimal point because you didn't see what else was possible, so that doesn't persuade me much. Worse, now it's a local optimal which might be entirely in your head.
There is some space for this "Estimate the theoretical maximum" as a high level insight, if the software needs to do a thing which your estimate says is impossible that's important to address up front but I think for practical software engineering you're much more likely to need profile-fix-repeat
Actually measuring is crucial and an estimate is not a measurement. I think one of the reasons many in the Handmade Community don't like measuring is that it too often ruins their "estimate" of how the crap they've written is achieving a "theoretical maximum" in favour of a boring fact that it's much worse than alternatives.
The best thing you could take away from this lecture is something a reasonable person should take away from the original "root of all evil" saying anyway. Measure. Measure. Measure. If you aren't measuring that's not optimization it's masturbation.
Ironically in the process of measuring for a third time yesterday I tripped a bug in Bill's language for which I opened an issue. This is not the goal of measuring three times but merely a happy accident.
Along the way Casey discovers (?) that Structured Programming means just what we today call programming†, that software was a lot smaller in the days when 4096 bytes of RAM was a good entry level option and that loads of these famous people from 1970s computer science knew each other.
† And knowing about this is one reason the Structured Concurrency people want that everywhere. Very possibly there's a future where it seems silly that people once wrote programs which did not use structured concurrency.
The problem is knowing what to measure. There's another saying
"When a measure becomes a target, it ceases to be a good measure."
As an example from memory, there was a game dev company that celebrated they had maxed out the cores on the PS3. That didn't mean anything though, anyone can max out the cores by filing them with bad code. But hey, their "measurement" told them they had maxed out the machine
This can be a problem, but much less so because so often we're doing "easy mode" where we don't need a proxy. The "it ceases to be a good measure" is because you're measuring a proxy. You wanted to deliver happiness, you measured wealth because it was easier to measure but seemed correlated and now you've got rich miserable people, oops. But software engineers can often measure the actual thing they want to improve directly, not a proxy and so it cannot cease to be a good measure.
Latency and hitching are annoying to measure, so for a long time, they were pretty much ignored. That has improved and hopefully will continue to improve. There is the bufferbloat initiative, gaming magazines take frame time histograms now, input devices and screens commonly have their latency measured. But latency is still under-measured - for GPUs / GPU drivers, for all software, for games and particularly for websites.
1. Your hottest loop is spilling registers which only shows up as non-local cache thrashing (i.e. some other random code becomes slow) or randomly slow instructions i.e. "why is this xorps to initialise this int suddenly slow" due to pipeline stalls
2. Your code stops fitting into cache due to the code bloat, there's no one method which is slow, everything is slowed down by a percentage factor
3. A lot of useless work being done like temporary strings being copied everywhere
4. Your code is "I/O bound" because all the data you're accessing is scattered all across memory, leading to completely predictable TLB stalls
It's very easy to make a large program, quite a bit harder to make a small one...
There is some space for this "Estimate the theoretical maximum" as a high level insight, if the software needs to do a thing which your estimate says is impossible that's important to address up front but I think for practical software engineering you're much more likely to need profile-fix-repeat
Actually measuring is crucial and an estimate is not a measurement. I think one of the reasons many in the Handmade Community don't like measuring is that it too often ruins their "estimate" of how the crap they've written is achieving a "theoretical maximum" in favour of a boring fact that it's much worse than alternatives.