Yes, all true. By the way, using a pre-built grid may also mean not having to re-invent the wheel, of course. And in SASS a grid is a doddle to setup anyway. But remember: it might be easy for you and me, but rather difficult for others. And support for flexbox is just about there, which removes the float property out of the equation.
Optimizing the images and other media in a site will generally offer much higher yields. The trouble with Adobe software is that this is still a weak point: never trust Photoshop,. Edge Animate, or Illustrator to optimize your PNG images properly, for example. External (free) tools are (still) a necessity.
Anyway, I am digressing. I think if one's workflow allows for easy minify integration (such as a SASS/LESS workflow), it can be beneficial under circumstances (for example, high traffic sites and related bandwidth costs). Otherwise, it would be better to focus on other obvious and less obvious areas to optimize the site speed.