That’s a good question, Sam. What I’m looking for are the elements that slow down the program, and make it “stutter”, for example, when you move around the perspective viewport, or make moving from named view to named view super slow. I noticed (and the technical side of Rhino is not my forte) that this does not seem to have to do with the overall size of the file, but with the nature of some of the elements in it. For example, I have a model I built of a marina in Boston, with boats, trees and street furniture imported from the Google 3d warehouse, about 400mb – was slow. But when I built a pile of lobster traps (basically a 18"x18"x30" cage of pipes) repeated 30 times, it was virtually impossible to move through the model-- even though I used blocks. I had to put the “heavy” stuff on one layer and turn it off every time I wanted to move the camera (and switch to wireframe).
You asked “Does this just mean the amount of bytes it will take to store the object, or also how hard it is to shove down the display pipeline?” I think it is display we are talking about, right? What one sees in the viewports and how it slows down our ability to move the camera around.