The 5A808w WIP reverts to the previous drawing technique in pretty specific circumstances. You can force the regression with a test command. Run TestUseBackBuffer and change the mode to Use D I B. This setting does not persist between sessions, so you need to run this test command each time you start Rhino.
Rhino likes to use a lot of GPU memory, but it also has to share with the OS and all the other applications that are running. The Rhino antialiasing setting and the number of open Rhino windows will also affect how much GPU memory Rhino needs.