You won’t be able to solve the limited RAM capacity… But needing 3Gb to open a 20Mb file is very extreme… Shouldn’t need anywhere near that much.
I’m wondering if the file has a lot of linetypes or blocks or something that is making it very hard for Rhino to read it in. Also, what version is the dwg? Rhino 4 is rather old, might not correctly read the latest dwg files.
If you can zip the file and post it or upload it somewhere and post a link, I can look at it and perhaps figure out why this is happening; and at least be able to export an openable Rhino V4 file for you…
–Mitch