Ok, so it seems the consensus is h264 for video codec. Why AAC for audio Brad? And mp4 for the container.
AAC is more efficient than AC3 or MPEG audio. Plus its part of the extensions to MPEG4, so as MPEG2/AC3 went together, so do h.264/AAC.
The profile of what you are encoding is very important, for example I encode in the PS3/XBOX360 compatible profile which also happens to be DXVA compatible, so your graphics card will do the playback. MP4 container for the same reason, MKV has the benefits I listed above, which mean shit to me. You don't need aftermarket splitters with Windows 7 for MP4 and PS3/XBOX support the playback natively.
I guess ultimately what i'm trying to do is this. Backup DVDs to ~700mb files. AutoGK was doing a good job, but the technology appears to be outdated, both in the codecs it outputs in, and the application itself. I'd be interested in your findings Brad. No point re-inventing the wheel
This is actually where I digress from the 'mainstream' if you like.
Rather than encoding to a specific filesize, which is quite wasteful, I now encode to a given quality level, and let the filesize sort itself out. My quality level is _awesome_. As in indistinguishable from the MPEG2 broadcast files I rip.
A Family Guy ep for example will come out around 70mb at full res, and quality will be better than equivalent 175mb "scene" rip.
A big secret I discovered is the filters you run before you encode. Most of the guides written by 'experts' over-sharpen the video, enhancing the artefacts, I found with a very slight softening you can get video that's amazingly close to the original, BUT I use 3-pass encoding which takes a long time. For SD-res though not an issue at all.
I also only encode my HD rips to 960x540 as a good filesize balance. Each step up in res is an exponential increase in compression time.
Grab xvid4psp and get me on MSN and I'll talk you through your first encode.