I have more than one GP2X Caanoo for NES, SNES, Genesis, Neo-Geo, GBA, Turbo-Grafx emulation. My favorite is GBA.
JXD makes an inexpensive Android handheld based on 2.6.35 Linux kernel (Aug 1, 2010) for old 8-bit, 16-bit, N64, PS1, and Saturn emulators. It's got 512 MB of RAM and a 1 GHz RISC CPU. It plays JPEG and all the image formats. It has 802.11. The $120 model has a better touch screen for touch screen games.
I admit there is no Android 3.0 or Android 4.0 game handheld with newer Linux kernels on Ebay.
Caanoo can do this with 2.6.22 kernel (24 January 2008). Caanoo kernel is as old as openSUSE Linux 10.3.