If you're going the DOS machine with floppy disk era stuff....
If you are strapped for cash, the early PIII era on back to the Pentium 1 era is still reasonably inexpensive for the most part and fairly compatible with most of the late era DOS and Windows 9x games that were popular during that time (Doom, Quake, Duke Nukem 3D, Diablo....etc), it'll also be less "fiddly" for the newer software.
If you are willing to spend a little more, I consider the 80486 era to be the perfect "middle ground" when it comes to DOS/Win9x and Floppy drive enabled stuff. They have enough CPU variants available that if you go early (ie 486 DX-33, 486 SX) you have basically a top notch DOS gaming machine, but if you go into the later stuff (DX2, DX4, AMD 5x86 133 - the era of VLB and later PCi, and the dawn of high-speed disk access, large capacity drives, and high speed video accelerator cards) you can cover most of the Pentium era ground as well. 486 stuff WILL set you back though as it is extremely popular these days for retro-gaming, and that era is the most fiddly when it comes to jumpers, and so fourth, so if you are looking for a plug N' play scenario.
The further back you go in the DOS PC genre it will get more expensive as the supplies have been getting lower and lower, but some bargains can still be had from time to time depending on where you look. Price inflation on this stuff is largely false or created by the fact gold scrappers were sucking up the supply years ago to accumulate gold from the chips inside these systems. The Compaq BillDeg posted above me is a good 386 era machine that does not go for too much. The older bigger Deskpros ask for $$$ on E-bay though.
If you want cheaper, I'd avoid IBM as EVERYONE (me included) loves IBM for DOS stuff up to or maybe even through the PS/2 era. Compaq can be good, depends on whose selling it. Whitebox clones can be had cheaply if you know what to look for, just be caerful because there are a lot of those out there that will be hiding newer hardware under the hood. One of my biggest laughs was that "386 DX" tower I bought a million years ago that was actually an AMD K6 with a pre-release of Windows ME on it.