I'm not exactly sure, but I am going to take a few guesses.
First off, programmers back in the day were very limited in the resources they could bring to the table. They usually had 8 bit processors, low-rez graphics, minimal sound and, at best, 48K or so of RAM to work with. Without all of the bells and whistles the gameplay was about all that they could use to attract and keep players.
Secondly, programmers used to work alone or in very small teams. Nowadays a typical big-name video game is created by a shop of 15, 20 or even more developers each working on some aspect of the game (graphics, sound, level creation, etc.) One or two are responsible for maintianing order in the chaos, but they don't have the same focus or control of the old game "craftsmen."
Finally, and this might be the biggest problem, game creation has gone corporate. Back in the day a programmer would create their game and market it to a software company for sale. They worked on it until they were done and that was that. Today game makers work towards a variety of deadlines most of which have nothing to do with when the game is finished. These newer games are, therefore, often released incomplete or even completely non-functional.
That's my $.02 anyway.
Erik