Well, I think it was partly due to IBM themselves, launching systems such as the AT and the PCjr, which were not 100% compatible either (even though the PCjr shared the 4.77 MHz 8088, it did not execute code at the same speed as PC/XT systems, and the video circuit was only partially compatible with CGA), which probably led to most developers writing somewhat more 'defensive' code in terms of hardware requirements. Variations in speed and other specs became the hallmark of the PC platform.
I guess the big success of clones was after 1984, when IBM themselves already 'corrupted' the PC/MS-DOS platform with incompatible systems.
Other system were different anyway. It was much harder to clone a system like the C64, because of all the custom chips.