How is the second HDD installed? Looking at the PS2 manual for the 350, the 5.25" drive installs in the second 5.25" drive bay. The BIOS should support the drive. You may need to get a different floppy cable; I am not sure if the supplied cable is long enough to connect to both floppy drives.
If the external 5.25" bays are filled, an external drive mounting is needed. The backpack parallel port drives would be the easiest option but hard to find now and can't be booted from limiting the usefulness with early PC games. It is possible to have an external drive attached to the floppy controller but it could be complicated keeping the data cable from suffering interference and making sure the drive is powered.