BASIC was typically loaded from paper tape, either from an ASR-33 Teletype or a standalone paper tape reader like the Oliver Audio OP-80A, or an adapted surplus paper tape reader. You could've toggled the whole thing in, but that's a lot of toggling!
I/O was through a console, which could be a number of output devices. Serial terminals, Teletypes, or TV Typewriter devices were probably the most common. The great thing about a Teletype, especially one with a punch, is it was a multifunction device that fulfilled a lot of needs on a small system like the Altair 8800 -- system console, printer, and storage system via paper tape punch/reader. Storing code on paper tape was really easy, since as a quick and dirty approach an application could just interpret the paper tape as typing at the Teletype keyboard, and handle it just like any other user input. Storing code was just a matter of turning the punch on, and listing the program.
You could of course burn BASIC into ROMs so that it didn't have to be read in every time.
The lights and switches are usable for small programs and debugging, but beyond that they're not so great. Even on minicomputers like the PDP-11, the lights and switches weren't usually the main interface for writing serious programs. Sure, you could, but it's an error-prone process at best.