Converting Sinclair Basic to CP/M

[Eudimorphodon]

(Such hardware were a (not particularly common) add-on for the Commodore VIC 20 and 64 back in the days).

80 column cards/cartridges existed for almost every 8-bit home computer that reached any level of popularity, so I guess I'd be kind of surprised if there was *never* some kind of external dingus to give a ZX Spectrum an 80-column screen, but perhaps the fact that the machine was so explicitly designed to be minimal and cheap rendered the concept moot. (These 80 column cards were almost universally text-only and consisted of little more than a CRTC, character generator ROM, and a couple K of SRAM, but as cheap as that was it'd probably cost almost as much as a whole ZX spectrum to build.) Even if they were available for your particular machine it was *very rare* for these cards to actually be worth buying, they were never popular enough to get much software support.

(I remember the ads for 80 column cartridges for the TRS-80 Color Computer in The Rainbow magazine; a TRS-80 plus one of these might have been one heckuva deal for running FLEX or OS9 compared to a tricked out SWTPC or GIMIX 6809 computer, but the audience for such a thing was probably in the lowest four digits at most.)

The ZX Spectrum is kind of unique even in the realm of cheap TV computers because it doesn't even have a hardware text mode, all you have is the one graphics mode. (With its slightly nightmarish memory map.) Most of the other computers that did have graphics also offered various software options to cram more characters on the screen, but of course readability really starts to go in the toilet once you're dealing with 5 or fewer horizontal pixels per cell, especially on a TV if you're going through an RF modulator.

 

[cj7hawk]

The ZX Spectrum is kind of unique even in the realm of cheap TV computers because it doesn't even have a hardware text mode, all you have is the one graphics mode. (With its slightly nightmarish memory map.) Most of the other computers that did have graphics also offered various software options to cram more characters on the screen, but of course readability really starts to go in the toilet once you're dealing with 5 or fewer horizontal pixels per cell, especially on a TV if you're going through an RF modulator.

The screen layout was an attempt to speed up text and simplify the text process.

Once you located the start position, you just incremented h instead of l ( or d instead of e ) for subsequent writes and it would go down vertically.

But it slowed down plotting points and made calculating screen positions more difficult.

 

[MiaM]

Also IIRC that layout was common in the full graphics modes of other computers too, like the VIC-II in the C64 and so on. I bet that in those cases it's the result of sharing address encoding logic that fits the text modes.

 

[Eudimorphodon]

Once you located the start position, you just incremented h instead of l ( or d instead of e ) for subsequent writes and it would go down vertically.

Strictly speaking the Commodore 64's memory map would be marginally more efficient for a Z80 splatting glyphs because all 8 bytes within a character position are consecutive; you could use one of the Z80 rep move instructions. (Granted those aren't really much faster than a discrete loop.) But it wouldn't make it any better for plotting arbitrary points; honestly, I think you could make a case that the Spectrum is a little better than the Commodore for that; at least horizontal lines are straightforward.(*) FWIW, describing the memory map as "nightmarish" was actually kind of a joke because terrible memory maps in graphics mode were more the rule than the exception with 8 bit machines. The Spectrum is basically just a slightly more complicated-than-usual case of "weird interlacing". If you really want to witness pure, abject horror the Apple II's memory map wins almost every time.

(* Memory maps like the C64's were fairly common because they're a natural consequence of taking a fundamentally character generator-based system and bolting graphics onto it by assigning a block of RAM to the hole the character generator goes in and saying that's your bitmap memory. The C64's VIC-II is essentially doing that to save gates internally.)

Supposedly the reason the Spectrum shuffled the "Y" bits the *specific* way it does relates to being able to leverage "page mode" on the DRAM to save a cycle; it has to fetch two byte for every 8 pixels it spits out, the pixels themselves and the attribute byte for the 8x8 grid square they live in, and the weird memory map facilitates that. (Essentially the layout is optimized for the attributes bytes, which are a nice sensible 32x24 grid of 768 contiguous cells, and the weird addressing gymnastics for the bitmap bytes means that you can read any set of pixels+attributes with the same 7 bits at the bottom of the address, which is the requirement for page mode.) This does make it somewhat analogous to the Apple II, which has such weird addressing ("memory holes" and very weird interlacing) partially to save counters and partially to to ensure that every DRAM row gets exercised often enough to meet the refresh requirements.

 

[cj7hawk]

So after trying to write routines from scratch for FORMAT, CAT, ERASE and MOVE, I decided to shift them in the command table and stick them next to SAVE, LOAD, MERGE and VERIFY since they now all pertain to disk services.

Now I have 8 disk handling routines all within the same handler, and so far it works pretty well. All of them are now Spectrum Group B and are handled by the SAVE program.

If I type CAT "" it shows the CP/M disk directory - for any files marked with a "z". Next is to have it open and mount a TAP if I do something like CAT "PROGRAM" it will open the program zPROGRAM and mount it as a TAP, then scan the TAP to see what files lurk within. After that, I can mount the TAP directly and LOAD "" will work by loading the next BASIC program from the TAP. It will also be possible to both mount and load a program straight from LOAD if a filename is specified. Also the z in the CP/M directory is implicit - BASIC isn't aware of it and so SAVE "TEST" will create a file called zTEST and load "TEST" will mount zTEST and then load TEST.

Possibly the most useful function however is that it makes the file handling routines consistent - LOAD and SAVE work like CAT and ERASE. I may need to recheck MOVE still works within this structure though without triggering the syntax checking. Although if required I can bypass the handler at the start before the syntax checking occurs.

 

[MiaM]

Great progress!

Possible feature creep suggestion: Giving CAT a text file as an argument prints the text file on screen, with a "more" style feature. Not being able to look what various files contain is perhaps the biggest constraint when using old 8-bit microcomputers.

Re locations in memory and whatnot: I know that you can't aim for full binary compatibility with existing programs, but still, if you want a rabbit hole to dive into you could look at how various third party disk interfaces did implement these functions. Or maybe not, perhaps all of them used the interrupt thingie like the Interface 1 did?

Speaking of the Interface 1: Will your implementation be able to use AUX: as a source for LOAD and destination for SAVE? This was a feature with the ZXnet of the Interface 1, that I think few if any used (as it required you to type load on the receiver and save on the sender, I.E. there weren't any dedicated file server background process or a dedicated file server machine - a missed opportunity).

 

[cj7hawk]

"More" is the default on ZX Spectrums... They say "Scroll?" and wait for either enter, or space. Space breaks. Enter continues.

But I will find a way to make that voluntary - ie, set it on or off. Maybe offer a third choice like "C" for continuous. I think continuous scroll would be ideal on a Spectrum at times. It was a way to create a "scroller" game - Except you go top down and reprint the character at the top of the screen and each new line scrolls upwards.

Existing disk services aren't very practical for a CP/M conversion. They were like extensions that added new disk load and save functions - while I want load and save to work the same way, so BASIC programs will still work when carried across - especially if they load in variables or other data. Most existing disk systems also work by hooking the error handler, but I'm just adding new code and rewriting BASIC - so I also don't have the extra memory the original disk interfaces had.

Also, it would be useful to be able to manipulate TAP files like disk partitions on a modern machine - since a lot of software is distributed on TAP and using the backwards compatible interface, it means I can run a tape and convert it to disk in real time as long as it uses the Spectrum load and save routines - no matter how many files are on the tape - which also means you might get several programs on a tape and need to break these up later. I'll even add screen loading support - or rather an API for it, so systems that support graphics can include screen$ support regardless of how their video is structured.

I'll add network functions too, but they will be handled by an external processor that allows "disk" space to be shared, so you can either map your BASIC to someone else's share, or share your own, though that will be an OS function - BASIC also won't be aware of it. It will solve the problem of having to be ready to receive the program over the LAN though.

I'm not really aiming at Binary compatible, but definitely code compatible. I'm changing too much to make it binary compatible - eg, no screen memory. In the end, it will have slightly more space than a 48K spectrum for program, even with the CP/M BDOS/BIOS at the top of memory.

 

[MiaM]

Re creating scroller games: On the Commodore 8-bits you could also go to the upper right corner and print an insert character code, and then do a POKE telling the screen editor that line 1 actually isn't linked with line 2. That way everything except the top line would scroll downwards. It was fully possible to write "racing" style games this way in BASIC

Oh, yes, it was the error handling I was thinking about when I wrote "the interrupt thingie"

Re software written for tape: Maybe you might want to add code that remember the last file name loaded (and also saved?) and if LOAD is done with an empty string it searches the directory for the next file name sorted alphabetically? That way a two part loader would work as long as the files are named correctly, even if it doesn't specify the name of the second part but rather just expects the next file from tape. (Don't know if Sinclair allows unnamed files to be saved, if so you'd perhaps want to generate alphabetically sequential file names for SAVE too).

 

[cj7hawk]

You can't save an unnamed file, at least it's not supposed to be possible and is trapped, but you can save a blank filename. You can also include non-printable codes, terminal codes etc. You could even add in up to ten tokens.

I thought about having Load "" work automatically, and might set in a default using the CP/M autofind - ie, it loads z???????.??? and then the first program it finds. Who knows which program it will find first though. I also might get it to remount the last TAP viewed with CAT - so CAT becomes the selector - so if you do "CAT "Program2" then LOAD "" it will mount Program2, then load the first file in Program 2. This gives me close to backwards compatibility with the usual Spectrum way of working. Using CAT as a defacto "Change Directory" type of command seems the easiest way to do it without breaking anything.

Having sequential files within the TAP files on CP/M works well with the Spectrum LOAD method. You can still load programs natively directly, but at least it's an option to save all the files to the disk directly. I'll include the possibility to turn them into a single tap with MOVE also.

 

[cj7hawk]

Great progress!

Possible feature creep suggestion: Giving CAT a text file as an argument prints the text file on screen, with a "more" style feature. Not being able to look what various files contain is perhaps the biggest constraint when using old 8-bit microcomputers.

Yep, since a catalog of everything in a TAP file can be examined, I like this idea and will implement it. I should include the filename, the type, and it's size in bytes. Since I won't get a huge number of files in a TAP (hopefully) I can show them vertically, even if I show the CP/M names horizontally - Suggestion onboarded

 

[MiaM]

Even more feature creep:
If every file I/O command works on a specific TAP file, and if CAT is the only command that user programs will likely not use (at least giving it special arguments) you could also use CAT with some special syntax to get a directory listing "outside" the TAP file, and also to switch between different TAP files, perhaps?

 

[cj7hawk]

Even more feature creep:
If every file I/O command works on a specific TAP file, and if CAT is the only command that user programs will likely not use (at least giving it special arguments) you could also use CAT with some special syntax to get a directory listing "outside" the TAP file, and also to switch between different TAP files, perhaps?

Very Astute - yes, it already does this.

In keeping with Sinclair structures, CAT "" will produce a directory of any sinclair-related files on the disk. I can add extra data after the "" if necessary and have the save structure identify and analyze it which would allow for TAPs to be switched mid-operation an a, examining that as a likely path. I can also use OPEN and CLOSE and FORMAT. But when the TAP filename is included eg, CAT "PROGRAM" then it mounts "PROGRAM" and scans the TAP for files within the TAP.

At the moment, if I type CAT "" it will show me any TAP files on the CP/M machine, which are identified with a z at the start, rather than with the TAP extension. The reason for that is LOAD "File" is double-action. If the TAP is loaded, it searches the TAP. If the TAP isn't loaded, it searches the directory for a TAP of the same name, then loads it, then proceeds to scan the TAP for the file of the same name - and when I save, it saves both a TAP filename and a directory filename that is compatible, so there is zero requirement to pre-load a TAP prior to loading the BASIC program. It will happily access the disk directly just like it's a tape system.

Though I will also allow streams in a modified version, so it might be possible for a BASIC program to execute CAT commands directly and capture this to a buffer for input. Quite how I'm going to do that, in a single-threaded system, I'm not sure. But if I can create a buffer for streams then it should be possible - Although Sinclair BASIC should do it native, the concept of a buffer is pretty alien to it. Most buffers are a single character at most. Not enough for a full string. If I left it up to the default mechanisms it would only transfer the first character. I may need a cooperative multitasking element added in, where if the CAT is called, and it's a single "character" then I suspend the BDOS processes until the next character is called and so on, and rather than transferring the output to a buffer, I hold off on reading the input further.

I could solve the issue in the underlying OS, but then it wouldn't be a CP/M compatible solution. For now, the simplest way seems to be to drive the output direct to the screen, so that a program can call to display the files, but can't directly interact with them, much like how Wordstar did it.

 

[cj7hawk]

So feedback time - looking for comments.

When a "TAP" is viewed, what should the information structure look like?

eg,

<type> <size> <name >

or

<name> <type> <size> ?

Or what other information?

The Spectrum data types are BASIC, VARIABLE, STRING and DATA and I was thinking of simplifying this to "BAS" "VAR" "STR" "DAT" but this brings up the question as to whether I should add a fifth for the data block itself?

And should I separate header information from block information across two lines? Or put the block information on the same line following the header?

Of note, sometimes programs didn't use the header - they just created multiple runs of data, though it's probably not an issue here.

Not many programs had a lot of files, but programs like music makers often had different files for each different tune following the main program, and you often stopped the tape before loading.

But I'm curious as to what "Best Practice" might be when listing the files in a TAP file?

 

[MiaM]

I would recommend either name, type, size or size, name, type. I.E. type always goes immediately after type for every directory listing I've ever come across.
Don't know about contents in the files. Perhaps just show the ascii/printable parts from the start, including detokenized basic perhaps?

 

[cj7hawk]

I would recommend either name, type, size or size, name, type. I.E. type always goes immediately after type for every directory listing I've ever come across.
Don't know about contents in the files. Perhaps just show the ascii/printable parts from the start, including detokenized basic perhaps?

I had a detokenized "type" command I wrote for CP/M long ago - but that doesn't run as Sinclair BASIC - it's just how I wrote the type command under CP/M - to be Sinclair-aware.

This is more for the CAT command - which can show the TAP files in the CP/M directory, or the Sinclair files in the TAP file, since a TAP is a bit like a Sinclair version of the LBR function in CP/M.

At the moment, it looks something like this;

 

[cj7hawk]

I'll add that it also shows how the Sinclair editor currently works ( ie, the inverse block showing the mode, that moves back and forth on the line being edited ) and the TAP is the Mark Time Music Box TAP, which I chose because it has a good mix of files in a single TAP as the original TAPE had quite a few files on it. The read past the EOL is another bug - I forgot to adjust the stack, so rather than check where it was and correct, I just let it read to EOF. Though it does show how the extra errors work in the updated BASIC.

 

[cj7hawk]

So an interesting observation. The Spectrum "LOAD" command will generate a list of all the entries in a TAP file as well, as long as it can't find the program it's looking for. It says "Program" or "Data" for different save types. Interestingly, now it can't reprint the same line over and over, it shows each entry twice - once for the header, and once for the data block. I imagine once it doesn't recognize the header, it just ignores the datablock and already has the name in the buffer, so just reprints it via the "failure to find file" routine. Though I haven't checked that yet.

I still need a way to lock a TAP in, so it gets used as a primary directory to allow the files to be loaded non-linearly. I'm not really sure on what to do there command-wise. Load now both mounts a TAP and looks for a filename within a TAP. I think it's important to keep these functions together - so that new files can be saved separately, but bundling them also makes a lot of sense. If it were from scratch, it would be simple - just leave it like that, but even with bundling the software needs a way to break the mount and mount a new tap.

I only have "stand-in" routines at the moment, and they aren't optimized as they just fake-read the entire file to get the entries, and I can replace that with random file I/O to only read the entries themselves.

Another interesting thought that arises from this - At the moment I've changed "OPEN #" and "CLOSE #" to "OPEN#" and "CLOSE#" without the space - but I wonder if I should drop the # as well. As an interesting "aside" the commands under Sinclair BASIC can be changed without any loss of functionality as they are all tokenized, so if one system sees it as "OPEN #" and another sees it as "OPEN" there is no problem and no interoperability is lost. Though it might be confusing if people type OPEN# when the system just expects OPEN.

At the moment, I'm leaning towards allowing a TAP to be permanently mounted as #16, which doesn't break Sinclair conventions - It simply won't work if transferred to a normal spectrum, but they don't have a disk anyway. So Load "FILE" will mount and load FILE, but Open#16,"FILE" will simply mount it, and defer all fill commands to the TAP -meaning subsequent CAT commands would just cycle through the TAP listing instead of the root directory until a "CLOSE#16" was encountered.

If I do it this way, it may also be possible to break the "fourth wall" of the OS and read from the stream directly - reading and writing to the CP/M file rather than the BASIC OS file, allowing a method of opening CP/M files and writing them outside of the TAP format, or to load a TAP format file and process it in binary, including the vectors at the start of each program. Doing it this way, I could allow the BASIC program to even go as far as opening up the CAT "" command to show the full CP/M directory listing rather than just the spectrum files.