• Please review our updated Terms and Rules here

Question about CP/M Behavior on serial writes (to disk)

cj7hawk

Veteran Member
Joined
Jan 25, 2022
Messages
1,058
Location
Perth, Western Australia.
Hi All,

Once again I am looking for assistance ( Won't be the last time, and thank you so very much for all the help so far, Especially Doug and Chuck and John on both forums )
I'm looking to better understand CP/Ms write behavior, since its not so easy to observe. And please if there's a convention that exists, but isn't necessary, let me know also.

I'm trying to get my head around what is normal behaviour for the OS - this doesn't seem so well defined in the documentation as it's more related to the code itself and how it is written than it is to the instructions on how to open files and write to them. I wrote some code to perform and then examined the memory between BDOS calls.

*Do any files manipulate the record and extent counter to perform defacto "random" disk I/O operations? eg, Rewrite the same record before closing?
* Does any software read and write sequentially to a file at the same time? (eg, manipulating the FCB as per above).
* Is the FCB ever copied back to the disk with each write operation, or only on closing the file? On Blocking/Deblocking? When an extent is filled? Only when the OS chooses?
* Are multiple open files for write supported or required on one disk?
* Are multiple open files for write usually supported or required across multiple disks?
* Are the allocations in the FCB really as arbitrary as they seem? It seems like as long as the disk handler has the FCB, it can translate the disk however it wants, and only the BDOS calls matter, along with the three trailing bytes for truly random I/O.

Some quirks I've noticed that I am not sure about... (on a real machine) is this how it should be?
1) It seems entirely valid to open a file on a subsequent extent (eg, open Extent 2 rather than Extent 0) and to write it, although it obviously doesn't show up in directories and is difficult to delete.
2) It seems that it's also possible to open multiple extents of the same file independently - eg, a file could be written backwards, starting from the last record.
3) What the disk system does with an open file seems to be entirely up to the disk system. Including when (prior to closing) anything is written to disk.

Thanks again,
David
 
So, prior to the introduction of the READ RANDOM and WRITE RANDOM, and associated, BDOS calls, yes programs did directly manipulate the FCB to do random I/O. DRI certainly strongly recommended that all programs stopped doing that, but there might be some old programs out there. Ugly stuff, and especially ugly to try and emulate.

Regarding the FCB, it is NOT the directory entries used to store a file on disk. The FCB is never directly read or written to disk. But, parts of the FCB are copied to the directory entry. However, that is only done when the (16K) extent is "closed". For example, if you write 47K to a file and don't close it, there will be 2 (16K) extents fully saved on disk ,and only the last 15K will be lost because you didn't close. In the old days when you could actually hear the head stepping on your floppy drive, you'd hear after every 16K was written the heads would zip back to the directory track - where the previous extent was closed and the next one was opened.

With conventional CP/M, there are essentially an infinite number of open files, and usually CP/M doesn't know if you plan to write to them or not. The only limit on the number of open files is the amount of memory you have for FCBs. And, technically, you could swap FCBs in and out of memory and when you bring one back it is still "open".

DRI states that a user program must never touch bytes 16-31 of the FCB (starting at 0). The BDOS "owns" those bytes and that's where the allocations take place. If those allocations do not get saved to disk, then loging in the disk will show them as free space again. The BDOS uses that information in those allocations to compute track and sector for each block (and adds +1 for each subsequent sector within the block, with carry over to track number). But, if you are replacing the BDOS with your own invention, you may do something different - although you have the potential to be incompatible with a program that pretends to "know" too much about the FCB. For my CP/NET servers, I implant a file descriptor ID in those bytes (along with some check/validation bytes). Only the directory entries returned by SEARCH functions put fake allocation data there. MP/M keeps a checksum and will throw an error if a program alters those bytes.

I would suggest you draw a line in the sand about how errant a program is allowed to be on your system. You probably can't support every program ever written unless you use the actual DRI BDOS. Supporting programs that don't use RANDOM access function might be a line you draw.
 
Hi Doug, that makes sense. I am using the allocation vectors in the FCB to store real allocations at the moment, as my current disk system follows CP/M - and should be possible to read/write on other CP/M machines assuming the DPH and DPB match up. But I only want to do that for the "disk" drives, which will be drive A/B/K/L which all share a single AV table space in memory so I'll need to rebuild the table each and every time a write changes disks and commits a record, which won't be all that efficient, but the inbuilt drives before expansion cards are installed will be based on battery backed static ram or EPROMS ( or maybe a mix of both, since it seems that's supportable under CP/M, eg, both read-only and read-write disk space )

And I'm planning on having the subsequent disk systems (eg, real floppy, HDD etc ) maintain their own memory space and doing the blocking/deblocking outside of BDOS so everything will resemble 128 byte sectors, but I don't think that causes any compatability issues since 128byte sectors are common on some older disk systems, and if the disk driver does the blocking/deblocking outside of CP/M I don't think that's an issue either, as long as the files close correctly.

Speaking of which, did CP/M ever assign a logical drive to sequential storage? eg, Tape, Punch, Infinite Loop, etc? So that files could be loaded and copied from these? Or was it all via bootstrap software to load from things like punched tape.

I don't want to deviate too far from CP/M, but only to the extent that I don't miss anything important. After that, it's all extensions ( eg, the case insensitivity, network disks, IP etc ) and I still need it all to fit in the same space that BIOS/BDOS normally takes up, including IP support, but I don't want to end up with MS-DOS either, which is ironic as the path I'm taking to rewrite CP/M is closer to the way MS-DOS was created.

The disk write routines are the next thing I'm working on, now that the disk read routines are complete and seem to work reliably.

Thanks
David
 
CP/M did not have the "unified file API" like Unix. Files were files and character (stream) devices were character devices.

CP/M 3 BDOS was capable of doing the blocking/deblocking of large sectors itself, and it also provided a buffer-management scheme for deblocking buffers. The really relieved the BIOS of a lot of that mess. Otherwise, the BIOS must manage physical sectors larger than 128, and try and optimize that (e.g. to avoid disk access on every 128-byte record). The BDOS file API is 128-byte records already (with a slight caveat for multi-sector I/O). The BIOS disk API for CP/M 3 could handle physical sectors or strictly 128-byte "sectors". For older CP/M, the BIOS disk API is strictly 128-byte "sectors".

Note, allocation vectors are a transient bitmap showing only which blocks are in use (for an entire disk). The "disk map" is the section of a file's FCB/directory-entry used to store the block numbers that were allocated to the file (and their order of allocation).
 
*Do any files manipulate the record and extent counter to perform defacto "random" disk I/O operations? eg, Rewrite the same record before closing?
On this point: Digital Research's LINK does this rather than use the random access API, presumably so it can do random access on CP/M 1.
 
On this point: Digital Research's LINK does this rather than use the random access API, presumably so it can do random access on CP/M 1.

Hi John, I find this rather Ironic given Doug's earlier comments about DR warning people not to do it, but it makes sense from an OS perspective, because that's exactly how I'm going to write my random IO - I'm just going to mask, shift, and load the results straight up into the EX and CR fields in the FCB and do it from there.

:)

It's a long, slow path, but I've come a long way since I started trying to write some CP/M disk handling routines in Basic earlier this year so I could read/write/copy my amstrad/sinclair disk images onto the PC. All the help along the way has genuinely been appreciated.

Out of curiosity, do you know of many people who have rewritten the CP/M OS entirely ( rather than building the BIOS and copying/modifying the BDOS and CCP ) from scratch?

I'm imagining I'm not the only one, even excepting the famous MS-DOS case, but I can't find any records on the Internet so I'm guessing those who did probably did so very long ago, long before blogging or even recording one's efforts got recorded online?

David.
 
The BIOS disk API for CP/M 3 could handle physical sectors or strictly 128-byte "sectors". For older CP/M, the BIOS disk API is strictly 128-byte "sectors".

LoL! You have no idea how hard this was for me to figure out. It's been messing up my planning of code writing for months. I only just figured this out a couple of weeks ago.

It was all the documentation that called the records "Sectors" that did me in. I figured they had to be Disk sectors, since all the relevant masks and other information is in the DPH/DPB but it was only when I looked into early disk drives with 128 byte sectors that I realized it probably wasn't even a consideration early on. Then things started to click and the penny dropped and I figured out what they were doing.

I think you did mention this earlier, but it's very easy to miss important details when you're still learning, even when an answer is specific and obvious in hindsight.

I got my sequential write routines working this morning, I needed to work on another project over the weekend, but the power was out while I was getting a generator put in for Summer and I was interrupted a lot, so my Christmas sprint got moved to yesterday with clean up this morning.

I still have to do code optimisation, because I'm pretty sure a lot of the read operation and write operation can reuse the same routines, but writing the CP/M OS from scratch in z80 is teaching me a lot about why DR did the things they way they did. So far every time I try to change something, I realize that there's quite a few reasons they did things they way they did.

I still remember how long it took me to debug code dry in the 80s and I've been working in my emulator to speed things up, which lets me break the code anywhere, see what is going on and what's on disk and the like, and while I'm writing all of the debug into my CCP, I'm amazed something a complex as CP/M got updated so often by a single person. I was still a kid in the 80s just learning z80 and I'm slowly gaining the benefits of having once attended failed university course that imagined people would still write OSs in the future while I write my own CP/M, but I'm in awe of what they managed to do back then as solo programmers. From scratch...

To use an analogy, the weeds over the paths may have grown back over the trails forged long ago, but it's still less work refinding them than it would have been for the original trailblazers who created them.

Now I have BDOS sequential read and write working, a CCP half-written, can load and execute .COM files and have a functional skeletal bios, it feels like I've reached the halfway point in the project progress... :)
 
A quick question on this topic to ask a further question if I may - When serial writes fill up the disk space, and no free blocks remain, then what happens?

Does this routine just return a FF, or does it bomb out with a visible error (eg, DISK FULL or BDOS ERROR). or does it do both (visible error and return code )?

Not sure if there is a consistent behaviour across CP/Ms for this ?

Thanks
David
 
CP/M 2.2 provided a semi-format way to patch the BDOS to control error handling. CP/M 3 provides a formal mechanism via a BDOS call. However, CP/M 2.2 defines the return code for WRITE SEQUENTIAL as:
Register A = 00H upon return from a successful write operation, while a nonzero value indicates an unsuccessful write caused by a full disk.
CP/M 3 and MP/M (and CP/NET) support more detailed error codes.

The only errors that the BDOS traps specifically are "Bad Sector", "Select", and "Read Only". And those can be overriden by the above techniques.
 
Can I reasonably assume then that the BDOS errors I get from my Amstrad is because it's CP/M3 then? It prints error codes for BDOS calls regardless then crashes back to CCP. Or is that normal CP/M3 behavior? (eg, trying to create a file that already exists )
 
Can I reasonably assume then that the BDOS errors I get from my Amstrad is because it's CP/M3 then? It prints error codes for BDOS calls regardless then crashes back to CCP. Or is that normal CP/M3 behavior? (eg, trying to create a file that already exists )
That sounds a bit draconian for default CP/M 3 behavior - the BDOS forcing an abort because a file exists. Unless what you are seeing is that the *PROGRAM* detects that a file exists and it decides to abort. Traditionally, BAD SECTOR, SELECT, and R/O DISK were the errors that the BDOS forced an abort over. But, CP/M does have extended error codes and the BDOS messages do print those error codes. I'd have to read more on the CP/M 3 error handling feature to see what the options are.
 
It's my own program opening the file, so I know it's not my own error handler, because I don't have one. The code blindly follows each step then opens a monitor so I can see what the code is doing / has done. :) I never get the monitor after this, so I think it returned to CCP via Reset.

The exact text on screen is;
CP/M Error on A: File Exists
BDOS Function = 22 File = NEWFILE .TXT
A>

Verbatim. Including the space between NEWFILE and .TXT

It might be a little draconian, but it's how the PCW9512 responds - though they call it CP/M+ and not CP/M3, though from what I can tell of books of the era, the terms were interchangeable?
 
Back
Top