is there a chkdsk type tool for CP/M?

[cj7hawk]

Thanks to Chris Radek, FSCK has now been tested on vintage hardware and not just my one-of-a-kind lockdown project. This is an Imsai, a Z80 S100 machine running CP/M 2.2 with an 8" single sided single density (standard IBM 3740 format) floppy. There were some issues, but these have been fixed with the help of the owner and we now have FSCK 1.2. See: https://github.com/rundel-tech/ZARC/tree/main/software/cpm/cpm_dev/fsck for details.

Nice project - I'll pull it down and have a play with it later. From the source comments; (which provides an idea of what it does.)

signon_msg byte "*** CP/M File System Checker V1.2 ***", 0x0d, 0x0a, 0x0d, 0x0a, "$" no_z80_msg byte "Z80 CPU required", 0x0d, 0x0a, "$" op_err_msg byte "Operand error. Expected format: fsck <drive>[:] [/s]", 0x0d, 0x0a, "$" exit_msg byte "Returning to CP/M.", 0x0d, 0x0a, "$" memory_msg byte "Insufficient memory", 0x0d, 0x0a, "$" sel_err_msg byte "Can't select drive.", 0x0d, 0x0a, "$" disk_err_msg byte "Disk I/O error.", 0x0d, 0x0a, "$" dir_done_msg byte "Directory checks complete", 0x0d, 0x0a, "$" files_msg byte "Files found: $" unused_msg byte "Unused directory entries: $" blocks_used_msg byte "Blocks used: $" block_map_msg byte 0x0d, 0x0a, "Block map ('D': directory, 'F': file and '-': unused).", 0x0d, 0x0a, "$" errors_msg byte "Errors and warnings: $" no_ss_msg byte "Surface scan skipped", 0x0d, 0x0a, "$" surf_scan_msg byte 0x0d, 0x0a, "Checking all blocks are readable (surface scan)", 0x0d, 0x0a, "$" track_msg byte 0x0d, "Track: $" sector_msg byte ", sector: $" scan_done_msg byte 0x0d, 0x0a, "Scan complete", 0x0d, 0x0a, "$" ; Directory entry error messages bad_user_msg byte " - bad user number$" bad_name_msg byte " - bad file name$" bad_ex_cnt_msg byte " - bad extent count$" bad_s1_msg byte " - bad S1$" ext_dup_msg byte " - duplicated physical extent$" dble_blk_msg byte " - block not unique$" sparse_msg byte " - is sparse (holed) [warning]$" blk_too_big_msg byte " - block number out of range$" no_alloc_msg byte " - no allocations in physical extent [warning]$" bad_rc_msg byte " - RC not 0x80 in intermediate extent$" empty_msg byte " - has no data [warning]$"

I wrote some consistency checks into my PC based DSK editor I wrote last year to move files to and from Spectrum CP/M disks ( Spectum +3 reads Amstrad disks ) - and perhaps the best use of the cross-linked files check was to tell me if I had the disk structure correct.

But different applications seem to use the OS access to disk differently. I noted Wordstar creates extent 3F for temp files so that they don't typically show on a "Directory" listing but are still valid and cannot be overwritten until deleted, and they show on STAT - so the lack of rules around CP/M directory structure probably leads to issues with a FSCK program in some cases - so reporting stuff is nice, and not trying to fix it is nicer -

I intentionally cross-link files in my new OS too, as it's completely valid for that architecture to cross-link CP/M files if the disk is memory - as it just means that multiple programs are using that same page - so the flexibility of CP/M and the lack of rules is actually a really powerful tool and capability. It also means that when I switch back to real CP/M it doesn't cause errors or issues as anything CP/M accepts seems valid by design.

Thanks for sharing - Looking forward to checking out the binary later.

David.

 

[Rundel]

Hi David. I hope you enjoy it. There is a write-up (fsckGuide.pdf, in the same folder as the .COM file) so you don't have to examine the source (but feel free, obviously).

 

[cj7hawk]

I like it - it works well. If there was anything I'd like more in it, it would be;
* Case Insensitivity on the CLI and
* If no drive is specified, default to the current logged drive.
* A way to stop it scrolling when it needs to display more lines than the screen has available - especially when there's a lot of files with problems.

Case insensivity probably isn't a problem with DR's CP/M but might affect other BDOS/CCPs. In my case it just means I have to type in the command in caps.

 

[Svenska]

Regular CCP always upper-cases the command line. I'm fairly sure many programs rely on that behaviour.

 

[rlauzon]

Regular CCP always upper-cases the command line. I'm fairly sure many programs rely on that behaviour.

MBASIC will let you create files with lower case letters.

 

[cj7hawk]

Regular CCP always upper-cases the command line. I'm fairly sure many programs rely on that behaviour.

I Imagine that's exactly the reason. There is not a lot of value in sanitising the input if the OS can be relied upon to do it for you. It just means more code required to do the same thing.

CP/M Plus was the same, though I wonder, had CP/M continued and we got 4,5,6 etc, would it have become case insensitive - After all, history was written by then and DRDOS was out - IIRC, it didn't fix case and wasn't case sensitive. But my memory of that could be wrong.

But as was just pointed out, the BDOS would let you put other characters into the FCB, yet it is case sensitive. BDOS being case-sensitive and CCP being Case Setting is a bit of a long-term issue IMHO.

Also, implementing a new CCP that was case insensitive made sense around that time, so I assume some of the CCP replacements that came out may have possibly supported case insensitivity in filenames also. That's certainly what I did, but mainly for compatability with non-CP/M systems which was also a goal when I wrote mine. I can't imagine I wrote the only CCP/BDOS that lets a=A

Where's an online emulator of every version of DOS that ever came out when I want one !

Anyway, I really like this program. Though it highlights the issues of producing more characters than the display can handle. Did CP/M ever solve that like DOS did in later years when it started to implement more unix-like commands?

 

[Svenska]

MBASIC will let you create files with lower case letters.

Not through the CCP-provided command line, on regular CP/M. But it puts lower-case letters in the FCB, which is arguably a bug or at least a very bad design decision.

Though it highlights the issues of producing more characters than the display can handle. Did CP/M ever solve that like DOS did in later years when it started to implement more unix-like commands?

No, terminal handling always was an application issue; codepages did not exist on CP/M. DOS only solved this when they introduced NLS, and the world was standardizing on PC-compatibles by then.

Where's an online emulator of every version of DOS that ever came out when I want one !

Isn't pcjs good enough?

 

[cj7hawk]

Not through the CCP-provided command line, on regular CP/M. But it puts lower-case letters in the FCB, which is arguably a bug or at least a very bad design decision.

What he said was correct though - MBASIC doesn't work through the CCP CLI. It has it's own CLI.
So if you do something like;
10 PRINT "Funny Filename"
SAVE "FunnyCHR.Bas"
then it will save it as FunnyCHR.Bas - with the mixed case.

Then you go and do something like ERA FunnyCHR.Bas and it says something like "FUNNYCHR.BAS Not Found" - I can't remember the exact message. Basically, BDOS is case sensitive and it gets rewritten as CAPs. It's a bit messy like that.

So it's a good example of when the CCP CLI fails CP/M - espcially given MBASIC is a pretty common basic - You can manage the files through BASIC too, but given the CCP is the entry point to the OS, it's a problem. You can use Wildcards in some cases to deal with the filenames, but that's just a workaround.

Thank you for the PCJs mention! That's perfect.

 

[tskemper]

Hello Rundel,
I just found your fsck-program in this forum, because i was looking for a tool to find out directory errors on my CP/M-machine.
I had to make some corrections in it to get it working properly for me:
- I use not the standard CP/M-BDOS but ZS-DOS, so I make use of some special file attributes like "archived", "public access", "no-date". Your program dropped thousands of "Bad file name" errors because of attribute-bits set to 1 which aren't allowed in standard-CP/M. I had to eliminate the test for all the 7th bits in the filename, then it worked better.
- another thing was, that I couldn't use ^S to interrupt the program und not use ^P to print the output. This is, because the output is done by the BIOS not by the BDOS. I fixed the conwch routine in cpm_utility.z80 to use the BDOS, than it worked fine.
Perhaps someone is interested in these hints.
Thank you to writing this tool!
Thomas

 

[Rundel]

Hi Thomas (tskemper). I'm glad you took the time to have a look at fsck. I have updated it to work with CP/M 3 and made the changes you suggested. I have no (easy) way of testing it with ZSDOS so am interested in how you get on. There are some more options to control the output which go some way to address cj7hawk's issue that the messages scroll off the screen. Normal BDOS flow control ^S / ^Q now works as well. fsck will also help if you have password protected files but have forgotten the password (/V option).
Merlin

 

[Svenska]

MBASIC doesn't work through the CCP CLI. It has it's own CLI.

I never wrote that MBASIC uses the CCP CLI. Not sure where you got the idea. But MBASIC, like many other programs, reads the CCP-provided command line.

You can type "mbasic myfile.bas" and MBASIC.COM will automatically load and run MYFILE.BAS. File names are uppercase.

Basically, BDOS is case sensitive and it gets rewritten as CAPs. It's a bit messy like that.

The CP/M documentation never mentions mixed-case or lower-case filenames. It is strongly implied that only upper-case filenames are valid. The same is true for 8-bit characters: In file names, they are explicitly forbidden, but they are not officially supported anywhere else, either.

I do not consider BDOS to be "case sensitive". Instead, I consider applications using lower-case filenames "broken". They misuse the FCB API.

 

[Rundel]

I will look into case insensitivity on the CLI (requested by cj7hawk). It should be an easy change. I use DRI's standard CCP which always converts to upper case so it isn't an issue.

 

[cj7hawk]

The CP/M documentation never mentions mixed-case or lower-case filenames. It is strongly implied that only upper-case filenames are valid. The same is true for 8-bit characters: In file names, they are explicitly forbidden, but they are not officially supported anywhere else, either.
I do not consider BDOS to be "case sensitive". Instead, I consider applications using lower-case filenames "broken". They misuse the FCB API.

IIRC, CP/M forces the command line to upper case, which is fine as long as you're using the command line. CP/M specifically defines what characters are valid.

Once you use other software, it's possible to write directory entries with non-standard characters. Even under CP/M. Though I can't recall whether it's the CLI or BDOS that enforces this... ( I think it might be the CCP ? )

Filenames that don't respect this as "broken?" well, it's not wrong from a purely CP/M perspective... Though how about programs that misuse the allocations to create "sparse" files that don't have a starting block and don't exist when you do a directory? Like Wordstar - is that broken too?

 

[Rundel]

This is the situation as I understand it:

* The DRI CCP forces the command line to upper case. Hence all filenames referred to by this method will be upper case. Replacement CCPs don't necessarily do this. My code will not currently recognise lower-case letters, so can have trouble if invoked with a non-DRI CCP.
* Some applications (like Microsoft Basic) allow you to create and use files with mixed case, which the BDOS permits. They are just a pain because you can't (easily) refer to them with commands typed from the DRI CCP command line. It is of course debateable that this is an error. Perhaps I should make it a warning.
* I chose to report sparse files as a warning, not an error. It is a bit odd, and I don't think many applications do it. I have seen it occur as a result of bugs (mine).

I am aware that there are many variants of CP/M, replacement CCPs and the like. I don't think I can reasonably cover them all, and testing would be problematic. Some systems will likely report "unrecognised directory entry ..." for things that it doesn't understand. That shouldn't affect the basic functions such as detecting doubly-allocated blocks and the like.

 

[durgadas311]

...
* I chose to report sparse files as a warning, not an error. It is a bit odd, and I don't think many applications do it. I have seen it occur as a result of bugs (mine).
...

DRI does discuss/mention sparse files in the Programmer's Guide, and they are supported and expected to work. I think the main use for them was database programs (which could include apps written in MBASIC). But, they could be sparse "by design". Still, a (softly-worded) warning seems OK as long as you can verify the integrity of such files.

 

[Svenska]

Programs that rely on unspecified and obviously unintended behaviour are broken, yes.

The default FCBs provided to the application by the BDOS are guaranteed to have upper-case file names. This is a strong hint that filenames are expected to be upper-case, especially when taking upper-case-only terminals or printers into account.

CP/M did not waste bytes or clock cycles on further input sanitation. It expected operators to do the right thing, as was common back then. This does not mean "anything goes". Both Intel and Microsoft had to learn this the hard way, too; Digital Research did not live long enough.

Perhaps I should make it a warning.

Sparse files are non-standard, but legal. If you think they are worth a warning (which I think is correct), then files with non-standard names should also be worth a warning. Same as files without a beginning (no first block).

I am aware that there are many variants of CP/M, replacement CCPs and the like. I don't think I can reasonably cover them all, and testing would be problematic. Some systems will likely report "unrecognised directory entry ..." for things that it doesn't understand. That shouldn't affect the basic functions such as detecting doubly-allocated blocks and the like.

Intentionally breaking the file system to force a specific implementation to behave in a non-standard manner... makes it hard to reason about the file system's validity. Taking the executive decision that incompatible CP/M variants are "not CP/M" is fine.

I'd recommend trying to detect the currently running system and disable unsuitable checks, and print at least a warning if you cannot identify the system (because any "repair" might break something). Some people can get incredibly creative, as this thread shows.

 

[durgadas311]

Obviously, "right" or "wrong" is irrelevant here. What matters is that there are/were useful applications that produced some "oddities" but remained usable, and any prospective 'chkdsk' utility would want to handle those in order to be useful.

 

[cj7hawk]

It's a nice utility Thanks @Rundel

 

[norwestrzh]

Yep. Looks like a good tool. I tried it on my 20 MHz Z80 SBC. Runs in a flash. I don't understand some of the messages. I don't have the .ASM files for ERAQ20 and UNERA33. I probably erased them after building the applications. There are .COM files however. ZCPRDEMO.MAC is a rather large file, and I can TYPE it with no problems.

Edit: My bad! I found a zero length (empty) file in A5: with the name ZCPRDEMO.MAC (which I deleted), and also the two .ASM files indicated in another user area. The two .ASM files TYPE out OK, so I gather that the S1 error isn't fatal?

 

[cj7hawk]

Yep. Looks like a good tool. I tried it on my 20 MHz Z80 SBC. Runs in a flash. I don't understand some of the messages. I don't have the .ASM files for ERAQ20 and UNERA33. I probably erased them after building the applications. There are .COM files however. ZCPRDEMO.MAC is a rather large file, and I can TYPE it with no problems.

Edit: My bad! I found a zero length (empty) file in A5: with the name ZCPRDEMO.MAC (which I deleted), and also the two .ASM files indicated in another user area. The two .ASM files TYPE out OK, so I gather that the S1 error isn't fatal?

IIRC, Sometimes S1 get used for "binary" files and records the bytes used in a record, so that it shows how big the file's binary size is.

But CP/M doesn't use it at all - So all files are whole multiples of 128 bytes and the file ends when the operating system encounters a Ctrl-Z character... Regardless of how many more records the file itself holds.

I guess binary files weren't a thing back in the 1970s... CPM's file system has a lot of ASCII influences - and ASCII is only 7 bits.

So it's not that weird for S1 to hold something other than what a specific version of CP/M is expecting - and if you're using text files terminated with Ctrl-Z, then S1 doesn't matter... As long as it doesn't clash with the current OS you're using.