Model 1 Disk Archiving Project

qbarnes · Nov 26, 2024

New release of gw2dmk: https://github.com/qbarnes/gw2dmk/releases/tag/v0.0.1.alpha.8. Please report any problems found.

On MS Windows, when Greaseweazle was on a COM device with 2 numbers (e.g. "COM57") and running on an old version of the OS, gw2dmk wasn't using the "\\.\" prefix hack so had to specify the device name with the prefix as a workaround. Update should make this prefix unnecessary.
Plugged some memory leaks.
Command fails now if "-d <drive number>" is not provided on the command line instead of defaulting to 0. Will autodetect drive number soon.
Incompatible sides check (--[no]compat) wasn't detecting properly.
Crashed when writing out DMKs to HD floppies.
All kinds of drives (-k option with 1 to 4) should now be fully supported. Before only -k 1 was supported (1.2MB HD drive writing to single and double density diskettes.

richarjo · Nov 26, 2024

Successfully wrote a TRSDOS 2.2 DMK to disk with 360k drive with version alpha_8 DMK2GW and read back to DMK with GW2DMK reporting 10 good sectors per track. The disks (2) are labelled TRSDOS 2.2 but had 2.3 on them. It's good to be able to restore them.

Thankyou @qbarnes for an awesome bit of kit. I've been tearing my hair out with GW.exe to write more than 9/10 sectors/track with 35T TRSDOS DMK iimages.

NB: I haven't tried the written disks on a real drive, just read back, run in emulation. My TRS-80 model 1 is Gotek powered but I have a few system disks.

richarjo · Thursday at 2:43 AM

Successfully wrote a TRSDOS 2.2 DMK to disk with 360k drive with version alpha_8 DMK2GW and read back to DMK with GW2DMK reporting 10 good sectors per track.

Thankyou @qbarnes for an awesome bit of kit. I've been tearing my hair out with GW.exe to write more than 9/10 sectors/track with 35T DMK iimages.

NB: I haven't tried the written disks on a real drive, just read back, run in emulation. My TRS-80 model 1 is Gotek powered but I have a few system disks.

richarjo · Thursday at 2:46 AM

Keir Fraser has done a development build of GW.exe v1.22 fixing DMK handling. He would like some testing on a DMK written to disk and tested on real hardware. As I have a Gotek in my Model 1 would you be able to test it? Link: https://github.com/keirf/greaseweaz...quNPZNiDFczn0L17yg_aem_uJeny97McwkUpUOL5kQang

qbarnes · Thursday at 4:12 PM

richarjo said:
Keir Fraser has done a development build of GW.exe v1.22 fixing DMK handling. He would like some testing on a DMK written to disk and tested on real hardware. As I have a Gotek in my Model 1 would you be able to test it? Link: https://github.com/keirf/greaseweazle/actions/runs/12054415540?fbclid=IwY2xjawG0Y1NleHRuA2FlbQIxMAABHcFballNlbcsHRHUQZ8lWV8JlLjCVzzbP_JfsVLdOeequNPZNiDFczn0L17yg_aem_uJeny97McwkUpUOL5kQang

Is something up with this link? When I saw it earlier, it was some facebook link that took me to a GitHub page with a GW build. Now it just breaks.

What kind of DMK?

There's dozens of kinds of DMKs. Does Keir just want a proof-of-concept quick test of one DMK that `gw` will write and run on real hardware? Or does he want to know all the DMKs he still has yet to support? Is there a PR or Issue on GitHub that's tracking this?

syzygy · Thursday at 6:50 PM

So, to continue reporting on my archiving project...

The newer version of gw2dmk written and posted by @qbarnes has been working *very* well for me!

First, I had about a dozen NewDos disks that I had not yet archived. Earlier, I made a copy of a NewDos system disk on a NOS floppy (using SU) and used that a few times to recover the files on the ND disks. IOW, I would boot the Model 1 with NewDos and then read the disks and copy the files.

gw2dmk, however, can read NewDos disks straight away! The resulting .dmk files can be read by TRS80Tools without problem.

gw2dmk has let me breeze through my Model III disks without problems! Granted, these were newer and better stored so much less cleaning needed and much greater 100% reads.

I used gw2dm to read a M1 disk that I had already archived. Then I used dmk2gw to write the disk on a NOS floppy. Then I tried the floppy in the M1 and it worked! This means that the DAM that had caused problems previously (e.g., with the GW .dsk files) appears to be written just fine. Sweet!

In fact, I really have no suggestions, but will report an experience....

I have archived so many floppies, I may just be confused, but my recollection is that If T0 is not formatted, gw2dmk appears to give up and tells you T0 is not formatted. When dealing with a flippy where I only formatted one side, this is helpful. In only a couple of cases was it an issue (and a small one).

Again, I have to go back and check it, but when I read that early version of Trakcess (I posted a photo in the thread), the map looks like this:

Code:

Cyl-> 0         1         2         3
H. S: 01234567890123456789012345678901234
0. 0: ...XX.X.....................XXXXXXX
0. 1: X..XX.X.....................XXXXXXX
0. 2: X..XX.X.....................XXXXXXX
0. 3: X..XX.X.....................XXXXXXX
0. 4: X..XX.X.....................XXXXXXX
0. 5: X..XX.X.....................XXXXXXX
0. 6: X..XX.X.....................XXXXXXX
0. 7: X..XX.X.....................XXXXXXX
0. 8: X..XX.X.....................XXXXXXX
0. 9: X..XX.X.....................XXXXXXX

Of course, with SU, one can easily deal with it - but seeing no formatted T0 gw2dmk said 'no'. Granted, you have so much in the program, there may be another way to deal with it that I don't know about. Now, it is possible that I am getting confused and it could have been another floppy where T0 was blown away (probably by me long ago and for some long forgotten reason).

Thanks so much @qbarnes. I have a dedicated use, but I have not had any real problems and nothing that I would call a bug, at least so far. Respect!

qbarnes · Friday at 2:34 AM

syzygy said:
So, to continue reporting on my archiving project...

The newer version of gw2dmk written and posted by @qbarnes has been working *very* well for me!

First, I had about a dozen NewDos disks that I had not yet archived. Earlier, I made a copy of a NewDos system disk on a NOS floppy (using SU) and used that a few times to recover the files on the ND disks. IOW, I would boot the Model 1 with NewDos and then read the disks and copy the files.

gw2dmk, however, can read NewDos disks straight away! The resulting .dmk files can be read by TRS80Tools without problem.

gw2dmk has let me breeze through my Model III disks without problems! Granted, these were newer and better stored so much less cleaning needed and much greater 100% reads.

I used gw2dm to read a M1 disk that I had already archived. Then I used dmk2gw to write the disk on a NOS floppy. Then I tried the floppy in the M1 and it worked! This means that the DAM that had caused problems previously (e.g., with the GW .dsk files) appears to be written just fine. Sweet!

In fact, I really have no suggestions, but will report an experience....

I have archived so many floppies, I may just be confused, but my recollection is that If T0 is not formatted, gw2dmk appears to give up and tells you T0 is not formatted. When dealing with a flippy where I only formatted one side, this is helpful. In only a couple of cases was it an issue (and a small one).

Again, I have to go back and check it, but when I read that early version of Trakcess (I posted a photo in the thread), the map looks like this:

Code:

Cyl-> 0 1 2 3 H. S: 01234567890123456789012345678901234 0. 0: ...XX.X.....................XXXXXXX 0. 1: X..XX.X.....................XXXXXXX 0. 2: X..XX.X.....................XXXXXXX 0. 3: X..XX.X.....................XXXXXXX 0. 4: X..XX.X.....................XXXXXXX 0. 5: X..XX.X.....................XXXXXXX 0. 6: X..XX.X.....................XXXXXXX 0. 7: X..XX.X.....................XXXXXXX 0. 8: X..XX.X.....................XXXXXXX 0. 9: X..XX.X.....................XXXXXXX

Of course, with SU, one can easily deal with it - but seeing no formatted T0 gw2dmk said 'no'. Granted, you have so much in the program, there may be another way to deal with it that I don't know about. Now, it is possible that I am getting confused and it could have been another floppy where T0 was blown away (probably by me long ago and for some long forgotten reason).

Thanks so much @qbarnes. I have a dedicated use, but I have not had any real problems and nothing that I would call a bug, at least so far. Respect!

Not sure how to completely read the above map. I'd assume "." means something was found (an ID Record) but unclear if it had a valid ID or data CRC for that sector, and I'd assume "X" means nothing found (no matching ID Record).

but seeing no formatted T0 gw2dmk said 'no'.

Looks like from the map that T0S0 had one sector (sector #0). That's good enough for gw2dmk to find that track formatted. That is, unless the sector had a bad CRC when gw2dmk attempted to read it, then it'll give up if no valid sectors on track 0 exist.

If the disk did have only one sector on T0S0 and it was bad, you can always tell gw2dmk to not check the disk for valid tracks (by checking track 0 for valid sectors) and just read the disk for so many tracks with the "-t X" option.

{c,g}w2dmk is filled with heuristics to make best guesses, but when those guesses need nudging, there's always a option to tell it to do what you need.

When dealing with a flippy where I only formatted one side, this is helpful.

I'm not sure I follow. You you made a disk physically flippy by punching another mirror index hole and write protect notch in the jacket, but then never formatted it?

syzygy · Friday at 4:30 AM

qbarnes said:
I'm not sure I follow. You you made a disk physically flippy by punching another mirror index hole and write protect notch in the jacket, but then never formatted it?

Straight from the archives and I made .dmk files from these diskettes during the last week using yor software and there is no evidence that I was tabbed with acid (Don't you hate that ?

. I did not make these disks. I remember people selling "notchers", but I either bought these like this or they were given to me from a software company (Friends from the old Med Systems Software - they may have had them made to put different versions on different sides of a disk). But I am 99% sure I never made them - granted forty years ago can leave memory holes in brains. You have never seen one of these before??

Or do you just find it incredulous that forty years ago I only formatted one side of a flippy? Hey, maybe I was at the end of a study or maybe I just forgot about a side and used another disk. It is a mystery.

I remember once getting paid a $100 a master to put Atari and C-64 programs on the same side of a disk for a guy from what was then Virgin Mastertronic. This, so he would not have to sit on a pile of disks on the duplicator and run the stack again turned over for the other side.. I remember doing this for 4-6 titles for them. He loved it because he could get them done in half the time and I loved it, because in Grad school in 1983, $100 went a long way!

qbarnes · Friday at 3:18 PM

You have never seen one of these before??

Or do you just find it incredulous that forty years ago I only formatted one side of a flippy?

The latter. I've never seen a disk punched to be a flippy and then its flip side was unformatted. But if you didn't make them, that would make more sense.

BTW, I used to have a Aerocomp flippy drive so I didn't have to punch my flippy disks. (But the drive along with my M1 got stolen out of my apartment in college. Grrrr!) To recover my flippy disks, I had to then punch mirror index holes in them years later so I could access them in a regular drive.

My query though was to make sure gw2dmk's flippy detection algorithm for the flip side and user notification was working properly when claiming the flip side really was unformatted.

syzygy · Friday at 3:55 PM

qbarnes said:
The latter. I've never seen a disk punched to be a flippy and then its flip side was unformatted. But if you didn't make them, that would make more sense.

BTW, I used to have a Aerocomp flippy drive so I didn't have to punch my flippy disks. (But the drive along with my M1 got stolen out of my apartment in college. Grrrr!) To recover my flippy disks, I had to then punch mirror index holes in them years later so I could access them in a regular drive.

My query though was to make sure gw2dmk's flippy detection algorithm for the flip side and user notification was working properly when claiming the flip side really was unformatted.

These have only a single sector index hole in the media and two holes in the the case. I don't even remember seeing if they had "notchers" to make your own like that.

I will try to run some tests on that Trackcess diskette and report back.

TRS-80.com · Friday at 5:37 PM

Syzygy - The media will always just have a single index hole; otherwise neither side would work as the index hole would come up twice as fast as it should. IIRC, "flippy" was more what Quentin is describing - a disk which can be flipped without a second index hole being punched in the case. When the second index hole is punched, its just a two sided disk. I had a 5.25" PC-style drive that could read flippy, but it broke a long time ago and I wish I could remember which brand it was.

Quentin - I can think of a few reasons why side 2 wouldn't be formatted on a punched disk; most notedly that if you are going to punch a disk, you do it before you write data on either side, in case you scratch the media; so the disks would be punched long before whatever was planned for side 2 would have occurred. I punch 100% of the disks I send out, so that I can put a 2nd (backup) copy of each DOS in the set. But sometimes one side might just be bad, and only one side would then get written to.

syzygy · Friday at 7:06 PM

Interesting, Do you have a pick of the notch tool?

TRS-80.com · Friday at 8:36 PM

@syzygy - Nope. I have a bunch of empty jackets, so I flip the jacket and draw on the disk index holes, and then use a regular old hand hole punch to punch the index holes. As for the write protect, I took a two hole desktop punch and some tape, and set up guides, so I could just put a disk in, and push down to punch the hole in the right place. Punching the write protect hole is much harder so I found using the desk one much easier.

richarjo · Saturday at 5:00 AM

qbarnes said:
Is something up with this link? When I saw it earlier, it was some facebook link that took me to a GitHub page with a GW build. Now it just breaks.

What kind of DMK?

There's dozens of kinds of DMKs. Does Keir just want a proof-of-concept quick test of one DMK that `gw` will write and run on real hardware? Or does he want to know all the DMKs he still has yet to support? Is there a PR or Issue on GitHub that's tracking this?

Keir toted a development build of GW.exe 1.22 that addresses the TRS-80 DMK DAM issue. He's since pulled it so may be doing further work. He wanted a written disk tested on real M1 hardware.
I've uploaded the Win x64 build to Onedrive here: greaseweazle-ci-1.22.dev3+g0d768a5-win64.zip
This dev 1.22 build denies a DMK without a DAM, such as the Super Utility DMK. How does DMK2GW compare? How is such a protected disk read to DMK? I was able to write the Super Utility 3.2 DMK with DMK2GW without verification but reading it back to DMK has lost sectors and the drive head hits the buffer on later tracks. DMK doesn't load. I haven't tested on a TRS-80 M1. I also downloaded the CMD version of Super Utility 3.2 on a TRSDOS 2.3 DMK which would read/write normally.

Apologies to @syzygy for the interruption.

richarjo · Saturday at 10:09 AM

There is an earlier VCFed thread on DMK transfer also : https://forum.vcfed.org/index.php?threads/transferring-dmk-files-to-real-trs-80-disks.34901/

syzygy · Saturday at 4:33 PM

So, I ran across a few M1 stranglers. I know these were given to me. They were in a notebook using those plastacine (or whatever) floppy holders. I dutifully use gw2dmk and it is dutifully doing its thing. Except, TRSTools refuses to open the .dmk ?! WTF? Sure there was a sector error here and there, but the ,dmk was written ok and, by now, please understand I have been through these steps *many* times.

I go to SU and start nosing around. T17 is completely different (from what I am used to) as is the boot sector. These are marked as Pascal and 35T TRS80....hmm getting closer. A single disk was marked, subtly I add, as 'boot'. So I boot it on the M1 and get...

Aha! Going here, I do some background reading.

Apparently, they had their own dedicated DOS, which is quite cool, I think. Furthermore, I can see now why TRSTools did not like the .dmk and it is likely that it has nothing to do with gw2dmk.

I may very well have written a simple Pascal program around this time, but I was never keen on Pascal (no offense at all to those who loved it). Soon after the TRS-80s, I got a clone-XT and that meant the advent of Borland's Turbo-C.

TRS-80.com · Saturday at 4:53 PM

@syzygy - TRSTools is quite long in the tooth, and there are PLENTY of bugs in it. As you have seen, a disk image isn't really a 'problem' unless it won't boot or run or be read in an emulator AND it can't be opened by TRSTools. When I run into those, I then move over to Miguel Dutra's VDK-80 to see if it can extract the files. If it won't boot/read, and TRSTools won't work, and VDK-80 won't work, then I look at the disk in a file viewer to see if there is anything really on it.

syzygy · Saturday at 5:12 PM

TRS-80.com said:
@syzygy - TRSTools is quite long in the tooth, and there are PLENTY of bugs in it. As you have seen, a disk image isn't really a 'problem' unless it won't boot or run or be read in an emulator AND it can't be opened by TRSTools. When I run into those, I then move over to Miguel Dutra's VDK-80 to see if it can extract the files. If it won't boot/read, and TRSTools won't work, and VDK-80 won't work, then I look at the disk in a file viewer to see if there is anything really on it.

I hear ya. I actually did try VDK-80 some time ago and, as I recall, it wanted nothing to do with my Win7 system. Apparently, I am using a retro computer to read/write even more retro floppies

But hey, why emulate when you can do the real thing. This morning I played (well, booted up and got frustrated) this talking (using the term loosely) game. I actually had to go read about it in order to understand the command "GET OUT". *sigh*

I'm having a blast with stuff.

syzygy · Saturday at 5:45 PM

More mindless and nostalgic musing from the archives....

In grad school we were *invited* to teach an Intro course [full deal - 90 students - the freshman were great and the seniors were often whiny PITAs]. I did not go on to Academia, but I taught several classes and they were a very worthwhile experience.

BTW: When I say 'invited', read that as required and you were paid less than half of what a regular assistant prof was paid.

Here are some extra credit opportunities (in Scripsit, but you can read them...and I know that you can save a Scripsit file in ascii but I am lazy).

>C=YEXTRA CREDIT OPPORTUNITY #2** 2 POINTS **Name.....................................................>C=N
Chapter 2 of your text describes the use of twins in research. Go to the library, find and learn how to use PSYCHOLOGICAL ABSTRACTS. Copy or Xerox any abstract which describes an experiment using twins. Attach the copy to this page. ABSOLUTELY NO CREDIT WILL BE GIVEN IF YOU CUT THE ABSTRACT OUT OF THE VOLUMES IN THE LIBRARY!!!!>C=YMUST BE IN MY HANDS BY 9/19Ž

>C=YEXTRA CREDIT OPPORTUNITY #1**** 4 POINTS ****Name.....................................................>C=N
Chapter 1 of your text describes the use of the scientific method and statistics in research. In lecture, I discussed the mis-use of statistics and the difficulty of drawing "big" conclusions from a "little" research. Find an article (any magazine, newspaper etc.. is a good source) which describes some behavioral research and draws unfounded conclusions. Copy or Xerox the article and attach it to this page. Also, in ONE paragraph state why you believe the conclusions are misleading.>C=YMUST BE IN MY HANDS BY 9/9/83Ž

*note*: These opportunities are expired so stop whining about your grade!

qbarnes · Saturday at 7:42 PM

richarjo said:
Keir toted a development build of GW.exe 1.22 that addresses the TRS-80 DMK DAM issue.

I'm not sure what you mean by "DAM issue". Is there a GitHub PR or open Issue on it?

richarjo said:
He's since pulled it so may be doing further work.

I doubt that's the reason. Those quick builds like that auto-delete on GitHub after a short period of time. I think the default is 48 hours.

richarjo said:
He wanted a written disk tested on real M1 hardware.
I've uploaded the Win x64 build to Onedrive here: greaseweazle-ci-1.22.dev3+g0d768a5-win64.zip

If that's for me, it doesn't do me any good. My machines are Linux. Even the gw2dmk .exe binaries for MS Windows are done via cross-building on Linux.

richarjo said:
This dev 1.22 build denies a DMK without a DAM, such as the Super Utility DMK. How does DMK2GW compare?

Works for me. I took a DMK that @TRS-80.com gave me some time ago for Super Utility v3.2 dual boot and wrote it out with dmk2gw. The disk booted on my physical Model 1. (Yay!)

Then I read the newly written disk back to a DMK with gw2dmk. Then I wrote that new DMK back out to another disk, and that disk booted on my Model 1. (Yay!)

So I'd say that's a good test of dmk2gw/gw2dmk with a multi-boot, copy-protected disk.

richarjo said:
How is such a protected disk read to DMK?

I'm not exactly sure what you're asking.

DMK format just captures the byte stream from each track of a disk noting which portions of the byte stream are in FM or MFM and marks in a lookup table where the track's IDAM bytes are. Those marks are also used to tell whether the byte stream is regular bytes or might have special marker bytes.

Disks that are missing DAMs or have unusual address marks are just bytes in the stream. gw2dmk will keep retrying attempting to read the missing DAM bytes, but will eventually give up, write out the byte stream as-is, and go on reading the next track. As long as the IDAM bytes are decoded and marked in the lookup table correctly, everything in the DMK will work.

richarjo said:
I was able to write the Super Utility 3.2 DMK with DMK2GW without verification but reading it back to DMK has lost sectors and the drive head hits the buffer on later tracks.

I'm not sure what you mean by "has lost sectors". What said the DMK has lost sectors? What do you mean by "drive head hits the buffer on later tracks"?

There's going to be a lot of missing sectors with SU disks because of its copy-protection. For example from reading back in the SU disk with gw2dmk:
Track 3, side 0, pass 1:
#fe cyl=80 side=38 sec=02 size=3e [bad ID CRC] [missing DAM]
#fe cyl=c9 side=c3 sec=0c size=30 [bad ID CRC] #fa [incomplete sector data]
#fe cyl=0d side=20 sec=f4 size=c9 [bad ID CRC] #fb [incomplete sector data]
#fe cyl=55 side=20 sec=f9 size=06 [bad ID CRC] [missing DAM]
#fe cyl=3c side=20 sec=a1 size=cd [bad ID CRC] [missing DAM]
#fe cyl=0a side=20 sec=03 size=3e [bad ID CRC] [missing DAM]
[...]

There's bad ID Record CRC's, unusual DAMs, missing DAMs. Any utility reading this track is going to complain a lot!

Also, make sure you have the right -k and -m options when running dmk2gw for your disk drive.

richarjo said:
DMK doesn't load. I haven't tested on a TRS-80 M1.

Something went wrong for you that didn't for me. What do you mean by "DMK doesn't load" when you hadn't tested it on a Model 1? Doesn't load on what?

richarjo said:
I also downloaded the CMD version of Super Utility 3.2 on a TRSDOS 2.3 DMK which would read/write normally.

Apologies to @syzygy for the interruption.

Model 1 Disk Archiving Project

New Member

Experienced Member

Experienced Member

Experienced Member

New Member

Veteran Member

New Member

Veteran Member

New Member

Veteran Member

Member

Veteran Member

Member

Experienced Member

Experienced Member

Veteran Member

Member

Veteran Member

Veteran Member

New Member