• Please review our updated Terms and Rules here

Quick and dirty Python viewer for character generator ROM dumps

Eudimorphodon

Veteran Member
Joined
May 9, 2011
Messages
5,445
Location
Upper Triassic
On the slight chance anyone's interested here's a minimal program I threw together in Python3 using "Pillow" (an actually supported fork of the older Python Image Library, PIL to render a graphic dump of the contents of a byte-wide character generator ROM.

(I wrote this because I wanted to do some "comparative anatomy" of a few different machine's character sets for a project I've been working on, and I wanted to make sure I was getting the data straight from the horses' mouths. Tried it with several character ROMs from Tandy and Commodore machines and it works. Round "ROM_ROWS" to the next biggest byte, IE, say "16" for a 10 row font like the Model II.)

Code:
#!/usr/bin/env python

from PIL import Image, ImageDraw

image=Image.new("1",[256,256])

# TRS_80 Model I
fontfile = open("ROMS/trs80m1.chr", "rb").read()
ROM_ROWS=8
NUM_CHARS=128
OUTFILE="TRS-80_modelI_chars.png"

# TRS_80 Model III ; this ROM has twice as many characters
# fontfile = open("ROMS/trs80m3/8044316a.u36", "rb").read()
# ROM_ROWS=8
# NUM_CHARS=256
# OUTFILE="TRS80MODELIII_CHARS.PNG"

# Commodore PET, uses same settings as Model III
# fontfile = open("characters-2.901447-10.bin", "rb").read()
# ROM_ROWS=8
# NUM_CHARS=256
# OUTFILE="PETchars.png"

# Example for a 4-bit row font
#fontfile = open("ROMS/m2/8043316.u9", "rb").read()
#ROM_ROWS=16
#NUM_CHARS=128
#OUTFILE="TRS80MODELII_CHARS.PNG"

gridposx=0
gridposy=0

d=ImageDraw.Draw(image)

for fontchar in range (NUM_CHARS):
    for y in range(ROM_ROWS):
        for bitpos in range(8):
            if (fontfile[(fontchar*ROM_ROWS)+y] >> bitpos & 1):
                d.point([(gridposx+(7-bitpos)),(gridposy+y)],1)
    gridposx=gridposx+8
    if gridposx > 255:
        gridposx=0
        gridposy=gridposy+ROM_ROWS

image.show()
#image.save(OUTFILE)
 

VileR

Veteran Member
Joined
Jul 21, 2011
Messages
603
Location
Israel
Nice!

I was considering Pillow for something very similar (generating the bitmap font previews found here), but I ended up being even lazier, and wrote my script using ImageMagick (and ffmpeg, just because it's so much faster at handling multiple .png files simultaneously).

As a bitmap typography geek, your project there sounds like something interesting.
 

Eudimorphodon

Veteran Member
Joined
May 9, 2011
Messages
5,445
Location
Upper Triassic
As a bitmap typography geek, your project there sounds like something interesting.

Heh. Alas it's not really about the typography, per se... well, maybe to some degree. My little pet project is to build a "video card" that can emulate various 1970's vintage "pre-CRTC" memory-mapped video systems, like the SOL-20/VDB-1, SSM VB1B, Polymorphic Systems, TRS-80, original Commodore PET, etc. (With the addition of bitmapped graphics modes because SRAM is a lot cheaper these days.) For the character generator ROM I was planning to use a 128Kbyte flash chip, which means a ton of room for switching between different font/character sets, so I'm trying to figure out a reasonable set that would cover most possibilities.

Here's a shot of the prototype displaying a test plate I generated on a Tandy 1000 with a VGA card in it (using EGA's 640x200 mode), so you can see how the CGA font would look. ;)

attachment.php


(Since the card as implemented so far is 1 byte characters, no attribute byte, if I were going to put the CGA font onto it I think it might be worth shuffling the characters around to make room for a contiguous set of 16 2x2 semigraphic characters that don't depend on reverse video to do psuedo-pixels...

Pixel clock here is 12mhz, because it gives a nicely proportioned 512 pixel wide screen, which is good for the 64x16 displays of the early 8080/Z80 machines I want to target, but I'm thinking about how to also offer 80/40 without changing crystals. I've tested a "70 column" display and it fits between the overscan, so one thought I had was implementing a 7-bit wide character mode. Apparently Kaypros did that...)
 

ldkraemer

Veteran Member
Joined
Mar 14, 2013
Messages
2,317
Location
Chaffee, MO
Eudimorphodon,
I've been trying to get your Python code to work with a TRS-80 Model 1 or Model 3 Character ROM file,
where the bits for a character are in a 5x8 matrix. I'd like to be able to use it to view several different
Character ROMS of that type.

My Model 1 has this for the first character.

Do you have a modification for this type file?


LK-Char.png


Thanks.

Larry
 

Eudimorphodon

Veteran Member
Joined
May 9, 2011
Messages
5,445
Location
Upper Triassic
Eudimorphodon,
I've been trying to get your Python code to work with a TRS-80 Model 1 or Model 3 Character ROM file,
where the bits for a character are in a 5x8 matrix. I'd like to be able to use it to view several different
Character ROMS of that type.

Do the dump files you have actually only have six bit words for each character row? One of the files I tested was the .chr file from a MAME ROM set for the TRS-80 model I, and it worked; here's the resulting render:

TRS-80_modelI_chars.png

(Slightly annoyingly this is the only Model I character generator dump I've been able to find; comparing it to the XTRS source this appears to be the "second" character generator, the one that has the "raised descenders" lowercase letters like the original MCM6674 but has the square brackets replaced by arrow keys.) It renders it fine because even though the physical part had only five output data lines the dump has the data stored in 8 bit bytes. Do you have dump files that actually only have five bit words packed tightly? (What's the size of your dump file, 1K, or something smaller?)

The Model I character generator that has an @ in the first position is, I believe, the version that came if you got the official Radio Shack lowercase mod, where to work around a ROM issue they duplicated characters 64-95 into the 0-31 slot. If you'd be willing to share a copy of your dump I'd love to take a look at it. Depending on how the ROM was dumped it's possible that the three unused data bits in each byte might be "1"s instead of "0"s, but I would be really surprised if it doesn't have 8 bit words in the dump...

The Model III had 8-bit wide characters; here's my dump of the file from the MAME ROM collection. (Likewise I seem to really be striking out on finding independent dumps. There's an old thread that claims they're on bitsavers, but I don't know where...)

TRS80MODELIII_CHARS.PNG

Likewise if you have dumps that differ from this one I'd be curious to see them...
 

Eudimorphodon

Veteran Member
Joined
May 9, 2011
Messages
5,445
Location
Upper Triassic
FWIW, in another fruitless attempt to try finding original dumps I ran across this thread talking about a "GenDon 3" enhanced character ROM for the Model I. I tried running my dumper against a .bin file for that and it likewise reads okay; have to set "ROM_ROWS" to 16 instead of 8 because that one uses a 6x12 matrix. (With, again, the unused bits of the 8-bit wide rows being zeros in this dump.)
 

ldkraemer

Veteran Member
Joined
Mar 14, 2013
Messages
2,317
Location
Chaffee, MO
I've got the OEM DUMP of my Ver 1.1 ROM Model 1. I've also got the Dump of the Lowercase Mod by
KSG Adapter Board that I bought at the Dayton Hamfest. I've got Model 3 Dumps and the Model 1
8046673.ROM Dump, along with several others.

I put these in Libre Office Calc so I could see the differences in all the versions.

I can send you the dumps so you can process them.

Larry
 

ldkraemer

Veteran Member
Joined
Mar 14, 2013
Messages
2,317
Location
Chaffee, MO
You have mail, and lots of files to play with.

If anyone else has a problem with the GOOD Python code, use a hex editor to verify
your file is in the format of 0011 vs 30303131. That was my problem............xxd to
the rescue in Linux.

THANKS.

Larry
 

Eudimorphodon

Veteran Member
Joined
May 9, 2011
Messages
5,445
Location
Upper Triassic
You have mail, and lots of files to play with.

Thanks!

For fun last night I dumped a random selection of character generator ROMs I found on a mostly CP/M-focused site, and I did find a few interesting specimens that produced anomalous results, but I suspect issues with how the ROM was dumped. There were a couple where, as I suspected could happen, the bits that were outside the actual matrix covered by the generator (for instance, the 3 "spare" bits from reading a 5-bit wide character generator into an 8-bit file) were rendered as white instead of black; I'm betting that was the reading hardware's inputs floating the un-anchored lines high. There was also an interesting case where the glyphs were backwards; I wonder if on that machine they really did have the outputs from the character generator wired to an output shift register in the opposite of the usual order? But, well, that's not the sort of thing I can really program around easily. ;)

(Although if you know the ROM you're looking at has these anomalies after the first read it should be easy enough to modify the code to fix that, it's pretty straightforward.)

(Oh, there was also another one where the characters were reverse video, black-on-white. I'm curious if that likewise was an actual quirk in the original hardware or some kind of reading glitch.)
 
Last edited:

ldkraemer

Veteran Member
Joined
Mar 14, 2013
Messages
2,317
Location
Chaffee, MO
Yes, the KSG Tech Character Generator Board has a mixed up layout too for the column of bits.
Here is what I had to decode to get the characters. (I suspect they did it to be different so no
Radio Shack law suits etc.).

The columns with the X are not used, so I reordered the Columns. It was an interesting decode.
It would have been impossible with out Nedit and Macro's.


Code:
[SIZE=2]        0         1         2         3         4         5         6         7         8         9         A         B         C         D         E         F 
   X X  X    X X  X    X X  X    X X  X    X X  X    X X  X    X X  X    X X  X    X X  X    X X  X    X X  X    X X  X    X X  X    X X  X    X X  X    X X  X 
7654 3210 7654 3210 7654 3210 7654 3210 7654 3210 7654 3210 7654 3210 7654 3210 7654 3210 7654 3210 7654 3210 7654 3210 7654 3210 7654 3210 7654 3210 7654 3210 
120   43  120   43  120   43  120   43  120   43  120   43  120   43  120   43  120   43  120   43  120   43  120   43  120   43  120   43  120   43  120   43  
1100 1011 0010 1101 0010 1001 0110 1011 0110 1101 1100 1101 0100 1011 0000 1001 0100 1001 1000 1011 0010 1101 0010 1101 1110 1111 0010 1101 0010 1101 0000 1001 
1100 1111 0010 1011 0010 1011 1100 1011 0010 1011 0010 1011 1100 1111 0000 1001 1100 1011 0010 1101 0000 1101 0000 1101 0000 1101 0010 1101 1100 1011 0000 1001 
1100 1111 0010 1011 0010 1011 0010 1011 0010 1011 0010 1011 1100 1111 0000 1001 1110 1111 0000 1101 0000 1101 0100 1111 0000 1101 0000 1101 1110 1111 0000 1001 
1110 1111 0000 1101 0000 1101 0100 1111 0000 1101 0000 1101 0000 1101 0000 1001 1110 1011 0000 1101 0000 1101 1010 1101 0010 1101 0010 1101 1110 1011 0000 1001 
0010 1101 0010 1101 0010 1101 1110 1111 0010 1101 0010 1101 0010 1101 0000 1001 1100 1011 0100 1001 0100 1001 0100 1001 0100 1001 0100 1001 1100 1011 0000 1001 
0011 1001 0011 1001 0011 1001 0011 1001 0011 1001 0011 1101 1101 1011 0001 1001 0011 1101 1001 1101 0101 1101 0001 1111 0101 1101 1001 1101 0011 1101 0001 1001 
0001 1101 0001 1101 0001 1101 0001 1101 0001 1101 0001 1101 1111 1111 0001 1001 0011 1101 1011 1111 0111 1101 0111 1101 0011 1101 0011 1101 0011 1101 0001 1001 
0011 1101 0011 1111 0111 1101 1011 1101 0011 1101 0011 1101 0011 1101 0001 1001 1101 1011 0011 1101 0011 1101 0011 1101 0011 1101 0011 1101 1101 1011 0001 1001 
1101 1111 0011 1101 0011 1101 1101 1111 0001 1101 0001 1101 0001 1101 0001 1001 1101 1011 0011 1101 0011 1101 0011 1101 0111 1101 1001 1101 0111 1011 0001 1001 
1101 1111 0011 1101 0011 1101 1101 1111 0101 1101 1001 1101 0011 1101 0001 1001 1101 1011 0011 1101 0001 1101 1101 1011 0011 1001 0011 1101 1101 1011 0001 1001 
1111 1111 0101 1001 0101 1001 0101 1001 0101 1001 0101 1001 0101 1001 0001 1001 0011 1101 0011 1101 0011 1101 0011 1101 0011 1101 0011 1101 1101 1011 0001 1001 
0011 1101 0011 1101 0011 1101 1001 1011 1001 1011 0101 1001 0101 1001 0001 1001 0011 1101 0011 1101 0011 1101 0011 1101 0111 1101 1011 1111 0011 1101 0001 1001 
0011 1101 0011 1101 1001 1011 0101 1001 1001 1011 0011 1101 0011 1101 0001 1001 0011 1101 0011 1101 1001 1011 0101 1001 0101 1001 0101 1001 0101 1001 0001 1001 
1111 1111 0011 1001 1001 1001 0101 1001 0001 1011 0001 1101 1111 1111 0001 1001 0101 1001 1101 1011 0111 1101 0101 1001 0101 1001 0101 1001 0101 1001 0001 1001 [/SIZE]



Larry
 

Eudimorphodon

Veteran Member
Joined
May 9, 2011
Messages
5,445
Location
Upper Triassic
Guess I found another kind of fascinating formatting thing in the dump file that's out there for the original IBM MDA and CGA cards. That ROM has has three fonts in it; thin and thick versions of the CGA 8x8 font and the MDA 8x14 font. The way it's arranged is the first 2K of the ROM has the top 8 rows of the MDA set, the next 2K is the *bottom* six rows of the MDA font, and then that's followed by the two CGA fonts. (In other words the MDA card apparently attaches the fourth "row" address line to the ROM after instead of before the 8 "character" address lines.) That results in a kind of goofy looking dump if you do the whole 8K:

attachment.php


To get a properly rendered picture of a ROM like this with this code it'd probably be easiest to duplicate the "for" loop that fills in the pixels on the image and do an even/odd set of 8 rows. Something like this:

Code:
#!/usr/bin/env python

from PIL import Image, ImageDraw

image=Image.new("1",[256,256])


# Dump the MDA font from the first 4K of the IBM CGA/MDA font ROM
fontfile = open("ROM2/IBM_5788005_AM9264_1981_CGA_MDA_CARD.BIN", "rb").read()
ROM_ROWS=8
NUM_CHARS=256
OUTFILE="MDAfont.png"

gridposx=0
gridposy=0

d=ImageDraw.Draw(image)

group2_offset=0

for fontchar in range (NUM_CHARS):
    # print ("looking at character ", fontchar)
    for y in range(ROM_ROWS):
        #print ("looking at line", y)
        for bitpos in range(8):
            # print ("looking at bit position", bitpos)
            if (fontfile[(fontchar*ROM_ROWS)+y] >> bitpos & 1):
                d.point([(gridposx+(7-bitpos)),(gridposy+y)],1)
        group2_offset+=1
    gridposx=gridposx+8
    if gridposx > 255:
        gridposx=0
        gridposy=gridposy+(ROM_ROWS * 2)
# Second row
gridposx=0
gridposy=8

d=ImageDraw.Draw(image)

for fontchar in range (NUM_CHARS):
    # print ("looking at character ", fontchar)
    for y in range(ROM_ROWS):
        #print ("looking at line", y)
        for bitpos in range(8):
            # print ("looking at bit position", bitpos)
            if (fontfile[((fontchar*ROM_ROWS)+group2_offset)+y] >> bitpos & 1):
                d.point([(gridposx+(7-bitpos)),(gridposy+y)],1)
    gridposx=gridposx+8
    if gridposx > 255:
        gridposx=0
        gridposy=gridposy+(ROM_ROWS * 2)


image.show()
image.save(OUTFILE)

That seems to do the needful.

attachment.php


Also note the glyphs are all touching because this font is a legit 8 dots wide and relies on the hardware to insert a 1 pixel space between the character cells.
 

Attachments

  • IBM_CGA_fulldump.png
    IBM_CGA_fulldump.png
    4.4 KB · Views: 9
  • MDAfont.png
    MDAfont.png
    1.8 KB · Views: 9
Joined
Jan 9, 2022
Messages
30
Care if I wake an old thread? I'm trying to decode a rom from a Datamedia DT80 that I am restoring to confirm that it is good or bad. However when I use your script all I get is this.
ImageC.png
The Diamond and solid square at the bottom right is data I added myself to confirm that the script is working as it should. Now, while messing around with the DT80 I can get some characters to show up on the display by poking around a little although it's finicky due to being buried for who knows how long. I think the char rom is mostly good, but I might be wrong. It looks to not be in any format that's standard afaikt. It appears to be 7 bits wide with the 8th bit always set to 1, but I don't know yet how each char line is organized in rom. I edited the binary file to remove all of the 0x80 bytes and replace them with 0x00 for now to make viewing the images generated a little easier. I'll upload what I was able to recover for anyone who is interested at picking their brain.
 

Attachments

  • DT80char.zip
    5.8 KB · Views: 6

resman

Veteran Member
Joined
Jan 1, 2014
Messages
530
Location
Lake Tahoe
Here is a horrible, horrible hack I came up to view and edit character generator ROM images: prepend a Portable BitMap header to the file. I used this for Apple ][ character ROM files to create a lower case image and alternative character ROM image. Using vi to edit the ROM image, simply insert (for a 2K ROM image):

Code:
P4
8 2048

in front of the raw data and save it. You can load and edit the result using GIMP. It will look like a very tall and skinny bitmap, so zoom in to see the individual characters. You can edit and save the file, too. When complete, simply re-edit the file in vi and remove the two line PBM header and re-save it.
 

Eudimorphodon

Veteran Member
Joined
May 9, 2011
Messages
5,445
Location
Upper Triassic
Here is a horrible, horrible hack I came up to view and edit character generator ROM images: prepend a Portable BitMap header to the file. I used this for Apple ][ character ROM files to create a lower case image and alternative character ROM image. Using vi to edit the ROM image, simply insert (for a 2K ROM image):
Hey, sounds like a legit solution to me, since the ROM data for that is effectively an 8x2048 monochrome bitmap anyway.

I guess it looks like I didn’t mention it, but as the project I wrote this python code advanced I actually ended up using it to essentially “rotate” the fonts for the display system I was building so instead of being arranged like that the resulting character data is a “horizontal” bitmap instead of a vertical one, IE, the 256 character glyphs are arranged so the first 256 bytes are row #1, next 256 bytes row #2, etc, so the same 8x2048 bitmap instead becomes a 2048x8 one. This was a hack to save some gates on the memory addressing circuitry. (Probably need a diagram to fully explain it. A bonus of it is that character sets can be an arbitrary number of lines tall without wasting any space on blank lines in the memory image.)
 
Last edited:
Joined
Jan 9, 2022
Messages
30
Ah! I got it working with mine! Turns out the char select bits are bits 0-6 on this ROM and the line count bits are 7-9 with the 10th bit used for something else but I'm not quite sure yet. I'll post my edit of the script but do note I'm not well versed with python.

Image.png

As you can see, there is some data corruption, but it's fixable!

Code:
#!/usr/bin/env python

from PIL import Image, ImageDraw

image=Image.new("1",[256,256])

# TRS_80 Model I
fontfile = open("ROMS/CharROMread.BIN", "rb").read()
ROM_ROWS = 8
NUM_CHARS = 128
OUTFILE ="Image.png"


gridposx = 0
gridposy = 0
BitTen = 0

d=ImageDraw.Draw(image)

while BitTen <= 1:
    for ZeroThroughSix in range (NUM_CHARS):
        for SevenThroughNine in range(ROM_ROWS):
            for bitpos in range(8):
                if (fontfile[ZeroThroughSix + ((SevenThroughNine * 128) + (BitTen * 1024))] >> bitpos & 1):
                    d.point([(gridposx + (7 - bitpos)), gridposy + SevenThroughNine],1)   
        gridposx = gridposx + 8
        if gridposx > 255:
            gridposx = 0
            gridposy = gridposy + ROM_ROWS
    BitTen = BitTen + 1

#image.show()
image.save(OUTFILE)
 

ClassicHasClass

Veteran Member
Joined
Mar 22, 2013
Messages
1,632
Location
Sunny So Cal
Here is a horrible, horrible hack I came up to view and edit character generator ROM images: prepend a Portable BitMap header to the file. I used this for Apple ][ character ROM files to create a lower case image and alternative character ROM image. Using vi to edit the ROM image, simply insert (for a 2K ROM image):

Code:
P4
8 2048

in front of the raw data and save it. You can load and edit the result using GIMP. It will look like a very tall and skinny bitmap, so zoom in to see the individual characters. You can edit and save the file, too. When complete, simply re-edit the file in vi and remove the two line PBM header and re-save it.
I just want to say, this is super-clever.
 
Joined
Jan 9, 2022
Messages
30
Here is a horrible, horrible hack I came up to view and edit character generator ROM images: prepend a Portable BitMap header to the file. I used this for Apple ][ character ROM files to create a lower case image and alternative character ROM image. Using vi to edit the ROM image, simply insert (for a 2K ROM image):

Code:
P4
8 2048

in front of the raw data and save it. You can load and edit the result using GIMP. It will look like a very tall and skinny bitmap, so zoom in to see the individual characters. You can edit and save the file, too. When complete, simply re-edit the file in vi and remove the two line PBM header and re-save it.
I tried this out and it worked even better than expected. My char ROM was a little different so I'll link it as an example to anyone else who needs it.

 
Top