IBM System/23 Datamaster, display and character generation

[VileR]

I was looking into the available docs and firmware for this system (http://bitsavers.org/pdf/ibm/system23/), to try and figure out how this thing handles text display. Naturally I don’t have one of these – I’m just a certified nutter for bitmap font stuff, but the System/23 is particularly interesting, as an immediate relative of the 5150 in some respects.

The easy part:

The display is 80 columns by 24 rows; each character occupies 10x14 dots (the ROM format is 8x16, but the bottom two lines are unused, and the two extra columns contain either spacing or bit duplication, much like the 5150 MDA’s 9th dot column). The attribute generator provides blinking, inverse-video, underlined and highlighted (intense) text.

The native encoding is a form of EBCDIC, but the system provides several country-specific character sets (see Operator Reference, p304). You can input characters outside the selected set (with some restrictions) by holding down ALT and typing the character code on the keypad - another small thing which was carried over to the IBM PC.

The CRT section of the service manual helpfully provides an image for each charset:



The actual character ROM is found in firmware/System23.zip (CHR_73783bc7_8519412.bin). You might expect the contents to match the machine's EBCDIC-based encoding, but no - interestingly the ROM simply reproduces the country-specific sets one after the other.
The first group of 64 characters is common to all sets, followed by the second 64 chars for each set in order... with all the wasteful duplication that this entails:

---

The weird part:

So how does this all map onto the System/23's EBCDIC-based encoding?

This encoding is described in various places - most completely in the BASIC Language Reference, p. 53 (user_manuals/binder_07).
Problem is, it contains quite a few characters which don't appear anywhere in the country-specific sets, or in the character ROM - the missing ones are in yellow here:



Yes, the ROM does not contain the digit "1"(!).... as you might have noticed from the image of its contents. So how is it rendered on screen?

Looking at existing images and videos, it seems that the bitmap for lower-case L l is simply reused for the numeral 1:



But what about those other missing characters?
I could see how the underscore (6Dh) could be generated by simply enabling the underline attribute, for example.... and the tilde ~ (A1h) by displaying only a couple of scanlines from 'Ã', or something. But that doesn't explain the "¼", or the line-drawing characters, or most of the rest really.

It'd sure be interesting to know what's going on there. Even just conjecture: why would the use such a seemingly over-complicated and wasteful setup?

Who knows, maybe some day the information may help towards emulating this system (which hasn't been done yet AFAIK).

 

[RetroAND]

Hello,

The truth being said, there are still many unknowns in this system. There is another memory in the video subsystem, an Intel 8KB pseudostatic ram. As the computer won't even display anything if the dynamic memory cards are not present my hypothesis is that it "builds" the actual character generator in this ram region by copying them from the rom plus adding other characters from the ROS itself and that the character map may lie in the fixed ram area. However, this has still do be proved.

As of today I know the ports of the 8275, but not the ones from the 8257 nor the offsets of the character memories in the memory map.

Relating to the emulator, I have a MAME collaborator waiting for details at the low level to start the work, but it is very hard to obtain this information. With a bit of luck I may be able to obtain more data soon.

 

[VileR]

The truth being said, there are still many unknowns in this system. There is another memory in the video subsystem, an Intel 8KB pseudostatic ram. As the computer won't even display anything if the dynamic memory cards are not present my hypothesis is that it "builds" the actual character generator in this ram region by copying them from the rom plus adding other characters from the ROS itself and that the character map may lie in the fixed ram area. However, this has still do be proved.

Hm, that's interesting - the CRT section of the service manual only mentions a read-only character generator (as the part that is addressed by the character codes in the row buffer, to fetch the dot patterns which are then sent to the video circuits).

On the other hand, what you say would make sense if the character sets were meant to be an easily extensible feature (or software-loadable). And the CRT section of the service manual also says this (emphasis mine):
"The commands and character codes that are used to control the operation of the display unit are part of the microprograms and are not described in this service manual."

I also had a look at the 5322 ROS set, but it doesn't have anything that unambiguously looks like character bitmaps in a graphical dump. Of course that doesn't mean much, as the data may be interwoven with code as instruction operands, or generated in some other fashion (and the set does have one ROM marked bad).

Relating to the emulator, I have a MAME collaborator waiting for details at the low level to start the work, but it is very hard to obtain this information. With a bit of luck I may be able to obtain more data soon.

Nice... fingers crossed!

 

[RetroAND]

Hm, that's interesting - the CRT section of the service manual only mentions a read-only character generator (as the part that is addressed by the character codes in the row buffer, to fetch the dot patterns which are then sent to the video circuits).

On the other hand, what you say would make sense if the character sets were meant to be an easily extensible feature (or software-loadable). And the CRT section of the service manual also says this (emphasis mine):
"The commands and character codes that are used to control the operation of the display unit are part of the microprograms and are not described in this service manual."

I also had a look at the 5322 ROS set, but it doesn't have anything that unambiguously looks like character bitmaps in a graphical dump. Of course that doesn't mean much, as the data may be interwoven with code as instruction operands, or generated in some other fashion (and the set does have one ROM marked bad).



Nice... fingers crossed!

There is not a bad rom there: it's "KBD" for "Keyboard" and it's only 1KB long because the target is not 8085 but 8048. How do I know? I made those dumps; maybe I should have had to write a readme.txt, sorry for the confusion.

EDIT:

Sorry, there is not such pseudostatic ram. I got confused with the usart... as you may already know all the ICs are remarked.

EDIT 2:

The semigraphical characters in columns 0..3 are built-in features of the 8275 and displayed by enabling or disabling the "character attribute codes". You might want to see the datasheet for this CRTC.
About the other ones, I still don't know.

 

[VileR]

(EDIT: ha, I must be losing it -- the bitmaps for ¼, ¾ do appear in the ROM... so these characters don't belong in the 'missing' list, but I can't edit my original post anymore.)

There is not a bad rom there: it's "KBD" for "Keyboard" and it's only 1KB long because the target is not 8085 but 8048. How do I know? I made those dumps; maybe I should have had to write a readme.txt, sorry for the confusion.

Yeah, the ones in your set are all good - the bad one is in the other bitsavers set (http://bitsavers.org/pdf/ibm/system23/firmware/IBM_5322_ROS_ROMs.zip); "5322-13-4481706-bad.bin" is 8K there, but clearly not a good dump (unlucky 13?).

However, now I see the that the other ROMs in that set are all identical to yours. It's also missing the keyboard and character ROMs (which exist in yours), so that set can be considered redundant.

The semigraphical characters in columns 0..3 are built-in features of the 8275 and displayed by enabling or disabling the "character attribute codes". You might want to see the datasheet for this CRTC.
About the other ones, I still don't know.

Thanks for that. Yeah, I see now; per-line output for the line-drawing chars is generated by the CRTC (LA1/LA0/VSP/LTEN), but their actual appearance would depend on (1) the underline position, and (2) the external dot-level circuitry which actually acts on the CRTC's signals.

I just found your other thread about the S/23 - hadn't thought to look under 'Minis and Mainframes'... happy to see your success in resurrecting it!

If you can get the video output on a working monitor, perhaps it would be possible to simply print those "missing" non-ROM characters on screen, and then take some sort of close-up photo?
If they are printed next to other characters (whose appearance is known from the character ROM), that could help determine the actual bit patterns - and perhaps deduce how they're generated, too.

 

[vwestlife]

The use of the same character for the number 1 and lowercase letter l is quite surprising to me. I wonder if the software for it was designed to interpret any accidental keypress of a lowercase l in a numerical field as the number 1? Because the operator wouldn't be able to tell the difference between them on the screen.

 

[RetroAND]

(EDIT: ha, I must be losing it -- the bitmaps for ¼, ¾ do appear in the ROM... so these characters don't belong in the 'missing' list, but I can't edit my original post anymore.)


Yeah, the ones in your set are all good - the bad one is in the other bitsavers set (http://bitsavers.org/pdf/ibm/system23/firmware/IBM_5322_ROS_ROMs.zip); "5322-13-4481706-bad.bin" is 8K there, but clearly not a good dump (unlucky 13?).

However, now I see the that the other ROMs in that set are all identical to yours. It's also missing the keyboard and character ROMs (which exist in yours), so that set can be considered redundant.


Thanks for that. Yeah, I see now; per-line output for the line-drawing chars is generated by the CRTC (LA1/LA0/VSP/LTEN), but their actual appearance would depend on (1) the underline position, and (2) the external dot-level circuitry which actually acts on the CRTC's signals.

I just found your other thread about the S/23 - hadn't thought to look under 'Minis and Mainframes'... happy to see your success in resurrecting it!

If you can get the video output on a working monitor, perhaps it would be possible to simply print those "missing" non-ROM characters on screen, and then take some sort of close-up photo?
If they are printed next to other characters (whose appearance is known from the character ROM), that could help determine the actual bit patterns - and perhaps deduce how they're generated, too.

Fortunately the failure I faced was a joke with a near zero cost. Just some capacitors.
What flavor do you want your picture to be, b/w composite in a flat screen or the original green phosphor monitor?
I am not familiarized with this computer's BASIC, but is you provide a pseudocode or even a listing of actual BASIC I don't see why not.
I have a few days free next week, I plan to dedicate as much time to study the board. The video subsystem is of my interest at this moment so I may give it a try.

The use of the same character for the number 1 and lowercase letter l is quite surprising to me. I wonder if the software for it was designed to interpret any accidental keypress of a lowercase l in a numerical field as the number 1? Because the operator wouldn't be able to tell the difference between them on the screen.

Actually it is a very common practice among typewriters, where even other characters were sometimes missing (s/5, O/o, etc.). The Datamaster was a new concept for IBM, they wanted that the operator was more familiar with their common tasks than with their computer and for this reason they might had started with a digital typewriter in mind, then upgrading it to a full computer. Maybe a user with the printer could also corroborate if there is also reuse of characters there.

 

[vwestlife]

Actually it is a very common practice among typewriters, where even other characters were sometimes missing (s/5, O/o, etc.). The Datamaster was a new concept for IBM, they wanted that the operator was more familiar with their common tasks than with their computer and for this reason they might had started with a digital typewriter in mind, then upgrading it to a full computer. Maybe a user with the printer could also corroborate if there is also reuse of characters there.

I fully understand that, having used a typewriter that was missing the 1 and 0 keys because it expected l and O to serve double-purpose. But virtually every computer manual from the late '70s to early '90s stressed the importance of not mixing up these characters, because when the computer is expecting the number 10 and you type in lO instead, it's going to give you an error or unexpected results.

My question was since there is no visual distinction between 1 and l on the screen, if the Datamaster takes that into account and automatically corrects any input of l in a numerical field to a 1 (and possibly O to 0 as well, since you mention it). Seeing how the designers took pains to conduct user testing and improve its ease of use (see video below), I sure hope they would!

 

[RetroAND]

I fully understand that, having used a typewriter that was missing the 1 and 0 keys because it expected l and O to serve double-purpose. But virtually every computer manual from the late '70s to early '90s stressed the importance of not mixing up these characters, because when the computer is expecting the number 10 and you type in lO instead, it's going to give you an error or unexpected results.

My question was since there is no visual distinction between 1 and l on the screen, if the Datamaster takes that into account and automatically corrects any input of l in a numerical field to a 1 (and possibly O to 0 as well, since you mention it). Seeing how the designers took pains to conduct user testing and improve its ease of use (see video below), I sure hope they would!

Sorry, but I cannot give you a proper answer now. I imagine they spent time creating checks, but only by using commercial software (which I don't have) or by continue the disassembly (which requires time) I could answer. The keys are there for sure so maybe the ebcdic representation of the screen is buffered elsewhere or maybe they used some "clever" programming with the character attribute codes, who knows.

What I know for sure is that the strings I found inside are EBCDIC and contain separate I/1 and O/0 characters. I hope this helps to answer your question.

 

[VileR]

Fortunately the failure I faced was a joke with a near zero cost. Just some capacitors.
What flavor do you want your picture to be, b/w composite in a flat screen or the original green phosphor monitor?
I am not familiarized with this computer's BASIC, but is you provide a pseudocode or even a listing of actual BASIC I don't see why not.
I have a few days free next week, I plan to dedicate as much time to study the board. The video subsystem is of my interest at this moment so I may give it a try.

Nice, that'd be appreciated!
I think a picture of the original CRT would be preferable - flat screens tend to have fuzzy interpolation when not at their native resolution, so the dot patterns may be harder to establish. But either way would work for me, whatever is more convenient.

The BASIC reference is decent, and has some helpful sample programs... so I believe the following should work.
The idea is to display all printable characters, with alternating normal/inverse-video attributes, so character boundaries are easy to see. Some chars may show up as "blobs" (says the manual...) but the relevant ones should be fine.

10 OPTION BASE 1                                   ! match str/array pos.
20 D = DISPLY(1)                                   ! set US charset
30 LET EVEN$ = RPT$(CHR$(20)&" "&CHR$(8)&" ", 16)  ! 64 (32 chars+even attrs)
40 LET ODD$ = RPT$(CHR$(8)&" "&CHR$(20)&" ", 16)   ! 64 (32 chars+odd attrs)
50 LET TR$ = EVEN$&ODD$                            ! attrs for 2 rows
60 DIM LDC(11)                                     ! 11 line-drawing chars
70 DATA 27,32,33,34,40,41,44,48,49,52,62           ! their values
80 READ MAT LDC
90 FOR I = 1 TO 11
100 TR$(LDC(I)*2+2:LDC(I)*2+2) = CHR$(LDC(I))      ! in chars 0-63:
110 NEXT I                                         ! populate line-drawing only
120 GOSUB PRTTWO
130 C = 64                                         ! chars 64-255 are all valid
140 GOSUB TWOLN
150 C = 128
160 GOSUB TWOLN
170 C = 192
180 GOSUB TWOLN
190 STOP
200 TWOLN: FOR I = 1 TO 64                         ! populate 2 rows
210 TR$(I*2:I*2) = CHR$(I+C-1)
220 NEXT I                                         ! and fall through:
230 PRTTWO: PRINT TR$(1:64)                        ! print 2 rows
240 PRINT TR$(65:128)
250 RETURN
260 END

Sorry about the length- I tried to keep it short, but this BASIC dialect is a bit long-winded... and the reference seems to imply that strings shouldn't be longer than 255 chars, so I had to reuse and repeat.

(The "!"s designate comments, so no need to type those parts of course.)

Hope I didn't sneak any bugs in there. Anyway, if you find some time for this, it could be an informative experiment.

Seeing how the designers took pains to conduct user testing and improve its ease of use (see video below)

Ah yes, always loved the part where they figure out that "reverse image" is crappy terminology.

"She thought the characters would be backwards! Everybody went guffaw-guffaw-guffaw!"

 

[VileR]

Sorry about the length- I tried to keep it short

I should've tried harder... rolled another loop, so this one's a bit shorter:

10 OPTION BASE 1                                   ! match str/array pos.
20 D = DISPLY(1)                                   ! set US charset
30 LET EVEN$ = RPT$(CHR$(20)&" "&CHR$(8)&" ", 16)  ! 64 (32 chars+even attrs)
40 LET ODD$ = RPT$(CHR$(8)&" "&CHR$(20)&" ", 16)   ! 64 (32 chars+odd attrs)
50 LET TR$ = EVEN$&ODD$                            ! attrs for 2 rows
60 DIM LDC(11)                                     ! 11 line-drawing chars
70 DATA 27,32,33,34,40,41,44,48,49,52,62           ! their values
80 READ MAT LDC
90 FOR I = 1 TO 11
100 TR$(LDC(I)*2+2:LDC(I)*2+2) = CHR$(LDC(I))      ! in range 0-63:
110 NEXT I                                         ! populate line-drawing only
120 GOSUB PRTTWO
130 FOR C = 1 TO 3                                 ! chars 64-255 are all valid
140 GOSUB TWOLN                                    ! (do in 3 groups of 64)
150 NEXT C
160 STOP
170 TWOLN: FOR I = 1 TO 64                         ! populate 2 rows
180 TR$(I*2:I*2) = CHR$(I+C*64-1)
190 NEXT I                                         ! and fall through:
200 PRTTWO: PRINT TR$(1:64)                        ! print 2 rows
210 PRINT TR$(65:128)
220 RETURN
230 END

 

[RetroAND]

Hello,

I enclosed the board, so it has keyboard again! Unfortunately the program didn't execute successfully.
What I can say for sure is:

1. Really the operator cannot distinguish between "l" and "1".
2. The graphic characters are generated by a combination of the 8275 special chars plus underlining.
3. There is no issue to print graphic characters alongside text.

Sorry, no pictures this time, I made the tests in a haste and at the same time it seems my keyboard sends random scancodes at times and shall be revised.
There will be a three-week timespan before I will be able to access the 5322 as I was, but may be able to take some picture of the screen.

Regards

 

[VileR]

I enclosed the board, so it has keyboard again! Unfortunately the program didn't execute successfully.
What I can say for sure is:

1. Really the operator cannot distinguish between "l" and "1".
2. The graphic characters are generated by a combination of the 8275 special chars plus underlining.
3. There is no issue to print graphic characters alongside text.

Sorry, no pictures this time, I made the tests in a haste and at the same time it seems my keyboard sends random scancodes at times and shall be revised.
There will be a three-week timespan before I will be able to access the 5322 as I was, but may be able to take some picture of the screen.

Nice, thanks for having a go at it - good to have those 3 points confirmed.
Too bad about the program... I'd ask if there were any helpful error messages (syntax error at line #, issue with the GOSUB falling through, etc.), but troubleshooting without a way to test is flying blind anyway.

If you're able to get some photos next time, that'd still be appreciated. Good luck with the keyboard fix!