• Please review our updated Terms and Rules here

Looking for volunteers to help test a new benchmark

Ok, I actually read all 22 pages of this thread and that answered most of my questions: I see these numbers are how many milliseconds the operation took so obviously lower is better. Sill not sure on what some of them mean like MemEATest and 3DGameTest.

Sorry, I've let this project stall a bit. I plan on doing some coding in a few weeks to finish up a realtime tool that compares against systems; I've abandoned the idea of trying to do it with a very pretty interface in the interest of just Getting It Done.

Lower numbers are better in the five microsecond timing sections, but for the Score, which is the actual synthetic metric, higher is better. On extremely fast machines, the microsecond timings get close to 0 or hit 0, but the Score is still representative of which machine is faster than another.

The names of the tests represent the code that is being executed; in what you wrote above, "MemEATest" is a test of effective address calculation speed when accessing memory. "3DGameTest" is a mixture of opcodes in the same frequency as several 3-D DOS games that are not hard-coded (ie. they scale nicely as processor speed increases).

You can read some background info at http://dosbenchmark.wordpress.com and you can download the actual metric code sections at http://dosbenchmark.wordpress.com/downloads/ if you want to see exactly what they're doing. Even if you don't understand assembler, the 3DGameTest code has a lot of comments at the beginning that explain the methodology and what games were used to come up with the metric.

I have a theory about the Packard Bell 560 hangup – I tried this benchmark on one other system and got the same result – CAPSLOCK can be toggled but the program appears to be not responding. Both systems have a few things in common, but I think only one is the real issue as the “CPU speed detect” is the hang-point. It seems that there is a problem detecting newer “generation” processors that are underclocked – It is probably particular to the Pentium line as it was so long lived and there were numerous changes and die shrinks, etc.

That, and my CPU model/speed detection is not foolproof and can hang some machines if I haven't had access to them. For such machines, run with "-s" and those tests will be skipped. Detection of Pentiums and above is somewhat easy thanks to the CPUID instruction, but for CPUs earlier than that, you have to tickle all sorts of bugs and undocumented opcodes to hone in on what you have. Around 1-2% of the time, my code takes a wrong turn somewhere and hangs certain CPUs.

The CPU model detection is what actually hangs the CPU. The CPU MHz detection is based off of the model detection, so if it hangs, it's never getting to the part that figures out what MHz the CPU is running at.
 
Unless it runs at 8MHz or faster, it won't give truly "double" performance. If it runs at 7.16MHz (likely) it will give roughly 1.75x performance. In any case, feel free to post a score.

hello Number smasher rev 3.0 with switch on low speed:
;Data collected by: TOPBENCH | Benchmark and detection stub | Version 0.97b
;This file contains fingerprinting information about your computer. Please
;email this file to trixter@oldskool.org with a subject line of "Benchmark" to
;help test these routines and seed the TOPBENCH database.

[UID1F682575]
MemoryTest=3765
OpcodeTest=1817
VidramTest=2127
MemEATest=2040
3DGameTest=1936
Score=4
CPU=Intel 8086
CPUspeed=4.77 MHz
BIOSinfo=Copyright (c) 1984,1985,1986 Phoenix Technologies Ltd (09/24/87, rev. 0)
MachineModel=0000
BIOSdate=19870924
BIOSCRC16=1F68
VideoSystem=VGA
VideoAdapter=VGA, ATI 28800-5VGAWonderBios : 0.1, 256kb Video Memory (BIOS)
Machine=PC/XT

...and switch to high speed:

[UID1F681C91]
MemoryTest=1994
OpcodeTest=745
VidramTest=1640
MemEATest=995
3DGameTest=770
Score=8
CPU=Intel 8086
CPUspeed=10 MHz
BIOSinfo=Copyright (c) 1984,1985,1986 Phoenix Technologies Ltd (09/24/87, rev. 0)
MachineModel=0000
BIOSdate=19870924
BIOSCRC16=1F68
VideoSystem=VGA
VideoAdapter=VGA, ATI 28800-5VGAWonderBios : 0.1, 256kb Video Memory (BIOS)
Machine=PC/XT

Did a quick run with MIPS (v1.20) it says 0.41 mips in high speed.
/cimonvg
 
Unless it runs at 8MHz or faster, it won't give truly "double" performance. If it runs at 7.16MHz (likely) it will give roughly 1.75x performance. In any case, feel free to post a score.

1.5X actually. 4.77*1.5=7.155

Since the turbo in 7.16 mhz machines (like the various faster 1000's) is done by switching from the 14.52mhz clock crystal to an external 22mhz one, then putting a divide by six inline with the 8253 so the system clock stays at 2.38mhz, and a divide by 3 to the bus to keep it at 4.77.

Used to be instructions floating around for doing this to normal 4.77mhz PC's.

-- edit -- found one such instructions for doing it to an original 1000.
http://alternatezone.com/electronics/files/turbo_bd.txt
 
He lives!

He lives!

Don't fall out of your seats, but I resumed coding on my benchmark tool. I've given up trying to design it from the interface forward, and have just been going slowly from the ground up (data structures, test harnesses, etc.). I had forgotten how it is simultaneously fun and irritating to have to code every single thing yourself. Need to manage an in-memory database? Brush up on your balanced binary tree knowledge from college. Need to manipulate .INI files or export to .CSV? Guess what, you’re writing all that drudgery yourself. Ugh.

In any case, I’ve got a console/text-only utility that can do all of the above, so I was able to import all 80+ contributed system entries that people have listed in this thread and also emailed me personally into a single database, and it’s available here: ftp://ftp.oldskool.org/pub/TOPBENCH/database_20120729.zip In that file you'll find both the "ini" style format that is easy to read, and also a .CSV you can load into your favorite spreadsheet.

Every entry in both files is listed in order of speed, ascending
.
So the first entry is a stock PCjr (no surprise), and the last entry is an AMD K-6 @ 350MHz (which is surprising, because there are some 2GHz+ CPUs in the list as well -- just goes to show you that Intel and AMD abandoned 16-bit optimization a very long time ago).

As a reminder, I set up a website for this project a while ago with some analysis of other benchmarks and other info. You will be surprised (actually, maybe not) what the Norton SI benchmark code actually looks like.

I am still looking for results for a stock IBM AT @ 6MHz. The PC/XT-286 @ 6MHz is in there, and it uses 0-wait-state RAM; I really would like to know how the AT's 1-wait-state RAM compares.
 
Last edited:
I am still looking for results for a stock IBM AT @ 6MHz. ....... I really would like to know how the AT's 1-wait-state RAM compares.
Following are the results from my first model IBM 5170.
All parts are IBM supplied. Clean boot. Tool run from C:

[UID6DBA12B5A1]
MemoryTest=1245
OpcodeTest=542
VidramTest=1477
MemEATest=705
3DGameTest=526
Score=11
CPU=Intel 80286
CPUspeed=6 MHz
BIOSinfo=6181028 COPR. IBM 1984 (01/10/84, rev. 0)
MachineModel=0000
BIOSdate=19840110
BIOSCRC16=6DBA
VideoSystem=EGA
VideoAdapter=EGA
Machine=PC/AT
 
Sweet, thanks! This confirms my suspicion that the PC/XT-286 performs at nearly twice that of the AT, even though they both use a 6MHz 80286. I've wondered this for years... nice to check another mystery off the list.
 
Since the turbo in 7.16 mhz machines (like the various faster 1000's) is done by switching from the 14.52mhz clock crystal to an external 22mhz one, then putting a divide by six inline with the 8253 so the system clock stays at 2.38mhz, and a divide by 3 to the bus to keep it at 4.77.

That is not quite right. The 7.16MHz Tandy 1000s (SX, HX, EX), all derive their system timing from a 28.636MHz clock (as does the original 1000 & TX according to the Tech Ref). A divide by 2 gives 14.318MHz (ISA OSC Signal), 4 gives 7.16MHz (CPU Fast & DMA), 6 gives 4.77MHz (CPU Slow & DMA), 7 gives 3.58MHz (CGA NTSC Color) and 8 gives 2.38MHz (8253). The 8-bit Tandys have a 16MHz crystal for the Floppy Disk Controller, which may be with a separate crystal or combined with the 28.636 crystal.

Now that the mystery of the 6MHz 286s have been solved, how about a 8MHz 286 comparison. There is the IBM AT @ 8MHz, the Tandy 1000 TX, TL and TL/2, among other machines. None of them have reported benchmarks.
 
I have both a TX and a TL/2 so I can contribute those, obviously. I just need to get the full tool in a form that I'm ready to pass out (although I won't stop people from running the stub and contributing info). Already I've got error-checking functions to suck in one or more .INI files and merge them in-memory with sorting and dupe-checking, and also the ability to save to a new combined .INI and also export to .CSV. All I need to do is add the ability to fingerprint the existing machine, edit/delete existing entries, compare any two entries, realtime compare the existing machine against the database, and wrap it all in a nice interface, and I should be all set (famous last words, I know).
 
Now that the mystery of the 6MHz 286s have been solved, how about a 8MHz 286 comparison. There is the IBM AT @ 8MHz, the Tandy 1000 TX, TL and TL/2, among other machines. None of them have reported benchmarks.
Following are the results for my 8 MHz IBM 5170.
Clean boot. IBM EGA card.

[UID35CB359]
MemoryTest=923
OpcodeTest=397
VidramTest=1470
MemEATest=514
3DGameTest=388
Score=13
CPU=Intel 80286
CPUspeed=8 MHz
BIOSinfo=62X0820 COPR. IBM 1981, 1985 PARITY CHECK 1 (11/15/85, rev. 0)
MachineModel=01FC
BIOSdate=19851115
BIOSCRC16=35CB
VideoSystem=EGA
VideoAdapter=EGA
Machine=PC-AT 319 or 339, 8MHz
 
Awesome, since you're still accepting new machines I'm going to try to get my newly acquired Core Duo Macbook Pro done, probably won't work, but worth a shot.
 
Awesome, since you're still accepting new machines I'm going to try to get my newly acquired Core Duo Macbook Pro done, probably won't work, but worth a shot.

If it can boot DOS, it can run the stub. Your MacBook doesn't have a traditional PC BIOS, so it won't boot DOS, unless you know something I don't. I would prefer not to accept official results for results obtained under virtualPC sessions, emulators, etc. (I won't stop you from gathering your own results, but I'm not going to officially record them :)
 
I was actually going to try Windows 98 under bootcamp, why not? If I can get it to install, I can boot 98 into MS-DOS mode and (hopefully) run it :p.
 
I've been able to boot certain DOS versions on my Macs, but they all run into the same problem, I can't type anything, cause the BIOS emulation has no USB keyboard emulation support..

Even when trying to install XP on my 2008 Mac Pro, I can't get the keyboard to work in the text mode portion of setup.
 
Following are the results for my 8 MHz IBM 5170.
Clean boot. IBM EGA card.

[UID35CB359]
MemoryTest=923
OpcodeTest=397
VidramTest=1470
MemEATest=514
3DGameTest=388
Score=13
CPU=Intel 80286
CPUspeed=8 MHz
BIOSinfo=62X0820 COPR. IBM 1981, 1985 PARITY CHECK 1 (11/15/85, rev. 0)
MachineModel=01FC
BIOSdate=19851115
BIOSCRC16=35CB
VideoSystem=EGA
VideoAdapter=EGA
Machine=PC-AT 319 or 339, 8MHz

Fascinating to observe that the IBM PC AT is really only ever so slightly faster at 8MHz than 6MHz and tough to notice the difference with an XT-286 board.
 
One thing that comes out of these benchmarks is that the speed of the video card can make quite the difference. The Tandy Graphics Adapter is pretty speedy compared with its competition CGA, PCjr. and EGA. The Amstrad PC1512 in particular seems handicapped by a really, really slow graphics adapter, its slower than the IBM CGA card even though the processor is running at almost twice the speed. The 386DX/40 results are really unkind to the Trident VGA.
 
@commodorejohn: Thanks, I'll incorporate those (turbo only, I'm not officially recording any results with the turbo button off or intentionally underclocked).
 
So, I am stuck at a really stupid place: The interface. I want the benchmark to be friendly and easy to use, and have a nice interface that anybody can understand and navigate. I have three options on how to do the interface at this point in development, and would like a few opinions on how to proceed. Here's what I'm looking at:

TOT Library (pop-up menus)
- Pros: easy to implement (not fast, just easy)
- Cons: clunky API, inflexible, bloated code

Turbo Vision (pull-down menus)
- Pros: elegant flexible API, CUI-compliant interface
- Cons: longest development time (Turbo Vision is difficult to learn)

Spend-a-weekend-rolling-my-own:
- Pros: lean/small/fast code
- Cons: clumsy/inelegant, would shell out to an editor (I'm not spending weeks coding flexible form input)

Because I'm a perfectionist, I am leaning towards Turbo Vision, but this would delay release by several weeks. However, it would look the nicest and be the most flexible moving forward. Curious what everyone's thoughts are.

Yes, I know this is a stupid thing to get hung up on, but it's important.
 
Back
Top