• Please review our updated Terms and Rules here

Looking for volunteers to help test a new benchmark

I meant consistently faster than the XT. I was expecting it to be faster than without the expansion (and that really is horrific, how slow it is without). (To be fair, I was also expecting it to be a bit faster than the XT, if it was kept out of internal memory.)
 
Ah, I see. Well, the difference is about 3%, which is IMO not statistically significant enough to warrant a deep dive into why that might be :) but I agree it is interesting. It may have to do with the PCjr implementing a different DRAM refresh method than the PC, which may take up a tiny bit less cycles.
 
I ran this quite a while back and submitted results.. Would you like me to run it again on my V30 powered IBM?

EDIT: Nope doesn't seem I need to run it again, despite now having a 2GB Disk On Module, and an ATI VGA Wonder video card in my Model 25, it still matched my machine nearly perfectly and picked up my last submission instantly.
 
Last edited:
The benchmark isn't testing the storage subsystem, so it wouldn't affect things. Video, though, it is testing video memory bandwidth, so changing the video card does warrant a re-test.
 
Video memory bandwidth came out exactly identical however, there was no real change at all. In fact, it still thinks I have MCGA I think, and may have been testing the wrong video memory, unless my VGA Wonder has the same exact speed as the MCGA's RAM.
 
Well...

Per the PS/2 Model 25 technical reference, the on-board video subsystem is on the 8-bit section of the machine, and the 8-bit side ran with four wait states (read: half-speed). Just about any ISA video card other than CGA will handle a 4 MHz 8088 (which is what the card thinks it is) slamming things into it as fast as possible...
 
bhtooefr said:
Well...

Per the PS/2 Model 25 technical reference, the on-board video subsystem is on the 8-bit section of the machine, and the 8-bit side ran with four wait states (read: half-speed). Just about any ISA video card other than CGA will handle a 4 MHz 8088 (which is what the card thinks it is) slamming things into it as fast as possible...
Click to expand...

Technically, the card doesn't "think" it's anything. It's being run by the clock, which on a 25 is clocked at around 8MHz, twice than what you say it's claiming it is.

I hate to nitpick, but the ISA bus on the 25 is not clocked at 4MHz.
 
Per the technical reference, an 8-bit bus access is eight cycles, or 1000 ns. That's actually 1 MHz, but I was referring to MHz in terms of 8086 clock cycles, not bus accesses - an 8086 or 8088 will take four cycles assuming no wait states, so a 4 MHz machine at 0WS would take 1000 ns.
 
Ok. I was referring to the clock signal on the ISA bus in my 25, it measures 8MHz, and is actually tied to the same clock as the CPU.

As it is right now, on an XT-IDE I transfer around 640Kbps in transfer tests.
 
Alright, I've been thinking about TOPBENCH for quite some time now, but in a non-PC sort of way; hope that doesnt annoy everyone.

Before I dig too deeply into the code, has anyone done any investigation or work on porting to say CP/M for Z80? I've thought about trying a port to Z280 to get some numbers for the CPU280,. The Pascal portions might not be too difficult, but the asm portions of course would have to be rewritten, and the video benchmarks would be pretty much useless. But I've wondered about a more universal and non-PC-capable benchmark that could at least make a reasonably-close comparison between architectures (like the Byte Unix benchmarks do for the old-school Unix world).

And I know the difficulty of cross-arch benchmarking; but, again, the Byte Unixbench suite did that pretty successfully for the very architecture-diverse Unix world (successfully comparing performance between a VAX, a 68K workstation, and SPARC).

Perhaps someone has asked about this before in this 39-page thread; a quick (really quick!) skim didn't find anything, but I reserve the right to be wrong.....
 
lowen said:
Alright, I've been thinking about TOPBENCH for quite some time now, but in a non-PC sort of way; hope that doesnt annoy everyone.

Before I dig too deeply into the code, has anyone done any investigation or work on porting to say CP/M for Z80? I've thought about trying a port to Z280 to get some numbers for the CPU280,. The Pascal portions might not be too difficult, but the asm portions of course would have to be rewritten, and the video benchmarks would be pretty much useless. But I've wondered about a more universal and non-PC-capable benchmark that could at least make a reasonably-close comparison between architectures (like the Byte Unix benchmarks do for the old-school Unix world).

And I know the difficulty of cross-arch benchmarking; but, again, the Byte Unixbench suite did that pretty successfully for the very architecture-diverse Unix world (successfully comparing performance between a VAX, a 68K workstation, and SPARC).

Perhaps someone has asked about this before in this 39-page thread; a quick (really quick!) skim didn't find anything, but I reserve the right to be wrong.....
Click to expand...

I'd go for porting to odd varieties of non-IBM-PC DOS before porting for CP/M, but you'll run into the same problem in both cases. Neither DOS or CP/M have any native ways of precise timekeeping, so a benchmark must be tailor-made for a particular system. If I'm not mistaken, TOPBENCH uses some DMA hack to determine the CPU speed. Such things can be very difficult to replicate on non-PC systems.
 
lowen said:
Before I dig too deeply into the code, has anyone done any investigation or work on porting to say CP/M for Z80?
Click to expand...

I'd recommend against that; TOPBENCH was deliberately designed for x86 systems only, to create a synthetic score to use as a means to compare different x86 CPUs against each other based only on the time it takes for their instruction opcodes to execute in various situations. It was never meant to be portable to other CPU architectures or environments. If you want something to compare across all systems, a more generic workload-only benchmark would be more appropriate; see below:

But I've wondered about a more universal and non-PC-capable benchmark that could at least make a reasonably-close comparison between architectures (like the Byte Unix benchmarks do for the old-school Unix world).
Click to expand...

The "problem" with that is that if you create a portable piece of code, it is possible for that code to be compiled differently depending on what compiler is used (even if all the compilers you try are for the same architecture). That's why the TOPBENCH metric code was written in assembler. With very few and irrelevant exceptions, the code assembles the same no matter the assembler.

And I know the difficulty of cross-arch benchmarking; but, again, the Byte Unixbench suite did that pretty successfully for the very architecture-diverse Unix world (successfully comparing performance between a VAX, a 68K workstation, and SPARC).
Click to expand...

I'd say try porting that, except it contains a ton of unix-centric system calls (exec(), pipe-based context switching, etc.), as well as I/O-limited stuff (copying a file), so that won't be portable across all systems. It's portable across all UNIX systems, yes, but not all systems.

If I were you, making a benchmark to compare across all systems, I'd do the following:

  • Pick some common benchmark workloads (dhrystone, whetstone, etc.)
  • Determine if you are measuring the performance of the hardware only, or the entire system (hardware+OS+I/O)
  • ..If hardware only, describe the workload in pseudocode but allow anyone to write an optimized version of it for a CPU (maybe even in assembler) so that the CPU isn't penalized by bad compilers
  • ..If entire system, write portable C code
Here is a recent resurrection of the Byte Unixbench; it contains both a dhrystone and a whetstone, so maybe those could be lifted.

per said:
TOPBENCH uses some DMA hack to determine the CPU speed.
Click to expand...

It's nothing that heinous; TOPBENCH reprograms the INT 08 PIT interrupt to fire at 20Hz instead of 18.2Hz so that the time measurements are done with integers in milliseconds, then it programs it back so that DOS time doesn't skew too much. In retrospect, this was unnecessary; I could have gotten the same accuracy if I had just monitored the 8253 timer value ports. Oh well, can't change it now without potentially invalidating the numbers in the database.
 
Trixter said:
I'd recommend against that; TOPBENCH was deliberately designed for x86 systems only, to create a synthetic score to use as a means to compare different x86 CPUs against each other based only on the time it takes for their instruction opcodes to execute in various situations. It was never meant to be portable to other CPU architectures or environments. If you want something to compare across all systems, a more generic workload-only benchmark would be more appropriate; see below:.....
Click to expand...

Thanks for the nicely worded reply. I suspected as much, but it never hurts to ask. I really would like to find a comparison between some of the newer turbo-Z80's (eZ80@50Hz, Z8S180@ 49MHz (there is at least one P112 out there that has run at 49MHz!), Z280@12MHz even though it's not newer) and the x86 family for some common operations. Especially the comparison between a 286 @12 and a Z280 @12.
 
Okay I finally got my Zenith Z-159 up and running and Topbench was the 2nd program I ran after Commander Keen. Results with turbo on and off:

Code: 
[UID50471A772]
MemoryTest=3887
OpcodeTest=1842
VidramTest=2743
MemEATest=2049
3DGameTest=1924
Score=4
CPU=Intel 8088-2
CPUspeed=4.77 MHz
BIOSinfo= Copyright (C) 1986, by Zenith Data Systems (
BIOSdate=20000101
BIOSCRC16=5047
VideoSystem=EGA
VideoAdapter=Paradise PEGA 1A
Machine=XT Clone (Zenith Z-159 @4.77MHz)
Description=A British Telecom rebadge of a Z-159 in normal mode running DOS 3.20
Submitter=brassicGamer (VCF)

Code: 
[UID50472959A]
MemoryTest=2307
OpcodeTest=1087
VidramTest=2113
MemEATest=1210
3DGameTest=1141
Score=6
CPU=Intel 8088-2
CPUspeed=8 MHz
BIOSinfo=Copyright (C) 1986, by Zenith Data Systems (
BIOSdate=20000101
BIOSCRC16=5047
VideoSystem=EGA
VideoAdapter=Paradise PEGA 1A
Machine=XT Clone (Zenith Z-159 @8MHz)
Description=A British Telecom rebadge of a Z-159 in turbo mode running DOS 3.2
Submitter=brassicGamer (VCF)

I have to say I was very impressed with the performance of this computer @ 8MHz - the zero wait states really makes a difference versus a comparable AT.
 
Added a Panasonic CF-53 to my DOS line-up. Dual-Boot DOS 7.1 (w98se in real mode) and XP SP3, FAT-32 512GByte SATA drive. This machine has native USB support in BIOS, works in DOS. Used FreeDos FDISK, then FreeDOS Format, then Win98se Format/s to make the bootable drive. Then install XP for the dual boot. XP makes use of 3.2GBytes of RAM, PharLap DOS uses all 4GBytes.

[UID35981200DD]
MemoryTest=1
OpcodeTest=1
VidramTest=62
MemEATest=1
3DGameTest=1
Score=779
CPU=Intel CORE I3
CPUspeed=2095 MHz
BIOSinfo=(C)1985-2009,American Megatrends, Inc.All Rights Reserved. (06/16/10, rev. 0)
BIOSdate=20100616
BIOSCRC16=3598
VideoSystem=VGA
VideoAdapter=VGA, VESA, 256kb Video Memory (BIOS)
Machine=PANASONIC CF-53E
Description= Intel i3-2310M 2.10GHz
Submitter= Brian
 
FYI, new yearly update here: https://github.com/MobyGamer/TOPBENCH/releases/tag/0.38.h
There are now 260 systems in the database.

I'd like to add at this point that I am a lot less likely to accept submissions for anything over a 486. The benchmark had to be compatible with the 8088, so it only uses 8088 instructions. Trying to use TOPBENCH to compare anything like a Pentium with a K6, etc. is not prudent because most applications that run on those systems are 32-bit protected-mode programs. There are other benchmarks more suited to that kind of comparison.
 
With the above - I will not bother posting the benchmark for a Core I5 2.7GHz machine. My RM Fortran-77 code has never run faster. Sometimes you do not need a lot of memory, just raw computational speed and ease of implementing code. The Panasonic CF-53 mk3 supports USB drives, card readers, and USB memory sticks under DOS. I'm using a FAT-32 512GByte drive to dual-boot XP and DOS 7.1 (w98, Bootgui= 0). The CF-52 and CF-53 offer a full range of extended graphics modes, unlike the earlier CF-50 and CF-51.
 
Trixter said:
FYI, new yearly update here: https://github.com/MobyGamer/TOPBENCH/releases/tag/0.38.h
There are now 260 systems in the database.

I'd like to add at this point that I am a lot less likely to accept submissions for anything over a 486. The benchmark had to be compatible with the 8088, so it only uses 8088 instructions. Trying to use TOPBENCH to compare anything like a Pentium with a K6, etc. is not prudent because most applications that run on those systems are 32-bit protected-mode programs. There are other benchmarks more suited to that kind of comparison.
Click to expand...

Older benchmarks can give really weird results. I have an AMD Athlon XP 1.4Gz system I've been testing as a tweener. The biggest minus is no ISA slots. On other other hand it supports two floppy drives, sits in a huge Antec tower, and is paid for.

It's tickles my sense of humor to see Norton's Utilities 4.5 SI report a machine that runs 3,100 times as fast as an XT. Even stranger (before I turned off the cache and throttled back the bus) was seeing that it ran 63,000 times faster than an XT. Landmark (v2 & v6) truncated to 3 digits in reporting the speed, and MIPS gave gibberish. I suspect a formatting issue as they didn't anticipate 1.4Gz processors back then.

Fun fact: this system has 1Gb of DDR memory on board. Contrary to legend, Win98SE has no problems running. And (again) it tickles my sense of humor to run "mem /c" under DOS and see a full gigabyte of memory.
 
You must log in or register to reply here.
Back
Top