Some time ago, I came across an old Computer World article from 1980 that was comparing “$15,000 computers” (high end equipment at the time). That discussion is here. And the point of those articles was showing how some of the then-new-to-the-world microcomputers were seemingly catching up with the performance of those “high end” machines, at substantially less cost.
I then came across an article describing a kind of “performance benchmark” that had been implemented in BASIC, first introduced in 1983 in Creative Computing (see here November 1983 edition, page 260), and known as Ahl’s Benchmark since the code was submitted by David Ahl.
A have a number of “boots-up-to-BASIC” systems around, so I thought it would be interesting to run this same benchmark on those systems, and compare some 2024-results with those 1983-results. Listed below is the version of the Ahl-Benchmark code as I used it. A summary of my changes are:
- Disregarded the comment on line 10.
- Did not use “:” to do compound lines of code on the same line (because the IBM5100 version of BASIC doesn’t support doing so). For those lines, I offset the additional line to x5 (i.e. 25, 45, 75).
- On the Color Computer only, I had to replace RND(1) with RND(0). A few systems support RND(.) but I wanted to keep the program as consistent as possible across the test systems.
20 FOR N=1 TO 100
25 A=N
30 FOR I=1 TO 10
40 A=SQR(A)
45 R=R+RND(1)
50 NEXT I
60 FOR I=1 TO 10
70 A=A^2
75 R=R+RND(1)
80 NEXT I
90 S=S+A
95 NEXT N
100 PRINT ABS(1010-S/5)
110 PRINT ABS(1000-R)The original Creative Computing article describes the working of the benchmark program (and also discloses known limitations and issues about it). No single benchmark is ever perfect, but I wanted to just stick with this original as-is to give it a try. For the benchmark results, essentially the first number is a measure of the “accuracy” of the floating point (such that closer to 0 is better, this number will not change between runs). The second number is a measure of the “quality of randomness” and would also ideally be 0 (so again, lower is better, but the number will vary and so a few runs is necessary to see the typical trend).
| Float Accuracy | Randomness | Time (MM:SS.ms) | |
| HP 9830A (1973) | 8.89000E-06 | 1.3102 | 01:51.85 |
| IBM 5100 (1975) | 1.566E-6 | 782.326940 | 03:04.32 |
| IBM 5110 (1978) | 3.458354E-10 1.447754 (single) | 954.616583 954.5898 | 00:40.38 00:38.12 |
| HP 85 (1980) | 0.00000002 (2.0E-8) | 993.982626014 | 01:19.098 |
| Commander X16 (8MHz 65C02) (2024) | 5.65648079E-4 | 1.26747227 (best) 22.1375535 (worse) | 00:07.75 |
| Commodore 64 (stock 1MHz) (1982) | 1.04141235E-3 | 1.15579295 (best) 13.7690766 (worst) | 02:02.39 |
| Tandy Color Computer 3 (6309 upgraded) (1986) | 3.96284867E-4 | 5.8566258 | 02:22.48 |
| IBM 5150 (4.77MHz stock) (1983) | 1.159668E-2 | 9.317505 | 00:23.88 |
| Commodore PET (4032 from 1980) | 1.04141235E-3 | 7.32327056 | 02:00.12 |
| NEC PC-8001 mk II (1983) | 0.0338745 | 1.76392 | 01:28.14 |
| Tandy Model 102 (1986) | 2.058E-7 | 12.4944259276 | 04:55.84 |
| Apple IIe (1986 Platinum) | 1.04141235E-3 | 0.821743011 | 01:53.71 |
| Sharp PC-5000 (1983) | 5.859375E-3 | 7.188416 | 00:29.75 |
| NEC PC-8001 (1979) | 0.0338745 | 1.76392 | 01:30.72 |
| Aquarius+ (2024) | 0.187805 | 1.20343 | 01:11.34 |
| Agon Lite 2 (2023) (bbcbasic 1.06) | 2.57968903E-4 | 3.26272535 | 00:01.31 |
Here is a text-code version of the same data:
Some concluding notes and observations:
- For the NEC PC-8001, it is the only system that continues to flash the BASIC console cursor throughout the runtime of the benchmark program. This makes it more apparent that during the runtime of any BASIC program, system interrupts are still being invoked even if they have fairly benign activity.
- Of these, only the Commodore systems and Color Computer support the RND(.) “optimization”, while all the other systems treat RND(.) as an RNG init (and thus for those systems, RND(.) always returns the same value).
- Note how the Apple IIe, Commodore PET, and Commodore C64 all have the same floating point accuracy result. This is because not only are they using the same 6502 processor, but the floating point code in each of them is derived from the same original Microsoft BASIC source.
- For the bolded systems, the results here match the original 1983 article.
- The LCD screens of the Sharp PC-5000 and Tandy Model 102 are hard to photograph without reflections. The IBM 5100/5110 CRTs are also difficult for similar reasons (the glass cover of the CRTs has a lot of reflection). For the Commodore PET I had to just turn off the room lights to avoid its reflections. The Color Computer is “famous” for its nuclear green default background color.
- Both the NEC and IBM 5150 (IBM PC) feature a set of “short cut keys” as indicated at the bottom of the screen.
- I ran both the NEC PC-8001’s at the Japanese 100V. But I ran them also at US 115V and got the same results.
- The Aquarius+ is a modern remake of the original Mattel Aquarius (a Z80A based system from June 1983).
The runtime here corresponds to that systems’ ability to tokenize the BASIC program, then interpret the tokens during the runtime of the program. Some systems will tokenize as the program is being entered (and typically on those systems, the BASIC program is reformatted when it is LISTed such that it differs from what was originally typed in). As an example of tokenizing, a keyword such as “SQR” or action such as * (multiply) is encoded into a byte, and variables (such as R and S) are allocated to a storage spot in the BASIC “workspace” (in most BASIC’s that reserved space is 5 bytes for each declared variable).
Some systems are obviously more efficient in their implementation of BASIC than others. Another factor on the runtime is the implementation of the SQR and RND function, which can vary widely across the systems. The RND call in particular might use a clock-related counter or may rely on some algorithm techniques (which those techniques may be both slower and less random). The presence of any clock requires some custom machine code to access (wrapped in the System ROM’s implementation of BASIC for that system), but the clock (or counter) can be thought of as a “hardware-accelerated” random number generator.
SPECIAL: HP9830A and Ahl’s Benchmark
Using GO9800 emulator, determined the following is syntax valid for the HP9830A. Some interesting notes:
- Variables need to be initialized explicitly before being used. Like R=0, S=0. They don’t have to be initialized to 0, but they need to be initialized to something (like A=N in the example below).
- RND requires an extra expansion module.
- Use DISP instead of PRINT to show results on the LED screen instead of the printer.
- “END” keyword must appear at the end of the program or else the LED display is overwritten with an “ERROR 50”
Results (in GO9800 emulator) were displayed in 8.83 seconds, with decent benchmark results. We’ll see how the real hardware compares.
On the real hardware, the results were like this:
- Using -0.8675309 as the RND seed (yes, the numbers from the song! HP manual recommends using a “long sequence” for the seed, and better results if the last digit ends in 9)
- The code is LIST’d to the printer page
- The timing result is approximately 1 minute and 52 seconds. Float “quality” is 8.89000E-06 and the RNG “quality” is 1.3102 . These are respectable results, but mostly I’m glad to see the task completed at all.
