• Please review our updated Terms and Rules here

Audio compression for the 8086/88

carlos12

Experienced Member
Joined
May 10, 2020
Messages
171
Location
Madrid, Spain
I've being doing a research for PCM samples compression for such an early processor as the 8086/88, for being played on the internal speaker, parallel port devices and SB and compatibles . The requirements I look for are: it must be fast, as the target processors I chose are quite slow; it must work in real mode and without 286+ instructions; the binary code must be small as on those platforms every byte counts, and every byte used on the program is a byte that cannot be used to store graphics and sounds. The first solutions I found, without using compressors, are so far:

- Using 8 bit mono samples, which is fine as I plan to target the lowest common denominator.
- Reducing the sample rate so the files will be smaller (although worse quality)
- Trimming beginning and end silences

But aside these I wanted to do some real compression. Unfortunately, the general use compressors doesn't seem to compress a thing with audio. So I found a few specific audio compressors/decompressors that work on the 8088:

- Audio Compression by John W. Ratclif. I liked a lot because there's a version written in assembly language, so it's fast and the generated code is quite small, and it really compresses a lot while not reducing, apparently, the quality of the sound. The problem is that while the decompressed sound plays perfectly on the speaker and LPT DAC, it does not play on the Sound Blaster, at least not trough the CT-VOICE driver. I haven't yet tried playing sending the PCM data directly to the DMA channel. I found it here: https://www.drdobbs.com/database/audio-compression/184408798 On this moment it's the one I liked the most.

- Creative ADPCM. As it was said on another thread on this forum, it's compressed using a Creative utility. The sample loses quite quality and, as it was said on the other thread, it's compressed to a format that can be used only with real Sound Blasters. So I discarded it.

- MPEG Layer II (MP2). Copyright (c) 1991 MPEG/audio software simulation group. I think this may be the best solution, as it makes so tiny files. I found the sources in C language but I didn't tried yet as it looks very complex to implement, there are many C files and they are quite large so while I have no warranty it will work well on the 8088, they may take a lot of room from my program.

- ADPCM Stichting Mathematisch Centrum, Amsterdam, The Netherlands. Intel/DVI ADPCM coder/decoder. I haven't tried it yet. It's written in C but at least the decoder function is quite small.

Do you know other audio compression algorithms that may work for the 8086/88?

Thank you very much for your opinions and help.
 

bakemono

Experienced Member
Joined
Jan 19, 2020
Messages
177
Location
USA
Creative ADPCM should be easy to decode in software (both in terms of complexity and CPU time) according to the description here https://wiki.multimedia.cx/index.php/Creative_8_bits_ADPCM

I'd say the ACOMP one in your Dr Dobbs link would be a step up, and would be comparable to Dialogic ADPCM which can be decoded using lookup tables instead of (potentially slow) multiplication. Another step up would be Yamaha or Microsoft ADPCM which need multiplication to decode. I don't think an 8086 can handle MP2 at all.

Some schemes would be more or less suitable depending on the type of audio. Are you planning to use long samples with music or speech? Or is it more like drum samples and sound effects?
 

Trixter

Veteran Member
Joined
Aug 31, 2006
Messages
7,413
Location
Chicagoland, Illinois, USA
Today's your lucky day: This is my wheelhouse.

- Audio Compression by John W. Ratclif. I liked a lot because there's a version written in assembly language, so it's fast and the generated code is quite small, and it really compresses a lot while not reducing, apparently, the quality of the sound. The problem is that while the decompressed sound plays perfectly on the speaker and LPT DAC, it does not play on the Sound Blaster, at least not trough the CT-VOICE driver. I haven't yet tried playing sending the PCM data directly to the DMA channel. I found it here: https://www.drdobbs.com/database/audio-compression/184408798 On this moment it's the one I liked the most.

ACOMP is indeed a nice scheme, although it doesn't play back realtime on an 808x class system; there is a 2-3 second pause for decompression. The playback program uses the DIGPAK series of drivers (also by Ratcliff) and it definitely works through Sound Blaster (and others); you just need to have the correct driver loaded. ACOMP is not the *best* scheme, but if you don't want to code up anything for yourself, it can achieve 2:1 compression for music/sound and 3:1 for speech with very little audible loss.

- Creative ADPCM. As it was said on another thread on this forum, it's compressed using a Creative utility. The sample loses quite quality and, as it was said on the other thread, it's compressed to a format that can be used only with real Sound Blasters. So I discarded it.

It is also limited to 12KHz playback on real sound blaster cards. The quality isn't great.

- MPEG Layer II (MP2)

No, frequency-based systems are too complex for 808x systems. You'd spend 20 seconds decoding, with a math coprocessor, to play a 5-second sound.

- ADPCM Stichting Mathematisch Centrum, Amsterdam, The Netherlands. Intel/DVI ADPCM coder/decoder. I haven't tried it yet. It's written in C but at least the decoder function is quite small.

IMA ADPCM was engineered to be high quality and fast playback for 16-bit sample data and for 12 MHz+ class systems. I'm not sure it can play back realtime on 808x class systems; maybe 10MHz XTs might. But because the scheme was created for 16-bit sample data and a 4:1 compression ratio, you have two choices for adapting it for 8-bit PCM data: Either discard the lower 8 bits on playback (and drop the effective data compression ratio to 2:1), or try to adapt the tables for real 8-bit input and output (also 2:1 compression unfortunately). Michael J. Mahon did the latter attempt, which you can find here in 6502: http://michaeljmahon.com/adpcm.zip
However, I'm not sure the routines are fast enough for realtime decode.

So, what are software options for realtime decompression on 808x class hardware? I've done a lot of original research on this over the last decade, sometimes taking inspiration from the demoscener Algorithm of Onslaught who has dabbled in this for lower sample rates (8KHz) on the C64 (4-bit output target). To design something like this, you need to take the capabilities of the target hardware into account and design a decompression routine FIRST, so you can test it with random data to ensure it is capable of decoding your samples fast enough for realtime output through the PC speaker or an LPT DAC. Then you write the compressor.

I've successfully tested two methods that produce acceptable results on a 4.77 MHz 8088: One that is only good for noisy music or sound effects so I won't mention it here, and another based on vector quantization. Essentially, in a VQ scheme, the compressed data consists of a codebook that contains N entries of samples, each M in length, followed by bytes that index into the codebook. So for a simple 4:1 compression scheme, if we have 256 (N) entries each 4 bytes (M) in length, a (256*4 = 1024 bytes) codebook of samples is transmitted first, and then each byte that follows is an index into that codebook that reproduces 4 output bytes of sampled audio. Since decoding is only lookups, it's crazy fast.

Here is my 2013 experiment using VQ with 4096 entries, each 12 bytes in length, that plays realtime through the PC speaker on any 4.77MHz 8088 PC: ftp://ftp.oldskool.org/pub/misc/temp/gb.zip
The source was 16572 Hz, and the compression ratio was roughly 6:1 to fit 3 minutes of audio on a 360K disk.
HOWEVER: It is not optimal; there are tradeoffs. For every 4:1 compression of the data, there is roughly 2:1 loss of fidelity which exhibits as distortion. However, unlike simply reducing 8-bit PCM sample rates by half, the loss and distortion is not the result of a low-pass filter, so it sounds somewhat subjectively better.

Can this be improved? Yes; for one thing, I coded the VQ compressor myself, and I'm a terrible programmer. A proper implementation using python and scipy, numpy, or something doing a proper K-means (or better) would likely produce better output quality. Taking into account the deltas between samples and using that as a quality metric would also help, something I haven't gotten to yet, but something this person DID: https://brokenbytes.blogspot.com/2018/03/a-48khz-digital-music-player-for.html
and his results (3x sample rate not withstanding) sound better than mine.

I have designed a scheme that combines *all* of the above disciplines along with order-0 through order-3 prediction, something none of the above do, which should be able to achieve 4:1 with better -- not transparent, but much better -- quality than both VQ and ACOMP while still being able to decode in realtime at 4.77 MHz. However, I want to use it in my own production/code first before I release it.

There might be other solutions, if you tell us more about what you're trying to achieve. I'll say this though: Assembly will be required. Compiled C code won't be fast enough.
 

carlos12

Experienced Member
Joined
May 10, 2020
Messages
171
Location
Madrid, Spain
Thank you so much for your responses! I've read them very carefully.

What I intent to do with the samples is:

- Playing some music with the speaker or LPT DAC, only on the start screen (which is static). If Sound Blaster is selected, the music would play through OPL. My intention is doing something like that old PCM music of the 80's games, looping, alternating and pasting just a few chunks of music (as several bars just repeat), like the ones showed on this video:


That poses another problem: the segmentation. If a chunk takes more than 64 Kb, I have to split it into several sub-chunks to fit it. I just don't want to use Turbo C huge pointers as I think they are slow... and buggy.

- A few special effects, very short, most of them lasting less than 1 sec.

The problem is that I want my project to fit on a 720 kb diskette (or two 360 kb) and uncompressed PCM samples just take too much room. Even worse, they can also decimate the RAM. So I would like to find a solution that at least saves some of the precious diskette space. But if it could decompress the audio in real time, saving also some Kbs of RAM (it is, not needing to previously decompress the complete data on RAM), that would be terrific. Anyway, if the last one is not achievable on the 8088, would be happy just saving disk space.

Thanks again.
 
Last edited:

Trixter

Veteran Member
Joined
Aug 31, 2006
Messages
7,413
Location
Chicagoland, Illinois, USA
Ah, so you want to reproduce what Mach 3, Space Racer, etc. did on their title screens. Nice :) Well, they didn't use any compression at all; they used samples that had a playback rate of 6 to 9 KHz.

As mentioned before, realtime decompression+playback in this domain is a specialized thing. I had to write my own. If you're not comfortable with assembler, you may want to abandon that idea for now and just strive for compressing samples on disk. If you can adapt the ACOMP decompression code to decompress samples after they're loaded (it's not fast enough for realtime on 8088), do that. ACOMP can compress music 2:1 mostly transparently; anything beyond that will introduce some distortion but that might be acceptable depending on the music itself.

If you'd like to try realtime decompression anyway and want something super quick to get your feet wet, simply use 4 bits to store each sample instead of 8 (literally >> 4 to "compress", put two 4-bit samples in a single byte, and << 4 to "decompress" before sending to the 8-bit output device). The quality will suffer, but it's a foundation to build on.

Whatever scheme you land on, lossy 2:1 compression is generally transparent, but beyond that the audio quality starts to suffer. One experiment you can try that has open source C code is the G.726 ADPCM codec. It's slightly heavier to decode than IMA, but it supports 2, 3, 4, or 5 bits per sample for 4:1, 2.66:1, 2:1, or 1.6:1 compression ratios respectively. You can play with this in ffmpeg:

Code:
# encode:
ffmpeg -i input.wav -acodec g726 -ab 16 g726_16k.wav
# default -ab is 32k which is 4 bits per sample, we use 16 to get 2 bits per sample

# decode:
ffmpeg -i g726_16k.wav output.wav

The G.726 codec forces an 8khz playback rate, but you can get around that by changing the playback rate (not resampling!) of whatever you feed it to 8KHz and then change it back when done.
 
Last edited:

bakemono

Experienced Member
Joined
Jan 19, 2020
Messages
177
Location
USA
For title screen music, you might even consider an Amiga module or similar. There is a thread on VOGONS about Modmaster which runs on an 8088, albeit with limited sampling rate and number of channels. If you already composed an Adlib version then the patterns are basically done, you just need samples and a player routine...
 

Trixter

Veteran Member
Joined
Aug 31, 2006
Messages
7,413
Location
Chicagoland, Illinois, USA
This isn't practical on 808x-class hardware. Mod Master XT is one of only two viable players that can do this on an 8088-class system (Galaxy Player 2.12 is the other), and their source code isn't public. Mod Master's source is a mixture of Turbo Pascal and assembler. Also, @ 4.77 MHz, the best output rate you could hope for playing on an LPT DAC or internal speaker, playing a 4-channel .mod, is around 5 KHz.
 

Trixter

Veteran Member
Joined
Aug 31, 2006
Messages
7,413
Location
Chicagoland, Illinois, USA
If Sound Blaster is selected, the music would play through OPL.

A real sound blaster has a real 8-bit DAC; you should use it. You can drive it like an LPT DAC if you want, feeding samples at a constant rate, but there is a speed penalty (bigger than the LPT DAC!) because you have to acknowledge each sample sent with a second command, so using the DMA mode is way better.

If you did mean OPL because you want to drive an Adlib like a DAC too, just offer that for Adlib. If you mentioned it because you thought you could support only OPL DAC-like playback and it would work for both SB and Adlib, that's not advisable because bit-banging an OPL2 chip like a DAC produces much quieter output than just using the SB natively.
 

carlos12

Experienced Member
Joined
May 10, 2020
Messages
171
Location
Madrid, Spain
Yes, by OPL2 I mean Adlib too. I'm using the Id/Apogee IMF playing code, which is quite low CPU intensive as it can run as low as at 70hz (maybe less? Didn't try it). I prefer not to complicate this too much. Regarding the SB, when I use it for sound effects, I send them to the DMA via the CT-VOICE, just releasing the CPU for another tasks.

Trixter said:
As mentioned before, realtime decompression+playback in this domain is a specialized thing. I had to write my own. If you're not comfortable with assembler, you may want to abandon that idea for now and just strive for compressing samples on disk. If you can adapt the ACOMP decompression code to decompress samples after they're loaded (it's not fast enough for realtime on 8088), do that.

Well, I'm not Michael Abrash but I don't feel too bad on the assembler fields :D The problem is that the core of the game is taking a lot, lot of time, so I prefer to focus on that in order to make it as much efficient as I can, while not dedicating too much time to other tasks. I try not to reinvent the wheel as much as possible. Also audio compression is a totally new subject to me, so it would become an enormous task. So I will follow your advice and will stick on the ACOMP for at least saving some real estate from the disk, as it lighweight and relatively easy to implement.

bakemono said:
For title screen music, you might even consider an Amiga module or similar
Trixter said:
This isn't practical on 808x-class hardware. Mod Master XT is one of only two viable players that can do this on an 8088-class system (Galaxy Player 2.12 is the other), and their source code isn't public. Mod Master's source is a mixture of Turbo Pascal and assembler. Also, @ 4.77 MHz, the best output rate you could hope for playing on an LPT DAC or internal speaker, playing a 4-channel .mod, is around 5 KHz.

I was also thinking on that idea too. Since many years ago I'm aware of a mod player that works on the 8088, MOD-OBJ, by Mark J. Cox. But it's too slow to be usable at 4,77 Mhz (even 8 Mhz), the source code has never been released and it weights 17 Kb, too much for my project.

Also the 8088Mph Mod player source code seems to be available https://github.com/reenigne/reenigne/blob/master/8088/demo/credits/mod.asm but I don't want to make things even more complicated so I abandoned the idea of playing MOD files.

Ah, so you want to reproduce what Mach 3, Space Racer, etc. did on their title screens. Nice :) Well, they didn't use any compression at all; they used samples that had a playback rate of 6 to 9 KHz.

Hehe, yes! :p I think I'll continue with that idea.

By the way, I listened to your Ghostbusters theme on a 5,25" diskette. Very impressive to fit it on that, and being played on the 8088 through the speaker!

Out of curiosity, did you use some kind of audio compression for 8088 Corruption and 8088 Domination? Thanks!
 

Trixter

Veteran Member
Joined
Aug 31, 2006
Messages
7,413
Location
Chicagoland, Illinois, USA
I was also thinking on that idea too. Since many years ago I'm aware of a mod player that works on the 8088, MOD-OBJ, by Mark J. Cox. But it's too slow to be usable at 4,77 Mhz (even 8 Mhz), the source code has never been released and it weights 17 Kb, too much for my project.

Your memory is slightly muddled; Mark Cox's playback code relies on a MUL per sample, and no 8088-class system's MUL is fast enough. His code was meant for 286s and later. A 10 MHz NEC V30 might be acceptable.

His code was early, so it shouldn't be judged too harshly; modplay itself was a nice total package including nice displays, but his techniques have been far surpassed in the current state of the art (mod master XT).

Also the 8088Mph Mod player source code seems to be available https://github.com/reenigne/reenigne/blob/master/8088/demo/credits/mod.asm but I don't want to make things even more complicated so I abandoned the idea of playing MOD files.

It also has limitations; it is not a general-purpose modplayer, for example.

Out of curiosity, did you use some kind of audio compression for 8088 Corruption and 8088 Domination? Thanks!

Nope. Speed was the focus of both, and even realtime decompression techniques at even 4 instructions per sample (my current record, quality not great though) takes too much time. It would have meant a 24fps video would have played at 15fps or lower and I was going for maximum framerate.
 

xaby

New Member
Joined
Jul 5, 2022
Messages
1
Location
Berlin
Don't know, if this thread is still in tact.
Running on a C64, 1 MHz plays back music and video
I found this:

And another thing is, I was using back in the days http://www.bluemoon.ee/history/scdos/index.html
Sound Club for DOS. That was able to re-play music and sound on a PC-speaker / quaker on a 80286 with only 512 kByte memory and a VGA GPU.

But I guess, you are aiming for even lower hardware. But replaying MOD is not a bad idea. C64, NES and some other low-end consoles also try that.
I my self did some ADPCM tests, but not on real-hardware.

I also don't know, how much performance it would cost, to use Bink Audio.

I guess, if you get better than this, all is fine :)
 

Chartreuse

Member
Joined
Feb 22, 2013
Messages
22
Location
Calgary, AB, Canada
Don't know, if this thread is still in tact.
Running on a C64, 1 MHz plays back music and video
I found this:

And another thing is, I was using back in the days http://www.bluemoon.ee/history/scdos/index.html
Sound Club for DOS. That was able to re-play music and sound on a PC-speaker / quaker on a 80286 with only 512 kByte memory and a VGA GPU.

But I guess, you are aiming for even lower hardware. But replaying MOD is not a bad idea. C64, NES and some other low-end consoles also try that.
I my self did some ADPCM tests, but not on real-hardware.

I also don't know, how much performance it would cost, to use Bink Audio.

I guess, if you get better than this, all is fine :)
Algorithms like the one used in that C64 demo are quite neat. They're not something that would have been practical when the machines were new though as they require a HUGE amount of computing power to do the initial compression but very little to decompress. A codec like that could probably be used with a Adlib as you have a large number of waveforms to play with with the FM chip, so you could set up the model to use a selection of say 9 waveforms that were chosen for best performance (or even change the waveforms on the fly for each sample).

It's kind of doing a FFT but instead of layering sine waves it's layering arbitrary waveforms to best approximate the original data.
Either way it takes a lot of computing power to create the data in the first place so could only be used for pre-recorded playback. Though that's not a terrible issue for games or what not. The quality isn't super high but can be encoded to very low bitrates for the quality.
 
Top