sqpat
Experienced Member
RealDOOM is a port of vanilla Doom (forked from PCDoomv2, to be specific) made to run in Real Mode. (Coincidentally, another project named Doom8088 was being worked on at the same time, with a similar goal but a different starting point.)
(rendered from a 16 bit program! photo representative of actual FPS!)
You obviously cant just take the original source and build it to 16 bit. Development was mostly done by continuing to build for 32-bit but with more and more 16-bit style restrictions and code. A few weeks back I was able to build the binary for 16-bit, but it crashed in initialization. Fast forward to this past week, and16-bit binary was stable enough to make it out of initialization and run for a bit in text mode. Over the last couple days I fixed several bugs in the video code, and now the game kind of runs in DOSBOX. Bugfixing continues, though. The main task involved moving the program's heap to run off of EMS.
I have been working on this for over 3 months, and now I decided to finally announce the project since it "sort-of" running properly in 16-bit mode. Timedemos (playback recordings) diverge after a couple hundred frames, so there's definitely still bugs and this is still what I would call an 'alpha' release. Honestly I wanted a bit more polish on this before I posted about it, but I'm going to be traveling for a month very soon wanted to post this now as I might not be able to put as much time into it until I'm back.
Below is mostly technical details especially for those knowledgeable about or interested in the DOOM engine.
Technical Details - What was done
As a note - going into this project I'd never written anything for x86 assembly (and still haven't) and had never written 16 bit DOS software. It turned out that this has entirely been a C project so far.
DOOM allocates a huge block of memory at startup then runs its own allocation scheme within the big block of memory in z_zone.c This was rewritten to use EMS and return "MEMREFs" instead of pointers. MEMREFs can be used to get a pointer from the memory manager, which internally keeps track of which pages every variable is in. As references to anything in the EMS heap go stale quickly, MEMREFs get stored and passed around instead of pointers. There is a lot of juggling going on to support 4 page frames and 64k of active EMS memory. If you build the project for 32-bit, there is still a big allocation done at startup and the EMS support is emulated and you can "cheat" and have dozens of page frames.
Aside from that - lots of changing of variables from 32 bit to 8 or 16 bit when possible... lots of removal of 'minor' features and dead code... really, too much to remember. FastDOOM's early commit history was helpful.
RealDOOM's Goal
The goal is accuracy first. RealDOOM is meant to be able to run with the same graphical fidelity and engine accuracy as the original DOOM. It should support the WADs released by id software. There are some limits to things like texture sizes, map sizes, things like that - so not every custom WAD out there will work - but it should support the original id software content. Timedemos should also play back accurately.
About Speed and Performance
It's very slow now. Right now, dosbox running the 16-bit executable on my Ryzen 3950x probably gets 0.3-0.5 FPS or so with full detail. At some point when the engine is as stable in 16-bit as it is in 32-bit, I'll probably go ham with optimizations. I think 20x or so speed improvements are pretty straightforward - there's a lot of potentially easy stuff to do (see below). I doubt this will be running playable speeds on a 286 with the same level of detail as the original game, but we can always strive for that. Things like lower resolution textures and potato level detail and such can be considered, or optimizations based around fixed smaller resolutions.
Performance issues for this port have less to do with how many pixels there are to draw and a lot more to do with memory swapping. It's really not just a matter of reducing the view size and drawing lower quality. Most of the speed improvements will come down to making fewer page switches. In the worst case, hundreds or thousands of EMS page switches are being done per frame. I recently added a conventional memory allocator using whatever conventional memory is left at initialization for certain heavily used variables (lines, nodes, etc) , which helps a ton. As the binary size keeps decreasing and more memory is made available, more and more fits in conventional and doesn't have to be paged in and out. There's 20k or so unused bytes in the default data segment too, which could be used for something - I just haven't done it yet. If page frames are increased from 4 to 8 or 10 (Using c800-EFFF) then things will run even faster. Meanwhile, there's no assembly in there yet either. I'm noticing that the 32 bit and 16 bit versions respond differently to different optimizations, so as the 16-bit version becomes more stable there will be a lot more to try.
A note: on real hardware with a 286, hardware EMS support will be pretty important. (Imagine a poor 286 trying to copy 16KB back and forth thousands of times per frame!) I have a 286 that I've run over 35 mhz before that I would like to run this on at some point... but that's a story for another time.
Project Status
Known (Major) Issues
- Savegames dont work
- No sound (need to find a 16 bit compatible library or write from scratch?)
- Multiplayer support removed
- Joystick support removed
- Untested outside of doom1 shareware for now.
- 16 bit mode is a bit of a bugfest still
Planned Memory Optimizations
- EMS 4.0 style support for more page frames
- Use of the default data segment - there is over 20k free in there mostly unused. Maybe map objects can all be put in there, or blocklists, or something.
- Continued optimization of binary size + better use of conventional memory to reduce swaps
Planned Assembly Optimizations
- Rewriting FixedMul and FixedDiv in assembly. We are doing thousands of 32 and 64 bit integer multiplication/division calls per gametic/frame... I'm sure this is a huge performance drain. If they can be made any faster in ASM it should help a lot
- R_DrawColumn, etc in assembly
- Other math functions in assembly (eg multiplying by constants, multiplying 16 bit by 32 bit, multiplying 16 bit by 16 bit into 32 bit, etc)
Planned Feature Optimizations
- At some point I'd like to add some flags for no textures and flats and/or potato quality
- Hybrid EMS/conventional visplanes solution. I have each working independently, but EMS visplanes are notably slow. Maybe 48 or so can be in conventional, and then anything beyond that in EMS, so that the rest of that conventional memory can be used for better purposes.
Well, that's the current state of things.. as I noted a little earlier, there are still stability issues but its almost there.
I also want to shout out Viti95, who wrote FastDOOM and contributed a couple of optimizations to RealDOOM as well. If anyone else wants to contribute in some way I'd very much appreciate it. Especially anyone with 8088/286 assembly experience who could knock out assembly versions of those math functions.
(rendered from a 16 bit program! photo representative of actual FPS!)
You obviously cant just take the original source and build it to 16 bit. Development was mostly done by continuing to build for 32-bit but with more and more 16-bit style restrictions and code. A few weeks back I was able to build the binary for 16-bit, but it crashed in initialization. Fast forward to this past week, and16-bit binary was stable enough to make it out of initialization and run for a bit in text mode. Over the last couple days I fixed several bugs in the video code, and now the game kind of runs in DOSBOX. Bugfixing continues, though. The main task involved moving the program's heap to run off of EMS.
I have been working on this for over 3 months, and now I decided to finally announce the project since it "sort-of" running properly in 16-bit mode. Timedemos (playback recordings) diverge after a couple hundred frames, so there's definitely still bugs and this is still what I would call an 'alpha' release. Honestly I wanted a bit more polish on this before I posted about it, but I'm going to be traveling for a month very soon wanted to post this now as I might not be able to put as much time into it until I'm back.
Below is mostly technical details especially for those knowledgeable about or interested in the DOOM engine.
Technical Details - What was done
As a note - going into this project I'd never written anything for x86 assembly (and still haven't) and had never written 16 bit DOS software. It turned out that this has entirely been a C project so far.
DOOM allocates a huge block of memory at startup then runs its own allocation scheme within the big block of memory in z_zone.c This was rewritten to use EMS and return "MEMREFs" instead of pointers. MEMREFs can be used to get a pointer from the memory manager, which internally keeps track of which pages every variable is in. As references to anything in the EMS heap go stale quickly, MEMREFs get stored and passed around instead of pointers. There is a lot of juggling going on to support 4 page frames and 64k of active EMS memory. If you build the project for 32-bit, there is still a big allocation done at startup and the EMS support is emulated and you can "cheat" and have dozens of page frames.
Aside from that - lots of changing of variables from 32 bit to 8 or 16 bit when possible... lots of removal of 'minor' features and dead code... really, too much to remember. FastDOOM's early commit history was helpful.
RealDOOM's Goal
The goal is accuracy first. RealDOOM is meant to be able to run with the same graphical fidelity and engine accuracy as the original DOOM. It should support the WADs released by id software. There are some limits to things like texture sizes, map sizes, things like that - so not every custom WAD out there will work - but it should support the original id software content. Timedemos should also play back accurately.
About Speed and Performance
It's very slow now. Right now, dosbox running the 16-bit executable on my Ryzen 3950x probably gets 0.3-0.5 FPS or so with full detail. At some point when the engine is as stable in 16-bit as it is in 32-bit, I'll probably go ham with optimizations. I think 20x or so speed improvements are pretty straightforward - there's a lot of potentially easy stuff to do (see below). I doubt this will be running playable speeds on a 286 with the same level of detail as the original game, but we can always strive for that. Things like lower resolution textures and potato level detail and such can be considered, or optimizations based around fixed smaller resolutions.
Performance issues for this port have less to do with how many pixels there are to draw and a lot more to do with memory swapping. It's really not just a matter of reducing the view size and drawing lower quality. Most of the speed improvements will come down to making fewer page switches. In the worst case, hundreds or thousands of EMS page switches are being done per frame. I recently added a conventional memory allocator using whatever conventional memory is left at initialization for certain heavily used variables (lines, nodes, etc) , which helps a ton. As the binary size keeps decreasing and more memory is made available, more and more fits in conventional and doesn't have to be paged in and out. There's 20k or so unused bytes in the default data segment too, which could be used for something - I just haven't done it yet. If page frames are increased from 4 to 8 or 10 (Using c800-EFFF) then things will run even faster. Meanwhile, there's no assembly in there yet either. I'm noticing that the 32 bit and 16 bit versions respond differently to different optimizations, so as the 16-bit version becomes more stable there will be a lot more to try.
A note: on real hardware with a 286, hardware EMS support will be pretty important. (Imagine a poor 286 trying to copy 16KB back and forth thousands of times per frame!) I have a 286 that I've run over 35 mhz before that I would like to run this on at some point... but that's a story for another time.
Project Status
Known (Major) Issues
- Savegames dont work
- No sound (need to find a 16 bit compatible library or write from scratch?)
- Multiplayer support removed
- Joystick support removed
- Untested outside of doom1 shareware for now.
- 16 bit mode is a bit of a bugfest still
Planned Memory Optimizations
- EMS 4.0 style support for more page frames
- Use of the default data segment - there is over 20k free in there mostly unused. Maybe map objects can all be put in there, or blocklists, or something.
- Continued optimization of binary size + better use of conventional memory to reduce swaps
Planned Assembly Optimizations
- Rewriting FixedMul and FixedDiv in assembly. We are doing thousands of 32 and 64 bit integer multiplication/division calls per gametic/frame... I'm sure this is a huge performance drain. If they can be made any faster in ASM it should help a lot
- R_DrawColumn, etc in assembly
- Other math functions in assembly (eg multiplying by constants, multiplying 16 bit by 32 bit, multiplying 16 bit by 16 bit into 32 bit, etc)
Planned Feature Optimizations
- At some point I'd like to add some flags for no textures and flats and/or potato quality
- Hybrid EMS/conventional visplanes solution. I have each working independently, but EMS visplanes are notably slow. Maybe 48 or so can be in conventional, and then anything beyond that in EMS, so that the rest of that conventional memory can be used for better purposes.
Well, that's the current state of things.. as I noted a little earlier, there are still stability issues but its almost there.
I also want to shout out Viti95, who wrote FastDOOM and contributed a couple of optimizations to RealDOOM as well. If anyone else wants to contribute in some way I'd very much appreciate it. Especially anyone with 8088/286 assembly experience who could knock out assembly versions of those math functions.
