It's been 6 long months since our last status update, but for good reason - we've been hard at work on plenty of new features, improvements and optimizations that we'd like to talk about today.

Compatibility Improvements

Over the past 6 months, there has been a huge focus on manually testing games, and using these test results to help guide development in order to substantially improve redream's compatibility.

To that end, emulator testing legend Literalmente_Game (with help from bigboo from our Discord) has been hard at work testing the entire Dreamcast library. Thanks to this, every single one of the Dreamcast's 650+ games have been manually tested in redream. Multiple months were spent fixing compatibility issues as they were reported during this testing, and the results were recorded and used to create our new compatibility list.

As it stands at the time of this writing, nearly 70% of games in the Dreamcast's library are playable from start to finish, with the full breakdown looking like:

The value provided by this testing can't be overstated - a huge thanks to everyone who's been involved.

Usability Improvements

User Interface

In October of last year, there was a push to get an initial user interface going to make the emulator available to a new audience of users. The goal was to make an interface that would work across all platforms as well as the upcoming Android app.

 

The result of this work was a controller-navigable, big picture style interface. The interface enables you to adjust most of redream's options, and to manage / run games from your library, but for now the interface is only accessible before launching a game. We're looking to soon provide a slimmed down variant that's accessible in-game.

BIOS Emulation Improvements

In the last progress report, we announced the debut of our HLE BIOS, which is a high-level emulation of the original BIOS intended to replace the need for having an original BIOS in order to run redream.

Several major fixes have went in to the HLE BIOS since then, but perhaps most importantly, the ability to boot CDI images was added. With these fixes and CDI support landing, the HLE BIOS is now only a handful of games away from being on par with the real thing.

Automatic Region Unlocking

The Dreamcast implemented region locking, which limited the use of games from regions outside of your own. The result being that PAL games often only ran on PAL machine, JAP games often only ran on JAP machines, etc.

The region lock check is now automatically patched at runtime such that any region game will run, regardless of the region of the BIOS you use.

Automatic Time Sync

During the Dreamcas'ts boot, if a large discrepency was detected between the real-time clock and the last boot time, the system time menu would pop up asking you to set the time.

Every Dreamcast owner with a dead CMOS battery is all-too-familiar with this menu.

 

While only a minor inconvenience, support has now been added to synchronize the Dreamcast's time with the local machine time on startup, eliminating this menu once and for all.

CHD Support Added

Thanks to Romain Tisserand's efforts, support for loading MAME's CHD (Compressed Hunks of Data) disc format landed in August.

This is often desirable for users who have a large game collection, because as its name implies, it is a compressed format which can drastically cut down on each game's total size. To illustrate, here's a look at both Skies of Arcadia and Sonic Adventure in the GDI and CHD formats:

size in bytes | filename
--------------|-------------------------------------
           87 | Skies of Arcadia (USA) (Disc 1).gdi
      1470000 | track01.bin
      3455088 | track02.raw
   1185760800 | track03.bin
--------------|-------------------------------------
    523960829 | Skies of Arcadia (USA) (Disc 1).chd
--------------|-------------------------------------
                CHD is 55% smaller
    

size in bytes | filename
--------------|-------------------------------------
           89 | Sonic Adventure (USA).gdi
     34254528 | track01.bin
      2147376 | track02.raw
   1185760800 | track03.bin
--------------|-------------------------------------
    913745140 | Sonic Adventure (USA).chd
--------------|-------------------------------------
                CHD is 25% smaller
    

CDI Support Revamped

In November the CDI code was revamped, and support was added for loading Mode 1 2336/2352 and Mode 2 2352 tracks. Before this, CDI support was often hit or miss, but now it's finally a first class disc format in redream.

CPU Improvements

While numerous minor changes have went into the CPU emulation as part of our compatibility push, one feature in particular deserves a special shout out.

Invalidate Self Modifying Code

When emulating the Dreamcast's SH4 CPU, the original SH4 code is dynamically recompiled into code for the host CPU, and the results of the compilation are cached and used again in the future if the same SH4 code is again ran.

What this means is that our emulated CPU effectively has an infinitely large instruction cache, where as the original SH4 CPU has a contextually small instruction cache.

For most games, their code is loaded into memory from disc and never touched again. This means it's always safe to reuse the cached compilation results, as the original code never changes.

For a small number of games, after the code has been loaded and ran, it is modified. In most cases, after the game modifies the code it explicitly clears the SH4's instruction cache. When the SH4's cache is cleared, we clear our own internal cache, causing the new code to be compiled and executed as expected when it is next encountered.

For a very small number of games, after the code has been modified, the games do not explicitly clear the SH4's instruction cache. These games instead rely on the SH4's small instruction cache being organically flushed of the old code by the time the new code is encountered. Due to our infinite cache size however, old code is never organically flushed, and the old code instead continues to be ran often resulting in a crash.

To fix this, each time a block of code is recompiled, a memory watch is setup to monitor the SH4 code used to compile the block. If this watch detects a modification to the SH4 code, the corresponding cached code is invalidated. With this in place, some very notable games are finally playable:

Marvel vs. Capcom 2 no longer crashes after the first match

Shenmue is now completely playable

Graphics Improvements

In early December, the v1.1.0 milestone was created, with the goal of implementing many of the missing features from our PowerVR emulation. The work for the milestone ended up really taking off, resulting in improvements well beyond what was originally scoped for the milestone.

High-Definition Rendering

One of the most requested features has finally landed for premium users.

The option is accessible under Options -> Video -> Internal Resolution and is used to control the resolution that the original Dreamcast video output is rendered at. By rendering at higher resolutions "internally" and then downsampling to the window's size, aliasing is drastically reduced and texture quality is substantially improved.

Click each of the below shots to see them scaled to 1920 x 1440 in order to get an idea of the difference:

Power Stone 2 at 640 x 480

Power Stone 2 at 5120 x 3840

Opaque Modifier Volumes

Support has landed for a feature the Dreamcast's PowerVR provided called "modifier volumes." These modifier volumes were user-defined 3D volumes that could be programmed to change how lighting was calculated for geometry that was either inside or outside of the volume.

The most common use of modifier volumes is to render shadows on the geometry they contain:

The raw modifier volumes in red.

For shadows, the modifier volumes are programmed to darken the geometry contained inside of them.

With this support finally landing, a large number of games feel much more immersive.

Raw Framebuffer Writes

When rendering 3D geometry, games on the Dreamcast primarily did so with the PowerVR's Tile Accelerator. In short, the Tile Accelerator takes in lists of geometry, rasterizes them and writes the output to video memory. Eventually this video memory is read back and output to the connected monitor.

However, when rendering 2D geometry (e.g. loading screens), many games skip using the Tile Accelerator and instead write directly to video memory.

Games that did this presented a problem. As an optimization, redream has always skipped the process of copying the rasterized output to and from emulated video memory. Instead, the 3D geometry would be rendered with OpenGL and directly output to the screen.

Due to this, any data that was directly written to the emulated video memory by games was never output. This caused the SEGA license screen to not be rendered on boot, many loading screens in games to just be black and most homebrew software to not render at all.

This was fixed by now monitoring the emulated video memory for changes, and if writes are detected, synchronizing the emulated video memory and OpenGL at that time.

Render To Texture Support

For the same reasons that raw framebuffer writes were unsupported, render to texture has also long been unsupported.

As mentioned above, when rendering 3D geometry the Tile Accelerator would take in lists of geometry, rasterize them, and copy the results to video memory. Normally, the location in video memory was the location of the primary or secondary framebuffer, but it was also possible to copy the results to an arbitrary location in the region of video memory meant for textures.

The use case for this is to render a scene, copy the output to texture memory, and then use that rendered scene as a texture in a subsequent frame. This is what Crazy Taxi (pictured above) does when rendering the pause menu. The last frame (before opening the menu) is rendered to a texture, and when the menu is drawn that texture is used as a background (and the main scene stops rendering). Many racing games also use this technique to render the rear and side mirrors, and games such as Tony Hawk use this to render shadows.

After setting up the plumbing for copying the OpenGL results back to the emulated video memory to support raw framebuffer writes, it was quick work to also support copying the OpenGL results back for render to texture requests.

Multipass Rendering Support

When games pass geometry to the Tile Accelerator, they do so in passes. Each pass can have different geometry sorting rules, and certain operations can be performed between each pass such as clearing the depth buffer.

Previously, these passes were entirely ignored and the geometry was instead merged together as one single pass. Support has now been added to group the geometry by pass, and to perform the inter-pass operations correctly.

Clipping Support

On the Dreamcast, games were able to specify custom rectangles that were used to mask geometry when rendering. These rectangles would either be inclusive, in which case only geometry contained inside of the rectangle was output, or exclusive, in which case only geometry contained outside of the rectangle was output.

Now that these clipping rectangles are supported, the menus of several games have been fixed, and rear-view mirrors in many driving games are now rendered correctly.

Translucent Polygon Fixes

The Dreamcast was known for its hardware level order-independent transparency support.

Because the hardware sorted translucent geometry at a per-pixel level, games themselves did not have to handle sorting the geometry before rendering it. This was great for game developers on the Dreamcast, but a thorn to emulate without taking a hefty performance hit.

Up until recently, redream would sort each triangle strip passed to the PowerVR based on the minimum Z value of any vertex in the strip. As shown in the above screenshot, this wasn't acceptable for most games with large amounts of translucent polygons.

While we want to soon offer an accurate but expensive option for emulating this at a per-pixel level, we also wanted to offer a better solution that remained performant and worked for the majority of games. To this end, we've swapped from sorting at the triangle strip level to sorting each individual triangle in each triangle strip.

Bumpmap Texture Support

Support for bumpmap textures has landed, turning games that used them such as Rayman into an entirely new experience.

Floating-Point Color Conversion Saturation Fixes

Some games specify vertex colors as floating-point values (as opposed to integers) that are typically in the range of [0.0, 1.0]. Before passing these to OpenGL, they are converted to integer values in the range of [0, 255]. When performing the conversion from floating-point to integer, the result was not being saturated, meaning that values less than 0.0 or greater than 1.0 were not being clamped to 0 or 255 respectively.

Specular Lighting Fixes

A small but important fix, some surfaces were incorrectly disabling contributions from specular light sources.

Where Development Is Going

The next milestone for v1.2.0 is just starting to get organized. Some of the current high-level tasks involve save states for premium users, getting the Android app out the door, support for WinCE games and expanding our new user interface.

Join us in our Discord server if you'd like to chat about it!