Bad Apple CD+G on a karaoke machine

12 hours ago 1

In 2023 I got my first CD Graphics disc. These are standard audio CDs, but they also have graphics data in space that otherwise goes unused. I was drawn to the idea of a hidden unobtrusive bonus for those who know where to look, so since then I've been digging into the technical details and collecting every CD+G disc I can find.

Soon I hit on the idea of using it for animation, specifically the famous Bad Apple!! music video, common fodder for demos on underpowered hardware. CD+G mostly fell into a niche of displaying karaoke lyrics, and I thought it'd be fun to show the video running on a karaoke machine. I got that working well enough in November 2024, but I wanted to present it with a bit of background and explanation of the techniques. Bear with me and I'll try to avoid waxing encyclopedic. (Or skip ahead to the video if you prefer.)

Drawing less, less often

This project was a challenge because CD+G is very much not designed for animation. There's bandwidth for only 300 commands per second, which take effect immediately, and if a slot isn't used for a command it's wasted. The visible display is 48 x 16 blocks, and each of those takes at least one command to write, so if you were to draw a full screen image it would take at least 2.5 seconds. [aside-scrolling]

If we want to update the display at animation rates we'll need to cut some corners. The main compromise is to use a comically low resolution, I'm rendering only 11 x 4 blocks, 66 x 48 pixels. But even updating that small part of the screen in monochrome can only happen at 6.8 FPS. A fundamental animation technique is to only update the parts of the image that change from one frame to the next; a lot of the image is background or within solid-colored figures. Only making updates for changed blocks runs at an average 17 FPS. It increases to 20 FPS if we allow some jitter, starting the next frame a little early if there's time after the previous one.

But there's still a major problem: Even if we were able to update at the full 30 FPS of the source video, it still often takes a noticeable amount of time to update the image, and that whole time the frame is being displayed. If we could hold for a bit on the finished image it might not be too bad, but to hit that framerate we need to be constantly updating. It becomes a flickering mess. We want to hold on one frame while we're drawing the next one; we want to double buffer. However CD+G only has the one buffer… [CD+EG]

Invisible updates

Let's talk about colors. Each pixel is 4 bits, CD+G has a 16 color lookup table (CLUT, a.k.a. palette) that gives the RGB value to use for each color index. We can freely update the CLUT, it only takes two commands to set all 16 entries, and that will affect the whole screen. This allows us to use some very powerful techniques, all of which are based on setting multiple entries to the same color.

A common effect is to set all the CLUT entries to the same RGB value (e.g. black), draw an image (however long that takes), and then make rapid updates to the CLUT to transition the colors from black to the proper value for the image. This hides the initial drawing and creates a fade-in effect, often this is used to make a good first impression with a fancy title screen. The downside is that nothing is displayed on screen while the image is being drawn invisibly.

If we're dealing with an image where we don't need 16 distinct colors, we can do some more tricks. Each of the 4 lines of text in my Bad Apple!! demo is drawn with a different color index, though they are all only displayed in white. The CLUT has only 1 of those colors set to white at a time, the other 3 colors are set to the same black as the background, thus rendering them invisible while they're being drawn. When it's time to display the next line of text I issue a single CLUT command, changing which line is white.

Creating a double buffer

The text trick only works easily because I have the text lines draw at non-overlapping locations. For our double buffered movie we want to reuse the same location in the middle of the screen. This requires a different technique: bitplane extraction. [bitplane-term]

The color index of each pixel will be set depending on the combination of colors in two independent images, call them A and B:

image
Aimage
BCLUT
indexbinary
blackblack000
blackwhite101
whiteblack210
whitewhite311

We can choose which image is displayed just by setting the CLUT. To display image A we set the colors as in the image A column: If image A is black at a pixel it could have value either 0 or 1, depending on B; it won't matter which because they both are displayed as black. We could switch to displaying image B by setting the CLUT to the colors in the B column.

So image A is displayed, and we want to update image B. I mentioned that it takes at least one command to draw a block of graphics. An unusual feature of CD+G is that the command to set a block can use any pattern, but only two colors. If you want to have more than two colors in one block you need to issue a series of XOR commands, which allows flipping bits of the color index in certain pixels. Switching the color of image B is just an XOR with 1, that toggles between 0 and 1 (both A black), or between 2 and 3 (both A white). This XOR is the same regardless of what color is in A, we just need to specify what pixels we want to toggle in image B. [aside-corruption]

So, now we're double buffering! This has a subtle impact on framerate. We're updating two buffers, but we only need to update each for half the frames, and we're still just using one command per block update. But one buffer has the even frames and the other has the odd frames, so they're further apart in time; the separate buffers each need to change more on each update because generally more distant frames have less in common. In this case we go from 20 to 18.5 FPS.

Final touches

I ended up using triple buffering. When I tested the CD+G in the ares Mega CD emulator it seemed like the palette change didn't take effect immediately, so drawing the next frame was often visible. With three buffers there can be one frame drawing and the previous two possibly displaying. This supports a wider range of player implementations because it doesn't matter exactly when the display switch happens between the two complete frames. The bitplane XOR update works the same as with two buffers, and the 8 colors needed can all be updated by a single CLUT command. The only downside is this further increases the distance between consecutive frames in a buffer, making the partial updates slower; the average drops to 17.3 FPS.

Finally, I added in the lyrics. These work more like subtitles than karaoke lyrics, but I think it gives a good impression. Mainly the lyrics are loaded when there is bandwidth left over from the animation. I split it across 4 lines because it needed to have 3 lines saved up; there's little bandwidth left over during busy scenes, but the words keep coming! Even with that pre-buffered it needs to sometimes preempt the animation, stealing command slots from frames that otherwise would render the fastest. This puts the final average at 16.3 FPS.

In motion!

Here it is, I hope it's a little more interesting with that background!


Also check out the rainbow version which uses unique colors for all the color indexes, this shows all the stuff going on behind the scenes. [bonus]

Resources

  • bad-apple-cdg-mk7 is the latest version of the .cdg file I produced
  • CD+G is part of the CD Audio spec, where it's called TV-Graphics mode. It's easiest to find this as the international standard IEC 60908.
  • cdgraphics is a fast JavaScript CD+G renderer with high accuracy
  • CD+G disc info


Read Entire Article