In general, the life of a software developer is a pain. What makes a person a successful developer? Ability to go through the pain and do it persistently. Let me give you a recent example.

As part of the Director development, I prefer to work on a specific title. What makes me more motivated is if there is someone eagerly looking into a specific title to be supported. This is why I was happy to help even some of my patreons in the past, so if you are one of these, please feel free to drop me a line. And one of such titles for Director 6 is Master of Elements. Felsqualle, my fellow co-lead, is a particularly big fan of it and was wrestling to support the original under modern Windows.

It's a title full of advanced Lingo, and earlier I already reworked tempo settings for D6, thanks to its advanced use even in the intro. But then there was a small visual issue, demonstrated in the screenshot below:

Please note how the book does not seem to have proper keycolor transparency.

Let me explain a little bit what a keycolor is and how it is meant to work in the Director.

30 years ago, having lots of memory was an unreachable luxury for ordinary people. Only supercomputers could have gigabytes of RAM. Typical computers had it in megabytes. Normal configuration in 1997, the year when Director 6 was released, had merely 16MB of RAM (now I have 64GB on my laptop. That is FOUR THOUSAND times more).

So, as a result, every byte counted. And this is why the Director was showing how many kilobytes each cast member was using. This is the reason why, when possible, it was preferable to use 8-bit images, that is, images with 256 colors and a palette. In the modern world, everything is 32 bits, which is 4x more info per pixel. And those 32 bits contain information about color, RGB, and transparency, Alpha. As you can imagine, there was no transparency information in 8-bit images. This is why the concept of keycolor was invented.

Keycolor is a color, picked from the 256 colors, which is considered fully transparent; that is, when rendering, it leaves holes in place of those pixels. This is how the picture of a book above is constructed. Speaking of transparency, the hand cursor is also constructed with keycolor, but as you see, it is rendered properly.

In Director, when you put cast members on the stage, you can specify Ink, which is a method of drawing this cast member. Some examples of ink effects are Copy (copy it as is), Reverse (inverts the colors), Darkest (selects a darker pixel from the image and background), etc., totaling 18 types. And that includes Matte ink.

Here is the official description from the Director 3.1 manual:

Essentially, as you can see, the white color at the edges of an image is converted to transparency.

So, what is the problem with the book in the screenshot above?

The ink for the book is Copy.

In other words, there should not be any transparency, but it is there!

This was my finding on the first day when I was looking into the problem. And it took me 1.5 weeks to finally solve.

It should be noted, that Master of Elements ignores all the Director Score and manages the sprites exclusively from Lingo. That INTRO.DXR movie contained only one frame, and to make the animation, everything was moved around. All the cast members are initially put outside of the screen, with coordinates like 2000 pixels, and then are moved by the following Lingo:

As you see, first, it moves some sprites to the screen (values less than 2000), and then moves other ones outside of the screen to coordinates (2000, 2000). Then, it assigns to sprite 5, cast #61 from the cast library iassets. This is how the book is displayed.

I spent two days trying to find, is there any Lingo-related trick, which sets the ink from Copy to Matte. But there is nothing.

Then, I decided to try to launch the original and see what is going on there. For that, I needed to decompile DXRs into DIR with ProjectorRay, launch Director 6 under Wine, and check what was going on. Nothing. It was pure Copy link, which should have NO transparency!

Another day has passed, and I decided to create a minimal movie, containing only the book and nothing else. Here I was faced with the problem, my Wine-supported Director 6 just crashed on save! So, I had nothing to do, but fire up a 86box with Windows 95, install Director 6 there, and thus, I was able to work on creating the example movie.

And it was STILL COPY INK! But with clear transparency!

What made things worse, is that when I tried to add another copy of the same bitmap to the stage, that one had white corner!

This was the movie:

It contains one cast member, the book. And it is put twice on the stage. Both have Copy ink, but one had transparency and another one does not.

Then I realized, that Director 8 and above, had a nice feature. It can display all cast member properties in a list. That is, there is a way to see everything instead of trying to guess what is missing. I launched Director 10, loaded the movie in there, and here is the result in both condensed form and in the list:

And here is the info about the bitmap itself:

As you see, there are no traces for Matte. Also, Director has Mask Ink, when there is another 1-bit image coupled with the main one and it designates which pixels are transparent. That was not present either.

I was banging my head against the wall.

What made it even more bizarre, is that when I copied the book with transparency, I could paste it with transparency. But when I added it via normal methods, the corner was white. So, somehow Director knew that these images do differ.

Then, an idea struck my head. It was clear that in this movie, there is only one cast member. As such, all of the differences in info should be recorded in the Score data. I decided to create an animation on the same spot, in the same channel, thus, the Director could show me what was that difference between them.

I extended debug logs with comprehensive info on what is being loaded at each frame in ScummVM.

And here is the resulting log (I left out irrelevant info):

Yes, as you noticed, the difference between the images was that it was flipping between thicknesses of 16 and 0!

As it appears, Director 5 was a special quirk: when ink was Copy, but the fourth thickness bit, which normally means that the sprite has Blend, was forcing the ink to Matte!

That means, that I spent ten days on searching the difference of ONE BIT!

I quickly added the fix, tested, and now Master of Elements renders perfectly at this place.

There are further problems with this title, like Clover rendering is fully missing, or the jumping ball shadow in the Warmth screen sometimes shows weird artefacts, but those are yet to be solved.

These were my 1.5 weeks of pain.

For a number of years, ScummVM is using a free online static analysis tool, Coverity Scan (https://scan.coverity.com/projects/scummvm). Static analysis is a kind of tool which analyses the code without actually running it. Usually, it goes from simple pattern matching to complex simulations of the code path executions. Coverity is one of the most advanced ones for C++ and it can find bugs that otherwise would be relatively difficult to spot.

When I am kind of feeling stuck with the current projects, I go and fix a bug or two.

Yesterday I fixed, among others, these interesting beasts:

In startrek/sound.cpp file, there was the following declaration:

  byte *loadedSoundData;

And then later in the code:

  loadedSoundData = new byte[soundStream->size()];

But later, when the actual music is requested to be loaded, the following construct is used:

  _midiSlots[slot].midiParser->loadMusic(loadedSoundData, sizeof(loadedSoundData));

What is interesting here, is that "sizeof()" call. Despite the immediate logic of a programmer who wrote this line in 2018, it was not doing what is intended. The thing is that the loadedSoundData variable contains just a pointer to the allocated memory chunk, and thus, its size is the size of a pointer, 8 bytes on a 64-bit system. Thus, we were telling the midiParser that all of our music is just that small.

This was part of refactoring and perhaps was not tested well enough. The engine has been in limbo for a while and hasn't undergone public testing.

Another similar error was in our recent rewrite of dIMUSE for FT/Dig and COMI.

In one particular place there was the following call:

  Common::strlcpy(_textBuffer, marker, sizeof(marker));

Of course, sizeof(marker) is the same size of a memory pointer. I fixed it with

  Common::strlcpy(_textBuffer, marker, sizeof(_textBuffer));

"_textBuffer" in this case is defined as "char textBuffer[256]" and as such, its size is determined correctly, while "marker" is passed as "char *marker" in one of the parameters.

In general, the life of a software developer is a pain. What makes a person a successful developer? Ability to go through the pain and do it persistently. Let me give you a recent example.

As part of the Director development, I prefer to work on a specific title. What makes me more motivated is if there is someone eagerly looking into a specific title to be supported. This is why I was happy to help even some of my patreons in the past, so if you are one of these, please feel free to drop me a line. And one of such titles for Director 6 is Master of Elements. Felsqualle, my fellow co-lead, is a particularly big fan of it and was wrestling to support the original under modern Windows.

It's a title full of advanced Lingo, and earlier I already reworked tempo settings for D6, thanks to its advanced use even in the intro. But then there was a small visual issue, demonstrated in the screenshot below:

Please note how the book does not seem to have proper keycolor transparency.

Let me explain a little bit what a keycolor is and how it is meant to work in the Director.

30 years ago, having lots of memory was an unreachable luxury for ordinary people. Only supercomputers could have gigabytes of RAM. Typical computers had it in megabytes. Normal configuration in 1997, the year when Director 6 was released, had merely 16MB of RAM (now I have 64GB on my laptop. That is FOUR THOUSAND times more).

So, as a result, every byte counted. And this is why the Director was showing how many kilobytes each cast member was using. This is the reason why, when possible, it was preferable to use 8-bit images, that is, images with 256 colors and a palette. In the modern world, everything is 32 bits, which is 4x more info per pixel. And those 32 bits contain information about color, RGB, and transparency, Alpha. As you can imagine, there was no transparency information in 8-bit images. This is why the concept of keycolor was invented.

Keycolor is a color, picked from the 256 colors, which is considered fully transparent; that is, when rendering, it leaves holes in place of those pixels. This is how the picture of a book above is constructed. Speaking of transparency, the hand cursor is also constructed with keycolor, but as you see, it is rendered properly.

In Director, when you put cast members on the stage, you can specify Ink, which is a method of drawing this cast member. Some examples of ink effects are Copy (copy it as is), Reverse (inverts the colors), Darkest (selects a darker pixel from the image and background), etc., totaling 18 types. And that includes Matte ink.

Here is the official description from the Director 3.1 manual:

Essentially, as you can see, the white color at the edges of an image is converted to transparency.

So, what is the problem with the book in the screenshot above?

The ink for the book is Copy.

In other words, there should not be any transparency, but it is there!

This was my finding on the first day when I was looking into the problem. And it took me 1.5 weeks to finally solve.

It should be noted, that Master of Elements ignores all the Director Score and manages the sprites exclusively from Lingo. That INTRO.DXR movie contained only one frame, and to make the animation, everything was moved around. All the cast members are initially put outside of the screen, with coordinates like 2000 pixels, and then are moved by the following Lingo:

As you see, first, it moves some sprites to the screen (values less than 2000), and then moves other ones outside of the screen to coordinates (2000, 2000). Then, it assigns to sprite 5, cast #61 from the cast library iassets. This is how the book is displayed.

I spent two days trying to find, is there any Lingo-related trick, which sets the ink from Copy to Matte. But there is nothing.

Then, I decided to try to launch the original and see what is going on there. For that, I needed to decompile DXRs into DIR with ProjectorRay, launch Director 6 under Wine, and check what was going on. Nothing. It was pure Copy link, which should have NO transparency!

Another day has passed, and I decided to create a minimal movie, containing only the book and nothing else. Here I was faced with the problem, my Wine-supported Director 6 just crashed on save! So, I had nothing to do, but fire up a 86box with Windows 95, install Director 6 there, and thus, I was able to work on creating the example movie.

And it was STILL COPY INK! But with clear transparency!

What made things worse, is that when I tried to add another copy of the same bitmap to the stage, that one had white corner!

This was the movie:

It contains one cast member, the book. And it is put twice on the stage. Both have Copy ink, but one had transparency and another one does not.

Then I realized, that Director 8 and above, had a nice feature. It can display all cast member properties in a list. That is, there is a way to see everything instead of trying to guess what is missing. I launched Director 10, loaded the movie in there, and here is the result in both condensed form and in the list:

And here is the info about the bitmap itself:

As you see, there are no traces for Matte. Also, Director has Mask Ink, when there is another 1-bit image coupled with the main one and it designates which pixels are transparent. That was not present either.

I was banging my head against the wall.

What made it even more bizarre, is that when I copied the book with transparency, I could paste it with transparency. But when I added it via normal methods, the corner was white. So, somehow Director knew that these images do differ.

Then, an idea struck my head. It was clear that in this movie, there is only one cast member. As such, all of the differences in info should be recorded in the Score data. I decided to create an animation on the same spot, in the same channel, thus, the Director could show me what was that difference between them.

I extended debug logs with comprehensive info on what is being loaded at each frame in ScummVM.

And here is the resulting log (I left out irrelevant info):

Yes, as you noticed, the difference between the images was that it was flipping between thicknesses of 16 and 0!

As it appears, Director 5 was a special quirk: when ink was Copy, but the fourth thickness bit, which normally means that the sprite has Blend, was forcing the ink to Matte!

That means, that I spent ten days on searching the difference of ONE BIT!

I quickly added the fix, tested, and now Master of Elements renders perfectly at this place.

There are further problems with this title, like Clover rendering is fully missing, or the jumping ball shadow in the Warmth screen sometimes shows weird artefacts, but those are yet to be solved.

These were my 1.5 weeks of pain.

I was quite busy with the project in recent months, and that led to a breakthrough in QT Virtual Reality 1.0 rendering.

This is an old and now obsolete technology (and ScummVM is all about old and obsolete) to the point that there is no way besides running old copies of Windows or MacOS on an emulator or 20-year-old hardware. It was officially dropped after QuickTime 7, e.g., in 2005. Moreover, there were two major versions of QTVR, and version 1.0, released in Jan 1995, was pretty quickly surpassed by QTVR 2.0 in May 1996.

The QTVR has two major types of movies: panoramas and object movies. Panoramas are typically 360-degree vertical cylindrical projections (in QTVR 1.0) and cube and horizontal cylindrical projections in QTVR 2.0. And Safecracker used both. You may see on the screenshot above the phone booth in the main window, which is a panorama, and there is a banana which is an object movie. You may rotate objects across all angles freely, which simulates a 3D look.

The problem with that is that practically all scarce documentation on the internet explains format 2.0, and we had to basically figure out the 1.0 format by ourselves.

The whole quest started with Roland van Laar's desire to support Safecracker. He joined the team around 5 years ago with ambitions to make the game run normally on modern OSes. As part of his exploratory efforts, two years ago, he implemented a Python-based renderer for panoramas and figured out how to render object movies. His panorama viewer, however, cheated a bit in the rendering part, as it simply used OpenGL.

The QTVR 1.0 used vertical projection for its panoramic movies. There is an associated patent (quite hard to read) describing their 1-dimensional caching mechanism that allowed them to project these panoramas on machines with 16 MHz CPUs in real-time. So, Roland's code skipped all that jazz and simply instantiated an OpenGL cylinder, slapped texture on it, and left all the rendering details to your GPU.

Of course, that was not very helpful since I did not want to add OpenGL to the Director as a dependency, so we had to implement a software renderer.

Last year, we had a GSoC student who attempted to add the renderer. He started his bumpy road by extending our QuickTime parser and loading some of the atoms (data chunks). Then he implemented an object viewer, since that is relatively simple, the object movies are a matrix of all pre-rendered views and you just switch between the frames. Unfortunately, the student struggled so much with panoramas that he stalled completely, and we failed him from the GSoC.

Thus, we were left with half-working code for object movies and very little code related to panoramas. This is where we were standing till January of this year. Then, I decided to bite the bullet and take a look into the matter.

So, after 2 months of work I have almost feature-complete QTVR renderer. And you can even browse and navigate an arbitrary movie with our 'testbed' engine. Then, I plugged it into Director, so far into two titles, "Safecracker" and "Star Trek: The Next Generation Interactive Technical Manual". The latter is one of the first Director titles using the technology, if not the first.

During the development, I had tremendous help from Eric Lasota, aka OneEightHundred, a developer who implemented vcruise engine. Thus, he is pretty familiar with the panoramas. I myself tried to follow the patent, but my rusty trigonometric calculus did not allow me to implement proper vertical tilting of the cylinders. I stuck after projecting the cylinder to a sphere, so the panoramas looked unnatural.

Both object and panorama movies are supported. However, there are a few things missing:

1. The QTVR player was using 177 (!) cursors for enhanced interactivity and intuitive hinting of possible action. I figured out most of them, except the family of triangle-shaped cursors, which look like directional ones, which you can see in the modern UI for flipping pages or moving between locations. I haven't found a QTVR 1.0 movie that made those cursors visible in the original QTVR player.

2. 'navg' type nodes. The documentation does not explain it well and I was not able to directly figure out what the player is doing with them. They have an associated structure, but it is not clear to me what makes them different from 'link' type nodes which I successfully implemented

3. Rendering quality settings, particularly skipping rendering passes and implementing anti-aliasing. A curious thing is that my earlier failed attempt to project the sphere to a plane is actually one of the quality settings.

Other than that, I'd say we now have the one and only open-source implementation of the QTVR 1.0 movies player.

My following plans regarding the renderer would be to complete the implementation; I already have ideas in regards to anti-aliasing; when we proceed further in safecracker, later in the game, there will be 'navg' nodes popping up, so we could study its behavior. And hopefully, we find an example with those triangle cursors.

Then, if we find a title that uses QTVR 2.0 movies, I will be curious to implement those as well.