Doom3[CC]: Current state and what still needs to be done

[DanielGibson]

Ok, the current state compiles and kinda works..
You can find it in this repository in the doom3cc branch

Still TODO (likely incomplete): likely only d3cclib/* needs to be modified, and the integration into the rest of the gamecode code is complete - but of course it's no problem if we need to do additional changes to the gamecode after all:

• Make language selection work, I think ccBst::initLanguages() in d3cclib/CCBst.cpp was supposed to do that, but it's not called from anywhere in the code I got from Víctor - likely it was still WIP or he didn't get the final code?
• memory leak: ccBst::Tree is not deleted completely (only the root node, not the rest of the tree)
• several memory leaks, see all the TODOs and FIXMEs I added to the code
• Resurrection of Evil (d3xp) support - easy enough, we only need to replicate the changes done in game/ there and do like two small additions/modifications to the cmake file
• English translations that are missing and for RoE
• Document how to integrate D3CC into other mods; there is a PDF describing the process but there's at least one additional thing that I added (I can do this part, should not take me too long)
• On/Off option (CVar, maybe setting in menu) for the radar in the HUD
• proper sub appearances or proper sub line counting (see Doom3[CC]: Current state and what still needs to be done #12 (comment))
• Separate GUI for subtitles (instead of in HUD), see same comment as before
• Subtitles canceling: When playing audio logs or videodiscs from within the PDA, stopping the playback does not stop the subtitles playback.
• Enhancing subtitle priority system somehow, let user choose what to see (e.g. no SFX descriptions), see Doom3[CC]: Current state and what still needs to be done #12 (comment) and following
• Something's wrong with the timings, should be fixed if possible (see Doom3[CC]: Current state and what still needs to be done #12 (comment))
• Animation speed for scrolling captions, depending on a mix of duration and text length (see Doom3[CC]: Current state and what still needs to be done #12 (comment))
• Some way to find out (in mod DLL) how many lines a string will need when rendered (see Doom3[CC]: Current state and what still needs to be done #12 (comment))
• Once it all works, port it back to the original Doom3 SDK so it also works with vanilla Doom3 1.3.1
  (I guess this is just diffing this branch against the dhewm3 SDK's master branch and then applying the resulting patch to the Doom3 SDK and fix the minor merge issues that might occur, shouldn't be too hard, we have no changes here that are dhewm3 specific)

[DanielGibson]

I just realized I had screwed up the line endings of the files in d3cclib/ and I found some uncommitted changes I did.
So I fixed the line endings and added the uncommitted stuff and I also prepared CMakeLists.txt for d3xp/ support, then I force pushed it to the doom3cc branch - sorry about that, if you had the branch checked out pulling might give you an error, in that case do:

• (if you have uncomitted changes yourself) git stash
• git fetch
• git reset --hard origin/doom3cc (if the git remote for this repo is origin)
• (if you had uncommitted changes, to get them back) git stash pop

The uncommitted changes included calling initLanguages() at some point that seemed to make sense, but I didn't test at all if language selected works now (and I'm not 100% sure how it's supposed to be used etc)

[IlDucci]

Okay, related to stuff that needs to be done (in my opinion):

• Radar Toggle: Folks that used the mod for subtitles only didn't liked the radar. In fact, the outsider guy who did the 1.3.1 quick patch for CC disabled it in the GUI file. So I'm thinking on adding a Radar On/Off option. I've tried to add the feature in the main menu and the main game GUIs, but this needs a new variable to be coded in.
• English texts that are missing: Right now, all the voices for the main game and RoE are transcribed. What's left is describing a lot of SFX that got untranscribed before, and then start adding the priority values before testing.
• Proper subtitle appearances: While doing some tests in Mars City Underground and Mars City 2, I've found out that subs that should be max priority are not displayed, which are the soldier voices that are heard beyond walls (combat_chatter). I have the theory that those audios are played from a source outside the map, and thus, the real distance is not being used. Either that or there's too much SFX that de-prioritize this.
• Proper character counting: The mod seems to use a hardcoded arbitrary value to measure out how many characters a subtitle line has in order to load the proper scrolling animation. This is a very hit-or-miss feature that doesn't do much well, but I'm fearing that the main way to figure out the amount of lines a subtitle has, or the amount of line jumps being added, is something done in the executable side, not on the DLL side.
• Decoupling the subtitle GUI from the rest of the ingame interface: The old team did a very simple trick to keep subtitles on screen: Disable the code that hide the entirety of the HUD whenever there's cutscenes. Also, since the code is in the HUD gui and not in the PDA gui file, PDA audio logs and video logs do not display subtitles until going back into the game. I'm thinking that the subtitles should have an independent GUI file that should be displayed on its own (Except in the main menu and intro FMV)
• Subtitles canceling: When playing audio logs or videodiscs from within the PDA, stopping the playback does not stop the subtitles playback.

There's other bones I'd want to pick, but I think I have to split things between "needed to get this looking proper" and "stuff that could mean rewrites/back to drawing board".

[DanielGibson]

An option to enable/disable SFX transcriptions/descriptions might be desirable.
Like, maybe people want subtitles because they don't understand spoken English too well, but can hear the environment sounds etc just fine. (Or does this already exist?)

[IlDucci]

It exists already, but the system in place is one of the areas that might fall into "back into the drawing board":

The current system uses only priority values, from 0 to A (in text, converted to hexa via code). Apparently, the system just disables printing out any values that are not 0 to do the switch between subtitles only or full closed captioning.

Though, in areas where there's a high density of voice lines, mostly during the start of the invasion in MC_Undeground and MarsCity2, there might be the need to add a secondary voice value. The part where this goes "back to the drawing board" is inspired by Half-Life 2's subtitles, where the priority value and the SFX tag were two different things, if I recall correctly.

Here's a copy of the priority system's help for caption writers:

// The closed captioning system makes use of priorities to determine which sounds get captioned.
// They are very important so that we make sure NPC dialog isn't cut off unexpectedly.
// PRIORITY GUIDE
// 0. Dialog
// 1. Enemy's gun fire and noises (alert moans, pain sounds, death sounds)
// 2. Explosives rattling on ground (grenade banging on the floor after an enemy threw it)
// 3. Explosions
// 4. Enemy's reloading gun
// 5. Enemy's footsteps
// 6. Doors, elevators.
// 7. Player gunfire
// 8. Player reloading
// 9. Player footsteps
// A. World sounds (Computers beeping, birds chirping, wind howling...)

In any case, once the mod's good enough to be tested, both subtitles and full CC will need to be tested and taken into account.

[IlDucci]

I've checked the updated code and the language selector is working again. That also reminds me of another thing that must be done: Get the "continues" substring out of the code and into a string so it's localizable. It seems to be used to detect when a sound is being played long enough by detecting if the substring's been there long enough.

EDIT: I'm attaching the current version of the mod's assets: doom3cc.zip

I just saw another reason why subtitles for the PDA menu should be enabled somehow: In order to compensate the lack of subtitles there, the original team decided to add transcriptions to the ingame emails. This causes two problems:

• For foreign languages, an extra copy of the newpdas folder with the corresponding transcriptions must be done, a copy that doesn't have the language split features of the game, such as the strings or the voice folder.
• For everyone, it includes adding critical, spoiler-laden content into the player's own PDA (the videodisc contents, which also need the transcriptions) with no way of linking the obtainement of those discs and the corresponding "email".

[DanielGibson]

The part where this goes "back to the drawing board" is inspired by Half-Life 2's subtitles, where the priority value and the SFX tag were two different things, if I recall correctly.

Or we make the priorities go up to F so we have 5 more for dialogue (and just move the current priorities 1, 2, ... back by 5) - for this all subtitles would have to be modified of course, no idea how much work that'd be and if it would be easy enough to do this with a script or something for everything that currently has priority 1 or higher.
(Or maybe instead of 5 more for dialogue just 2 more for dialogue and 3 for something else if needed, whatever)

Thanks for uploading the current gamedata, with this everyone who's interested in helping should be able to test the mod and maybe try to fix something :)
To test do the following:

• Check out this git repo and switch to the doom3cc branch
• Build it (see instructions in the readme)
• Copy doom3cc.dll (or doom3cc.so on Linux) next to your dhewm3 executable, where base.dll/so and d3xp.dll/so also lie
• Create a doom3cc/ directory, next to your base/ directory (that contains pak000.pk4 etc)
• Unzip doom3cc.zip (linked in @IlDucci's post right above this one) into that directory
• Start the game with ./dhewm3 +set fs_game doom3cc

[IlDucci]

By extending the system, I think only one or two extra values for dialogue should be enough. Changing the values in the dccs should be simple with a Search & Replace (for scripting, that's something that's out of my league).

[DanielGibson]

I documented in HowToIntegrateIntoMods.md how to integrate Doom3CC into (the C++ code) of other mods.
I also shortly described the purpose of each step, which might help understanding the API of ccBST a bit.

[IlDucci]

I have found a couple of additional possible issues:

• I have done a complete overhaul of the captions' sizes and timings. I've used an external tool, Aegisub, and then ported by hand the timings to make sure they were absolutely 1:1. Well, every caption that uses the timecode feature seems to display the captions earlier than they should (maybe a 1/10th of a second? I'd need to record this on video to give an exact value). Timing code should be verified to be 1:1.
• I have attempted to learn about how text is scrolled in CC mod. There's two points of contention here:
  • Seems like the captions are scrolled in the GUI by using predefined animations with a predefined, single speed. This means any caption, timecoded or not, that has a bit too much text in a small timing space might not display all the text properly. I have shortened the captions to avoid long walls of text, but this thing is still showing up. Maybe we could add two speeds of animation, depending on a mix of duration and text length?
  • Another thing that requires research is if the text rendering feature can be hooked from inside the DLL, or if the code can be ported from the executable. It seems that the mod decides the length of this string, and thus, how many lines are scrolled (selecting a specific scroll animation) by using arbitrary numbers for the amount of characters per caption, marked in the comments as "educated guesses". These numbers won't always work, as they don't account for actual character sizes, and while the best way to do this would be finding out how many lines a string has via the text rendering... is that actually feasible?

[DanielGibson]

Well, every caption that uses the timecode feature seems to display the captions earlier than they should (maybe a 1/10th of a second?

I debugged another issue and it turned out that between gamecode starting a sound and OpenAL actually being told to play it there can be a delay from up to about 100 milliseconds.
However, the delay isn't fixed, sometimes it's just 1ms, sometimes it's 100, often in between..
(dhewm/dhewm3#141 (comment) has the details)

[IlDucci]

Very interesting. Is OpenAL also used in the original EXE for the game? Because I just went there to check and the desync is, indeed, still existing. Just to rule out if it's an original game "bug".

[DanielGibson]

OpenAL support in the original game was optional.
However, this bug also exists in the other sound backends (that have been removed in dhewm3).

[DanielGibson]

Can you check if the delays get better with dhewm/dhewm3#296 (and if you notice any other new sound issues)?

[IlDucci]

I did just a quick check on timings (Loaded the first level, checked the synchronization of the first part of the cutscene, then on the video displayed on the kiosk and then on the Marine Command Report) and they don't seem to have changed their desync. Haven't played enough to hear any sound issues.

[DanielGibson]

So the sound still plays about 0.1s after the subtitle is shown?
Could be OpenAL itself adds an additional delay..

Thanks for testing! :)