How to get OBS Studio to work on GNOME and Wayland

Wayland it is since Fedora 32 my reference graphical server. As I’ve said before, I spend countless hours doing just about everything with it: web browsing, LibreOffice writing, scheduling, video editing, marathon Amazon Prime Video sessions, image editing, streaming music, and gaming (yes , even Windows-exclusive AAA with Proton). I do everything from Wayland with a very satisfactory result, except the recording with OBS Studio, which I had to continue delegating to Xorg due to the lack of official support for Wayland.

Luckily in OBS Studio 27 The foundations have been laid to close the deficit that the application was dragging when it comes to supporting Wayland, and in fact, if one wants, one can easily test the current beta version of the application using Flatpak. However, when I tested it in January I saw that the frame rate was very inconsistent, at least when shooting at 60fps.

Two months later I decided to give the beta version of OBS Flatpak a try again, and the truth is that the improvement in the quality of the recording has surprised me for good. However, the initial enthusiasm quickly turned to disappointment when I saw the recording of the image freeze when the focus was on a full screen application. The funny thing about it is that OBS was still working correctly, so when I moved the focus away from the application or when I closed it, everything returned to normal.

I made the decision to report that incorrect behavior to the maintainers of the Flatpak version of OBS Studio, who are not the same people who are inside the hard core of the application, although George Stavracas, the same one who is behind the official support for Wayland , is the lead developer of the Flatpak version.

After exchanging some messages and verifications, in the end they told me that the problem possibly came from the GNOME 3.38 configuration that I use in Fedora 33 Workstation, and the truth is that they were right. After entering a command that enables screen sharing with DMA-BUF, I can finally record full-screen applications via full desktop capture, which in my personal use mostly encompasses the Big Picture interface of Steam (except for first-person shooters and RTS I try to play everything with a controller) and video games.

So, seeing that the door is finally open to being able to record from the GNOME session on Wayland with OBS Studio, I am going to expose the steps to follow to get it up and running.

First of all you have to add the Flathub beta repository, which obviously requires before installing the Flaptak bracket.

flatpak remote-add flathub-beta https://flathub.org/beta-repo/flathub-beta.flatpakrepo

The second step consists of install beta version of OBS Studio in Flaptak format:

flatpak install flathub-beta com.obsproject.Studio//beta

Finally, if you are on GNOME 3.38 (I don’t know if it works in older versions of the environment), you have to activate experimental screen sharing support with DMA-BUF and restart the session user (or in extreme case, kill flies with cannonballs with a computer restart):

gsettings set org.gnome.mutter experimental-features "['dma-buf-screen-sharing']"

With this, it should be possible to record from the GNOME session on Wayland not only in Fedora 33 Workstation, but at least in any distribution that uses GNOME 3.38, so it should be applicable, if the default configuration does not work (already that on an Intel graphics should work without reconfiguring anything), to Mageia 8, Ubuntu 20.10, Ubuntu 21.04 and distributions rolling release like Arch Linux and Manjaro as long as they don’t migrate to GNOME 40 (which they won’t immediately). On the other hand, the GNOME reconfiguration part may still be valid even after OBS Studio 27 has been released as stable.

With the addition of OBS Studio, Wayland will take a giant step forward to establish itself as the replacement for Xorg in the near future, at least in GNOME. I still need to do more demanding tests (such as a recording or a long transmission), but things, at least for now, are promising.