emacs: Switch to lucid as the default toolkit

[adisbladis]

Description of changes

Because of long standing bugs and stability issues & an
uncollaborative upstream there has been talk on the emacs-devel
mailing list to switch the default toolkit to
Lucid (https://lists.gnu.org/archive/html/emacs-devel/2022-08/msg00752.html).
The GTK build also has issues with Xinput2, something that both we and
upstream want to enable by default in Emacs 29.

This situation has prompted me to use both Lucid an no-toolkit (pure X11) Emacs
as a daily driver in recent weeks to evaluate what the
advantages/drawbacks are and I have concluded that, at least for me,
switching the toolkit to Lucid is strictly an upgrade.
It has resulted in better stability (there are far fewer tiny UX
issues that are hard to understand/identify) & a snappier UI.
On top of that the closure size is reduced by ~10%.

In the pure X11 build I noticed some unsharpness around fonts so this
is not a good default choice.

As with everything there is a cost, and that is uglier (I think most
would agree but of course this is subjective) menu bars for
those that use them and no GTK scroll bars.

For anyone who still wants to use GTK they could of course still
choose to do so via the new emacs-gtk attribute but I think this
is a bad default.

A note to Wayland users:
This does not affect Wayland compatibility in any way since that will
already need a PGTK build variant in the future.

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 22.11 Release Notes (or backporting 22.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
  • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
• Fits CONTRIBUTING.md.

cc @AndersonTorres @tadfisher @lovek323 @jwiegley
cc @ckiee @thiagokokada

[AndersonTorres]

Waiting for OfBorg.

[kurnevsky]

Because of long standing bugs and stability issues & an
uncollaborative upstream there has been talk on the emacs-devel
mailing list to switch the default toolkit to
Lucid (https://lists.gnu.org/archive/html/emacs-devel/2022-08/msg00752.html).
The GTK build also has issues with Xinput2, something that both we and
upstream want to enable by default in Emacs 29.

Has they already made this decision? I haven't read through all those mails but most of what I read was not about lucid. So isn't it premature a bit? (as of myself I wouldn't want to invest in a gui library that will never support wayland so I will definitely continue using gtk)

[adisbladis]

Has they already made this decision? I haven't read through all those mails but most of what I read was not about lucid. So isn't it premature a bit? (as of myself I wouldn't want to invest in a gui library that will never support wayland so I will definitely continue using gtk)

No, upstream haven't changed it yet but I do not think it's premature.
We as a distribution don't have to be in line with upstream defaults and switching toolkits would solve issues our users have today that will get even worse in 29 because of unfixed GTK issues.

Regarding Wayland we will already have to introduce a distinct Pgtk attribute for that when the time comes as the Pgtk build is not to be used with X11 making that entire point moot.
Here's another resource stating the same points.

The fact is that GTK support for X11 is degrading and upstream no longer cares about X11 and has even gone so far as to suggest to drop X11 entirely.

[wentasah]

I'm not against this change, as I experience emacs crashes with the Gtk toolkit from time to time. However, I've a few minor points to consider:

• I agree that the fonts are ugly. For some reason, the Nix package has them way uglier than the Debian package. In Nix, labels in menu sometimes overlap. See the screenshots below:
  • Nixpkgs:
    
  • Debian
    
    I've not investigated the reason. Maybe somebody would know. If not, I can try looking at it.
• Also note that the version from this PR has an ugly scroll bar. In my config, I'm building Lucid Emacs with Xaw3d = pkgs.xorg.libXaw3d, which makes the scroll bars look like in Debian. This could be made the default.

[adisbladis]

I agree that the fonts are ugly. For some reason, the Nix package has them way uglier than the Debian package. In Nix, labels in menu sometimes overlap. See the screenshots below:

I was talking specifically about the "no" toolkit build looking worse than both Lucid and GTK builds (all from Nix).
Is the regression you're talking about a general problem with Emacs from Nixpkgs or is it specific to this PR?

Also note that the version from this PR has an ugly scroll bar. In my config, I'm building Lucid Emacs with Xaw3d = pkgs.xorg.libXaw3d, which makes the scroll bars look like in Debian. This could be made the default.

I just tried this to check the closure size impact which is an increase of 7M. Considering we just saved ~124M by removing GTK this is a drop in the bucket.

[thiagokokada]

I would prefer that we simply expose emacs-lucid for now, allowing us to have a cache for those that want, but doesn't make it the default yet. A few reasons for this:

• As already said, this is a regression for anyone that is starting Emacs for the first time since it affect the menus. I think it is still important for new Emacs users to have the best experience they can get as possible (even if Emacs is not nice to look/use anyway for a first time user)
• I used the GTK port for a long time and none of the issues that affects it ever affect me. I know some people that few things like Emacs daemon is really important, but generally those people are already advanced Emacs users and once they get there they can switch toolkits themselves
• I eventually expect that we switch to the PGTK port as default because even on X11 I find that it works much better than the old GTK2/3 port. Yeah, this may not happen once Emacs 29 is released since it seems that the port is really bug right now, but I expect most of the bugs to be ironed out once a stable release is made and this port has more attention from users (or maybe not, someone can dream)

[thiagokokada]

The fact is that GTK support for X11 is degrading and upstream no longer cares about X11 and has even gone so far as to suggest to drop X11 entirely.

BTW, reading the issue it seems that this is being considered on GTK5, and PGTK is a strict GTK3 port no?

[wentasah]

Is the regression you're talking about a general problem with Emacs from Nixpkgs or is it specific to this PR?

Hard to say. Strictly speaking, it's a general problem with Emacs from Nixpkgs when built with the Lucid toolkit. However, given that previously, Lucid Emacs was not built by default (was not present in the binary cache), I'd say it's also related to this PR.

[adisbladis]

As already said, this is a regression for anyone that is starting Emacs for the first time since it affect the menus. I think it is still important for new Emacs users to have the best experience they can get as possible (even if Emacs is not nice to look/use anyway for a first time user)

Given the choices "looks worse but much better stability" and "looks better but crashes" I think the choice should clearly be the former.
It might be a visual regression (it's not in my setup, YMMV) but overall stability should be the top priority.

I used the GTK port for a long time and none of the issues that affects it ever affect me. I know some people that few things like Emacs daemon is really important, but generally those people are already advanced Emacs users and once they get there they can switch toolkits themselves

As the most active maintainer of Emacs in nixpkgs I know this to be untrue from the bug reports I receive both here and on chat (sometimes in private messages).
There are lots of people using the daemon mode that are not advanced users who suffer from these instabilities.

The daemon mode is a core feature of Emacs, GTK is not (and I'd like to point out that I don't use this myself so I'm not arguing in my own self interest here).

I eventually expect that we switch to the PGTK port as default because even on X11 I find that it works much better than the old GTK2/3 port. Yeah, this may not happen once Emacs 29 is released since it seems that the port is really bug right now, but I expect most of the bugs to be ironed out once a stable release is made and this port has more attention from users (or maybe not, someone can dream)

I on the other hand do not expect to switch defaults to the pgtk build any time soon, certainly not within the next couple of years and given how reluctant the GTK upstream is to fix X11 related bugs.
It's better to rip this band-aid off earlier and deal with the fact that GTK is serving the wishes of Gnome and not other downstream software.

BTW, reading the issue it seems that this is being considered on GTK5, and PGTK is a strict GTK3 port no?

Yes, this is something I wanted to point out as a potential future issue that we can entirely avoid by not relying on GTK where it's not necessary.

[thiagokokada]

Given the choices "looks worse but much better stability" and "looks better but crashes" I think the choice should clearly be the former.

Well, I never had Emacs-GTK crash with me, at least not because GTK itself (most of my crashes that I had with Emacs I remember that I could reproduce in other toolkits too). But I will take your word here.

The daemon mode is a core feature of Emacs, GTK is not (and I'd like to point out that I don't use this myself so I'm not arguing in my own self interest here).

I could also argue that GTK is a core feature. Most users expect for all of their applications to respect their theme settings (I am talking about the Desktop here, not theming inside Emacs), and the GTK port does (for the vast majority of users at least, since most people have a GTK theme setup) while the Lucid port doesn't.

I on the other hand do not expect to switch defaults to the pgtk build any time soon, certainly not within the next couple of years and given how reluctant the GTK upstream is to fix X11 related bugs.

This is what I meant actually (this is why I said that we will not do this on Emacs 29). I think this will be probably on Emacs 30 or 31 (so from 2 to 4 years), hopefully.

===

To make it clear, I am not completely against this change, but I still think this is premature. I would prefer that we either:

• Follow what other distros are doing
• Follow upstream

And why? Because I think familiarity matters too, and if you're switching from other distro and them you found out that the emacs from distro X is different from emacs in nixpkgs, this is one more thing to get confused when you're starting. And I think both other distros and upstream still use the GTK port as the default one.

[wentasah]

I agree that the fonts are ugly. For some reason, the Nix package has them way uglier than the Debian package. In Nix, labels in menu sometimes overlap. See the screenshots below:

My comment about ugly fonts can be ignored, because it was caused by prehistoric content in my ~/.Xresources.

[AndersonTorres]

The rebuilds are because of the new politics of building Emacs packages?

[adisbladis]

The rebuilds are because of the new politics of building Emacs packages?

Yes.

[AndersonTorres]

OK then. I will merge.

[thiagokokada]

OK then. I will merge.

Any reason for this fast merge? I would like this to at least have more feedback from other members.

[yuuyins]

I eventually expect that we switch to the PGTK port as default because even on X11 I find that it works much better than the old GTK2/3 port. Yeah, this may not happen once Emacs 29 is released since it seems that the port is really bug right now, but I expect most of the bugs to be ironed out once a stable release is made and this port has more attention from users (or maybe not, someone can dream)

@thiagokokada

People running X should not use the PGTK port. The only use it has is
to support Wayland and Broadway.

And it's not intentional. Rather, it's a bug in the various GTK input
modules out there that isn't likely to ever be fixed. Just treat it as
a limitation of the PGTK port and don't use it unless you have to.

---Po Lu, https://lists.gnu.org/archive/html/emacs-devel/2022-02/msg00596.html

[thiagokokada]

People running X should not use the PGTK port. The only use it has is
to support Wayland and Broadway.
And it's not intentional. Rather, it's a bug in the various GTK input
modules out there that isn't likely to ever be fixed. Just treat it as
a limitation of the PGTK port and don't use it unless you have to.
---Po Lu, https://lists.gnu.org/archive/html/emacs-devel/2022-02/msg00596.html

I read this comment before posting my comment above.

My instance is more that people will start running PGTK in X, specially because PGTK fixes a bunch of long term issues in the GTK3 port (less redrawings, better font rendering, etc.), and this will, I hope, result in bug fixes being done in the PGTK port for X once it became widely available (after Emacs 29 is released).

So while I don't expect us switching to PGTK by default in 29, I can see PGTK becoming stable enough in the future that it will be recommended by upstream eventually.

[yuuyins]

I don't expect us switching to PGTK by default in 29, I can see PGTK becoming stable enough in the future that it will be recommended by upstream

@thiagokokada i see. my experience with PGTK on X always awful regarding issues with system clipboard. so as long such issues are sorted out I'm all for it.

[vcunat]

Why was a such a large rebuild merged directly to master?

[vcunat]

I mean, the description doesn't sound like the change needs to hurry so much, so I'd suggest reverting and moving to staging branch, like every other large rebuild.

[adisbladis]

Why was a such a large rebuild merged directly to master?

We recently started building Emacs packages on Hydra. Most of these packages are absolutely tiny.
This change doesn't really impact a lot of other things and the high number makes it seem worse than it actually is.

[vcunat]

If you merge such a change every few days, it will most likely still be significant. I mean PR #190056 now. Naturally now it's worse than usual and certainly significant, as Hydra builders have been in a very bad shape in the past several days.