@jazzdelightsme any idea on what other popular terminals support this?

See also https://gitlab.gnome.org/GNOME/vte/issues/23.

There is also an open PR for microsoft/terminal.

Hmm should not be hard to support those, still the concerns in the vte issue make them looking weakly specced. Is the attribs stack terminal global or bound to a buffer? Will have to look at xterm source...

Further note that, as handy as this might seem - it looses ground as soon as multiple producers use this interleaved, they will screw up the stack.

Edit: Why is the latter a concern? Isnt terminal state always global? Yes, but here the stack behavior might suggest to app devs to get a reliable way to refer to some older attrib values, which is not true for the reliability part. This is misleading at best, and produces wrong formatting at worst.

I think these sequences are interesting for tools that repeat several attribs often - like loggers or debuggers with certain output formatting. But those often operate as "collectors" - they might grab output from other commands to put it into a joint output. Now they cannot assume anymore to still get their wanted attribs with XTPOPSGR - a sub command might have "screwed up" the stack - and they again have to style explicitly (and to do that to memorize those attribs themselves). So nothing was won in the end.

I somewhat question the usefulness of those sequences. 😸

Edit2 (slightly offtopic): Recently I have the impression, that ppl want to see more and more stuff in terminals which allows a tighter coupling of app side and terminal side (this partly reminds me of the OSC 7 discussion in terminal-wg). This is not a bad thing per se, but simply not possible from within the current terminal interface - it cannot be achieved in clean way with some new sequences, it would need changes on much more basic interfaces (process and pty interface, which is not an easy task at all). I think we should restrain from adding stuff that looks promising in a certain way but will fail badly as soon as a more general use case gets applied. A sequence, that has several "works only if ... and if ..., never when ..." restrictions is a bad one.

I have found these sequences in practice to be more than what I would describe as merely useful; they enable a completely new model for text composition.

It is certainly true that interleaved output would produce nonsense (assuming the two components are not deliberately working together; in other words the definition of "separate components" here captures this fact).

To be clear, I believe this is the sequence of events we are referring to:

1. component A: write some output, including a push, and a color change
2. component B: write some output, including a push, and a color change
3. component A: writes some output, including a pop
4. component B: writes some output, including a pop

That this produces nonsense is expected. And, importantly, no different than the state of the world before the push/pop sequences were available--the following sequence is also not likely to produce desired results:

1. component A: write some output, including a push, and a color change
2. component B: write some output, including a push, and a color change
3. component A: writes some output, including a pop
4. component B: writes some output, including a pop

The advantage of the push/pop is that now we have the ability to write code that does this:

1. component A: write some output, including a push, and a color change, more content, and a pop
2. component B: write some output, including a push, and a color change, more content, and a pop
3. component A: write some output, including a push, and a color change, more content, and a pop
4. component B: write some output, including a push, and a color change, more content, and a pop

In each case, each component does not need to know what the "current" text attributes are--they can independently style some (or all) of their output.

So what we derive from this observation is that for these sequences to be useful, an independent component should use a push and a pop (or multiple) together in a single chunk. This is not burdensome... it's the whole point!

the concerns in the vte issue make them looking weakly specced

I felt like the specific questions raised were answered (for instance in Martin's comment). My read of the situation is that there was just disagreement from VTE maintainers about how aspects of it should have been done (especially selective saves) (correct me if I'm wrong). There's not much I can do about that, except to preach about how useful the capabilities bestowed by these sequences are. But even if other terminal implementations decided to pass on supporting selective saves, the full-save feature alone would be very useful.

@jazzdelightsme Yes, if the components restore the stack to the their entry level, thus all pushs done by the component are popped before returning, it should be fine. Its very much like not forgetting closing tags in wellformed XML.
I still see a burden here, app/component devs now have to track that outer state as well, and have to deal with it gracefully to keep the stack aligned. This means they cannot simply exit early anymore (in case of errors), now they have to to cleanup the terminal state as well in an exception or exit handler. In ASM we have a stack pointer to cleanup pretty fast, here we have to track dangling pushes, urgh. This is not failsafe, thus an outer component mostly likely will have to fall back to a more "traditional styling" with a prehand cleanup SGR (see next section).

Not so with the "traditional styling" - here a nested component would just style stuff prehand (e.g. SGR xy + some output; SGR z + other output) and just leave when done. The outer component can deal with it likewise up to the root component, which would have a final SGR 0 as a good citizen before exiting to the shell.

Plz dont get me wrong - I see a benefit in your examples above, mainly for pushs/pops done in close successions, where it can save the need to hold some of the previous intermediate attribs. Still not sure if these savings are worth the headaches introduced by spreaded terminal state manipulations, further illustration:

const SGR_0 = 'SGR 0';
const SGR_ERROR = 'SGR red|bold';
const SGR_WARNING = 'SGR yellow';
const SGR_INFO = 'SGR green';

// traditional way
output(SGR_ERROR, 'something bad happened...');
output(SGR_0);
some_other_call();  // code controlled by project
output(SGR_WARNING, 'just a warning...');
output(SGR_0);
foreign_call();  // output out of control
output(SGR_INFO, 'we are done.');
output(SGR_0);

// with push/pop SGR
PUSH()  // push SGR_0
output(SGR_ERROR, 'something bad happened...');
POP();  // restore SGR_0
some_other_call();  // code controlled by project
output(SGR_0);  // needed if we did not ensure the call returns with SGR_0 on top
PUSH()  // push SGR_0
output(SGR_WARNING, 'just a warning...');
POP();  // restore SGR_0
foreign_call();  // output out of control
output(SGR_0);  // needed since we cannot control the stack in foreign_call
PUSH()  // push SGR_0
output(SGR_INFO, 'we are done.');
POP();  // restore SGR_0

I think this shows the problem - as long as the stack usage happens within comprehensible code (close succession), its fine. But as soon as a complicated call enters the stage, the stack should be re-initialized as no one (beside compilers) can track reliably a mutable global state over complicated control flow. This example feels abit like facing problems again from good old goto days. Not sure if incorporating a wonky outer global state helps in terms of testability, imho the functional programming paradigms are on the better track here.

To sum this up - the benefit I see only holds for very close PUSH+output+POP actions, otherwise it should be discouraged.

1. component A: write some output, including a push, and a color change, more content, and a pop
2. component B: write some output, including a push, and a color change, more content, and a pop
3. component A: write some output, including a push, and a color change, more content, and a pop
4. component B: write some output, including a push, and a color change, more content, and a pop

This can just as well be written as

1. component A: write some output, including color changes, and finally an sgr0
2. component B: write some output, including color changes, and finally an sgr0
3. component A: write some output, including color changes, and finally an sgr0
4. component B: write some output, including color changes, and finally an sgr0

can't it? Maybe it takes a more complex setup to showcase your point.

Apart from the great observations from @jerch, and the ones pointed out in the VTE bug, I also cannot see where and how this feature wants to go. I don't see it reasonable for this feature to get implemented in the vast majority of terminals in the next 10 years.

An independent story: look at truecolor delimiter, even though the standard says to use colons which would be technically superior (more compatible with future extensions), using semicolons instead have become so popular that terminals have about zero incentive to support colons, too. It's easy to say that all of them should support colons, too, the world would become a better place. But it's just not happening.

Another independent story: a popular terminal emulator (I don't want to blame them so I'm not providing a link, you can sure easily find it if you want to) didn't support ST (string terminator for OSC). Took them 7 years to implement it (counting from the day it was reported). It was buggy. They claimed to fix that bug quickly afterwards. It was still buggy. Took another 1 year to discover that it was still buggy, and yet another 1 year to really fix it. That's 9.5 years from the initial bugreport to a working solution, for recognizing a freaking two-byte closing sequence. Based on this: Would they be interested in implementing a new escape sequence with close to zero use? Maybe, maybe not, I don't know.

We already have a pretty good solution for changing colors and attributes. Push+pop is not part of that. Why would any software author write a piece of software that runs only in a small fragment of terminals, rather than one that (perhaps with a tiny little bit more complicated code) runs on all of them? No one who aims to share their work with others would do so. Which, in turn, means that terminals don't have much incentive to implement support either because there aren't apps that would need it. It's a chicken and egg problem, and is very hard to break out of the current situation because it's developer effort not well spent, you'd need dozens of parties to agree and work in the same direction for some tiny benefit.

In the terminal ecosystem, every application currently assumes that the terminal is at the default colors+attributes when it's started up, and accordingly it's a fair game to return to the default (sgr0). E.g. you switch to yellow background from your shell prompt, executing a utility that colors its output (ls, gcc, grep, you name it...) won't revert to yellow but to the default color right after the first colored piece of text. It might be a great goal to fix it, but that requires changing all these utilities. Which, due to lack of generic support in terminals, means a temporary (for a decade or two) solution of sgr0+pop to restore, so that it pops correctly in terminals that support it, and falls back to the old sgr0 in terminals that don't. And then maybe in 20 years from now these utilities will be able to go with popping only.

The other use case is when your application can theoretically know the previous color, because it was this very application setting it, it's just cumbersome for your code to carry it and you'd rather push+pop. Actually, you could also relatively easily implement this stack itself in your application, or carrying and knowing the state, whichever you prefer. The difference: while for XT{PUSH,POP}SGR my estimate for becoming widely usable is 10–20 years if you push for it hard in all the terminals that matter, the other can be implemented in your app in one rainy evening.

In the VTE bug, for a side issue (the exact choice of escape sequence) you showed a semantically incorrect piece of code, and you convinced xterm's author to modify the escape sequence to make that incorrect code work. Over here you showed a theoretical example, a piece of code that could just as easily go with sgr0. I'm yet to see an actual example where push+pop would offer a notable advantage, let alone one that is worth the price of pushing for this feature across all terminals for many-many years, rather than just going a less convenient solution in your app.

That being said, I don't have concerns with push+pop per se, and by no means do I want to discourage other terminals from implementing it. I have concerns with the exact details how this escape sequence looks like, and I believe that the entire effort of getting this feature universally available is doomed to fail. Would be glad to see being proven wrong over the next few years.

Stressing the testability aspect one more time - as @egmontkob already noted, you can always build such a stack on your own and carry it around. Now you have full control over the stack, it is much easier to test. No need for uncertain outer state here. I am pretty sure haskell devs would avoid these sequences for very much the same reason - no need to write an ugly monad if you can actually control the full state reliably.

Going one step back from the sequences at hand - it seems we have a need for addressing certain attribs sets during runtime. I am not a big fan of putting to much app related stuff into the terminal state, still I wonder if "SGR/attrib slots" on terminal side would be a better way to deal with this:

• they are reentrant - an interrupt handler cannot screw up values anymore (the stack variant will go bonkers here)
• can be easily set up, e.g. by loading current attribs into a slot, maybe even make an explicit store sequence
• can be requested independently by slot numbers
• still needs discipline on app dev side, but its much easier to comprehend as an app global or subcomponent-local contract

Just a wild idea...

First, you are absolutely right that there is a "chicken and egg" problem: who will use these sequences if they are not widely supported, and who will support them, if few people use them? I agree with your assessment that it will take a long time for these to be adopted. That said, I do still hope to be writing code in 10-20 years. :D

Why sprinkling SGR0 (reset to default colors) is not enough

Resetting current colors to default works in many cases. This would be equivalent to a color stack that never has more than one item on it. But if your stack ever grows beyond a single item, SGR0 no longer does the correct thing.

For example, one place I have used this in my own code is when dynamically "highlighting" a particular line of output. Some layer of my program generates objects which, when printed, contain color sequences to help make information more clear. A higher layer that is consuming and writing out these objects may decide, for one reason or another, to "highlight" a particular object / line of output, which it does by effectively doing printf( "<push><bgDarkRed; fgBlack> %s <pop>", objWhatever.ToString() ). Because the color sequences used by objWhatever.ToString() use push/pop when changing the color, instead of SGR0, this all works dandy. If it used SGR0 instead, it would not work (the "highlight" would be "forgotten" after the first bit of color in the %s insert).

Alternate ways to handle this: certainly, my app could maintain its own internal stack of SGR state. Functionally, it would work exactly the same. However, there would be a couple of MAJOR drawbacks to doing it that way:

Problem 1: solving this myself is likely NOT simple

For a "simple" implementation, all output would have to be generated "in order": if we can ensure that objWhatever.ToString() is always called (and it dynamically formats the object), then all code that deals with pushing/popping can just reach out into some global g_colorStack to determine what the "current" color should be, and all is well. As you say, this is a "one rainy evening" problem. In fact, I spent one such rainy evening implementing this for VTE. (Side note: actually, now that I remember it, it was two rainy evenings: the first 1.5 were just getting a proper build environment set up. Have y'all ever considered a docker build container solution?)

However, I have no code which is so simple. Many objects may not actually dynamically generate a new string when .ToString() is called; they may want to be able to cache a previous representation. Furthermore (even without cached strings), there are many other reasons why output may be dynamically composited, such that stuff is outputted in a different order than it was generated in, which simply will not work with a "simple" global stack implementation.

The only solution to that, then, is to be able to embed conceptual push/pop sequences into your output, and then dynamically interpret them on their way out the door to the terminal. This is a considerably more complicated (and annoying) piece of code to write. This is semantically equivalent to writing your own VT parser (though you only have to handle "your own" sequences, and let the others flow out to the terminal). In addition to being complicated, it's simply wasteful.

Problem 2: solving it myself is not reusable

The second problem is that no matter how you "do it yourself" (whether the "simple" implementation, or the more complicated, "streaming" implementation), your code is no longer reusable. Any code that wants in on this scheme has to be baked together, either knowing to use the same g_colorStack or the same embedded codes and intermediate output streaming layer. Even among code that I completely write myself, I would (and do) find this to be an almost insurmountable burden, and it simply means that such bodies of code are just not shareable. No shared lib can do this stuff--no shared lib can take colorized output from any other shared lib, and add "highlight".

I'll go ahead and reiterate that last statement, because I allege it also applies to the current state of the world:

Without a standard solution, no shared lib can do this stuff--no shared lib can take colorized output from any other shared lib, and add "highlight".

In my opinion, the way to solve this, and thereby enable entirely new scenarios for textual output composition, is to bake that "standard solution" into the terminal itself: the code becomes "one rainy evening" simple, and it's usable by everything (using that terminal). We've got the ball rolling in xterm, and I know everybody (e.g. you, @egmontkob :D) is not happy with every aspect of that, but I intend to continue advocating for it elsewhere.

That said, I do still hope to be writing code in 10-20 years. :D

First, as they say: hope is not a strategy. I wrote down why I believe the attempt to address this issue is likely to fail. Maybe you'll prove me wrong.

Second: I bet you a beer that in 15 years from now you'll be working on something completely different and no longer care about this issue. Deal? :)

Without a standard solution, no shared lib can do this stuff--no shared lib can take colorized output from any other shared lib, and add "highlight".

This is correct – assuming you're not willing to enter the territory of parsing the SGR sequences. Again, you have two choices: spend a few days implementing a library that parses SGR and you have every chance to release it this year; or try to convince dozens of developers who would all need to agree and all do some work so that it's maybe widely usable in 10 years from now. If I were you, I would choose the former. Or maybe you can do these two in parallel, and in 10 years from now you can clean up your code and remove the ugly SGR-parsing hack.

I intend to continue advocating for it elsewhere.

I sincerely wish you good luck! (Which does not imply that I can help you by quickly adding this feature to VTE.)

Or maybe you can do these two in parallel, and in 10 years from now you can clean up your code and remove the ugly SGR-parsing hack.

Yes, this is exactly what is happening. In fact I think it's been close to ten years since I wrote my implementation. Which is why I am very confident that a) this feature is very useful, and b) the "do it yourself" approach does not scale, because it only works if all code running in a process agrees to send output through the hack, which is a pretty egregious requirement.

@jazzdelightsme This pretty much sounds like you want to daisy chain the SGR modifications with different stack levels across components or even separate processes. There is one big flaw in this idea - at no point in this chain the terminal is isolated, it is always a global thing any process with enough rights can hook in and screw up your stack and idea of SGRs. Its not even needed to be done by outer processes, even an interrupt handler triggered by one component could do this (that you might have forgotten in your big color control flow diagram).

What you describe is a big stack machine for SGR handling across several processes/components, while the held state is not secured or self-contained by any means, nope its OS global. This is doomed to fail. At least in POSIX terminals cannot be bound/restricted to certain processes. It works the way around, the controlling terminal is a notion of a process (group).

Perhaps I don't understand the concern. Yes, multiple processes can have access to the terminal... Any uncoordinated access from multiple processes is bound to result in garbage on the screen (and a poor user experience), no matter whether the output contains anything SGR-related or not. This is not a problem unique to an SGR stack...

It seems like there are several concerns here. I'm sorry; I still don't understand the concern about "atomic writes". I maintain that there is no additional burden on app devs, or even on the terminal devs, for the push/pop control sequences, any more than for regular SGR sequences.

Yes, multiple processes can access the same terminal, but I can't imagine a terminal implementation that allowed such concurrent access at a low enough level that it would cause a problem.

Consider a plain SGR sequence: a CSI (escape and [, for example), some parameters, and m. Suppose two processes each send a buffer with such an SGR sequence to the terminal "at the same time". Any reasonable terminal implementation out there will process one, then the other. You wouldn't worry about processing the CSI from one, then the CSI from another, then the parameters from one, then the parameters from another, in a way that would mess up the VT parsing state. And the push/pop sequences are just more VT sequences. Necessary parsing state and data structures will be just as protected against concurrent access as any other VT processing state. Or if the apps did "broken writes" (where the control sequences were broken in between separate writes), then yes, you could end up with what appears (in the terminal) to be malformed sequences... just as you would for plain SGR sequences. So it really should pose no additional problem.

The next concern is about the effect of interleaved output from different apps. If two apps are not coordinating their access, then you'll get garbage on the screen, at least for a while. Assuming the apps are well-behaved (balanced pushes and pops), things will self-heal with pushes and pops, just as for plain SGR sequences.

The third concern is how to deal with "bad" apps--how to recover if pushes exceed stack storage, if pushes exceed pops, or if pops exceed pushes. "Too many pops" is pretty easy: the obvious thing is to do nothing in response to a pop when the stack is empty. For the other situations, there are a few ways one could deal with this.

The route xterm took is that stack storage is fixed at ten slots, but the push counter is a full integer. Any pushes "over the [storage] limit" will not cause anything to be pushed onto the stack (because the stack is "full"), and pops while over the limit will not pop anything off the stack, either... but those over-the-limit pushes (and pops) are still tracked by the counter. The effect is that as long as pushes and pops are balanced ("well-behaved app"), it naturally recovers just fine, even if you temporarily exceed the storage limit.

The downside to that approach is that if you want to recover from a bad app that just did "lots" of pushes, you have to do "lots" of pops... and who knows how much "lots" is. (Imagine accidentally cat-ing a binary file out to the terminal, which has lots of data that looks like pushes.)

So the strategy envisioned by the VTE people (chpe? dunno), as indicated in comments, is instead of letting pushes and pops above the stack storage limit remain balanced, if you do another push when the stack is already full, it just drops the bottom of the stack (so it's a ring stack). That way, the way to get back to a "clean" (empty stack) state is to just do N pops, where N is the size of the storage stack. But an app that is otherwise well-behaved (balanced pushes and pops) that just happens to [want to] use more stack than is available will end up in a messed-up state.

You can also combine these approaches: for a stack of size N, allow pushes to continue past N (albeit without actually storing anything on the stack) up to some higher limit K. When the app does enough pops and the push counter drops back to the range of the storage, the pops start "working" again, and the app is left in a good state, provided it never does more than K pushes. So the advantage here is that K can be bigger than N, and yet still be a small enough number (and a known bound) that it isn't unreasonable to do K pops to recover from a garbage app that just did nothing but pushes.

All that said... I don't think K needs to be very big. When I first saw that xterm only allowed ten saves, I thought it seemed stingy, but then after having used it, for my scenarios it is plenty. But it's still possible you could have a script or something that does pushes in a loop or something a little nutty, so it could still be useful to have a K that's a bit higher than N.

I agree that these issues could use a little more spelling out. I'll write an "XTPUSHSGR / XTPOPSGR implementor's guide" and link to it here for your convenience.

I agree that we shouldn't worry about non-atomic writes per se, i.e., two applications writing to the terminal concurrently, without coordination. Not because it's a non-issue, quite the opposite. We shouldn't worry about it because it's hopelessly broken to begin with, therefore I couldn't care less if a new escape sequence breaks it even more or not.

Suppose two processes each send a buffer with such an SGR sequence to the terminal "at the same time". Any reasonable terminal implementation out there will process one, then the other.

No, there's nothing the terminal implementation can do about it. The terminal receives the input over one single channel. The quesiton is: is the data interleaved there, or is it one after the other? If it's interleaved, it's hopeless for the terminal to figure out what it was meant to be.

If tool X writes \e[31m atomically, and tool Y writes \e[32m atomically, you can at least be sure that the output will eventually be either red or green (which "either-or" is already a broken user experience). However, as the output chunks written become larger and larger, the chance of one of these getting broken (e.g. \e[3<something_irrelevant>1m or \e[3\e[32m1m etc.) increases, and there are many reasons for that, including stdout/stderr buffering happening to cut it there, or some transport layer (e.g. ssh, TCP) cutting into smaller fragments there, giving a chance for someone to interleave. Also the chance obviously increases with longer escape sequences (e.g. OSC 0 to set the title, OSC 8 to set a hyperlink etc.), and even longer if we introduce semantical pairs of escape sequences (e.g. push-pop) and expect these pairs to stay intact, not to interleave with conflicting pairs.

Not only escape sequences can be cut, e.g. a UTF-8 character can be cut in half, or a combining accent or variation selector or emoji skin tone modifier might be placed on top of an undesired base character, or a ZW(N)J can (non)join undesired characters, or text might end up inside an undesired BiDi context, or BiDi control characters may not pair up anymore because one of its UTF-8 sequences gets broken, etc...

And again: there's nothing the terminal could do to prevent these from happening.

Let alone: even if none of these happen, the result of two interleaving outputs is probably anything but user-friendly.

However, what we really should care about is having a decent terminal state recovered easily (as automatically as possible); in case of faulty, crashing applications, broken ssh connection, accidentally cated binary files etc., as well as the previous use case: after two apps writing concurrently in weird interleaving ways.

xterm [...] The downside to that approach is that if you want to recover from a bad app that just did "lots" of pushes, you have to do "lots" of pops... and who knows how much "lots" is.

Exactly. That's why xterm's approach is not suitable to build upon, and if you want to push for this feature getting widely adopted, you have to push for xterm to fix it.

Imagine this scenario: App X occasionally leaves something on the stack. Now, app Y or terminal Z receive bug reports that after using the terminal for a while, colors start to fall apart, behave incorrectly more and more often. Imagine dealing with this issue as the developer of app Y or terminal Z. Imagine seeing this issue on various support forums, and users suggesting to switch from your app Y to someone else's app W because that doesn't suffer from this problem. Would you want to see and deal with these? I wouldn't.

envisioned by the VTE people [...] That way, the way to get back to a "clean" (empty stack) state is to just do N pops, where N is the size of the storage stack.

This gets back to the "clean state" as seen by the terminal emulator; not something you should care about. For well-behaved applications (that never try to pop something they didn't push in) the state is always clean, that's why this model fixes xterm's bug. Applications should not begin by N pops, not only because you don't know the value of N, but more importantly because that would make it impossible to do cross-app push/pops (e.g. I push when I ssh to my remote system, will pop when closing the connection, until then I'll run apps on the remote host which might also push-pop).

But an app that is otherwise well-behaved (balanced pushes and pops) that just happens to [want to] use more stack than is available will end up in a messed-up state.

You mention this under VTE, but xterm's model obviously also fails here. (xterm eventually cleans up better here, nevertheless, the data printed by your app will be broken.) Not a surprise: if you fill up the stack then, well, you filled up the stack.

You can also combine these approaches:

Seems to me that this combines the disadvantages of both.

Let's talk about one more thing: a possible security/privacy implication.

If push/pop is supported along with XTREPORTSGR, a malicious app (e.g. a compromised site, right after I log in) can examine the contents of the stack. If another app is known to be buggy and leave a certain state on the stack, or is known to deliberately place something on the stack, the remote site could figure out that I was likely using that application (or that particular configuration, let's say a popular shell theme) locally on my computer, before I connected to the compromised site. Combine this with a remotely exploitable bug in that software. The attacker might gain knowledge about whose computers to further attack because are likely vulnerable.

I don't see much chance for this to happen, but there is some chance, so it needs to be taken into account. I firmly believe that if XTPUSH/POPSGR gets implemented then XTREPORTSGR must go.

About atomicity:
Imho the situation wasnt that bad in the beginning, but got watered more and more with features added over time like job control/pts layer and piping stuff through more and more transport layers, while the TTY is almost stuck on its early 70s incarnation. The process coupling is only half-heartedly done up to the session leader (a shell could insert an emoij every 100 bytes to any program output to look funny).

A cleaner approach would do at least: whenever a process context switch in foreground group happens, reset the tty + certain terminal aspects to the last state held by that process (with some sane defaults for new processes, eg. ICANON + SGR0). Ideally the terminal would be informed about that with some process notion, and maybe holds different pages for processes (not a big deal anymore memory-wise these days) or stays within line buffer/scrolling mode but can clearly mark the output as something different from another process. But thats more than unlikely to ever happen, it scratches at to many low level well established still malformed POSIX interfaces.
</POSIX rant>

Still I dont share this:

We shouldn't worry about it because it's hopelessly broken to begin with, therefore I couldn't care less if a new escape sequence breaks it even more or not.

Imho there is always a way to make it worse and less fixable, I tend to count the sequences at hand into this category.

About ringbuffer vs. counting unadressible excess:
As @egmontkob already mentioned - imho a ringbuffer is the only chance to start over with a (fake) clean state at any time, given than no one ever pops before he pushes. This is not possible with the integer excess counting, which renders the stack unusable if deeply nested.

About reports and security:
Back-reports of user/process generated content (here a stack of SGR values) is always a matter of trust and should at least be configurable, otherwise a CVE is waiting. Problem here: a piece of information of the past might cross processes with different privileges or even systems. Furthermore the stack makes it rather simple to pop and eval entries until you get no change anymore (prolly empty), and just push everything back - no one noticed (if done atomically or without other output in between). The terminal itself cannot help here as it cannot tell whether this was ok (again we have no process notion).

We shouldn't worry about it because it's hopelessly broken to begin with

That's a much better way of putting it; thank you.

For well-behaved applications (that never try to pop something they didn't push in) the state is always clean
Applications should not begin by N pops, [...] because that would make it impossible to do cross-app push/pops (e.g. I push when I ssh to my remote system, will pop when closing the connection, until then I'll run apps on the remote host which might also push-pop).

I had not looked at it this way before. I really like the ability for an app to be able to start "always clean".

It feel like it would be unfortunate to have to give up the "self-healing" nature of the xterm way. I can imagine a mechanism that would combine these two abilities (by splitting the storage into two stacks: an xterm-style, fixed stack on the bottom; and a floating ring stack on the top; and a counter to keep track of how much space is consumed in between the two stacks), but ... maybe that's overkill.

Re: the information disclosure: if XTPOPSGR allows up to ten drops of information to be harvested from the terminal, then even without the stack, XTREPORTSGR is disclosing one drop of info. Correct? Do you feel that XTREPORTSGR has to go, regardless of whether XTPUSHSGR is implemented?

That's a much better way of putting it; thank you.

And is something @jerch didn't agree with. Did any piece of this story had an agreement among all three of us? :)

It feel like it would be unfortunate to have to give up the "self-healing" nature of the xterm way.

I'm sorry, I must have missed something. In what sense is xterm's behavior "self-healing"? It seems to me it is not, VTE's proposed behavior is.

by splitting the storage into two stacks

I have serious doubts that a good solution could be built up on top of newly invented, never-seen-anywhere-else patterns.

even without the stack, XTREPORTSGR is disclosing one drop of info. Correct?

One drop of info which normally is the default state (that is: no info), and only contains some info when a previous app crashed or so. As opposed to if you combine it with a stack: in that case even the normal operation of things could expose some info. That being said,

Do you feel that XTREPORTSGR has to go, regardless of whether XTPUSHSGR is implemented?

I guess so, I tend to feel this way. I don't see how this sequence is useful at all, and if indeed isn't useful but has a questionable impact on privacy, I don't see a point in keeping/implementing it. Although I might miss useful use cases which might change my mind.

Wait a sec, I had wrong assumption about XTREPORTSGR. I though it reported the active SGR attributes. Nope, it reports the ones used within a rectangular area on the screen; that is, lets an app inspect the contents printed by a previous app. Yup, it's definitely a pointless yet dangerous one.

In what sense is xterm's behavior "self-healing"?

Suppose an app does 15 pushes, and then 15 pops (with SGR changes and content in between them all, and there are only 10 SGR stack storage slots). After the 10 storage slots are used up, pushes will not be stored. There won't be any noticeable effect, of course, until the pops start, at which point the first 5 pops will not update the current text attributes, making things look wrong. However, the last 10 pops will work.

With the VTE scheme (ring stack), the first 10 pops will work, but the final 5 pops will not.

I agree with your sentiment that "strange patterns" do not usually lead to good solutions, but I don't think the "split stacks" idea is actually very wild. The bottom "stack" could actually just be a single storage slot--the effect would be that the final pop would always succeed, no matter by how much you blew the stack in the middle, which would be nice. That said, I don't think it's crucial to have: like you said, if you blew the stack, you blew the stack; some breakage will occur.

If an app tries to place more items on the stack than a conforming terminal is required to support, the app is broken. Period.

Strictly speaking there is no self-healing here possible at all. The only difference in xterm's way and ringbuffer idea is where in a nested push/pop tree the stack stops working:

• ringbuffer - all PUSHs keep working, only deepest nested POPs keep working
• xterm's way - top level PUSH/POPs keep working

Still xterm's way is subpar for a simple reason - it will render the stack dysfunctional if PUSH/POPs are not balanced (e.g. you press Ctrl-C after some stack descent). Since its a global state the stack now is partly (or completely) taken, you'd have to trigger some clear action to make it fully working again. Sadly the world of terminal interaction is full Ctrl-C or similar hazards.

This feature gets less appealing to me, we already have identified tons of if and elses and even security concerns. Not the best cicumstances to get something rolling...