At RacketCon in July 2019, Racket architect Matthew Flatt proposed some ideas for what he called “Racket2”, the ostensible successor to today’s #lang racket. These ideas have caused ripples—and a little consternation—within the Racket community.
Most notably, Matthew floated the idea of moving away from Racket’s parenthesized S-expressions to notation more similar to other languages—what is sometimes called “infix notation” to contrast it with Racket’s “prefix notation”. Matthew’s rationale is that “parentheses are certainly an obstacle for some potential users of Racket … we should try to remove or reduce the obstacle.”
I’ve come to appreciate parenthesized S-expressions. But to be fair, what initially persuaded me to try Racket was the Scribble @-expression notation. This style of notation didn’t exist anywhere else, and allows you to write things like @func{text body} that combine invocations of code with ordinary text. (This notation system is also the foundation of my publishing language Pollen.)
So I agree with Matthew Flatt that notation matters. Indeed, I believe that the ability to design notation is a key benefit of language-oriented programming. + PL researchers are less excited by notation, but that’s fine—they need to care about proving results, whereas practitioners need to care about ergonomics. That’s why I spend a lot of time in Beautiful Racket on lexers and parsers, because these are the tools we use to implement that notation.
In the case of Lisp languages like Racket, Matthew Flatt is right: among a certain audience of potential users, parentheses are a deterrent. Who knows why? But in the last 60 years, simply grabbing these dissenters by the lapels and fumigating them with the stinky garlic breath of parenthesized S-expressions has not been a winning strategy. If Lisp partisans want different results in the next 60 years, we need to try something new.
Within the Racket community, some of the professed alarm about new notation has centered around the possibility of losing parenthesized S-expressions altogether. So far, I find these concerns overstated:
Racket is an interesting language precisely because it takes contrarian, even daring, positions—not because it follows the crowd. For example, the reason Racket exists at all is that it outgrew the boundaries of Scheme. For another example, the multiyear effort to reimplement the entire language on Chez Scheme, which is now bearing fruit. On that view, investigating alternate notation is consistent with Racket’s culture of curiosity. Certainly, if you want an intellectually stagnant programming language, you have plenty of other choices.
In any case, it’s not obvious to me that infix notation must necessarily displace parenthesized S-expressions. Again, consider Scribble: in addition to notation like @func{text body}, Scribble supports more standard-looking S-expressions like @(func "text body"). With Scribble, the Racket team already figured out how to let both styles coexist. There’s no reason to think they couldn’t do the same for Racket2.
But even if they don’t, the Racket team has already given assurances that vanilla #lang racket—parentheses and all—will continue to be supported for a long time. As I understand it, they’re not even promising that the new notation will “eventually take over”. Instead, it might remain part of a separate #lang indefinitely, and find whatever audience it finds. A wholesale displacement of parenthesized notation would only occur if and when a critical mass of users supports it.
Unfounded opposition to theoretical future notation is even less defensible than unfounded opposition to parentheses. If any engineering team can pull this off and blow everyone’s minds with how good it can be, it’s the Racket team. I’m content to reserve judgment until there’s a prototype that’s good enough to criticize. Certainly, I don’t want to get caught looking stupid by betting against Matthew Flatt and the other core Racket engineers.
I feel confident, based on their track record, that the Racket team will remain conscious of the costs and benefits of new notation. Certainly, it would be counterproductive to adopt new notation that alienates everyone who already likes Racket.
All that said, I do have a few concerns. If you think I’m wrong, I invite you to send me evidence & argument against them. But I’ll set them out here so I needn’t repeat myself in the years to come:
I expect that most of the parenthesis-hating programmers whom Racket hopes to attract with new notation will simply move on to some other pretext for dismissing Racket. That is, I believe that the objection to Racket (and other Lisp languages) often goes deeper than parentheses, but it’s shorthanded as “ugh, parentheses” because that’s easy and obvious.
It would be sad to divert a lot of time & effort to persuade people who are basically unpersuadable. That’s not an argument against trying anything new. Rather, it’s an argument in favor of proceeding on a prototype basis, and finding ways of validating that prototype with the intended audience.
The declining popularity of Lisp languages in CS teaching—and the related failure of these languages to ever crack industry in a big way—is a longstanding trend that goes beyond Racket.
For example: Scheme was originally created in 1975 for use in MIT’s introductory CS course, but in 2009, it was replaced by Python. Another example: programmer and startup investor Paul Graham has vigorously promoted Lisp for decades, citing it as the reason for his own startup success. Despite the weight of his endorsement (and his cash), there’s little evidence Lisp languages have made much of a dent in the tech industry. And another example: longtime Lisp advocate Peter Norvig switched from Lisp to Python because “it was better pseudocode” and because Python “has the edge … when the main goal is communication”.
“What about Clojure?” That is certainly the biggest Lisp success story in recent years. Still, though Clojure is a well-designed Lisp, its overwhelming advantage comes from the fact that it’s hosted on the JVM. That is, Clojure can be Trojan-horsed into workplaces that use Java (= about a zillion).
Against that backdrop, I think it’s fruitless to expect that Racket is ever going to be the choice of kajillions, or even mere jillions, of programmers. So what? Among programming languages, I think of it more like a McLaren supercar than a Honda Civic: it’s not for everyone, but if you need what it can do, you will love it!
If it weren’t obvious already, I believe that Racket’s great, unique strength—do people still use the phrase “killer app”?—is language-oriented programming, where it has no peer. That is why Beautiful Racket exists.
The interaction between the Racket community and Racket core engineering team has happened—at least in my six years of Racket experience—at something of a remove, like polite neighbors separated by a shoulder-high hedgerow. The core team is very responsive when it comes to, say, bug reports. But with larger issues, the core team makes the decisions, and there’s not huge visibility into how or why these decisions are made.
There is not, for instance, an official RFC process or other means of collecting and responding to community feedback in a sane and coherent manner (= roughly, a system that rewards reasoned thought over shoutiness and squabbling). + Typed Racket has been experimenting with an RFC system of its own, though activity has been light so far. Since RacketCon there’s been an effort to get an RFC system started. Fair enough. But this kind of change takes more than just committing some source files to a new repo. It requires a deeper cultural switch: the whole Racket core team has to commit to the new system, and then lead by example. + On this issue, unity matters. There are five members of Racket’s core team. Often, they speak as a group. Sometimes, they do not, which can be confusing to those trying to discern Official Racket Policy.
Can they? Will they? I have no idea. Of course, it’s the core team’s prerogative to run the project as they wish. The rest of us are, after all, freeloaders. But procedural transparency isn’t just a mechanism for gathering community feedback, nor is it bureaucratic foofaraw. It’s also a tool for convincing users, especially industry adopters, that Racket has procedures and values that are worthy of time & money commitments—that there is a “rule of law” that guides outcomes.
I may be a visible user and fan of Racket—and even a teacher at Racket School—but I don’t have any influence on the direction of Racket. I think the Racket community is now big enough, however, that pulling off this kind of major change in Racket requires a stronger procedural foundation. The absence of this foundation is, in this fan’s opinion, a greater risk to the success of Racket2 than any technical hurdle.
BTW, none of this is meant to imply any pessimism about Racket. Beautiful Racket has already introduced thousands of people to LOP and Racket. I plan to keep using Racket in my own work, and keep spreading the word.
Growth of an open-source language doesn’t just mean more commits to the code. It also means more commitments to the community. Racket has already evolved tremendously in the time I’ve been using it. I look forward to seeing where it goes next.
As I mentioned above, I think Racket’s facility for language-oriented programming (LOP) is its killer feature. The #lang abstraction, for instance, is a) amazing and b) available in no other language. Under the hood, these LOP tools are made possible by Racket’s extraordinary hygienic macro system—another feature unique to Racket.
Still, what we think of as Racket’s modern LOP tools weren’t there from the beginning. Racket, after all, started as a Scheme implementation. These extra pieces accrued over time.
Thus, to me—not that I have any say—the major opportunity of Racket2 would be to rebuild the language with these unique LOP ideas at the core, rather than tacked on from the outside.
Recently I was researching a question about certain differences between Rust and C. Someone quipped that “in Rust, they fill gaps in the language by making the language more complex; in C, they fill those gaps by writing more C.”
A cute idea, which applies nicely to Racket too: once you have the ability to make languages, why would you make the core language more complex? Why not just make more languages? Count me among those who thinks even racket/base has gotten too big.
My ideal Racket2 would take the idea of LOP abstraction to its natural extreme. I could foresee a system where the core language is smaller, and pieces are mixed in as needed. Racket core engineer Jay McCarthy has already explored this idea in his #lang remix project.
Maybe there isn’t even one “main” language—you pick the one that suits you. This also neatly solves the question of whether prefix or infix notation ought to prevail. Answer: neither. They coexist. Deal with it!
“But industry adopters want a language to have a canonical notation blah blah …” To a degree, I’m sure that’s true, but—
If the animating idea of Racket2 is to disguise the language’s best features to better attract people who don’t care about them, then it’s a waste. Let Racket be Racket.
In practice, network effects would cause one or two notational dialects to dominate for Racket at large (say, one with parentheses and one without). Other notations would persist for domain-specific purposes (like Pollen).
FWIW, there’s already a huge language community that embraces multiple notational dialects for the same underlying language: JavaScript. So perhaps LOP as a concept is not as foreign as it once was.
All that said, pulling this off would require substantial upgrades to deliver on the promise of a top-to-bottom LOP platform. For instance, the Scribble documentation system, though fantastic for languages that use parenthesized S-expressions, is mostly useless for those that don’t. Supporting arbitrary notation in Scribble documentation should be within reach, just as one can write custom indenters and syntax colorers for DrRacket.
Would this be a lot of work? Well, yeah. Am I the one who’s going to do that work? Well, no. + Though I contribute where I can, for instance improving the brag parser language. So feel free to ignore me. But as a consequence of writing Beautiful Racket, I’ve gotten a deeper look at Racket’s LOP tooling than many. I know the strengths; I know the weaknesses.
And from piles of fan mail, I also know that people really dig this material. Primarily because it uncorks a whole new way of thinking about programming. The reason I expect more from Racket is not because I’m dissatisfied with what’s there. On the contrary—it’s because Racket has already shown itself capable of so much, thus it’s the only language where those expectations seem achievable, rather than merely quixotic.
09 Aug 2019: First release.
12 Aug 2019: Added “if it were up to me” section.
Longtime Racket contributor Greg Hendershott has good points too.
Honu: A Syntactically Extensible Language: paper by Jon Rafkind and Matthew Flatt that describes how Racket macros would work in a notational environment without parentheses. This research would likely become the basis for infix notation in Racket.