JBoss, GlassFish, TomEE, Liberty Profile and Jetty respond to ‘The Great Java Application Server Debate’
Three months ago, RebelLabs released The Great Application Server Debate report and it got some really good attention and comments. But we wanted to hear back from the product/project owners, evangelists and enthusiasts themselves to see what they thought of our observations, scores and write up in general.
Firstly, let’s refresh our memory with the results of the report (unweighted):
Here are some of the responses we got back from each of the application server enthusiasts: (and I couldn’t help but add my 2 cents to some of the responses!)
1. For your Application Server specifically, which section were you surprised at seeing such a high/low score?
Liberty Profile
“I think the nicest surprise was being included. The Liberty profile was the only non-open source application server in the list and I know that WebSphere Application Server isn’t known for providing the same lightweight developer friendly experience as some of the alternatives like Tomcat. To be part of the comparison was awesome.
I think the bad surprise was the performance score. I was so surprised by it that I had to look into it. The good news is that the 8.5.0.2 release was better (this was released between the report being published and the data being gathered), and the latest 8.5.5 release got even better. I published my findings on WASdev.
The comments about the lack of an web UI didn’t really surprise me, although from a pure developer experience I think the eclipse based tools in WDT provide a much better experience, you don’t need to start the server to configure it, and you don’t have to leave the IDE.”
Simon’s two cents: “Without saying the phrase ‘we’re all winners’ I agree with the inclusion statement, “I think the nicest surprise was being included”. One of the reasons we looked into doing a report of this nature was due to the number of platforms that were available today which all proved to be very good developer platforms, and it’s difficult to choose between them. The Liberty Profile is still a young endeavour, but with established containers and services borrowed from the full WebSphere profile, so it’s inclusion to the report to comparing it against other servers that have been established in the developer market for so long shows how highly we regard it”
Jetty
“I would say we were most pleased with our ranking under Cost/Licensing. While it is obvious the cost will be low being an open source project, you accurately picked up that Jetty is indeed dual licensed since we migrated to the Eclipse Foundation. The EPL and Eclipse Foundation practices have a special place in the open source community since it exemplifies the importance of Intellectual Property and tracking not just the license of the code itself but all dependency libraries as well. The Eclipse Foundation represents the gold standard for IP cleanliness and auditing practices ensuring that their projects are safe to use.
As for the low score, we were disheartened to see the comments about slower starting. We have always prided ourselves on our fast startup time and to see that erode was not a happy thing. Unfortunately a lot of thing is due to developments in the Servlet API requiring that dependencies be scanned for annotations and like, which severely hamper our ability to startup fast. There are some thoughts on improving this in the future but most any advancement in that area will be at the cost of strict Servlet Api compliance so we need to tread carefully there. Additionally there might be an issue with some version so knowing a little more specifically what versions you were testing would have been helpful.”
Tomcat & TomEE
“I thought the rankings were actually quite fair.”
2. What did we not mention about your Application Server that we were crazy not to add to the report?
What if you no longer had to redeploy your Java code to see changes? The choice is yours. In just a few clicks you can Say Goodbye to Java Redeploys forever.
Liberty Profile
“We usually describe Liberty profile as fast, dynamic, lightweight, composible and developer friendly….and you kind of covered all those. I guess the two biggest things are the ability to package up a runtime that is exactly the right size for your application, and having developed your application on the Liberty profile, you can deploy to production on either the Liberty profile or the full profile (what you used to think of as just WAS) without having to make any application changes. We think these are pretty cool and worth a mention.”
Glassfish
“The report is highly subjective with little empirical evidence. Take documentation for example. How many pages of formal documentation? Are all areas of an appserver covered well (security, installation, dev guide, admin guide, etc). More well-defined, non-subjective metric measurements are required, IMHO, to really drive value into the report. With the defined use cases (see below) for each measured area, was the documentation easy-to-find and accurate in addressing the use cases?”
Simon’s two cents: We did a lot of behind the scenes research which was documented in more detail for the individual blogs but decided it wasn’t right to bloat the report with more information, as it’s already quite big. Maybe we should write a report on each category in uber depth and collate them as an online de facto App Server book – Would folks like something like this? Let us know in the comments below!
JBoss
“The command line on page 11 (you mention Glassfish’s asadmin) – we’ve tried really, really hard to ensure users never have to hack XML files – we have a very lightweight and responsive browser-based console and a super powerful command line (jboss-cli)
For startup times – don’t know which profile you used for JBoss – - obviously running the full EE profile wouldn’t be a fair comparison – Web profile would be closer to Jetty / Tomcat.”
Jetty
“The most glaring omission was the existence of a professional services or commercial support company for Jetty called Webtide, entirely composed of Jetty committers. We have existed for around 7 years and before that it was Mortbay which had existed for almost a decade supporting Jetty. While Webtide was bought by Intalio a few years back, the Webtide brand has been going strong and helping a great number of both large and small companies achieve success with Jetty and CometD.”
TomEE
“There are a couple features in TomEE specifically we’re quite proud of that I think have a strong impact on the Real Performance Metrics. While most App Servers have achieved very fast startup, this is still a ‘happy path’ metric. It focuses on the fast outcome when everything goes right.
As developers, we spend most the day doing things wrong. This is the nature of our job: to make things that don’t work, work; to make things that don’t exist, exist.
Let’s say your app server can deploy your app in 1 second. If you have 30 mistakes and issues in your app, this doesn’t mean you are 30 seconds away from a successful deploy. In TomEE we have strong focus on cutting that time down as much as possible. When you make mistakes in your app we do our best not just to tell you what you did wrong, but we’ll do our best to guess what you meant and tell what is right. We do this in compiler style, so if there are 30 mistakes in your app, you get all 30 on the first deploy.
As deploy and startup speeds reach zero, this is the next wave of innovation. I wouldn’t say we’re perfect at it, but we’re quite far and it’s something we’re always working towards.”
3. Do you think that the categories we chose are strongly representative of the developer needs of an application server today? What would you add/take away?
Liberty Profile
“Difficult to say. Although I spend my days developing an application server and need a good understanding of Java EE developers, I think people who live and breathe Java EE development would be better placed to answer this. As I said earlier, though, I don’t think the Administration & Management UI question is so relevant to developers.”
Glassfish
“We’d like to see defined use cases up front. Then apply the use cases to each section of the report. Example: Developing an application that creates a user interfaces, executes some simple business logic, and then interact with a database. Then come up with a more complex use case (distributed transactions, enterprise connectivity, etc). Apply these to each measured area.”
Simon’s two cents: I like this idea, a future productivity report for App Servers may be on the cards…
Your time is too valuable to sit around waiting for your code to redeploy. Seriously, imagine how much more you could accomplish if you choose to Eliminate Redeploys from your daily routine!
Jetty
“I think in general the categories were good, though I will say that I would have added a section on modern protocols since when it really gets down to it they will represent the largest advancement in both performance and scaling in the immediate future. With Jetty 9 we have taken this to heart and refactored those layers of Jetty to support HTTP, Websockets, SPDY and the forthcoming HTTP/2 (initially based on SPDY) protocols as equals. With SPDY (and our SPDY Push implementation) you can achieve up to a 30% improvement in latency without changing your web application…and we were first onto the Java scene with support for this protocol.”
Tomcat
“As the Developer Productivity Report shows, Tomcat was the preferred server. Not by one or two points, but beating all the other servers combined. You see this echoed in Java PaaS environments that have almost universally chosen to adopt Tomcat.
I suspect what this tells us is while the categories were spot on, the importance developers might place on each is likely not the same for each category. Some will be higher and some will be lower. It could make for a great follow-up poll; which categories are most important for developers. That information would certainly be a good way to get vendors to focus on what developers need the most.”
4. Who do you consider as your biggest competitor in the app server market today?
Liberty Profile
“I don’t think there is a “biggest” competitor today. There are a lot of very good products out there, they all have their good and bad points. When I look at the others, I tend to look at their strengths and weaknesses as a benchmark for making Liberty profile better.”
JBoss
“We compete with WebSphere Application Server (WAS) & WebLogic Server (WLS) commercially and probably Jetty for developer mindshare.”
Jetty
“In terms of market share, Tomcat has always been the leader in the servlet arena. They have a simple configuration mechanism and distribute with a management console that is one of the things that has lead operations folks to choose Tomcat. Though there has to be a reason that we so consistently hear that folks prefer developing using Jetty, but then they are forced to deploy onto Tomcat for production… This will be a focus moving forward for us once Jetty 9.1 is released.”
TomEE
“For TomEE specifically, I’d say our closest allies are JBoss and GlassFish. That wasn’t the exact question, but I’d say in a healthy market those answers should be the same. They do amazing things for Java EE and App Servers in general. We’ve seen days where App Servers would try and tear each other down and I think this hurts developers more than helps them. Sell your software based on how good you are, not by how terrible can make others appear.
Just this year, for example, we in the TomEE team worked very closely with the GlassFish team to get some key ideas in to Java EE 7 — yes, I’m a proud contributor to GlassFish! As well we were put up for a JBoss Community Award for TomEE’s support of Arquillian — a remarkable testing tool that benefits all app servers and developers. These are good times for developers.”
GlassFish
“.NET. We’re trying to grow the Java EE pie.”
5. How would you re-rank the app servers if you could choose? (we’ll take it for granted that your tool would be in first place ;-))
Liberty Profile
“I have to admit I was expecting Tomcat to come out on top, although I think Liberty’s dynamic behaviour means it grows with your applications needs much better, not to mention our continual focus on runtime performance. Other than that, I don’t really want to start any religious debates of the Emacs/Vi(m) persuasion…”
JBoss
“I wouldn’t re-rank but would include JBoss and Glassfish with WAS / WLS comparisons – all are full featured Java EE servers with rich operational capabilities. I wouldn’t want readers to think that WAS / WLS are in a class of their own – the hundreds of organizations who have moved to JBoss from WAS / WLS are evidence to the contrary.”
There’s a better way to code in Java. Learn more about how you can eliminate redeploys form your daily routine.
Simon’s two cents: I think this is actually more of a compliment, that although JBoss and GlassFish can be compared against WebSphere and WebLogic in production, they’re flexible enough to even be compared against Tomcat and Jetty from a developer perspective. Our next App Server report, based around production concerns will compare WebSphere and WebLogic against JBoss and GlassFish among others.
Jetty
“Jetty (Obviously)
JBoss (cause if you don’t care about footprint, get all the Java EE features you can!)
Tomcat (if its not Jetty or JBoss, then why not?)
Glassfish (OSGi architecture is sort of neat)
IBM Liberty Profile”
Tomcat & TomEE
“I wouldn’t :) We’re all innovating and that’s all that matters.”
Thanks to the following respondents for the quotes above:
Liberty Profile – Alasdair Nottingham, Liberty Profile Development Lead, IBM
GlassFish – John Clingan, Principal Product Manager, Oracle
JBoss – Rich Sharples, Product Management Director for Middleware, RedHat
Jetty – Jesse McConnell, Lead Engineer/webtide.com, Intalio
TomEE – David Blevins, Founder of Apache TomEE, OpenEJB and Geronimo
Get the Full Report here!