Results of the Grand C++ Error Explosion Competition

After much deliberation, the winners of the Grand C++ Error Explosion Competition are finally selected. There are two different award categories. The winners of the first category are those submissions that produced the largest error with the smallest amount of source code. These entries contain a multiplier number, which is just the size of the error message divided by the size of the source code. The second category is artistic merit.

Some of the code samples shown will overflow when displayed on the web. We apologize for any inconvenience this may cause.

Biggest error, category Anything

Name: Turtles all the way down
Author: Ed Hanway
Multiplier: 5.9 billion

This entry is the best implementation of the most common pattern, the double include self. Here’s what it looks like:

#include ".//.//.//.//jeh.cpp"
#include "jeh.cpp"
`

This implementation produced almost six times the amount of error messages of the second best entry of the same type.

Biggest error, Category Plain

Name: y u no backreference?
Author: Chris Hopman
Multiplier: 790 million

The rules permitted includes in the plain category, so obviously the double include was used in this category as well.

#include "set>.cpp"
#include "set>.cpp"

Biggest error, category Bare Hands

Name: Const Variadic Templates
Author: Marc Aldorasi
Multiplier: 657 million

The bare hands category did not allow for any use of the preprocessor, which lead most people to use recursive or variadic template initiations. This entry was the most compact of the lot.

template<class T,class...>class C{C<T*const,T,C>a;C<T,C>b;};C<int>c;

Best cheat

Name: What's perl?
Author: Chris Hopman

There were several interesting cheat attempts in this competition. For example did you know that in C++ digraph expansion happens after line continuation expansion? We sure did not.

Many tried to exploit a division by zero bug in the verification script. One submission even had a patch for this, which tried to changed the evaluator script to make their entry evaluate to infinite error. The best cheat went in a completely different direction, however.

The actual code consisted of only one space. Since this entry was in the anything category, it was accompanied by a list of header search directories. That file looked like this.

/usr/include; perl -e "@c=\"x\"x(2**16); while(1) {print @c}" 1>&2

When passed to the C++ compiler invocation line, this allows the shell code to escape the test harness sandbox. Extra credit for using Perl, which is the only language less readable than C++ templates.

Most surprising

Name: templates and nested classes are not best practice
Author: Aaron Grothe

This piece of code looks innocent but explodes in a completely unexpected manner. We also tested this with Clang, which detects correctly the missing semicolon, after which it anyway tries to evaluate the infinite template recursion and eventually segfaults. This entry gives a glimpse on the kinds of issues an IDE’s code completion engine needs to guard against.

template<class T>class L{L<T*>operator->()};L<int>i=i->

Most lifelike

Name: Bjarne's nightmare
Author: Victor Zverovich

Suppose you are given a task of adding some new functionality to an existing code base. You have been told that the guy who wrote it was “really smart” and that his code is of “enterprise quality”. You check out the code and open a random file in an editor. It appears on the screen. After just one microsecond of looking at the code you have lost your will to live and want nothing more than to beat your head against the table until you lose consciousness.

This entry could be that code. We’re glad we only needed to measure it rather than to understand and alter it.

#include <map>
#include <algorithm>
template<class T,class U>void f(T,U u){std::vector<std::vector<T>>v;auto i=end(v);find(i,i,u);find(i,i,&u);U*p,**q,r(),s(U);find(i,i,&p);find(i,i,&q);find(i,i,r);find(i,i,&r);find(i,i,s);find(i,i,&s);}template<class T>void f(T t){f(t,0);f(t,0l);f(t,0u);f(t,0ul);f(t,0ll);f(t,.0);f(t,.0l);f(t,.0f);f(t,' ');f(t,L' ');f(t,u' ');f(t,U' ');f(t,"");f(t,L"");}int main(){f(0);f(0l);f(0u);f(0ul);f(0ll);f(.0);f(.0l);f(.0f);f(' ');f(L' ');f(u' ');f(U' ');f("");f(L"");f(u"");f(U"");}

Barest hands

Title: whatever
Author: John Regehr

This entry does not have any template definitions or include recursion and yet it put up an admirable fight. This serves as an important reminder to all of us: when used correctly even the simplest of tools can be used to build impressive results.

struct x struct z<x(x(x(x(x(x(x(x(x(x(x(x(x(x(x(x(x(y,x(y><y*,x(y*w>v<y*,w,x{}

Epilogue

We would like to thank all people who participated in the competition. We hope that all participants as well as you readers have enjoyed this experience.

The final question now remaining is whether there will be a second TGCEEC next year?

The answer to this is simple: yes, if you, the people, demand it.

Till we meet again.

Statistics of week one

We have had a wonderful week here at tgceec headquarters. There have been a total of 6 submissions to the anything category, 11 to the plain category and 10 to the bare hands category. Entries have ranged from variadic template abuse and preprocessor trickery to plain old cheating which is just the way we like it.

Thanks to all those who have submitted their entry thus far. There are still three weeks remaining to participate so get those keyboards cracking. Feel free to spread links to the competition via any social or antisocial media of your choice.

Preliminary resource limiting test harness released

The test harness has been updated to limit the amount of resources each entry is allowed to take. The aim is to run the final tests on an i7 machine with 16 GB of RAM. The compile process is allowed to take 15.5 GB of RAM and use 5 minutes of CPU time. These limits are enforced with ulimit, those interested in the details can look at the source code.

We hope that these limits are the final ones. However we can not guarantee it at this time. Keep watching this page for final confirmation on this issue.

Test harness updated

There was a slight bug in the test harness code. Several people did the right thing and tried to exploit it to achieve infinite ratio of errors to code. This loophole has now been plugged.

To be pedantic, this would not have yielded infinite ratio, since an empty file produces no error messages. The ratio would thus have been 0 / 0, which is NaN.

The first entries are in!

It did not take even 12 hours for the first entries to arrive. Thanks to all those that have participated. For everyone else: there is still time so get coding.

We have also received questions about CPU and memory usage limits of the entries. The current plan is to run the tests on an i7 machine with 16 GB of memory. If we receive entries that break this massively then we will look into other avenues. These include, but are not limited to, compiler flags that limit GCC’s internal limits or just killing all processes that take too many resources.

We hope to have further information as soon as possible. Thank you for your understanding.

Introducing the Grand C++ Error Explosion Competition

All C++ coders have war stories of error messages that spawn dozens of
pages. But now the time has come to test who can conjure up the
grandest error message.

The challenge

Write a piece of C++ code. Measure the amount of bytes of error
messages it produces when compiled. Divide the given value by the
number of bytes in the source file. Beat others by obtaining the
highest ratio.

The Platform

All tests will be run on an up to date 64 bit install of Ubuntu 13/10
(saucy). The compiler used is the default GCC provided by the
distro. The flag -std=c++11 is used when compiling entries.

Categories

There are three categories in the competition. The first is “anything
goes”, where contestants are allowed to include any header in the
system and use the preprocessor freely. The maximum source size is 256 bytes.

The second category is “plain”, where any standard library header may
be included. Including other headers is forbidden as is any other use
of the preprocessor. The maximum source size is 512 bytes.

The last category is “bare hands”. In this category all uses of the
preprocessor are forbidden, even includes. The maximum source size is
1024 bytes.

Are there other categories?

The judges reserve the right to give out honorary mentions for
exceptional achievements. They also reserve the right not to.

What about cheating?

We prefer to think of it as creative problem solving and encourage it.

Rule changes

The organizers reserve the right to change the rules of the
competition at any time in any way. The most probable reason being a
fun-ruining loop hole in the rules. All entries submitted will remain
in the competition but if they do not pass the new requirements they
are silently disqualified. Rule changes are posted on this web site.

How can I participate?

Simply go to https://github.com/jpakkane/tgceec and check out the
test harness. It contains a text file explaining how to format,
measure and submit your entry.

Can I participate multiple times?

Sure, you can participate as many times as you like. Just send your
new entry to the address given. The judges ask that in order to save
everyone’s time and effort contestants would minimize the number of
times they submit code.

Prizes

Apart from fame(*) there are none.

(*) Fame not guaranteed.

Judging

The identity of the judges will be kept a secret as will the sordid
details of how they rank the contestants. However we can reveal that
the measured error to code ratio is the main tool in this ranking.

Any choices made by the judges are final and no correspondence will be
entered into. Also, no compensation is given to any participant or
non-participant for any reason whatsoever.

When is the deadline?

You must submit your entry by 2014/01/18 23:59EET. Any entries
received after that are silently rejected, even if the delay was
caused by unforeseen events such as server outages.

Will submissions be published?

Only those that win some sort of an award.

Can I remain anonymous?

Yes, you can submit your entry under an alias. See the competition git
repository for details.

Who are the organizers?

The competition was organized by Jussi Pakkanen with help from Jukka
Laurila and Juhani Simola.

What about copyright?

All submitters retain full copyright on their submissions. We only
need a permission to publish the source of winning entries. Submitters
are responsible for ensuring they have the necessary rights to their
submission (preferably by writing all of it themselves).

Can I include source from project X to my entry?

If the license of the project allows it. The submitter is responsible
for making sure that the entry fulfills any external license
requirements. Note that some licenses require you to keep a license
header in your source files, which is a big handicap (most license
headers are bigger than the maximum submission size, for example).

I found a bug in the rules or the validator script

Please send email to <tgceec@gmail.com> and we’ll get it fixed.

Is that it?

Yes. Now go out there and have fun!