Pitfalls to avoid in Robot Framework

I recently was working on a vulnerability assessment project, where the developers had used robot framework in the backend for their product. I hadn't had the pleasure to work with robot framework before, so I was eager to start. Robot Framework is a great project, which can be used to automate most of your testing needs.

What I found was a bit worrying for me and lead me to the conclusion that robot framework hasn't been designed to be used with external input.

I will now show you a simplified example of the problem I discovered. Although in my examples I pass values to the scripts variables from the command line, they work in the same fashion if robot framework is called programmatically.

Consider the following simple example:

The code should simply test if the variable "var" is empty and if it's not, the value is then printed to the console. You can run it as follows:

If the variable would've been empty, we would've been missing the "We got: foobar" output.

Simple enough, eh?

The example above contains the issue I discovered in the assessment. Let's say that the variable "var" comes from outside, originally from untrusted source. You might ask: so what? it's just a variable. Technically you would be correct, but what if I told you, that to parse the condition and determine if it's true or not, it will be passed to the pythons eval function? And what if I told you that since Robot Framework 2.8 the sys and os modules are imported by default when evaluating the condition? You might be shocked, but if you aren't let me explain the issue with a few more details.

While performing the assessment, I had a test case, where I only entered a double quote as the value and got the following error:

Evaluating expression '""" != ""' failed: SyntaxError: EOF while scanning triple-quoted string literal (, line 1)

My initial thought was that I managed to break some python script, because the SyntaxError is what python raises. But when you enter the whole exception to google, you see that it's actually coming from the Robot Framework (which calls python in the background).

So, what is entered to the variable called var, is passed to the pythons eval with the surrounding quotes. With the sys and os modules imported by default, we can trivially execute operating system commands, but the injections are blind. For the attacker the default imports are really helpful, after all you can't import new modules from the eval -function. Let's demonstrate it again with another example:

Here we see, that if the python syntax is seemingly correct, we get no errors. You might get the command output with for example os.system call, but that would only print the result to stdout and in the assessment, I never saw any output from my commands, no matter how I called them. When I tried to execute something along the lines of "sleep 20" I was able to confirm that os commands are actually executed. When I have access to the operating system, it becomes much easier:

First, I check that there is no file called bar and that it appears with the echoed contents after calling robot framework.

So, how could I exploit this? If the server is allowed to create TCP connections out to you, then the exploitation becomes trivial. Simply share a backdoor with http, then make the server download it with wget and finally execute it.

I used msfvenom to generate the payload, but anything goes here. Here's what I executed (yea, I'm using localhost for the demo here):

And here's what I see on my SimpleHTTPServer and nc shells:

I think the issue should be clear by now. Naturally I could've just told you to go RTFM.

According to the robot fw documentation:

Many keywords, such as Evaluate, Run Keyword If and Should Be True, accept an expression that is evaluated in Python. These expressions are evaluated using Python's eval function so that all Python built-ins like len() and int() are available. Evaluate allows configuring the execution namespace with custom modules, and other keywords have os and sys modules available automatically.

 -- http://robotframework.org/robotframework/latest/libraries/BuiltIn.html

Currently there's also another way for using variables:

Starting from Robot Framework 2.9, variables themselves are automatically available in the evaluation namespace. They can be accessed using special variable syntax without the curly braces like $variable. These variables should never be quoted, and in fact they are not even replaced inside strings.

Using the $variable syntax slows down expression evaluation a little. This should not typically matter, but should be taken into account if complex expressions are evaluated often and there are strict time constrains.

 -- http://robotframework.org/robotframework/latest/libraries/BuiltIn.html

In my personal experiments, when variables are used with the latter syntax ($var), the injections don't work anymore. So rather than trying to blacklist all the possible ways to inject python code, you might want to use the new syntax and take the the hit from slower evaluation. If I'm mistaken, please leave a comment.

In the examples I've used Robot Framework 3.0.2

From the builtins page you can see that at least the following keywords use evaluation:

• Continue For Loop If
• Evaluate
• Exit For Loop If
• Pass Execution If
• Return From Keyword If
• Run Keyword And Return If
• Run Keyword If
• Set Variable If
• Should Be True
• Should Not Be True

And no, I have no idea if it's used elsewhere, so don't take the above as a definitive list.

TL;DR think again before passing external data to the robot framework variables.