Python for the busy Java Developer

If you are reading this book, then chances are that you are a busy Java® developer who is interested in learning Python®. If so, I hope that by the time you are done reading this short book:

This book is not for the beginner programmer. I assume that you are comfortable programming in Java (or some other OOP language like C#) and hence, I will not bore you with explanations of basic concepts like variables, functions and classes.

This book is divided into three broad chapters:

In the first chapter, we will take a brief look at the Python language itself and understand what it has to offer. Next, we’ll get down to the details of The Syntax before wrapping up with a look at the wider Ecosystem surrounding the Python language.

I’ve been working as a software developer for more than a decade now, spanning multiple business domains in a variety of programming languages & environments. I’ve worked on several high-performance, server-side applications written in Java. I’ve also done web development and systems automation work in Python. I’ve had the opportunity to work in a startup, in Fortune 50 companies and a couple of places in between. My full resume is available on my website.

Offline, I live in Singapore with my lovely wife and our adorable daughter. Online, I can be found blogging at antrix.net and tweeting as @antrix.

I’d love to hear what you’ve to say about this book — please email me at deepak@antrix.net.

I’d always heard it being said, but only now do I truly realize it: writing a book is hard work! It would have been harder still had it not been for the support from my family and friends.

I wish to especially thank Hitesh Sarda, Rohit Sharma and Srijith Nair for the incredibly detailed and thoughtful feedback that they provided as I wrote this book. It is much better thanks to them.

I must also acknowledge the constant encouragement that I received from my wife, Sonika. I can’t thank her enough for her patience and support as I took time out to write this book.

Those are a lot of adjectives for one sentence! Let’s unpack them one at a time.

Python is distributed under an open source, BSD-style license called the Python Software Foundation License Agreement. This is a very permissive license that allows great flexibility in how Python can be used. Python’s development is done in the open by a large and diverse community of volunteers.

Python is general purpose in that you can use it to build a variety of applications running the gamut from simple scripts and command-line tools to web applications, network servers, GUIs, scientific applications, etc.

We know that Java is a statically typed language, i.e. the types are checked and enforced at compile time. In contrast, Python is dynamic which means that the types are checked only at runtime. But Python is also strongly typed, just like Java. You can only execute operations that are supported by the target type.

Another way to think about this is that while in Java, both variables and objects have types associated with them, in Python only objects have types, not the variables that they are bound to. In Java, when we declare:

The variable obj is declared of type MyType and then the newly instantiated object of type MyType is assigned to it. In contrast, in Python, the same declaration would read:

Ignoring the missing new keyword (which Python doesn’t have), obj is simply a name that is bound to the object on the right, which happens to be of type MyType. We can even reassign obj in the very next line – obj = MyOtherType() – and it wouldn’t be a problem. In Java, this re-assignment would fail to compile[1] while in Python, the program will run and will only fail at runtime if we try to execute an operation via obj that is incompatible with the type assigned to it at that point in time.

Python is object oriented and supports all the standard OOP features that Java has like creation of types using classes, encapsulation of state, inheritance, polymorphism, etc. It even goes beyond Java and supports features such as multiple inheritance, operator overloading, meta-programming, etc.

Python also supports a rich set of functional programming features and idioms. In Python, functions are first-class objects that can be created, manipulated and passed around just like any other object. While not as functional as say Clojure, Python certainly offers much more to the functional programmer than Java.[2]

Another similarity between the languages is in terms of manual memory management, in that there is none. The language runtime takes care of correctly allocating and freeing up memory, saving the programmer from the drudgery – and mistakes – of manually managing memory. Having said that, the JVM garbage collectors are much, much better performing than the Python GC. This can become a concern depending on the type of application you are building.

Finally, and above all, Python is fun and a joy to use. This is a strong claim to make but I hope that by the time you are done reading this book, you’ll agree with me and the millions of other Python programmers out there!

Python is the brain-child of a Dutch programmer named Guido van Rossum. He started working on it when he got frustrated with the ABC language in the late 80s and after some years of private development, released the first version of Python in 1994. This actually makes Python older than Java, the first version of which was released in 1996, a full two years later! Since then, the language has continued to refine and evolve, with Python 2.0 being released in 2000. As of this writing, the 2.x versions are the most widely deployed.

In version 3.0, the language designers decided to break backwards compatibility in order to clean up some of the accumulated language warts. Although this has been good from a language perspective, it has been a significant hindrance to those upgrading from 2.x to 3.x. Imagine if Sun had decided to introduce generics in Java 5 without type erasure, thus breaking backwards compatibility? The Java language would’ve been much nicer today but the transition period would’ve been difficult, to say the least. That is the kind of transition the Python user community is going through right now.

From the outset, Python’s development has been done in the open with a community of volunteers contributing to the language and the core libraries. Any language changes are proposed and discussed through a process called the Python Enhancement Proposals, with Guido having final say in deciding the outcome. For his stewardship and continuous involvement in the development of Python, Guido is affectionately called the Benevolent Dictator For Life. He also periodically writes a Python History blog chronicling the evolution of various language features.

This book is full of example code and the best way to follow along is to actually try out these examples by yourself. To do this, you’ll obviously need to install Python on your system. But an easier way is to check if you already have access to a system with Python installed! Almost all systems running Linux should have Python pre-installed. Recent versions of Mac OS X also come with Python pre-installed. Just open a command shell on either of these two systems and type in python. If you get a Python shell prompt, you are all set! The version of Python installed may be a bit outdated but it should be sufficient to get started.

If you can’t find a system with Python installed on it, just head over to python.org/downloads/ and follow the instructions to get Python.

Python source code is organized in files with a .py extension. The python executable interprets the source code and translates it into a Python language specific bytecode which is stored in .pyc files. This bytecode is then executed by the Python Virtual Machine which is also invoked by the same python executable. Although this sounds like two steps, in reality, it is just one step with the bytecode generation happening on the fly.

This is in contrast to Java, where the responsibilities for the parsing & compilation of source code and the actual execution of the compiled bytecode are split between javac and java respectively. In Python, the python executable handles both steps. In fact, .pyc files are in-effect just intermediate caches to hold the translated bytecode. They are not strictly necessary for execution.

There are multiple IDEs available for writing Python code. PyDev, based on the Eclipse framework and PyCharm, based on the IntelliJ IDEA framework are two of the more popular choices. While having an IDE is nice, it is perfectly feasible to write Python code using a plain text editor such as Vim[3] or Sublime Text.

One interesting feature of Python that’s missing in Java is the REPL, short for Read Eval Print Loop. A quick demo would be useful here. If you’ve got access to a Python installation following the instructions in Installation, go ahead and launch a python shell:

When you run the python executable in this manner, it starts up in an interactive mode. The first few lines contain information such as the version and the underlying operating system. After that, you are presented with the >>> prompt. This is where all your interaction with Python will occur. The python shell is running a loop which will read everything that you type in at this prompt, evaluate what it has read and then print the result. Thus the name, REPL.

Let’s try it out:

We typed in 10 + 10 at the prompt and hit the Enter key. The Python REPL read this value, evaluated it and printed out the result. Then it went back to the prompt waiting for our next input. Let’s try some variable assignment:

In this case, we didn’t see any output because what we entered was just a statement, not an expression. But it did modify the state of the python shell. If we query for x again, we’ll find:

Let us call one of the built-in functions named help:

Calling help on any object brings up a paged view of what is, effectively, the JavaDoc for the object’s class. To exit the help view, just type q at the : prompt and you’ll be back at the >>> prompt.

The full documentation view for an object can be quite verbose. If you just want a quick overview of what attributes an object exposes an object supports, use the dir function:

Ignoring the funky double-underscores for now, what dir(x) returned is effectively a directory of all the attributes available on the object. You can access any of them using the . syntax:

You can also use the dir() function without any argument to get a list of built-ins:

This gives a list of functions and other objects that are built-in and do not have to be imported from other packages. This is analogous to how everything defined in the java.lang package is available everywhere in Java without having to explicitly import it.

There’s one last thing I wish to show before we wrap up this section. Let’s create a new file named hello.py with the following contents:

Now execute python passing in this file as an argument:

This is more along the lines of a traditional development process: write code in a file and then execute that file. This also demonstrates how the python executable combines the role of javac and java in one process.

With that brief demo of Python, we are ready to dive into the nitty-gritty details of the language’s syntax!

Well, that was easy! Moving on …

Here’s a bit of Python code that .. well, I don’t have to tell you what it does, do I? Most Python code is like this, eminently readable and almost pseudo-code like. But you must have noticed a few things like the lack of semicolons as statement separators. Let’s work through this code one line at a time and see what else is new and different compared to Java.

Some of the basic data types in Python are numbers, strings and collections.

Lists are the workhorse data structure in Python and I exaggerate only slightly when I say that mastering them is the key to mastering Python! Although earlier I said that they are like java.util.ArrayList, they are truly much more than that. But first, let’s look at a short example demonstrating their use as a basic array:

Apart from single element access, what sets apart Python lists is the ability to extract element ranges from lists. This is accomplished using the slice syntax. Here’s how:

The copy returned by the slice notation is a shallow copy and we can demonstrate this with a quick example:

Now let us turn our eye towards performing operations on lists. The first thing one would want to do with a list is to iterate over its elements. There are a few different variants of list iteration in Python, either using the vanilla foreach style syntax or augmenting it with the enumerate, range and len built-in functions.

Next, let us see how we can mutate lists.

Lists can also be used as simple stacks and queues.

There are many more operations that a list provides, such as extend, insert and reverse. But let us now look at one of the most interesting features of lists: Comprehensions.

Consider the following code which computes the factorial of the first few integers:

The above procedural loop can be replaced by a functional one-liner using the built-in map function as follows:

Python defines list comprehensions using the syntax: new_list = [function(item) for item in L]. We can re-write the factorial loop using this syntax as follows:

Here’s another variant that introduces a conditional in the comprehension:

If the list/object being iterated over is large (or even unbounded), then a variant of the list comprehension syntax called generator expressions can be used. In this snippet, factorials_of_odds is lazily computed as you iterate over it.

Syntactically, the only difference between list comprehensions and generator expressions is that while the former are enclosed in square brackets, the latter are enclosed in round brackets.

With that rather verbose introduction to lists, let us turn our attention towards another building block of procedural programming: Functions.

Functions in Python are quite a bit more flexible than Java.

Here’s what a function definition in Python looks like:

Functions definitions begin with the def keyword (Hello Scala & Groovy!) followed by the parameter list in parentheses. Once again, there are no curly braces and just the indentation defines the scope of the function body.

The documentation within triple quotes is called a docstring, similar to Javadoc. Calling help(a_function) will display this docstring.

We don’t declare the types of the parameters, relying instead on duck typing i.e. as long as the parameter argument has the attributes our function expects to operate upon, we don’t care about its real type.

Let’s see how a_function as defined earlier behaves when called with different argument values.

Everything in Python is an Object and as you’d expect, the way to create objects is to start from Classes. Consider the following definition of a simple Person class.

As in Java, object is at the root of the class hierarchy but unlike Java, it needs to be specified explicitly in Python (although not in Python 3).[5] Inheritance declarations do not use a special keyword like extends. Instead, the parent class' name is enclosed within paranthesis after the declaring class' name.

The __init__ method is the initializer method and is analogous to the Java class constructor. There is also a constructor method called __new__ but you won’t use it unless you are doing metaprogramming like writing factory classes, etc. Within __init__, all the instance fields — called attributes in Python — are initialized. Note that we did not have to pre-declare all the attributes of the class.

The first argument of all instance methods is self. It is the this reference that is implicit in Java but made explicit in Python. Note that the literal name self isn’t mandatory; you could name it whatever you want. If you named it current, then full_name would be defined as:

The double underscore notation used in the __init__ method name is a Python convention for declaring special methods. The __str__ is another special method. Its behaviour is exactly like that of the toString() method in Java. I’ll explain what makes these methods special when we talk about Protocols.

Now here is some example usage of this Person class.

You’ll notice that we didn’t declare whether our fields are private or public. We just accessed them as if they are public. In fact, Python does not have the concept of visibility at all! Everything is just public. If you wish to indicate to the user of your class that a particular attribute or method is an internal implementation detail, then the convention is to prefix the attribute/method name with a single underscore and the person using the code will know to tread carefully.

Protocols are like Java interfaces in that they define one or more methods that offer a particular behaviour. However, unlike Java interfaces, protocols are not explicitly defined in source code. The closest equivalent would be the equals(), hashcode() and toString() methods in Java. These methods aren’t part of any explicit interface. Yet, we have an implicit convention[6] that these methods will be invoked in certain situations. So it is with Python Protocols.

The Python language defines several different protocols such as sequences, numeric, containers, etc. These protocols manifest themselves in terms of special syntactic support in the language grammar. Pretty much every language syntax in Python is implemented in terms of protocols and thus, can be made to work with your own types by implementing the relevant protocols.

Let me explain this further using one of the protocols, the Container protocol, as an example. Consider the following OrderRepository class definition which provides access to a database backed collection of Order objects.

I’ve elided the full class definition and only shown the three methods which are part of the Container protocol. Since OrderRepository can now be said to implement the Container protocol, it allows us to use it in the following way:

You can think of this as operator overloading or syntactic sugar, whichever fits your mental model!

Here’s a list of some of the other protocols that Python supports and the syntax that they power.

The most basic unit of source code organization in Python is the module which is just a .py file with Python code in it. This code could be functions, classes, statements or any combination thereof. Several modules can be collected together into a package. Python packages are just like Java packages with one difference: the directory corresponding to a package must contain an initializer file named __init__.py. This file can either be empty or can optionally contain some bootstrap code that is executed when the package is first imported.

Suppose we have a code base organized as follows:

Given this directory listing, we can see that there are two packages: db and model; and four modules: cart, mysql, postgresql and order.

We covered quite a bit of ground in this chapter. We started with a look at the basic syntax and built-in types that Python provides. We then moved on to the building blocks of functions and classes. We then familiarized ourselves with Python’s implicit interfaces, i.e. Protocols. Finally, we learned about organizing source code into modules and packages.

While this book ends our discussion of Python language syntax over here, there’s more to Python than this! Some of the advanced topics that we didn’t discuss include decorators, properties, generators, context managers and I/O. These are topics you’ll find yourself exploring once you get comfortable writing basic Python code. To help you along in your learning, I’ve compiled some useful resources in References at the end of this book.

In the Java world, we know that Java is much more than the language itself. There’s the JVM, the Java language runtime that allows deploying applications on a variety of hardware and operating system targets. There are the Java SE and EE Standard Libraries which provide a lot of useful functionality out of the box. And of course, there’s an immensely rich variety of third-party libraries and frameworks to choose from. It is the strength of this ecosystem that makes Java a great platform to build upon.

So it is with Python!

Python too can be deployed on different hardware targets like x86, ARM & MIPS; on multiple operating systems like Linux, Mac OS X, Windows & Solaris. It can even be deployed on other software runtimes like the .NET CLR (c.f. IronPython) or the Java Virtual Machine (c.f. Jython).

Python has got a great standard library with lots of functionality delivered out of the box. Batteries Included is often used to describe the richness of the standard library.

Beyond the standard library, the 3rd party libraries & frameworks landscape is similarly rich, ranging from numerical and scientific computing packages, to NLP, to networking, GUI and web frameworks. You’d be hard pressed to find a domain for which there isn’t already a Python library out there.

To help you get started, I’ve put together a couple of lists that’ll help you pick out the Python counterparts of the Java tools and libraries that you are already familiar with. These lists are by no means exhaustive but I do believe that they represent the most popular options in use today.

In the Python world, third-party library installation is typically done using a tool named pip. It can read the requisite third-party package dependencies for your project from a standardized requirements.txt file and install/upgrade packages as necessary. If needed, it can even download packages from a central repository named pypi, short for the Python Package Index. This is the Python world’s equivalent of Maven’s Central Repository.

With third-party packages comes the question of how they are located during application runtime. Java’s solution is the Classpath and Python’s equivalent is the PYTHONPATH. Conceptually, they are similar in that both specify a list of locations where the language runtime should search for packages being imported. Implemention wise, they differ a bit. PYTHONPATH is set as an environment variable, e.g. PYTHONPATH=/path/to/foo:/path/to/bar, a syntax similar to the typical system shell’s PATH variable. Another flexibility that Python provides is that this library search path can be modified at runtime just by manipulating the sys.path attribute exposed by the built-in system module. As you might have guessed, sys.path is initialized from the value set for the PYTHONPATH environment variable.

I mentioned earlier that there are multiple runtime implementations of the Python language. The canonical implementation is CPython, named so because the core of the language runtime is implemented in C. It is the most widely used implementation and regarded as the reference for the Python language. Alternative implementations include Jython, which is Python running on the JVM; IronPython which is Python running on the .NET Common Language Runtime, and PyPy which is an upcoming implementation of Python that provides a number of improvements like a runtime JIT compiler, secure sandboxing and green-threads based concurrency.

As a Java developer, you may wish to explore Jython a bit more. Since it is essentially Python code translated to Java bytecode and running inside a Java Virtual Machine, it allows for easy interop between Python and Java. As an example, you could write high-level business rules in Python and call into a Java rule engine to execute those rules.

Since Python is a dynamic language, there’s a class of programming errors that get revealed at runtime, instead of being caught at compile time like they would in a statically typed language. Thus, writing automated tests for your Python programs is even more important than it is in Java. The Python standard library ships with a unit testing framework creatively named unittest, that should serve as good start. If you find that unittest is missing some features you’d like or is just a bit too cumbersome to use, then nose is a good third-party alternative. In its own words, nose is nicer testing for Python.

Another tool in the modern developer’s arsenal for keeping code quality high is automated static analysis of source code. The findbugs and checkstyle equivalent for Python is pylint. It can enforce a uniform code formatting style, detect basic programming errors and duplicate blocks of code. And as you would expect, you can integrate it in your IDE as well as CI build server.

If you are building a desktop GUI, then the PyQT and PyGTK libraries are popular options. These are Python wrappers around the popular Qt and GTK libraries, respectively. Both these frameworks are cross-platform, just like Java Swing.

While desktop GUI applications still have their place, it isn’t a stretch to say that a majority of new applications being built today are web applications. Python is very well suited to this task. In fact, you may be surprised to know that many of the Internet’s most popular websites, like YouTube and Reddit, are written in Python.

Spring MVC is a popular choice for full stack web development in the Java world. By full stack, I mean a framework that handles everything from frontend user authentication to backend database connectivity and everything in between. The most popular full stack web development framework for Python is Django. If you are just dipping your toes into web development with Python, I strongly recommend that you start with Django. You’ll find it to be a very easy to learn and productive web development environment.

On the other hand, if you are the type of developer who finds full stack frameworks too restrictive to use, then you can mix and match best of breed libraries and roll your own stack. On the database access front, SQLAlchemy is the pre-eminent option. It provides a low-level DB interface like JDBC, a mid-level Spring JDBCTemplate like interface that provides convenient plumbing around writing raw SQL queries and finally, a high-level ORM layer like Hibernate. Depending on your requirements, you can choose just the level of abstraction that you need.

Another aspect of web development is generating responses for HTTP requests, typically HTML pages, using a template engine. This is the kind of work for which you’d reach for the Velocity or Thymeleaf libraries over in Java land. Jinja2 is the go-to templating library for Python applications. It has a pleasant syntax, lots of features and is fairly fast too.

Python web sites are generally deployed using either the Apache web server or a smaller Python specific web server like uWSGI. These web servers rely on a Python web development standard called WSGI – Web Server Gateway Interface. Think of it as the equivalent of Java Servlets in the Python world.