We remain excited about Docker as it evolves from a tool to a complex platform of technologies. Development teams love Docker, as the Docker image format makes it easier to achieve parity between development and production, making for reliable deployments. It is a natural fit in a microservices-style application as a packaging mechanism for self-contained services. On the operational front, Docker support in monitoring tools (Sensu, Prometheus, cAdvisor, etc.), orchestration tools (Kubernetes, Marathon, etc.) and deployment-automation tools reflect the growing maturity of the platform and its readiness for production use. A word of caution, though: There is a prevalent view of Docker and Linux containers in general as being "lightweight virtualization," but we would not recommend using Docker as a secure process-isolation mechanism, though we are paying attention to the introduction of user namespaces and seccomp profiles in version 1.10 in this regard.

Password security is still a hotly debated topic with the UK government advocating technical controls that let users remember simpler passwords and Edward Snowden’s password advice being described as only "borderline secure". Passwords are generally one of the weakest links in the security chain, so we recommend employing two-factor authentication, which can significantly improve security. Time-based One-Time Password (TOTP) is the standard algorithm in this space, with straightforward server-side implementations and free smartphone authenticator apps from Google and Microsoft.

Mesos is a platform that abstracts out underlying computing resources to make it easier to build massively scalable distributed systems. It can be used to provide a scheduling layer for Docker, or to act as an abstraction layer to things like AWS. Twitter has used it to great effect to help it scale its infrastructure. Tools built on top of Mesos are starting to appear, such as Chronos, which is a distributed, fault-tolerant cron replacement. Prominent success stories are appearing, such as Apple's Siri rearchitecting to use Mesos.

Our teams continue to enjoy using AWS Lambda and are beginning to use it to experiment with Serverless architectures, combining Lambda with the API Gateway to produce highly scalable systems with invisible infrastructure. We have run into significant problems using Java for Lambda functions, with erratic latencies up to several seconds as the Lambda container is started. We recommend sticking with JavaScript or Python for the time being.

Predictive analytics are used in more and more products, often directly in end user-facing functionality. H2O is an interesting open source package (with a startup behind it) that makes predictive analytics accessible to development teams, offering straightforward use of a wide variety of analytics, great performance and easy integration on JVM-based platforms. At the same time it integrates with the data scientists’ favorite tools, R and Python, as well as Hadoop and Spark.

HTTP Strict Transport Security (HSTS) is a now widely supported policy that allows websites to protect themselves from downgrade attacks. A downgrade attack in the context of HTTPS is one that can cause users of your site to fall back to HTTP rather than HTTPS, allowing for further attacks such as man-in-the-middle attacks. By using the server header, you inform browsers that they should only use HTTPS to access your website, and should ignore downgrade attempts to contact the site via HTTP. Browser support is now widespread enough that this easy-to-implement feature should be considered for any site using HTTPS.

Kubernetes is Google's answer to the problem of deploying containers into a cluster of machines, which is becoming an increasingly common scenario. It is not the solution used by Google internally but an open source project that originated at Google and has seen a fair number of external contributions. Since we mentioned Kubernetes on the previous Radar, our initial positive impressions have been confirmed, and we are seeing successful use of Kubernetes in production at our clients.

In earlier versions of the Radar, we have highlighted the value of Linux security modules, talking about how they enable people to think about server hardening as a part of their development workflow. More recently, with LXC and Docker containers now shipping with default AppArmor profiles on certain Linux distributions, it has forced the hand of many teams to understand how these tools work. In the event that teams use container images to run any process that they did not themselves create, these tools help them assess questions about who has access to what resources on the shared host and the capabilities that these contained services have, and be conservative in managing levels of access.

The PaaS space has seen a lot of movement since we last mentioned Cloud Foundry in 2012. While there are various distributions of the open source core, we have been impressed by the offering and ecosystem assembled as Pivotal Cloud Foundry. While we expect continued convergence between the unstructured approach (Docker, Mesos, Kubernetes, etc.) and the more structured and opinionated buildpack style offered by Cloud Foundry and others, we see real benefit for organizations that are willing to accept the constraints and rate of evolution to adopt a PaaS. Of particular interest is the speed of development that comes from the simplification and standardization of the interaction between development teams and platform operations.

The emerging Containers as a Service (CaaS) space is seeing a lot of movement and provides a useful option between basic IaaS (Infrastructure as a Service) and more opinionated PaaS (Platform as a Service). While Rancher creates less noise than some other players, we have enjoyed the simplicity that it brings to running Docker containers in production. It can run stand-alone as a full solution or in conjunction with tools like Kubernetes.

Amazon API Gateway is Amazon's offering enabling developers to expose API services to Internet clients, offering the usual API gateway features like traffic management, monitoring, authentication and authorization. Our teams have been using this service to front other AWS capabilities like AWS Lambda as part of serverless architectures. We continue to monitor for the challenges presented by over-ambitious API gateways, but at this stage Amazon's offering appears to be lightweight enough to avoid those problems.

The Elastic Container Service (ECS) is AWS’ entry into the multihost Docker space. Although there is a lot of competition in this area, there aren’t many off-premises managed solutions out there yet. Although ECS seems like a good first step, we are worried that it is overly complicated at the moment and lacks a good abstraction layer. If you want to run Docker on AWS, though, this tool should certainly be high on your list. Just don’t expect it to be easy to get started with.

While many deployments of smart devices rely on Wi-Fi connectivity, we have been seeing success with Bluetooth Mesh networks that don't necessitate a hub or gateway. With better energy usage than Wi-Fi and better smartphone adoption than ZigBee, Bluetooth LE deployed as a self-healing mesh provides interesting new approaches for connecting local device-area networks. We are still waiting for the formal approach to emerge from the Bluetooth SIG but have already had successful deployments. We particularly like the lack of infrastructure required to stand up a decentralized network but still retain the option to "progressively enhance" the system with the addition of a gateway and cloud services.

Ceph is a storage platform that can be used as object storage, as block storage, and as a file system, typically running on a cluster of commodity servers. With its first major release having been in July 2012, Ceph is certainly not a new product. We do want to highlight it on this Technology Radar as an important building block for private clouds. It is particularly attractive because its RADOS Gateway component can expose the object store through a RESTful interface that is compatible with Amazon S3 and the OpenStack Swift APIs.

Deflect is an open source service protecting NGOs, activist and independent media companies from DDoS attacks. Similar to a commercial CDN, it uses distributed reverse-proxy caching and also hides your server IP addresses and blocks public access to admin URLs. Particular effort is put in to combat the botnets typically used for extrajudicial censoring of independent voices.

Our growing ranks of hardware hackers have been excited by the ESP8266 Wi-Fi microcontroller. Rather than a specific technology innovation, it is the combination of low price point and small form factor that has sparked an inflection point in people's thinking about what is now feasible to achieve with custom hardware devices. Its main characteristics are: Wi-Fi capabilities (it can act as station, access point or a combination of both), low power, open hardware, Arduino SDK programmability, Lua programmability, huge community support and low cost compared with other IoT modules.

As Moore's Law predicts, we continue to increase the capacity of computer systems and reduce their cost, and so new processing techniques become possible that only a few years ago would have seemed out of reach. One of these techniques is the in-memory database: Instead of using slow disks or relatively slow SSDs to store data, we can keep it in memory for high performance. One such in-memory database, MemSQL, is making waves because it is horizontally scalable across a cluster and provides a familiar SQL-based query language. MemSQL also connects to Spark for analytics against real-time data, rather than stale data in a warehouse.

Mesosphere DCOS is a platform built on top of Mesos. It provides an abstraction over underling machines, giving you a pool of storage and compute that allows services built for DCOS to operate at massive scale (Support is already there for Hadoop, Spark and Cassandra, among others). This is probably overkill for more modest workloads at the moment (where plain old Mesos could still be a good fit), but it will be interesting to see if Mesosphere starts trying to position DCOS as a general-purpose system.

HashiCorp continues to turn out interesting software. The latest to catch our attention is Nomad, which is competing in the ever-more-populated scheduler arena. Major selling points include not just being limited to containerized workloads, and operating in multi–data center / multiregion deployments.

Presto is an open source distributed SQL query engine designed and optimized for running interactive analytics workloads. Presto's massively parallel processing architecture - combined with advanced code-generation techniques and in-memory processing pipelines - makes it highly scalable. It supports a large subset of ANSI SQL including complex queries, joins, aggregations and window functions. Presto comes with support for a wide range of data sources including Hive, Cassandra, MySQL and PostgreSQL, thereby unifying the interactive analytics interface across data stores of an organization. Applications can connect to Presto using its JDBC interface.

Realm is a database designed for use on mobile devices, with its own persistence engine to achieve high performance. Realm is marketed as a replacement for SQLite and Core Data, and our teams have enjoyed using it. Note that migrations are not quite as straightforward as the Realm documentation would have you believe. Still, Realm has us excited, and we suggest you take a look.

For people who want the benefit of cloud-based collaboration tools but don't want to inadvertently "become the product" of a major cloud provider, Sandstorm provides an interesting open source alternative with the potential for self-hosting. Of particular interest is the isolation approach, whereby containerization is applied per document rather than per application, and syscall whitelisting is added to further secure the sandbox.

The rise of containers, phoenix servers and continuous delivery has seen a move away from the usual approach to deploying web applications. Traditionally we have built an artifact and then installed that artifact into an application server. The result was long feedback loops for changes, increased build times and the not insignificant overhead of managing these application servers in production. Many of them are a pain to automate too. Most teams we work with favor bundling an embedded http server within your web application. There are plenty of options available: Jetty, SimpleWeb, Webbit and Owin Self-Host amongst others. Easier automation, easier deployment and a reduction in the amount of infrastructure you have to manage lead us to recommend embedded servers over application servers for future projects.

One of our common complaints is the pushing of business smarts into middleware, resulting in application servers and enterprise service buses with ambitions to run critical application logic. These require complex programming in environments not well suited to the purpose. We're seeing a worrying re-emergence of this disease with overambitious API Gateway products. API Gateways can provide utility in dealing with some generic concerns - for example, authentication and rate-limiting - but any domain smarts such as data transformation or rule processing should live in applications or services where they can be controlled by product teams working closely with the domains they support.

We've seen the indisputable productivity gains that come from deployment of applications and services into mature cloud providers. Much of that gain comes from the ability of teams to deploy and operate their own services with a high degree of autonomy and responsibility. We are now regularly coming across Superficial Private Cloud offerings within organizations, where basic virtualization platforms are being given the “cloud” label. Often teams can self-provision a restricted set of fixed service types with limited access and little ability to customize the centrally governed “enterprise blueprints,” leading to kludge solutions. Deployment pace regularly remains constrained by manually provisioned infrastructure such as network, firewall and storage. We encourage organizations to more fully consider the costs of mandating the use of an inadequate private cloud offering.