New technologies often simplify some aspect of life, an aspect which was previously painful. But then, as soon as it is adopted, a technology presents new challenges and new complexities.
With cloud computing, you can get a new machine in minutes—less than a minute, in fact, with some of the leading systems. Alternatively, you can provision a new virtual datacenter with secure VLAN and as much storage and “core-age” as you need. Once this is done, the problem of manually sourcing an environment for your application goes away, as it becomes increasingly easy to source one automatically.
With applications, however, there is a new challenge: their environment has to be managed. In the world of cloud, an application is expected to be able to flex when load increases, contract in quiet periods, and generally be efficient about how it’s consuming resources, and do so dynamically. Furthermore, this is not just the case in one site, because if it’s serving users globally, or operating on data-sets in different geographies, it needs to span across sites, adding and removing regions depending on demand, cost or compliance.
The consequence for architects and developers is that an application is no longer just its business logic; it has to carry with its operational logic.
– When should it change the resources it is using?
– How does it safely give up resources, and efficiently “on-board” new ones?
– Where are these policies expressed?
The only sane answer to the last question is that orchestration policies need to be part and parcel of the application. If an application doesn’t describe its deployment and its management—the operational logic—alongside its presentation tiers and its business logic, it is already legacy, and it will bring more complexity over time, not less. On the other hand, if the application embraces its fate, taking responsibility for its runtime lifecycle at dev-time, these operational challenges are addressed in a way app teams know well: by writing it as code—as part of the application—it can be tested, tracked in version control and reused. Continuous integration and testing applies to cloud complexity just as much as to other facets of application development.
There are a number of ways this can be done, with the most exciting activity occurring in the open-source world. Deployment frameworks, such as Puppet and Whirr, can ensure machine portability. Provisioning libraries, such as jclouds, can ensure cloud portability. PaaS offerings, including OpenShift and Cloud Foundry, can ensure application portability, and integrated deployment and management tools such as Brooklyn ensure policy portability.
For green-field applications, these tools—and an approach which treats deployment and ongoing management as core parts of development—give engineers what they need to tackle these challenges.
They enable complexity to be delegated to a PaaS or middleware for those components where it’s appropriate, and for the unavoidable complexity to be managed in other cases, where custom analytics or I/O is essential, or where we are faced with the wide-area realities of replication, consistency, bandwidth and security. By tying in with these open-source projects, application teams benefit from the best practices created by experts without being locked in to proprietary ways of running or managing.
For legacy applications, of course, it might not be practical to transition to new runtimes. However, many of these tools can still facilitate cloud-readiness. By adding a layer of operational logic to the application without touching existing code or business logic, deployment to cloud can be automated, and management introduced, with minimal risk. Eventually, re-architecting may make even better use of the cloud, but the benefits of a lightweight wrapper are significant, bringing at least the orchestration of legacy applications in line with post-cloud application development.
Coming back to our premise, if these tools solve current emerging complexities, it is inevitable that they will soon introduce new complexities. What will these be? Based on our experience, there are a few: but they’re healthy complexities!
Firstly, the policies become a source of complexity: we recommend keeping them as simple as possible and composing them. But we in the cloud industry (née software industry) are still figuring out best practices. Open-source policy frameworks are a great place for these conversations to happen. Secondly, everyone in the devops chain has to work together to make sure the policies are right for the app and right for the business. In some organizations this can be a challenge, but the rise of DevOps, more agile practices, and testable deployment are helping to ensure that the most valuable resource—people’s time—is spent as productively as possible.
By Alex Heneveld
