By DevOps can drive process benefits to an enterprise
For those of you that have been stuck under a rock for the last few years, DevOps is a way of developing, deploying and supporting applications in the most responsive and coordinated way possible. Once it becomes established on a running product, the pay-off is the ability to take a business idea or technical change and turn it into reality very quickly, typically within minutes of completing the tested code.
In practice, the DevOps movement (with continuous integration and delivery) is all about automation and applying development techniques to operations. It restores systems engineering to its position in the ‘90s before commoditisation narrowed the role of non-development engineers. The need to make systems bigger but also more agile have overcome years of layered processes that had become received wisdom.
Needs trump process, as they must.
Let me reveal the hidden benefit now. If you are a big organisation with many applications and a complex environment that has taken years or decades to build, the journey to DevOps will be long. However, once enough pioneers have completed their quest, a lot of unnecessary, repeated and sometimes incompatible processes will have been removed.
Everybody wins. Reforming the process landscape should benefit everybody, even if the majority of the applications are still manual.
The Enterprise Variant of DevOps
DevOps applied to an enterprise is a more pragmatic beast than the one a start-up would recognise. Typically due to layers of planning and coordination between an ecosystem of many interlinked applications.
Projects must experience similar benefits to non-enterprise teams: automated integration, testing, and deployments. But also, there is a broader responsibility of each service to provide more meta information. This can then be used in the wider ecosystem to coordinate data and provide feedback for tactical and strategic management — effectively logs and management information.
Conventionally, enterprises handle changes to inter-service coordination slowly in an attempt to stop incompatibility and drift by using architecture and change control. To gain more flexibility, many organisations are effectively employing hands-off mechanisms such as proxies and queues for data passing and more generic formats for data. It allows many technical types of change to carry on without the need to involve peer services.
Code quality is often factored into change control and this too can be reassessed. If every release candidate uses automated testing before it is deployed, then why is there a need for an external check that they are doing the right thing? If a new bug is found and coded into the continual integration test, then it will be checked forever.
However, some situations can’t be entirely automated. Data is fast evolving into an application in its own right with complex rules and a need to maintain traceable lineage. Architecture becomes more critical as services grow in number but become smaller in size (another industrial trend) and aligned to customer experience. Thus, there is a need to have a reformed set of manual interventions, although more efficient than before.
A Ball of String: The Impact of Process
A company of any size creates processes to cope with regular activities. Bigger companies have more of them and use them to link teams together to deliver outcomes without excessive delay.
However, keeping them updated and relevant is hard. Especially when the company is in a dynamic set of markets, is very large or there are a lot of processes to maintain. Changes to the rules require management consensus and work needs to be scheduled to get them implemented. Often, the most significant changes need to be funded with investment money, so when a company is having a period of budget constraint, they become less responsive to change and less able to adapt.
The focus of this context are processes that have become inappropriate, too long, or unmanageable. In these situations, the process can become some of the most disheartening parts of corporate life. At times, I have seen the procession of corporate rules and their delays kill off whole projects or impact payback times. The myriad of rules can be seen as a big ball of string that without trimming can tie up all involved in knots.
IT is full of delivery rules, which directly impact fundamental projects like DevOps. Effectively, DevOps is the new rule set and need to be embraced, replacing waterfall-style processes and modifying the ones that remain to be more appropriate.
Another modest proposal is that processes are automated as they come up for renewal. The automation should be attached to APIs to allow for computer initiation and integrated computer delivery. Speed is thus radically addressed, although at the cost of potential automated lock-in. Once automated, it becomes hard to reimplement manually or even reassess with the same degree of freedom. Process reform must be carried out at the same time as automation and any lock-in evaluated and abstracted.
Complications
Segregation of duties is important and vital within regulated industries. Imagine a crooked developer making rules to run your bank accounts without supervision. Many of us wouldn’t know until it was too late. In this situation, there would have to be safeguards during development, such as code review and disabling the continuous deployment part. Effectively, another set of eyes would need to approve the commits and potentially deploy them to a sandbox for a period. We are in good company: this is a similar model to the Linux kernel.
Another set of complications concern system and information security, which sit at the top of everyone’s agendas, and will be so for years. It generally requires a more system level view than developers typically embrace and would also impact the deployment cycle. The need is to provide a more architectural view of the finished product so that it can be shared with security teams. As maturity increases, we must embrace the tools needed to show architecture and carry out security evaluations.
Big organisations love their management reporting, so who takes care of it in the new world? A final set of complications we should consider is how to make sense of each product in the most streamlined way possible. My favourite solution would be an obligation on behalf of every app to publish keys statistics via a generic collection fabric. A management reporting system could then pick the data up and combine it with other product data to give holistic view of the enterprise.
The three issues raised here show that there will be difficulties to overcome. However, the spirit of DevOps must shine throughout any specific process put in place.
Embrace Agility
New recruits are eager to develop in Agile ways and are willing to embrace the increased responsibility it brings. Enterprises are evolving too, tantalised by the benefits of Continuous Integration and the prospect of small changes. Investment in DevOps can extend benefits to other apps, especially in process change as discussed above.
DevOps is not the only thing that has been happening in the world of IT. State regulation of data is bound to have a sizeable impact. Location of processing is becoming covered by statute, too. With cloud infrastructure, the range of vendors has increased but we must carefully model their costs, as they charge in ways that enterprises don’t expect.
IT has seen substantial changes in the past few years and more will come as it reforms. Enterprises should clear the decks. Take advantage of DevOps, agility, and cloud. Embrace a more streamlined and automated set of processes.
About the Author
Architect, PM and Strategist. Follow me on Twitter @nigelstuckey
System Garden
Agile Infrastructure for Enterprise DevOps
Design from diagrams, document and deploy to your cloud.
systemgarden.com, Twitter @systemgarden