Crowdsourced Architecture with IntegrationWorks

Crowdsourced Architecture: The IW Buzzword

"Open" this, anything "as-a-service", and "crowd" that. Do you ever feel as if you are caught in an avalanche of buzzwords? Crowd-sourcing is a hot topic, and for good reason. If conceived and done well it can harness the abilities and inputs of a group, and through iterative continuous improvement tend towards producing at a level and creativity far beyond the imagination of the individuals involved. This blog about crowdsourcing, was crowdsourced by our team of integration experts and written by our own collaboration enthusiast, Paul Evans.

Here at IntegrationWorks our architects have been exploring decentralised integration as an extension of decentralised IT. We took a good hard look at all the different facets of software delivery that can make up a decentralisation strategy, including governance and architecture. As we pored over potential patterns we kept returning to the importance of decentralisation needing to be led, and modelled, by leadership.

Here is the crux - how can architecture governance champion decentralisation when they themselves traditionally favour centralised governanceWe are not the first to use the term. In much the way that IT architecture is analogous to our cousins in the physical building architecture space, so it should come as no surprise that crowd-sourced architecture is a hotly contested subject for them as well. Only in their case it seems to be competition based - trying to chose the best discrete complete design from a variety of unrelated submissions. We believe this falls some way short of what crowd-sourcing can and should look like.  

And what could a crowd-sourced IT architecture delivery model look like? What should it look like? In the tradition of open-source delivery we would expect to see architecture that is open, collaborative, and iterative. And importantly we would expect, over time, for the best designs and revisions to emerge as successful through the market-force of adoption. Jumping ahead here, we might expect to see pull requests on an organisation's Enterprise Architecture or Solution Architecture framework, designs, and other key artifacts - where persons or teams beyond the architectural governing body can participate in defining and refining an organisation's IT architecture. Specifically - we would hope to see the greater body of the organisation harnessed to contribute to, and improve, architecture assets. 

Lets take a quick look at some of the problems associated with more traditional IT architecture delivery. What, exactly, could a crowd-sourced approach to architecture solve? There are a number of different problems that can, and do, occur in architecture teams - perhaps not all of them, and perhaps not all the time - but they happen. At the worst we have theoretical architects who are too far removed from the coal-face who painstakingly debate minor nuances of theoretical abstract reference models and split hairs over defining a canonical vocabulary. Over-engineering results in patterns that are difficult to implement - one size fits no-one. Slow decisions delay projects, or at least create convenient excuses for delays. Centralised architecture, with imposed, dictatorial blueprints and roadmaps can result in a lack of buy-in and independent design; and ultimately an overall IT that is not so much architect-ed - but segregated, held together with string and a prayer. With architecture being perceived as an inhibitor, not an enabler, of technology - a hoop for project delivery to jump through. 

There is clearly a demand for architecture to be taken out of it's sacrosanct box, and put it into the hands of ordinary IT practitioners. There is a need for architecture policy to pragmatically meet with the hands-on problem-solving that is the bread-and-butter of project analysts and designers. But how? These guys have their own jobs - aggressive deadlines and demanding clients. Perhaps by reducing their existing architecture overheads? A better way to put it might be, by leveraging and channeling their existing architecture effort. It is important to understand that architecture is already a significant overhead for these resources. They have to prepare architecture artifacts and present to architecture review boards. They have to polish architecture decision papers, often through multiple iterations, and second guessing the board to get approval. They have to iterate through and respond to feedback and queries from architects who are divorced from the project deliverable, and will mainly focus on alignment to their own personal silver-bullet. There are hoops that need to be jumped through to secure funding. Hoops to pass quality gates. And hoops to go live. Indeed, much of the problem is the culture where these controls are perceived as hoops. So, instead of promoting a culture where designers regard architecture as an overhead, we promote an environment where they can contribute to it.

Here are some of the elements IntegrationWorks recommends to help take the bureaucracy out of your architecture.  

Empower the teams who are grappling with these problems. Empowering does not just mean trusting the teams to do the right thing on their own (although trust is an important element). Architecture guidelines should not be regarded as an unsolvable problem. Project teams need to be brought to a point where they understand and internalise the concept that architecture is their responsibility too, and something they can help drive. Authenticity is key here - it will only work if the people implementing the architecture feel they are listened to in the design.

Promote architectural awareness through training, through feedback, through debate, and through collaboration. Take the time to equip the teams with sufficient awareness to engage in meaningful architectural dialogue. These are smart folk that you are trusting with the delivery of your strategic IT initiatives - they should be able to get their heads around this. 

Choose the right tooling for the job. Much the same as is needed for open source development - you need open, accessible repositories, wikis, and collaboration tools to facilitate dialog, debate and refinement. From the perspective of modelling and frameworks it is essential to provide access to a shared modelling repository. 

Mistakes are okay. In order for the 'best' approach to be allowed to emerge it is important to foster multiple approaches, most of which will have problems. Painstaking, bottleneck-causing, centralised architectural decisions do not in fact guarantee that correct or 'best' decisions are being made. All that is really happening is that distributed culpability is taking place so that everybody has a plausible explanation in the event that things go wrong. This is not a good reason for a project to be delayed. And it's not a good reason for architecture.  Far better to create an environment where it is safer to fail and failure at low cost (not necessarily fast) is seen as an enabler of value-creating progression. 

Embrace Agile.  Crowd-sourced design will not thrive in a traditional waterfall SDLC because of the lead time involved in iterations of design review to re-risk coarse grained and inflexible project delivery capabilities.  Post implementation reviews take place far too infrequently for any lessons to be learnt. Failing is too costly and takes too long.  Truly agile organisations can benefit from a being able to implement a low risk fail at low cost environment, from a reduction in architecture investment and from a greater reliance on emergent design - all of which are congruent with a crowd-sourced architecture.  

Most of our clients are embarking on or progressing through their transformation and integration journeys. The ultimate aim from IntegrationWorks is to allow businesses to change faster, deliver products and services to customers quicker, beat the competition and reduce risks. We work with our clients closely to develop strategies, powerful roadmaps for future paths, supplying thought-leadership and hands-on people-power in the form of integration architects, developers, testing services and operations analysts. We work with a variety of global technology companies, such as IBM, Dell Boomi, MuleSoft, Apigee and SAP to deliver innovative solutions that work. To learn more about us, check out www.integration.works

To organise a time to meet with our Chief Technology Officer, Ian Vanstone, make contact with him on LinkedIn by clicking here