How Agile ate SOA

How Agile ate SOA

Service Orientated Architecture (SOA) is a good idea. It has some great and enduring principles (e.g. loose coupling) and practices (e.g. service composition - still a rare skill today). SOA's intense focus on Architecture (big A) and service reuse were useful in days gone by but are rightfully questioned today. To delve deeper into that change let's first look at the past.

In the past, integration was slow and expensive because IT change was slow and expensive.  Consider the fictional example of adding a single new field (a voucher code) to a checkout web app.  This requires change to the web app itself, the ordering system, and the reusable integration service running on the ESB (Enterprise Service Bus).  Conservatively, in a large organisation, this would probably have involved the following phases and roles...

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

The cost of these minimal resources might have been $100k and they might have taken an elapsed 10 weeks to deliver.  All this for adding one field to a customer screen. Just imagine the costs and timelines of a major change! Of course, we don't have to imagine, anyone who has been in the IT industry for a few years has seen or at least heard of the high costs and elongated schedules of major change programmes.  The costs were high because deliver was slow.  And delivery was slow because:

  • people were assembled into a temporary team to deliver projects. 
  • people were likely working on multiple inflight projects and so part of several temporal project teams.
  • the processes were mainly manual.

In the world of the past, Architecture was a focus and a very good idea because the solutions requested by the business would not actually be delivered for some time.  i.e. a minimum of 10 weeks, as per the example above, but often well over 18 months.  So investing in Architecture (e.g. planning, business analysis, scoping, requirements, impact analysis, etc) was part of a massive de-risking exercise.  De-risking in the sense that we had to ensure that the solution would work at some defined time in the future.  So we had to predict what the business and technology architecture would look like in 10 weeks or 2 years.  And with multiple change programmes and projects in flight and overlapping at once, predicting the future was tough!

Given all that time and cost to deliver, once our solutions had gone live, we did not want to change them.  So we yearned for technology components that would endure and could be reused.  And so in the past our people, process, and technology were optimised for a world of slow technology delivery and enduring technology solutions.  

Agile

Agile is a great idea.  Not the kind of 'agile' whereby the IT leadership (or even a CEO) say 'We're agile and we do DevOps" yet under the covers there is widespread annual programme-based capex budgets and water-scrum-fall or wagile projects.  The kind of agile whereby products (more than programmes) are funded, continuous improvement and flexibility are baked into the operational model, and software can be released at any time.   

In the agile world, our people, process, and technology are optimised for a world of fast technology delivery and flexible business solutions.  This style of optimisation includes the loose coupling of people and process, in addition to the loose coupling of technology from the heyday of SOA.  With this style of optimisation, classic reusable services at the enterprise level can cause major issues because:

  • they often result in design-time and runtime dependencies between a reusable service provider and multiple service consumers spread across the enterprise.  This puts unwanted dependencies on people/teams and delivery processes. 
  • as service consumers change frequently and fluidly, a centralised reusable service that was originally designed to be the single source of service contract truth (spanning multiple bounded contexts) will fast bloat to the point that its data and service model become an amalgamation of big balls of service contract mud.  i.e. an unwieldy, non-agile mess.

Even where reusable services do not cause issues, they have less value in the agile world.  This is because in the agile world we can can deliver and refactor at speed. So if we get something wrong or deliver something that is not as great or efficient as it should be, we can fix/improve it relatively quickly - we are not stuck with it for years. 

So we travelled a path from the past and now live in a world of flexibility and shortened cycle times.  Along the way, agile ate SOA because:

  • Organisations that prioritise agile (over SOA) deliver code and value faster.
  • Agile prioritises continuous improvement and so organisations can adapt faster.
  • Agile takes loose coupling to a further level (e.g. to people/process as well as technology).
  • Agile actively discourages a major tenet of SOA (i.e. classical enterprise level reuse).

SOA and Agile

It would not be right to finish this blog with some further reality.... SOA is not dead.  SOA is often still of use in most well established organisations.  In well established organisations SOA and agile have to work well together as we as an industry continue to uncomfortably deal with legacy systems, architectures, operational models, and funding models.  It is not feasible to transform all shared services IT teams into agile product teams overnight.  Nor is it feasible to transform all architectures into event-driven microservices architectures overnight.   A good approach is to understand what's possible (in terms of target state and value), to understand our organisational constraints, and to collaborate in such a manner that results in lower cost and faster transformations.  When following this approach, having a good understanding of SOA's principles and practices (and their applicability) is of great benefit for a while yet. 

If you want help transforming your organisation or you want to hear more about continuous-transformation enablers, contact our CTO Ian Vanstone on LinkedIn.