Engineering has changed forever
"Almost everything we know about good software architecture has to do with making software easy to change."
- Mary Poppendieck, Software Development Expert and Author
In the last 6 months, the foundation of engineering has undergone a dramatic transformation, a change so profound that many organizations are only beginning to understand and adapt to its implications. Most companies have started to take a small incremental step to sprinkle some AI into their software, instead of diving deep into the possibilities.
In this article, let's focus on the architectural implications of AI. Broadly, this shift encompasses a move from traditional, rule-based architectures to more dynamic, AI-centric models, deeply altering the role of AI in software systems and challenging our foundational understanding of software architecture.
You can describe the architectural shift in 4 phases:
- Classic Software Services Architecture
- AI-Enabled Software Services Architecture
- AI Software Service Architecture
- Ai Engineering
Each phase differs significantly from the others in implementation and outcomes. Some companies will stumble from one phase into the next, other companies skip ahead. Regardless, most have yet to make a conscious choice among the options. Never mind that this choice could be the critical difference between surviving or being outpaced by their competition.
Note: in this article, when I refer to AI, I generally mean generative AI, however some readers might want to classify certain Machine Learning applications as such as well. An example from my own experience would be how FinTech companies use ML for credit decisioning and payment timing.
1. Classic software services architecture
We're wildly simplifying, but for purposes of this article we'll group a wide variety of software services architectures into the "classic" bucket. Predictable and precise, frequently adhering to a rule-based, service-oriented approach. Each component within this system serves a specific, predetermined function. This type of Architecture is mostly deterministic and modular, with the general purpose of being predictable and scalable.
The majority of software engineering in enterprise lives in this category. Each company is looking to enable their software with more AI functionality, and will mostly look one step ahead to incorporate some AI, which lands them in an…
2. AI-enabled software services architecture
AI-enabled software services architecture mixes a classic software architecture with select AI enhancements. While the core remains a service-oriented structure, certain services are augmented with AI capabilities, introducing elements of machine learning, predictive analytics, or generative AI. This integration retains the system's foundational modularity and predictability but adds a layer of adaptability, enabling it to respond dynamically to evolving data and requirements. The architecture strikes a balance, maintaining the reliability of traditional software services while embracing the flexibility and new functionality offered by AI.
Examples of enabling traditional services with AI are: transforming unstructured data into structure (extract consumer sentiment into a database), simple chatbot services, or simple AI/ML decisioning services (should we provide credit to this person). However, when you try to go deep to unleash the power of AI you soon discover that managing AI within old software constructs is strongly limiting its potential. Putting an orchestration service on top of AI restricts much of its use cases to input-output translation services.
3. AI Software Service Architecture
AI software service architecture embeds at least one fully AI-driven service within a traditional software framework. Here, the classic platform manages overall functions but utilizes an AI-specific service for complex, adaptive tasks. This AI service, autonomously processing inputs like text or data events, enables advanced decision-making by the system. Even though the core of the architecture remains rooted in classic software methodologies, this integration of an AI-centric service enhances functionality. It combines the structured orchestration of traditional software with AI's dynamic problem-solving capabilities.
Using AI in this fashion unlocks broader AI use cases. An example of an AI service would be one that can perform a mixed process of input transformation, reflection, decisioning, and action. For example, when a new file arrives, the AI takes a look at the file, compares it to its instruction set, determines it is an unstructured file, reads the first lines of content, determines that it needs to be manually inspected, and forwards it to a supervisory department.
4. AI Engineering
In truly native AI Engineering, the spotlight shifts entirely to AI as the core software, diverging significantly from traditional and hybrid architectures. In this paradigm, engineering efforts focus on empowering the AI itself, enhancing its capabilities and autonomy. The AI isn't just an element within the system; it is the system, tasked with platform-like responsibilities. Engineers work to expand the functionalities of the AI, not on orchestrating the AI. Examples of engineering work would be: integrating multimodal inputs, broadening its decision-making scope, and equipping it with a diverse range of tools. This approach is about nurturing the AI to seamlessly integrate into various workflows and real-world applications, ensuring it evolves into a more sophisticated, self-reliant entity. Unlike previous models where AI supplements or enhances traditional structures, here AI is the bedrock, redefining the very concept of software engineering with its boundless potential and adaptability.
Examples to imagine in this architecture category would be adaptive learning loops, end to end AI process automation, multi-modal decision scenarios with feedback, and an endless amount of other examples, with the only boundary being your imagination.
Architecture phases 3 and 4 have similarities, but they differ significantly. Let's clarity the difference a bit more. In AI engineering..
- … the focus of an engineer shifts to enabling the AI
- … the AI is the orchestrator of most activity, any necessary orchestration is very light with a main purpose to enable the user to AI interaction
- … SW services are tools/enablers for the AI instead of the AI being a tool for the SW service
An example of a true AI engineered service: "Ingest financial market changes, reflect on my portfolio, decide if any additional investment should be made, and then autonomously take action on the recommendation (e.g. selling stock ABC)".
Reflection
At the moment a lot of companies are still figuring out what AI means for them and they are experimenting with how to extract value. When companies move on to build real solutions, it is vital that they reflect on what engineering paradigm aligns best with their goals. Incrementally enabling AI solutions is where most companies will start, but in doing so they will miss out on the true capabilities that AI has to offer.
Soon, a new generation of companies emerges. Old and new companies which have deeply embraced AI native philosophies. Their development speed will far outpace that of any traditional player.
So choose wisely. In this high-stakes world of AI permeated engineering, your next move could set the course for success or obsolescence.
