Evolving Software Architectures

When we build complex systems before they're needed, we pay the costs of complexity without realizing the benefits. This isn't just wasted effort—it actively harms the organization by increasing cognitive load and slowing innovation.

• Reliability vs. Performance: Adding redundancy and validation improves reliability but often at the cost of performance
• Simplicity vs. Flexibility: Simpler solutions are easier to understand but may be more rigid to change
• Consistency vs. Availability: As discussed in the CAP theorem, distributed systems must choose which to prioritize
• Time-to-market vs. Technical debt: Moving quickly often means accumulating technical debt that will need to be paid later
• Coupling vs. Complexity: Reducing coupling between components tends to increase the overall system complexity

The evolution of architectures can be seen as increasingly sophisticated approaches to information management:

• Monoliths: Information is contained within process boundaries, with direct access through function calls
• SOA: Information is partitioned by domain, with access through explicit APIs that define contracts
• Event-Driven: Information is represented as events, with components publishing what they know and subscribing to what they need
• CQRS: Information for writing is separated from information for reading, optimizing each for its specific needs

Amazon Web Services applies these principles through their "Working Backwards" approach:

1. Teams define interfaces and contracts before writing implementation code
2. APIs are designed as if they're public, even for internal services
3. Service teams operate as if they have external customers
4. Documentation is written before implementation, clarifying what the service will do

This approach has enabled AWS to build hundreds of services that can evolve independently while maintaining compatibility.

Shopify started as a classic Rails monolith. As they grew, they evolved into a "modular monolith" with:

• Component boundaries enforced through code
• Domain owners responsible for specific areas
• Migration paths for gradually extracting services
• Shared core for common functionality

If a team couldn't be fed with two pizzas, it was too large. This organizational principle drove Amazon's service-oriented architecture:

• Teams owned specific business domains end-to-end
• Each team operated their own services independently
• Teams defined contracts with their "customers" (other internal teams)

Netflix built its A/B testing infrastructure on event-driven principles:

1. User interaction events flow into Kafka streams
2. Multiple independent consumers process these events:
   • Real-time dashboards showing experiment performance
   • ML models updating personalization algorithms
   • Analytics systems calculating business metrics

When a team wants to run a new experiment, they simply create new consumers of existing event streams.

Instead of storing current state:

// Traditional approach
account.balance -= 100;
account.save();

Event sourcing stores the event:

eventStore.append({
  type: "MONEY_WITHDRAWN",
  accountId: "123",
  amount: 100,
  timestamp: "2025-04-17T12:34:56Z"
});

Current state is derived by replaying events:

let balance = 0;
for(let event of accountEvents) {
  if(event.type === "MONEY_DEPOSITED") balance += event.amount;
  if(event.type === "MONEY_WITHDRAWN") balance -= event.amount;
}

• Atomicity: Transactions are all-or-nothing—they either complete entirely or fail completely, with no partial results.
• Consistency: Transactions move the database from one valid state to another, preserving all defined rules and constraints.
• Isolation: Concurrent transactions execute as if they were sequential, preventing interference between operations.
• Durability: Once committed, transaction results are permanent and survive system failures.

• Linearizability: Operations appear to occur instantaneously at some point between their invocation and completion
• Sequential Consistency: Operations appear to have executed in some sequential order, consistent with the order seen by individual processes
• Causal Consistency: Operations causally related appear in the same order to all processes, but concurrent operations may be seen in different orders
• Eventual Consistency: Given no new updates, all replicas will eventually converge to the same state

Amazon's DynamoDB exemplifies how consistency models can be practical implementation choices:

• Strongly Consistent Reads: Always reflect all successful writes, but have higher latency and may be unavailable during network partitions
• Eventually Consistent Reads: May not reflect recent writes, but offer lower latency and better availability
• Transaction APIs: Provide ACID guarantees for operations needing them, at a performance cost

DynamoDB allows developers to choose the right consistency model on a per-request basis, showing how consistency is a design parameter rather than a binary choice.

Airbnb evolved from a monolithic Rails application to a hybrid architecture:

1. Synchronous APIs handle user-facing operations (search, booking)
2. Event streams power analytics and personalization
3. CQRS patterns optimize search and listing displays
4. Core services remain synchronous where consistency is critical

First articulated by Peter Deutsch and others at Sun Microsystems, these fallacies highlight assumptions developers often incorrectly make:

1. The network is reliable
2. Latency is zero
3. Bandwidth is infinite
4. The network is secure
5. Topology doesn't change
6. There is one administrator
7. Transport cost is zero
8. The network is homogeneous

Effective distributed architectures must account for these realities rather than assuming an ideal environment.

Netflix pioneered chaos engineering through tools like Chaos Monkey, which deliberately terminates instances in production to ensure resilience:

• Teams are forced to build services that can withstand instance failures
• Resilience patterns like circuit breakers and retries are tested continuously
• Systems are regularly exercised in failure modes rather than only during actual outages
• The organization builds a culture that normalizes failure and recovery

This approach enables Netflix to maintain high availability despite running on distributed cloud infrastructure.

The Guardian newspaper successfully used the Strangler Fig pattern to migrate from their monolithic content management system:

1. Created a new API layer in front of their monolith
2. Built microservices for new features without touching the monolith
3. Gradually moved existing functionality to new services, one domain at a time
4. Used feature toggles to test new implementations with real traffic
5. Maintained backward compatibility during the multi-year transition

This incremental approach allowed them to continue delivering new features while modernizing their architecture without a risky "big bang" migration.

Successfully evolving architecture requires parallel evolution of teams:

• Component teams → Product teams: Shift from organizing around technical components to business capabilities
• Project → Product mindset: Move from time-bound projects to ongoing product ownership
• Specialist → T-shaped skills: Develop broader skill sets while maintaining depth in key areas
• Centralized → Federated governance: Replace centralized architecture boards with distributed decision-making
• Process-oriented → Outcome-oriented: Focus on business outcomes rather than adherence to processes

Spotify's famous "Squad" model evolved alongside their architecture:

1. Started with traditional teams organized by technical function
2. Evolved to cross-functional squads organized around product features
3. Grouped related squads into "tribes" with shared business domains
4. Maintained technical excellence through "chapters" that span squads
5. Used "guilds" to share knowledge across the organization

This model enabled autonomy while maintaining alignment, allowing teams to evolve their services independently while working toward common goals.

Amazon's journey from monolith to microservices wasn't planned from the beginning but evolved over time:

• Started as a monolithic C++ application in the late 1990s
• Gradually refactored into services based on actual scaling pain points
• Developed the "Two-Pizza Team" rule organically to address communication challenges
• Evolved from synchronous to asynchronous communication as scale increased
• Refined their approach through years of experience, not by following a predetermined blueprint

Jeff Bezos famously issued his API mandate not as a technical decision but as an organizational one. The resulting technical architecture emerged from this organizational principle.