This article is part of a series
- Part 1: Common Approaches in the Field of Socio-Technical Architectures
- Part 2: Platforms, Teams, and APIs: How Do They Fit Together?
- Part 3: Socio-Technical Architecture as a Competitive Advantage
- Part 4: Don’t Forget the People
- Part 5: How Much Thinking Can a Team Handle?
- Part 6: Internal Development Platforms (this article)
- Part 7: Enabling Stakeholders as a Success Factor
- Part 8: Socio-Technical Architectures: Informality from Mining to Today[1]
The Lowest Imaginable Autonomy
Those of us who have been around remember: Before the era of agile software development, we had large phase models like the waterfall model. Users would wait a long time before seeing any running software. Handoffs between teams, such as between development and QA, were complex and time-consuming. None of the teams involved had true autonomy. Each team within the development process was heavily dependent on at least the teams immediately before and after it. However, the mental load on developers was comparatively low. Developers primarily needed to know design principles, programming languages, and frameworks.
Agility Increases Autonomy, But Also Mental Load
Agility emerged in the late 1990s with models such as Extreme Programming, Scrum, and Crystal. At the time, these approaches promised rapid software delivery in 6-week cycles compared to the usual 6 to 24 months. This was partly because many handoffs were eliminated. The development team became “cross-functional,” incorporating different roles such as requirements engineering, development, and testing into a single team. However, security and operations still remained separate from the development team. The mental load increased significantly. Beyond software development, developers now had to write automated tests, often speak directly with users, estimate user stories, present results, and much more.
Shift Left, DevOps, and “You Build It, You Run It”
After the publication of the Continuous Delivery book (2010)[1] and Werner Vogels' famous interview about software development at Amazon (2006)[2], operations also moved into the development team’s realm (“You Build It, You Run It”). The term “Shift Left” became popular.
The term originated from the waterfall model’s visual representation from left to right. “Shift Left” means moving tasks such as testing or operations “to the left” into earlier phases of the process, such as architecture or development.
Development teams that successfully shifted many activities to the left could deliver high-quality software very quickly. Why? They had very high autonomy. Speed comes from shortening or even eliminating handoffs between teams.
However, the mental load for these teams increased considerably. Operations and infrastructure were added to their existing responsibilities, including maintaining CI/CD pipelines, managing cloud technology, participating in on-call rotations, and implementing observability.
Mental Overload: From Shift Left to Pile Left
With the introduction of concepts like DevSecOps, FinDev, and FinOps, development teams were expected to have more and more capabilities. Many companies found this was no longer manageable. It was no longer “Shift Left” but “Pile Left.”
Development teams suddenly had too many responsibilities. In the name of high autonomy and delivery speed, they were supposed to handle everything. In some cases, development teams were even expected to manage license purchases.
“Pile Left” inevitably led to problems. There are limits to what a development team can and should do. UX, security, QA, infrastructure, operations, and other areas are complex disciplines requiring specialized skills. Very experienced teams with a good mix of competencies might manage these tasks, but companies with multiple teams cannot expect all teams to be set up accordingly or to learn the necessary skills quickly enough. This approach isn’t efficient anyway.
The complexity of developers' lives has steadily increased over the past 25 years. What was unthinkable in the late 1990s during the emergence of agility—developers being responsible for test automation—is completely normal today. Additionally, we now work with larger and more complex software systems requiring more developers across more teams. Both factors make it worthwhile for many companies to invest in Enabling Teams and Internal Developer Platforms. These help better organize redundant work and knowledge building, thus managing the increasing complexity.
With Platforms and Enabling Toward Shift Down
If high autonomy creates a high mental load for many development teams, solutions must be found. Currently, there are two complementary approaches: Internal Developer Platforms (IDPs) and Enabling Teams[3].
IDPs try to “hide” as much of the specialized knowledge mentioned above in the platform as necessary (“Shift Down”). For example, they can handle infrastructure provisioning, monitoring, dashboards, log infrastructure, CI/CD, security (e.g., secrets management, vulnerability checks, authorization and authentication), license scanning, as well as application and service templates. This means developers no longer have to worry about “everything,” as the IDP handles the relevant work in the respective context.
Caution is needed here: Too much functionality in an IDP can drive up costs. Additionally, developers may be deprived of the necessary freedom to innovate on the platform.
IDPs can and should also address domain-specific infrastructure issues. An IDP could, for example, provide a “customer database” that meets operational requirements for locality, encryption, and data backup for customer data. This goes beyond providing a simple cloud database that would still need to be configured accordingly (Hohpe and Seitz)[4].
A platform can do a lot, but it cannot remove all mental load related to infrastructure, security, or testing from development teams. For these gaps, “enabling” is needed—coaching and support from specialized teams. These teams bring together experts with the task of empowering other teams to master specific challenges.
Success Factors for Internal Developer Platforms
Building and operating a successful IDP is often underestimated. An IDP is an internal product with its own customers (developers, operations, business) and therefore requires everything a product needs: besides a development team, it needs product management, good requirements engineering, support, an excellent developer experience, self-service capability, and internal marketing. Since this requires considerable effort, an IDP is usually only worthwhile from a certain size of development organization. The leaner the IDP, the sooner a company can afford one.
Looking Beyond: Domain-Specific Platforms
IDPs are not the only internal platforms that exist in companies. There are also internal domain-specific platforms, sometimes called “common services.” These offer functions that cover frequent and recurring requirements of many development teams in the corresponding domain, such as payment processing, fraud detection, or data analysis. These domain-specific platforms must be identified as important business components, developed, made known internally, and made discoverable. As with IDPs, the same rule applies: They are internal products and require the same organizational prerequisites and efforts to be successful.