The 12 Factor App: Principles Every Cloud Developer Should Know

Table of Contents

• Introduction
• Twelve factors:

1. Codebase
2. Dependencies
   • Concept
3. Config
4. Backing Services
5. Build, Release, Run
6. Processes
7. Port Binding
8. Concurrency
9. Disposability
10. Dev/Prod Parity
11. Logs
12. Admin Processes

• References

The 12factor.net website discusses the best technique to develop web apps or software as a service. So, I'm going over these 12 factors and using this list as the foundation, but I am going to cover items they don't mention.

Introduction

A group of people within Heroku developed the 12 Factors manifesto describing the rules and guidelines that needed to be followed to build a cloud-native application in 2011. Currently Azure, AWS, GCP, and Oracle are the leaders in the cloud, not Heroku. However, most of it is still relevant today.

How do you create a cloud-based app so that it is robust, scalable, efficient, and cost-effective?

First of all, let's understand what are an app and a cloud:

App:

• In any environment, a single instance can be run or deployed.

Cloud:

• The cloud is a distributed collection of servers, where users can manipulate, configure, and access the application or software over the internet.

Enter, the Twelve Factors:

• Objective guidelines and regulations
• A visual checklist and guidelines for evaluating designs during design sessions.

The 12 Factor App's main concepts are briefly explained in the section below.

1. Codebase

One codebase tracked in revision control, many deploys

• There must be a 1-1 relationship between the repository and the application. Whereas Google, Meta, Twitter, Microsoft, and many big company uses mono repo i.e., 1-many relationship.

• cloud-native apps must always consist of a single codebase that is tracked in a version control system like GitHub, GitLab or Bitbucket. The main goal of a single codebase is to produce any number of immutable releases(build artifacts that remain unchanged).

2. Dependencies

Explicitly declare and isolate dependencies

• Every Application needs dependencies to be built and deployed, such as external libraries(also known as linked libraries) from third parties that are not native code to the respective language. ex: Java given as a JAR, Apache Logging library as a log4j.jar, node.js. packages, .NET DLLs, etc or internal libraries (also known as embedded libraries) are the ones that the particular platform provides.

War (web application archive) vs Jar(java archive)

• a file called a "war" that is packaged and deployed to any servelet or Jsp container address. It includes web-related Jsp, HTML, Js, servlets, and other files required for the development of the web application. It is for a browser-based web application.

• Jar contains java related libraries, resources, & accessories files like property files. it is for desktop applications and works on machines.

Concept

1. declaring dependency & bundling (also called vendoring)

• Specify the dependencies your program requires.

  • dependency manifestation:
    • npm (node package manager) for JS - package.json file(javascript object notation),
    • maven for java - dependencies in an XML file, known as Project Object Model (POM),
    • gradle for java- build.gradle file,
    • NUGET for .net bundler for ruby,
    • godeps for go,
    • sbt for scala,
    • pip for python

2. dependency isolation

• App only uses the dependencies it explicitly mentions. You cannot presume that certain dependencies are using it without your knowledge because there are no implicit dependencies that leak in.

Main Goal

• It is simple to deploy, i.e. no pre-setup is required via container, with consistent builds across environments that can be repeated and easy for debugging and troubleshooting.

3. Config

store config in the environment

• Configuration varies depending on the deployment type (such as production, QA, or staging) and the time of application run. “config” here doesn’t mean the app’s internal information. If the value remains the same across all deployments then it is part of immutable builds.

• Some examples of configuration are:

  • URLs, hostname, port number, credentials
  • secret keys, API tokens
  • service config(Memcached, threat pool)
  • For the non-containerized environment:
    • configuration management tools like Ansible, Chef, Puppet, etc to install system-level dependencies.
  • For the containerized environment:
    • docker-compose.yml file, a Kubernetes manifest file, etc

Configuration manifest

Environment variables, zookeeper, spring cloud config, etc

4. Backing Services

Treat backing services as attached resources

• what is a backing service?
  • Any service on which the functioning of your application depends is a backing service.

• Services such as databases, caches, messaging systems, Simple Mail Transfer Protocol (SMTP) services, etc
• loosely couples the application - your source code does not couple with other services so, it never relies on any backing services and allows for scaling/flexibility.
• Locally maintained or provided by some third parties, these services have both options then it is possible to attach and detach backing services without redeploying the application.

5. Build, Release, Run

Strictly separate build & run stages

• Build stage
  • Input: source code
  • Output: executable Bundle
• Release stage
  • Input: build & configuration (immutable build)
  • Output: release in an executable environment
• Run stage
  • Input: release
  • Output: a running process
• Use CI/CD tools to automate the builds and deployment process
  • There is a 1:many relationship between builds & deployments.
• The entire process is ephemeral, which is the key to Build, Release, and Run. All artifacts and environments can be completely rebuilt from scratch if something in the pipeline is destroyed, utilising resources kept in the source code repository.
• Main goal
  • a one-directional flow from code to release
    • Note: Never work on production or staging. let it be a one-directional flow.

6. Processes

Executes the app as one or more stateless processes

• Stateless vs. stateful processes
  • Stateless: Without storing any data, a stateless system sends a request to the server and receives the response (or the state) back.
  • Stateful: monitor data, anticipate a response, and, in the absence of one, resend the request.

• Note:
  • all long-lasting states must be external to the application, provided by backing services.
  • Avoid using web servers (sticky sessions)
  • Pre-caching in memory, the data must be saved to a store and used from there rather than being kept in memory

7. Port Binding

Export services via port binding

• The twelve-factor app is entirely independent and does not depend on the runtime injection of a webserver into the execution environment to create a web-facing service.
• An app created to enable externalised, runtime port binding may serve as a backing service for another app. The flexibility is powerful.
• Using port forwarding, any problems resulting from a conflict/clash between a port number assigned privately to the network and one used publicly by another operation can be avoided.
• Containers like Tomcat, JBoss, Liberty, jetty, undertow and Web Sphere are common in the Java world. Other web applications might run inside other containers, like Microsoft Internet Information Server (IIS).
• In a non-cloud environment, These containers receive web application deployments and are responsible for allocating ports to the applications when they launch.

8. Concurrency

Scale out via the process model

• Processes should be separated after being organised according to their purposes so that they can be scaled up and down as required. The utilisation of resources might fluctuate.

• Old model: Vertical scaling is the process of adding additional CPUs, RAM, and other resources (physical or virtual) to a single monolithic application.

• New model: Scaling out, or horizontally, is the process of adding more nodes(workers) and is a much more contemporary strategy that is suitable for the type of elastic scalability that the cloud allows.

• Building disposable, stateless, share-nothing processes then use horizontal scaling not vertical, and run multiple, concurrent instances of your application.

9. Disposability

Maximize robustness with fast startup and graceful shutdown

• The processes of a cloud-native application are disposable, allowing for quick startup and shutdown. If an application cannot start quickly and shut down gracefully, it cannot scale, deploy, release, or recover quickly.
• High traffic scale out - When the load is high, new instances are created, and the time consumed by these instances affects user performance.
• Watch for lengthy startup efforts, and initialization work and externalize them to backing services.
• Fast shutdown is acceptable, but graceful shutdown is preferred and make sure that the entire state is unharmed and robust against sudden failure.

10. Dev/Prod Parity

Keep development, staging & production as similar as possible

• Reduce, minimize, and eliminate differences between an app’s development, staging, production environment, and so on as identical as possible.
• Although there are countless chances to create a gap between environments, the most common gap areas are:
• Time gap - Developers take a long time to move across environments.
  • There must be Ci/CD processes, automated pipelines, possibly one-click production deployment, and takes hours rather than months or days to deploy.
• Personnel gap - The developers team write the code, operation team deploy it.
  • Don't have manual procedures and there must be automated deployments, predictable deployments that can be repeated, and one-click deployment to production.
• Tools gap - The tech tools/stack/libraries used should be as similar as possible across environments.
  • To support this principle: the use of containers – a very powerful tool here, as they enable you to run the same execution environment from local development through production. Docker containerization and most of the tools are open-source and ideally all cloud environments (except development) then reduce it works in this environment issues, can easily catch issues and can fix/patch issues.

11. Logs

Treat logs as event streams

• Logging is essential for monitoring how your application behaves and here, it treats log entries as event streams routed to a separate service for analysis and archival. Every developer needs to read logs (to access it) ex: log4J config, etc.
• The App should not worry about how logs are written; instead, just write to stderr and stdout. The application is unable to predict factors like disc position, log size, rollover time, and others.
• Logs need to be non-functional requirements i.e., log aggregation, log storage, log processing, log access.
• To track and analyse your log emissions, you can use tools like the ELK stack (ElasticSearch, Logstash, and Kibana), Splunk, Sumologic, or other 3rd party applications. Logs become a separate service that helps in elastic scalability.

12. Admin Processes

Run admin tasks as one-off processes

• This rule states that management or administrative tasks should be carried out by applications in an environment that is the same as that of the app's normal and ongoing operations. It also recommends running such scripts on the production server using a tool included in the execution environment.
• Some examples of tasks that might require reevaluation include database migrations, interactive programming consoles (REPLs), running timed scripts like nightly batch jobs, and one-off jobs executing custom code.
• Moreover, it's crucial to align administrative processes with the cloud provider's features and avoid relying on traditional methods in cloud-native environments.

Instead

• Use cloud-enabled processes like cloud functions/lambdas to run instead of traditional administrative processes.
• All data migrations should be implemented as separate cloud-native processes.
• Don't use REPL as an admin.

Conclusion

In cloud computing, building applications demands agility, scalability, and resilience. The 12 Factor App guides developers towards creating applications that seamlessly produce in cloud environments. If you are just starting your cloud journey, the 12-Factor App methodology is worth exploring. Its principles offer a solid foundation for building modern, cloud-native applications that can adapt and flourish in the face of continuous change.

References:

• https://www.redhat.com/architect/12-factor-app
• https://12factor.net/
• https://www.oreilly.com/library/view/beyond-the-twelve-factor/9781492042631/

Send me a direct message on X , If you're curious or have any questions on the subject; I'd be pleased to chat with you!