Continuous Delivery and Feature Flags

Our current branching model includes two branches – master and stable. This allows changes not ready for production to sit on master while hotfixes can be cherry picked onto stable. While this is intuitive to understand as a developer, it is more complex when implemented in a deploy pipeline. Two sets of build artifacts are needed at any given moment, and there is always an extra cherry-pick step required to get anything to produciton. It also encourages devs to hold onto features in master for longer, building up a large changeset that has to be carefully tested and (potentially) orchestrated before cherry-picking onto stable.

In the new CI/CD pipeline managed by App SRE, there is only one branch involved in the deployment of code to stage and prod – main/master. This simplifies the build pipeline significantly – no cherry-picking or merging between two branches, and changes are pushed to prod more often (because there’s nowhere else for them to sit). Remember – one of the big aims of the DevProd team is to get app teams to push to prod early and often .

The code on the main branch is expected to be ready to be deployed to production at any time. This is particularly important for when a hotfix is required to go to prod. Any changes that have been pushed to main will go along with the hotfix. But how do you do this if you have a big feature that you don’t want hanging out on a side branch forever, particularly if you need to coordinate integration between other apps?

Enter feature flags.

Feature flags are like boolean environment variables on steriods. An app queries a feature flag server (Unleash in our case) for the value of a particular flag, and the server will respond with the appropriate value. The value could differ based on any number of variables (known in Unleash as Activation Strategies ): which environment (stage/prod/ephemeral), account number, internal users, beta UI, etc. We can even implement a “soft rollout” of features, where only a small subset of users can see a feature, with the flag being used between the frontend and backend to coordinate exposing features.

App SRE has built an integration with an Unleash server for tenants to use. We’ve already got an instance set up for both stage and prod . Clowder also supports deploying an Unleash server in ephemeral environments. At the time of this writing, no app has adopted feature flags, so we have yet to write up any guidelines beyond how to connect and fetch basic flag values. Unleash docs is a great place to start with integrating it into your app. App SRE wrote up a bit of doc for Unleash as well.

Lastly, DevProd knows that we have some work to get Clowder and UI integrated with Unleash in stage and prod .