How to ship the same JavaScript bundle across N environments

The problem: your client-side JavaScript app's deployment pipeline goes something like dev -> QA -> staging -> prod.

There's a huge variety of ways that JavaScript apps bundled for the browser are packaged and deployed, but the likely end result in production is that a user's web browser makes a request to a CDN, and the CDN fetches the bundle from one of many possible types of origin server and then returns the bundle (or a cached version of it):

Web browser: "hi can I please get a JS bundle?"

Regardless of the specific way your compiled bundle is packaged and sent off – for example, whether it's baked into a docker image that gets pushed to a container registry, or if you send it straight to S3 – it would be ideal to compile a bundle for a given git commit just once, and promote the exact same immutable artifact across all four environments (or deploys, or whatever you want to call them). Besides increasing our confidence in the fact that the code is the same everywhere, it will save quite a lot of time in the medium- and long- term as the app grows and the build becomes ever-slower.

But certain values in the app change across environments, for example, an apiUrl that references a backend service. We'll call these values that are likely to change configuration.

Because of these configuration values, we end up compiling different bundles for each environment, possibly leaving in our wake a giant pile of bundles with funny names like staging-alice-bljkjasdf.min.js or qa-bob-HACK-final-2-bljkjasdf.min.js, or maybe we resort to inflexible hacks to ask from within the bundle at runtime, "where am I?"

The goal of this post is to illustrate an approach that sidesteps the problems of those two approaches and prioritizes flexibility and time-savings.

Dealing with configuration: prior art

The 12-factor app's section on config offers a concrete prescription for dealing with this situation in the context of server-side apps: store config in environment variables. But alas, this is a client-side app and we can't just indiscriminately apply that advice. Still, if we look at the underlying 12-factor goals, we can extract two guiding principles:

  • A strict separation of config from code means we can change config (which there's a small amount of, and it doesn't need to take much time) without needing to change any code and recompile (which can be big and slow)
  • Using granular config can increase flexibility and prevent a "combinatorial explosion of config which makes managing deploys of the app very brittle"

Config coupled to code, two ways

You might have some configuration like this inside your bundle:

let apiUrl;
let analyticsUrl;

if (location.hostname === 'dogs.com') {
  apiUrl = 'api.dogs.com';
  analyticsUrl = 'analytics.dogs.com';
} else if (location.hostname === 'dev.dogs.com') {
  apiUrl = 'api.dev.dogs.com';
  analyticsUrl = 'analytics.dev.dogs.com';
}

With this kind of run-time "where am I?" test to decide on config values, you can promote a single bundle across environments. But it's inflexible: adding new config values (e.g., to add a new hostname pattern to match on) or adjusting the groupings (e.g., to get the combination of api.dogs.com plus analytics.dev.dogs.com for debugging) requires a code change and compiling a new bundle. Eventually this may become the combinatorial explosion of config we are trying to avoid.

Alternatively, you might have something like this in your source code:

const apiUrl = process.env.API_URL;
const analyticsUrl = process.env.ANALYTICS_URL;

And for each environment, a separate bundle is compiled, passing in environment variables at build-time for configuration values. By avoiding the grouping, this does achieve the granularity we're looking for. And at a glance, it kind of looks like that's not config inside of code, but in the compiled bundle each of those process.env values will be hardcoded strings, config values living inside the bundled code, so...it is 🙃.

Compiling different bundles for each environment is a popular approach but I don't understand why – it is convenient but confusing. Using build-time environment variables in this way seems to be often conflated with the 12-factor idea of storing config in environment variables, without reaping the main benefits of that approach. It also creates a footgun leading folks to accidentally put legit secrets into the bundle (notice the all-caps WARNING in the react docs).

Granular config, separated from code: a recipe

We talked earlier about the way the JS bundle might be deployed. Now we're more interested in the way the HTML page that references the bundle is deployed and served, which might be different.

If you're generating a static index.html file and shipping it to S3, directly to a CDN, etc, steps 1-3 will occur on the machine that deploys the file, just before the deployment.

If you're serving index.html from a VM instance/container,  steps 1-3 will happen there.

It's important that steps 1-3 are distinct from the build. Borrowing from build, release, run, we want to create a discrete stage that combines the build and the configuration.

Step 1

Store each config value that you want to deliver to the browser in an environment variable, under a prefix such as JS_BUNDLE_RUNTIME_CONFIG_. How these end up stored in environment variables depends on your specific situation, but here's one way:

export JS_BUNDLE_RUNTIME_CONFIG_API_URL="api.dogs.com"
export JS_BUNDLE_RUNTIME_CONFIG_ANALYTICS_URL="analytics.dogs.com"

Step 2

Create a method of extracting the environment variables under the JS_BUNDLE_RUNTIME_CONFIG_ prefix and transforming them into JSON string, an object of key/value pairs, with the prefix removed from the keys:

      
const PREFIX = 'JS_BUNDLE_RUNTIME_CONFIG_'

function jsBundleRuntimeConfigEnvVarsAsJSON() {
    const config = {};

    Object.entries(process.env).forEach(([name, value]) => {
      if (name.startsWith(PREFIX)) {
        config[name.slice(PREFIX.length)] = value;
      }
    });

    return JSON.stringify(config);
}

console.log(jsBundleRuntimeConfigEnvVarsAsJSON())
// {"ANALYTICS_URL":"analytics.dogs.com","API_URL":"api.dogs.com"}
      
      
      
import json
import os

PREFIX = 'JS_BUNDLE_RUNTIME_CONFIG_'


def js_bundle_runtime_config_env_vars_as_json():
    config = {}

    for name, value in os.environ.items():
        if name.startswith(PREFIX):
            config[name[len(PREFIX):]] = value

    return json.dumps(config)


print(js_bundle_runtime_config_env_vars_as_json())
# {"ANALYTICS_URL":"analytics.dogs.com","API_URL":"api.dogs.com"}
      
      
      
require 'json'

PREFIX = 'JS_BUNDLE_RUNTIME_CONFIG_'

def js_bundle_runtime_config_env_vars_as_json
    config = {}

    ENV.each do |name, value|
        if name.start_with? PREFIX
            config[name[PREFIX.length..-1]] = value
        end
    end

    JSON.dump(config)
end

puts js_bundle_runtime_config_env_vars_as_json
# {"ANALYTICS_URL":"analytics.dogs.com","API_URL":"api.dogs.com"}
       
      
      
#!/bin/bash

# requires https://github.com/jpmens/jo
PREFIX="JS_BUNDLE_RUNTIME_CONFIG_"
CONFIG_JSON=$(jo $(env | grep "$PREFIX" | cut -c $(expr ${#PREFIX} + 1)-))

echo $CONFIG_JSON
# {"ANALYTICS_URL":"analytics.dogs.com","API_URL":"api.dogs.com"}
       
      

Step 3

Use your function from step 2 to get the JSON string of config values and shlep them into your index.html template, as the contents of an inline JSON file, ensuring that it comes before your bundle:

<body>
    <div id="root"></div>
    <script type="application/json" id="js-bundle-runtime-config">
      {{ JS_BUNDLE_RUNTIME_CONFIG }}
    </script>
    <script src="my-bundle.min.js"></script>
</body>

The specifics here too will differ depending on your templating situation.

For something like create-react-app, where the default is that you're building an index.html file to be statically served, you can add the script tag above to public/index.html, use the shell command from step 2 to generate the JSON, and then use sed to inject that JSON string into the file:

sed -i "s/{{ JS_BUNDLE_RUNTIME_CONFIG }}/$CONFIG_JSON/" build/index.html

Tips on when exactly this step should happen if you're serving index.html from a VM instance or container:

  • If serving a static index.html, do it just once on startup before serving requests
  • If dynamically serving index.html, it'll happen on each request through the templating system (handlebars, jinja2, erb, etc)

The end result is that this gets sent to the browser:

<body>
    <div id="root"></div>
    <script type="application/json" id="js-bundle-runtime-config">
      {"ANALYTICS_URL":"analytics.dogs.com","API_URL":"api.dogs.com"}
    </script>
    <script src="my-bundle.min.js"></script>
</body>

Step 4

Within your bundle, parse the JSON script tag's contents and snag values from it as needed:

function getRuntimeConfig() {
    return JSON.parse(
      document.getElementById('js-bundle-runtime-config').innerHTML
    );
}

const config = getRuntimeConfig();

console.log(config);
// {"ANALYTICS_URL":"analytics.dogs.com","API_URL":"api.dogs.com"}

fetch(`${config.API_URL}/woof`).then(doStuff)

Step 4a

Wait what about local development? Who will injecteth the JSON? The specifics depend on exactly what you're using for a development server. Here's an example assuming the vanilla create-react-app setup, which means it's webpack-dev-server under the hood. We need to tweak the webpack config to do a string substitution, but I don't want to deal with ejecting, so here's a quick and dirty way to do it:

yarn add react-app-rewired raw-loader string-replace-loader --dev

Then create config-overrides.js in the root of the project, calling the same function we created in step 2 to get the JSON string:

module.exports = function override(config, env) {
  config.module.rules.push(
    {
      test: /\.html$/,
      loader: "raw-loader"
    },
    {
      test: /\index.html$/,
      loader: "string-replace-loader",
      options: {
        search: "{{ JS_BUNDLE_RUNTIME_CONFIG }}",
        replace: jsBundleRuntimeConfigEnvVarsAsJSON(),
      }
    }
  );
  return config;
};

And change package.json like so:

-    "start": "react-scripts start",
+    "start": "react-app-rewired start",

And voila! This assumes that those prefixed environment variables are set on your development machine. Perhaps use dotenv and a .env file outside version control to achieve this (or even use a .env file that is JSON and load it directly here, etc).

Why this is cool

We can now deploy/promote the same immutable artifact across any number of environments without recompiling bundles.

We are not polluting the global namespace by putting the config values on the window. The JSON is inline, which is convenient and requires no additional requests. It's clear that our config values are not "secret". And because the script is type=application/json, the browser will not execute it as JavaScript, and so it's compatible with a CSP that prevents inline scripts.

Variations: if your caching requirements are different (e.g. index.html is cached forever) then extract the JSON into a separate file. And if you don't mind living on the edge, store the config as a variable on the window.

How do you deal with config across environments in client-side JavaScript apps? Is there anything I'm missing about the coolness of build-time environment variables? And finally, I apologize for my contributions to the continued overloading of the word "environment".

Show Comments