Storing Sensitive Data for your DevOps Pipeline (in GitHub)

Never check your passwords and secrets into source control. We've all been told this plenty of times; Yet for some reason, people keep doing it. There are, of course, different levels of concern. If you have an on-premise source control solution, you might be less inclined to care. If the source control server is not accessible to the outside world, well, that's a little better than storing passwords in a cloud solution provider (still not great though).

How do we do this efficiently? In today's DevOps world of check-in often and automatically deploy, you're going to have multiple configuration files (one for each environment), so you're going to have different credentials to manage, and it's too "manual" to just put the configuration file on the deployment server and leave it there.

Let's look at an example using GitHub. In fact, we're going to use GitHub as the source control repository, GitHub Actions for our DevOps pipeline, and .NET framework for the language (this is so we can use ASP.NET's Web.config configuration file as an example).

In a typical ASP.NET project (let's use Framework instead of .NET Core here, but the reality is that this can apply to any application that requires a server configuration file) the Web.config file contains connection string information. When developing locally, it doesn't matter. Just shove that connection string right in the file. But what do you do when you deploy the application?

Your mileage will vary depending on your framework, but if you're using IIS as your web server, you'll likely have a Web.config, and in both ASP.NET and ASP.NET Core, you can use a transformation utility to transform the one Web.config into another. For example, if you have a "Dev" environment configuration, you likely have a Web.Dev.config file. When you publish (using Visual Studio's publish mechanism) or use a transform utility inside of MSBuild, the Web.Dev.config will replace certain components and values in the Web.config. This means you can have a "Dev" environment connection string in the Web.Dev.config file, and have that replace your local environment connection string.

Don't let the .NET lingo throw you off. You could also just do this with a standard .env file by having different versions of the file and a build task that renames something like dev.env to .env.

So what's the best way to handle this? There are two things we don't want. We don't want to check our connection strings into source control, and we don't want to have to manually manage files inside the different environments.

We can tackle this with GitHub Actions.

First let's take a look at a handy text replacement action in our GitHub workflow file:

- uses: cschleiden/replace-tokens@v1
            with:
              tokenPrefix: '___'
              tokenSuffix: '___'
              files: '["**/*.config"]'
            env:
              DBUSER: ${{secrets.DBUSER}}
              DBPASSWORD: ${{secrets.DBPASSWORD}}
              DBSERVER: ${{secrets.DBSERVER_DEV}}
              DATABASE:  ${{secrets.DBASE}}
              ADCONNECTION: ${{secrets.ADCONNECTION}}
              ADDOMAIN: ${{secrets.ADDOMAIN}}
              ADUSER: ${{secrets.ADUSER}}
              ADPASSWORD: ${{secrets.ADPASSWORD}}
      

This is a step inside of the build job in our GitHub workflow YAML file. The token prefix and suffix is a triple underscore (thanks, BEM designers for making me switch from a double underscore because of your crazy CSS classes). The files array is saying to look at any *.config file in the project directory. Finally, I'm setting some environment variables. This particular replacement action replaces text such as ___DBUSER___ with an environment variable that matches the name (sans token indicators).

The Web.config meanwhile, looks like the following (for the appSettings element):

<appSettings>
            <add key="WSBASEURL" value="___WSBASEURL___" />
            <add key="WSBASEFOLDER" value="___WSBASEFOLDER___" />
            <add key="DBUSER" value="___DBUSER___" />
            <add key="DBPASSWORD" value="___DBPASSWORD___" />
            <add key="DBSERVER" value="___DBSERVER___" />
            <add key="DATABASE" value="___DATABASE___" />
            <add key="ADCONNECTION" value="___ADCONNECTION___" />
            <add key="ADDOMAIN" value="___ADDOMAIN___" />
            <add key="ADUSER" value="___ADUSER___" />
            <add key="ADPASSWORD" value="___ADPASSWORD___" />
      </appSettings>
      

Again, remember that this doesn't have to be a Web.config and ASP.NET. You can easily do this with any configuration file, while using a build task to swap out the working file with a tokenized version.

Lastly, we need to take a look at those secrets in the GitHub workflow file.

This is a simple matter of setting up your project secrets inside of your GitHub repo. See the images below.

Of special note: These secrets need to reside in the same GitHub repository as the workflow file that you are running.

Simple enough?

The only beast is setting up all the secrets, putting in the replacement variables, and then adding them to the workflow file. If you have a large project, and you're introducing this for the first time, it can be a bit onerous to manually add all this data, but once set up, your passwords are safe and your DevOps pipeline is robust.