Enterprise Snafus Build Servers, Transpilers, Node.JS and Bower

We've all hit snafus when it comes to enterprise deployment, and even with the great strides that Microsoft has made with Visual Studio and Team Foundation Server, you can rest assured that their are instances when the compiler or build agent trips over its own feet trying to complete a successful build, or times when Visual Studio thinks it knows what you want, but offers little flexibility for you to get what you actually want.

The modern web is an era of DevOps and tooling. Programmers who have grown tired of the constraints of browser technology and updates have branched out and created technologies like TypeScript, Babel, Less, and Bower, and each of these can affect your tooling, workflow, and deployment.

At my current job, we went with a few of these options. TypeScript was a natural choice, since we're a Microsoft shop, and the static typing that TypeScript offers creates a nice strictness that fits well with our C# code. Less was another choice (for CSS transpilation) since much like TypeScript, Less is a superset of the language it compiles to, and not some alternate language. Bower, meanwhile, is a front end package management solution. It pulls packages from the latest stable release in GitHub, while also offering the flexibility of installing git packages not in their repository index. We could have just used Visual Studio's NuGet package management, but those aren't updated as often as the sources in GitHub, and even ASP.NET Core 1 tutorials are using Bower instead of NuGet for acquiring front end resources such as Bootstrap.

TypeScript will transpile directly in Visual Studio, but there is also a command line option that runs on Node.JS (node). Bower and Less are also utilities that run on node, which means that node needs to be installed on the development machine, and node modules need to be in the root directory of the application. This can cause some issues.

The Ideal Setup

Ideally, a project utilizing these tools would be set up in the following way.

• Only TypeScript's *.ts files should be in source control, not the generated JavaScript or source maps.
• Since Bower packages (installed under bower_components) are not meant to be used directly, but instead with something like Grunt, Gulp, or Brunch, this folder should not be included in source control--only the bower.init file.
• Much like the bower_components folder, the node_modules folder should not be checked into source control.
• Generated JavaScript from Grunt, Gulp, or Brunch task runners should not be in source control.

Using Bower can produce a lot of files in the bower_components directory structure, and if you have to publish a project locally, and manually move the files to a production server (or publish to a share), it can take an extended amount of time if the bower_components folder is included in the project. The same goes for node_modules.

In order for the above setup to work for deployment, Visual Studio's Package/Publish Web settings need to be set to publish all files in the project folder and not just files needed for the application to run. Visual Studio's TypeScript integration will ensure that the generated JavaScript files are published with the project, but any generated files from Grunt, et al. will not; that's why you need to publish all files in the project folder.

The problem is that the moment you do this, it'll try to publish the bower_components and node_modules folders that you've excluded from the project. You need a way to be able to publish everything in a project folder, but still be able to exclude certain folders. This is theoretically possible by editing either the *.csproj file for the project (which is an XML file), or by editing the *.pubxml file produced by the publish settings.

I say theoretically because it doesn't actually work. You can search StackOverflow to your hearts content, but no matter what variation of using the XML element <ExcludeFoldersFromDeployment> you try, you'll still get a failure on publishing. The error itself (The "CollectFilesinFolder" task failed unexpectedly.) is a copying error that essential points to a folder structure/file path being too long. In our particular instance this is being caused by the grunt-contrib-cssmin module. This is because node has decided that endlessly recursive node module library folders are a good thing.

Despite all the various attempts at excluding folders from a particular deployment, when you are doing a local file system publish, it still seems as if the Visual Studio build and publish process reads that directory before excluding it, and decides to choke on it.

There are only two viable solutions that seem to work:

1.) Add the generated JavaScript files to source control, and just deal with it. Then you can switch your publishing mode back to those items only in the project (or those items only needed to run the application).

or…

2.) Install the node modules needed for your Grunt operations globally, and then use npm install --link to create symbolic links inside of your project. For now, it seems like the build process will not follow those symbolic links.

Ultimately, this is for a local publish, which copies the files to the local file system for you to push elsewhere. A truly DevOps solution would have the deployment get pushed from a build server. Team Foundation Server allows you to create a bunch of various build steps in your build definition, including Grunt and Gulp task management. If you are running your publishing from a build server, you can have a Grunt step that transpiles your Less, Typescript, etc., and then have that output pushed with the rest of the build, avoiding the entire issue altogether.

This is just another reason to throw away the old publishing model, and move towards a DevOps and continuous integration solution.