Save Time with Scaffolding Using Yeoman
This is a text version compliment to the YouTube video of the same name. You can watch the video using the embedded player below.
Manufacturing jobs and supply chains are rife with examples of just-in-time processes and Lean process improvement. The idea is that by eliminating waste you can increase efficiency, productivity, and ultimately revenue. Although current hiccups in the supply chain due to COVID-19 disruptions have certainly drawn into question the value of just-in-time when the proverbial poop emoji hits the processor fan, productivity gurus didn't build a multi-billion dollar industry out of fluff. Well… most of them did, but that doesn't diminish the value of quality process improvement and appropriate time management.
Software has always benefited the most from acronyms that dictate frameworks (e.g., SOLID, DRY), but software engineering in abstract (or in academia) is different that software engineering run by finance departments, project managers, and basically anyone else that gives you a deadline before giving you the actual requirements. In those situations, you're often stuck repeating the same tasks because slack time is non-existent, and it's during this slack time where you would normally build tooling to assist your own work process.
I'm a big fan of tooling, but a huge critic of one-size-fits-all tooling. Each company is unique and the tooling that you need to produce the appropriate business value is thus, also unique. Once you get your feet wet in a new job or a new project you begin to put the productivity puzzle together. Completing a project is one thing. Making a project sustainable is another. Once comfortable, you'll see areas that would benefit not from more user features, but from better tooling under the hood. This is where true software productivity is won or lost.
In most of my current projects (both for work and for pleasure), I use very similar tools, frameworks, configurations, etc. With Node.JS applications, a lot of times this means loading in the same packages and copying the same configuration files all over the place. Whether in a monorepository or a brand new repo, this minimal step (although it might only take an hour or less) offers its fair share of time sinks, copy/paste errors, and various other issues.
This is where code generation comes in handy, but even more minimal that code generation is basic application scaffolding, and it's drop dead simple to do application scaffolding in Yeoman.
For a basic example, start by initializing a new project:
npm init -y
You can fill in the details of the package.json file later.
For your application structure, you're going to need a folder called generators and a folder inside of that one called app. This tells Yeoman where to find your generator code and template. If you had multiple generators associated with this generator package, you could have multiple folders under the generators folder and app would be the default generator.
You'll need to install the Yeoman generator package. I also recommend installing the Yeoman package itself locally for debugging purposes, and since we're just doing something basic, you can get by with just installing fs-extra as an added dependency.
npm install yo yo-generator fs-extra --save
With those installed, we can start building our generator. Do this in an index.js file in the root of the application directory (same directory that the generators folder sits in). Once you have this set up, we'll begin by creating our generator class.
const Generator = require('yeoman-generator');
const fs = require('fs-extra');
module.exports = class extends Generator {
...
};
Here we're just importing the base Generator class and the fs-extra module to give us some added power with file system work. When it comes to Yeoman, you can add whatever classes you want and it'll run those classes in the order in which they appear; however, we're going to stick to a number of baked-in life cycle classes that Yeoman defines.
First we need to initialize our class:
initializing() {
this.log(`Let's generate a Node.JS application!`);
this.props = {};
this.pkgs = {
defaultPkgs: [
...
],
expPkgs: [
...
],
restPkgs: [
...
],
defaultDevPkgs: [
...
],
expDevPkgs: [
...
],
restDevPkgs: [
...
]
};
}
All I'm doing here is defining an object with properties of arrays. I'm going to create a very basic scaffold that mostly just bootstraps the types of projects that I work with and the configuration files that go with them, so I don't need much more in here other than arrays of the packages that I want to install.
I've left the actual packages out, but if you want to see which ones (and want to see what the template files look like), you can find them in the GitHub repo.
One of the key things about Yeoman is that it can be interactive. It accomplishes this via including Inquirer.js and using the prompting functionality from that package. We'll work with this in our next method.
async prompting() {
const props = await this.prompt([{
type: 'input',
name: 'name',
message: `What is your application's name?`
},
{
type: 'input',
name: 'description',
message: 'Please enter a description for your project:'
},
{
type: 'input',
name: 'fullname',
message: 'Please enter your name:'
},
{
type: 'input',
name: 'email',
message: 'Please enter your email address:'
},
{
type: 'input',
name: 'repo',
message: 'Please enter the URL for the GitHub repository:'
},
{
type: 'list',
name: 'stack',
choices: ['nodejs'],
message: 'Which tech stack are you using?'
},
{
type: 'list',
name: 'app',
choices: ['package', 'express web app', 'restify API'],
message: 'What type of application are you building?'
}]);
this.props.name = props.name;
this.props.description = props.description;
this.props.fullname = props.fullname;
this.props.email = props.email;
this.props.repo = props.repo;
this.props.stack = props.stack;
this.props.app = props.app;
}
This method is async because the prompt() returns a Promise. We want to await the prompt() because we want to assign the results (the answers from the interactive session with the users) to properties on the class object.
The next built-in life cycle is configuring(). I won't be using it for any true configuration, but I'm going to use it here just because of it's position in the life cycle, and I'm going to use it to copy some files. Yes, I'm cheating here.
async configuring() {
this.log('Started copying files...');
try {
await fs.copy(`${ this.sourceRoot() }`, `${ this.destinationRoot() }`);
this.log('Finished copying template files.');
}
catch(e) {
this.log(`An error has occurred copying the template files: ${ e }`);
}
}
In this snippet, sourceRoot() is a special method to access where the templates are located and destinationRoot() is a special method to access where the Yeoman generator is being executed.
With the next life cycle method, I'm cheating a little less. I use writing() to take the projects package.json file as a template, set unnecessary properties to undefined and rewrite other properties.
writing() {
this.log('Writing package.json file...');
const pkg = require('../../package.json');
pkg.name = this.props.name;
pkg.version = '0.1.0';
pkg.description = this.props.description;
pkg.dependencies = undefined;
pkg.devDependencies = undefined;
pkg.files = undefined;
pkg.keywords = undefined;
pkg.publishConfig = undefined;
pkg.contributors = [ `${ this.props.fullname } <${ this.props.email }>` ];
if(this.props.repo != null && this.props.repo.trim() != '') {
pkg.repository = {
"type": "git",
"url": `git+${ this.props.repo }.git`
};
pkg.bugs = {
"url": `${ this.props.repo }/issues`
};
pkg.homepage = `${ this.props.repo }#readme`;
}
else {
pkg.repository = undefined;
pkg.bugs = undefined;
pkg.homepage = undefined;
}
pkg['scripts'] = {
build: "tsc",
test: "nyc mocha ./**/test/*.test.js --ignore ./**/node_modules/** --exit",
eslint: "eslint ./src/*.ts ./src/**/*.ts",
"eslint-fix": "eslint ./src/*.ts ./src/**/*.ts --fix",
};
this.fs.writeJSON(this.destinationPath('package.json'), pkg);
this.log('Finished writing package.json file.');
}
All this is doing is setting up my resulting scaffolded project's package.json file to have some information pulled from the responses from the prompting() method, while rewriting some other properties for standard usage. When finished, it uses the writeJSON() method to write the file to the output directory.
Remember those packages we outlined during initialization? Now we want to install them.
install() {
this.log('Installing NPM packages. Hang tight. This could take a few minutes.');
const pkgs = this.pkgs.defaultPkgs;
if(this.props.app === 'express web app') {
pkgs.push(...this.pkgs.expPkgs);
}
if(this.props.app === 'restify API') {
pkgs.push(...this.pkgs.restPkgs);
}
this.npmInstall(pkgs, { 'save': true });
const devPkgs = this.pkgs.defaultDevPkgs;
if(this.props.app === 'express web app') {
devPkgs.push(...this.pkgs.expDevPkgs);
}
if(this.props.app === 'restify API') {
devPkgs.push(...this.pkgs.restDevPkgs);
}
this.npmInstall(devPkgs, { 'save-dev': true });
}
Here we are simply checking the result of which type of application the end user selected and we're using the npmInstall() method to install the selected packages set aside for those types of applications. Notice the save versus save-dev properties that are used to specify where to place the dependencies.
Lastly, we wrap up the generator:
end() {
this.log('All done! Happy coding!');
}
That's great, but how do I debug this if something goes wrong? I use Visual Studio Code (Codium to be exact), so just add the following to your launch.json file:
{
"version": "0.2.0",
"configurations": [
{
"request": "launch",
"name": "Yeoman Generator",
"type": "node",
"program": "./generator-yo-bootstrapper/node_modules/yo/lib/cli.js",
"args": [ "@quietmath/yo-bootstrapper" ],
"stopOnEntry": true
}
]
}
With the above configuration in place, run npm link in the root directory to install the local package globally. Then when you launch the debugger, it'll run Yeoman with your newly installed generator. In this example, the package.json for the generator has @quietmath/yo-bootstrapper as the name of the package.
This generator is a quick-and-dirty one meant solely for my personal use to bootstrap new Node.JS applications, but you can see how with a little bit of coding, you can eliminate a lot of copy/paste. "Don't Repeat Yourself" doesn't have to be about code. It can include your processes as well. Once you get tired of copying the same configuration files all over the place, you head the generator route.