tl;dr: Before starting to read, I need to warn you, it is long :) so if you have no clue about webpack, webpack plugin development or don't have time, I woudn't want to bother you.

Hey guys,

I am having a bit trouble with my custom webpack plugin, and i hope someone can shed a light. So briefly my plugin has two parameters, sourceFolder, targetFile. It reads all json files from source folder ( config files per subject like api, websocket, routes etc ) then concats them and puts in src root folder. Also if environment is different than production, it looks in source folder if any folder name matches with current env and takes each file in folder and merges one way to production one. This way I can generate a configuration file which has all master data with modified values per environment. pretty straight forward.

Now here is how I did it. I tapped on entryOption if environment is production ( since for prod build it only needs to run once before anything to get config ready first ), and for dev/local it tapAsync to watchRun so each build will trigger to generate a new config file.

Here is some issues I had and reasons I gave up. I hope you guys can share some thoughts on that if you can think of any solution

1. Because configuration.js is imported in main index.js, Each time watchRun kicks in, it rewrites the file ( even nothing has changed, because it touches file and lastUpdated changes) which triggers a second build right after first one ( twin builds ).

2. Then I though maybe if I read the file at the beginning of build, then go through merge process and compare each result to decide if file needs to be updated or not. Theoretically makes sense, but then noticed my file never gets updated unless I restart webpack build. When I put some BPs around, I found out that even though it reads file with fs.readFileSync ( also tried require) it never changes, although file itself changed. Feels like something is caching whole file system in webpack then when you read/require a file it doesn't actually read the physical file but data comes from cache. If anyone know if I can override that, that would be awesome.

   • Because of above I decided to add configuration.js to watchIgnore to avoid this issue. I can't add it to webpack ignore since it is being imported, when I do that, it doesn't get resolved in babel. and if I place is somewhere outside of src, then webpack doesn't like that idea :)

3. Another issue is updating this file when json files changed in conf folder. Since nothing imports those files, changes or new additional files doesn't trigger webpack to rebuild. those are just source files to generate configuration.js file.

   • as a solution trial, I collected all file paths in a variable and tapped compiled to add compilation.fileDependencies so after build is complete, it will ( ideally ) start watching those files. but no luck. for some reason webpack ignored. which is very confusing since all the extra file watcher plugins uses same method, maybe I am sending wrong paths or sth.

While having these issues, today I was thinking maybe I am doing all wrong. Ignoring configuration.js is the only can't really do anything on that part unless someone has a better solution, but for watcher, I wonder if I add those files in array as imports on top of configuration file, would webpack start watching them. Not sure, once app is first downloaded and webpack started, there won't be a configuration.js so initial kick-off would probably fail or files won't be watched.

yeah I have been researching a while but all the resources I could find about webpack is how to use webpack, nothing much out there about plugin development other than webpack's few pages.

thanks for reading and help in advance.

/* global process */

const fs = require('fs');
const path = require('path');
const merge = require('lodash/merge');
const shallowEqual = require('shallow-equal-fuzzy');
const chalk = require('chalk');

function CommonConfigPlugin(options) {
    this.options = Object.assign({}, options);
    this.files = [];
}

CommonConfigPlugin.prototype.getConfig = function (config, location) {
    const _self = this;
    config = config || {};

    const filesAndFolders = fs.readdirSync(location, { withFileTypes: true });

    const files = filesAndFolders.filter(dirent => !dirent.isDirectory());

    files.forEach(({ name }) => {
        let currentFile = path.normalize(location + '/' + name);
        let data = {
            env: process.env.dev,
            NODE_ENV: process.env.NODE_ENV
        };

        data[path.basename(currentFile, '.json')] = JSON.parse(fs.readFileSync(currentFile, 'utf8'));
        _self.files.push(currentFile);
        merge(config, data);
    });

    return config;
};

CommonConfigPlugin.prototype.apply = function (c) {

    let _self = this;
    _self.lastUpdated = null;

    if (process.env.NODE_ENV === 'production') {
        c.hooks.entryOption.tap('CommonConfigPlugin', function (compiler, callback) {
            _self.createConfig.apply(_self, [compiler, callback]);
        });
    }
    else {
        c.hooks.watchRun.tap('CommonConfigPlugin', function (compiler, callback) {
            try {
                _self.createConfig.apply(_self, [compiler, callback]);
            }
            catch (e) {
                throw new Error(e);
            }
        });

        c.hooks.afterCompile.tapAsync('CommonConfigPlugin', function (compilation, callback) {
            if (Array.isArray(compilation.fileDependencies)) {
                _self.files.map(file => compilation.fileDependencies.push(file));
            }
            else {
                _self.files.map(file => compilation.fileDependencies.add(file));
            }
            callback();
        });
    }

};
CommonConfigPlugin.prototype.createConfig = function (compiler, callback) {
    const _self = this;
    const target = _self.options.target;
    const sourceDir = _self.options.source;
    let currentConfig = '';
    try {
        currentConfig = require(path.resolve(_self.options.target), 'utf8');
    }
    catch (e) {
        currentConfig = '';
        // throw new Error(e);
    }

    if (typeof target !== 'undefined') {

        const env = global.env || 'production';

        // first lets build up production based config
        let config = _self.getConfig('', path.resolve(sourceDir));

        if (env !== 'production') {
            config = _self.getConfig(config, path.resolve(`${sourceDir}/${env}`));
        }

        let header =
            '/* eslint-disable */\n/**\n' +
            ' * ===============================================================================\n' +
            ' * This file is auto generated from ' + global.env + ' environment based configuration files \n' +
            ' * Any custom changes you make on this file will be overridden in next build. \n' +
            ' * All configuration changes should be added to environment configuration files at ' + sourceDir + '\n' +
            ' * ===============================================================================\n' +
            ' */\n\n' +
            'module.exports = ';

        let source = header + JSON.stringify(config, null, 4) + ';\n';

        // fixme - because of file formatting this is not working. File content and generated content doesn't match 
        if (source === currentConfig) { 
            console.log(chalk.yellow(`Configuration for ${global.env} has not been changed, skipped...\n`));
        }
        else {
            fs.writeFileSync(target, source);
            console.log(chalk.cyan(`Configuration for ${global.env} has been generated...\n`));
        }
    }
    if (typeof callback === 'function') {
        callback();
    }
};
module.exports = CommonConfigPlugin;