Popular posts

This sounds like a simple solution, I'll implement this too then benchmark... Thanks

Williams Isaac

Just a rough draft without filesystem access, likely has typo's but should give you a general idea. No fancy threading, I'd save that for each separate data stream to handle your lines.

const
    csv = require('csv-parse');

var
    result = [],
    parser = parse({delimiter : ','});

parser.on('readable', function() {
    var line;
    if (line = parser.read()) {
        for (let i = 0, iLen = line.length; i < iLen; i++) fields[i] = {};
        do {
            for (var i = 0, iLen = line.length; i < iLen; i++) {
                if (!result[i][line[i]]) result[i][line[i]] = 1;
                else result[i][line[i]]++;
            }
        } while (line = parser.read());
    }
});

parser.on('end', function() {
    for (let i = 0, line; line = result[i]; i++) {
        let amt = 0, value = '';
        for (let j in line) {
            if (line[j] > amt) {
                amt = line[j];
                value = j;
            }
        }
        result[i] = value;
    }
});

/*
    here you would just parser.write your CSV file, OR pipe
    the stream into it. Only once you've read and written all
    the data would you:
*/

parser.end();

Again, probably the simplest approach. Uses csv-parse (so you have a bit more complete a handling). Notice how at the 'end' event we normalize out the search sets for just the result, re-using the same array.

I went with array of object instead of object of objects since we really don't know the fieldnames nor would/should you want them static. Though if I were to track fieldnames I'd use that initial 'if' to say the first row are the names and then swap from do/while to just plain while.