Discussion on "Cleanup the node_modules for a lighter Lambda Function"

Vikas Solegaonkar · 2020-12-09T05:54:41.898Z

Any nodejs project carries a bulky folder - the node_modules - that carries all the modules and dependencies that the application would need. If you try to peep into that folder, you can see a huge chunk of folders and files. That often makes me wond...

hey, that's a really awesome amount of value you shared with the community! Thanks for it, saved for later! it was much needed 😄

Nice article, I didn't know about the prune. Normally I just use webpack and serverless framework and run the optimisations on webpack itself.

but the deployment package has to restrict to 500MB

I think you mean 250Mb.

From the docs, the deployment package can be:

50 MB (zipped, for direct upload)
250 MB (unzipped, including layers)
3 MB (console editor)

or now up to 10Gb if you are using a container image.

Thanks for your inputs. docs.aws.amazon.com/lambda/latest/dg/runtimes-wal…

500 Mb (512 to be precise) is the net allowed - including the runtime itself - that is if you take up the a custom runtime.

Of course, things have changed with the container in Lambda. But that has impact on the pricing as well.

The serverless framework itself brings in so much to the deployment - it ends up blowing up the deployment size.

I had not tried the webpack before. Tried it after your comment. But I saw it had problems with non js code in the node_modules. It was wonderful with pure js code. Any way to overcome that?

In my opinion,

$ npm prune --production

is better to remove devDependencies because npm install downloads dependencies but npm prune only removes devDependencies, doesn't downloads dependencies.

Small Benchmark with your example package.json:

$ cat package.json
{
  "name": "layerjs",
  "version": "1.0.0",
  "description": "This is the lambda layer generated for the service",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "jsonwebtoken": "^8.5.1",
    "pdfkit": "^0.11.0",
    "uuid4": "^2.0.2",
    "xlsx": "^0.16.9"
  },
  "devDependencies": {
    "aws-sdk": "^2.805.0"
  }
}
$ npm install # reinstall all dependencies
...
$ rm -rf node_modules
$ time npm install --production
...
added 146 packages from 120 contributors and audited 158 packages in 3.491s
...
npm install --production  3.29s user 1.81s system 99% cpu 5.122 total
$ npm install # reinstall all Dependencies
...
$ time npm prune --production
...
removed 12 packages and audited 146 packages in 1.3s
...
npm prune --production  1.52s user 0.44s system 95% cpu 2.054 total

Thanks for the insight. What I showed above is just a reminder to delete the node_modules that is already collected in the previous step.

Ofcourse, in real live, we don't have to install, delete and install once again. A simple npm install --production is enougn. Perhaps I need to elaborate on that point.

Thanks for pointing out

Discussion

Cleanup the node_modules for a lighter Lambda Function

Responses(3)

Recent in Forum

Search Hashnode

Cleanup the node_modules for a lighter Lambda Function

Responses(3)

Recent in Forum