Resolving Serverless Webpack Issues

Image for post
Image for post

A story about how I debugged an issue with our webpack bundles on a serverless infrastructure and the key takeaways about developing JavaScript systems.

The Problem

Specifically, we were consistently seeing issues with our user’s service which contains roughly 22 functions at present. After some investigation, I discovered that our bundles being uploaded to AWS’s Lambda (λ) were ~5MB/function. λ can easily handle bundles of this size, but these bundles were significantly larger than what we expected. After some research, I discovered that webpack might just need more memory allocated to the process to complete successfully. However, I didn’t believe our code bundles should be large enough to require this change and I wanted to make sure we weren’t sweeping a different issue under the rug.

Around the same time, I was also beginning to see issues with longer cold starts which raised an alarm of concern since I know bundle sizes are directly correlated to cold start time. This lead me down a path of investigating why our bundles were larger than our expectations and how to reduce those sizes.

Background

As part of this, we developed a custom ORM for our database using ES6 classes that emulated the behaviors of libraries like Sequelize, Mongoose, or Rails’ ActiveRecord. This gave us classes with static members for record lookup and instance methods for entity queries and mutations.

We also created other classes around the codebase categorized into strategic types such as business logic, utilities, API wrappers, etc. All of these classes are imported/exported throughout the system using ES6 default exports. For instance, our UserModel file vaguely resembles:

As I’ll show you through this article, you’ll see that our decision to use ES6 classes and default exports created the severe issues that lead us to this investigation and fix.

Finding the root cause

1. Debugging the bundling step

Serverless framework provides the command sls package that packages all the functions and prepares them for deployment to your configured provider. This command is the first step that is typically run with the sls deploy command which also handles deploying the code to infrastructure. As such, I decided to start my investigation here and ran the package command locally to see what results it would produce. This produced 2 major findings:

  1. The aforementioned ~5MB bundles being uploaded to Lambda were actually the gzipped and compressed versions of the functions. I was reading the values from the AWS console and didn’t realize it was the gzipped value. This also meant our minified bundles of code were larger and I soon discovered that they were closer to ~7MB.
  2. The raw bundles were roughly ~16MB of unminified code! I don’t know exactly how much additional memory webpack requires to perform all its operations, but ~16MB of JavaScript definitely seemed like it would be a problem for a limited memory machine to accomplish.

Given all these findings, I decided to track down why the bundle sizes were so large. Our codebase isn’t that large and some of the functions in question definitely should have been significantly smaller than ~16MB.

2. Understanding Webpack

  1. We weren’t running minification at all.
  2. We were converting everything to commonjs.
  3. We are using large 3rd party packages, e.g. lodash and momentjs.
  4. We didn’t have tree shaking configured correctly.

These were all excellent candidates for further exploration but none explained how some of these functions were growing to be ~16MB. Hence, I needed delve deeper into how webpack’s bundling worked to understand the core issue.

3. Exploring the bundled code

These tools have their individual pros and cons, but I decided to go with the Webpack Vizualizer because I found its output to be easier to read.

I couldn’t run the sls package command on our user service without disabling most of its function which was the main problem so instead, I decided to run the tool on our graphql service which was also starting to experience some issues. This required some configuration and I ended up making some of the webpack optimizations before I got here which reduced our bundles by a bit, but the data still gave me some immediate data points to dig into further:

  • Our raw bundle was still too big at ~13+MB! 🤮
  • 97% of our bundle came from our node_modules 😑
  • The Twilio node library accounted for over 50% of our bundle at 7MB 🤯
Image for post
Image for post
Image for post
Image for post
After Minification: Left — graphql server handler, Right — currentUser handler

Twilio being the main culprit was exceedingly shocking since I was expecting either lodash or moment to be the main culprits as this service uses both those libraries, but, more importantly, the service in question doesn’t even use the Twilio package. In fact, Twilio is only used in 2 specific lambas in the user service so it had no place being in this bundle whatsoever. So why was Twilio being included in this bundle?

I stumbled up on the WHYBUNDLED? library which provides tooling to investigate the outputs from webpack stats and provide a path explanation of why a particular package was included through dependency trees. I hit a small snag at this point because the serverless-webpack plugin doesn’t provide the same CLI optinos as webpack so getting the stats.json used by WHYBUNDLED? wasn’t possible. I ended up finding the Webpack Stats Plugin and was able to get the needed stats.json produced with some minor configuration changes. Finally, I could run WHYBUNDLED? against our service and got the following output:

Image for post
Image for post
WHYBUNDLED output for twilio module

Accordingly to WHYBUNDLED?, Twilio was being imported by the graphql main handler function. Looking at that file’s imports, I didn’t see Twilio being imported directly, but I did see that our UserModel was being imported which does include the Twilio package. This indicated that maybe there was a flaw with our usage of ES6 classes and default exports, but we needed to try a few simple tests first.

Solution Attempt #1: Patch it for later?

AdonisJS’s Configuration

Google’s Closure Compiler

TypeScript

Solution Attempt #2: Quick Wins?

  1. Exporting everything as commonjs
  2. Large package misuse
  3. No minification

Exporting everything as commonjs

Image for post
Image for post

This was directly regarding minification and tree-shaking. Compiling to commonjs made it impossible to use either feature which was a big blocker to solving this problem.

As it turns out, we had installed the warmup plugin incorrectly. Fixing its installation and switching us back to the auto targeting mode quickly resolved this problem. Unfortunately, this didn’t really do much to fix the problem directly, but it did shave a few hundred kilobytes from the bundle. The bigger win was that I could now turn on minification and tree-shaking again so it was a move in the right direction.

Large Package Misuse

Starting with lodash, I read how tree shaking didn’t really work unless you did specific things to lodash. Then I discovered that those articles didn’t apply to Lodash v4 using ES6 module import statements correctly which is what we were doing 😄, but if you import chain anywhere, it pulls in the entirety of lodash because of how that function is written 😰. Turns out, we had a single instance of chain which I quickly replaced to alleviate this issue.

With moment, I read that by default it includes its locales which adds ~200KB of minified code to bundles. Reading further, I discovered we were already ignoring the moment locales correctly with our webpack configuration so this was a non-issue. 😅

With the two well-documented easier wins out of the way, this left Twilio’s node package. As it turns out, it was not tree-shakable and adds ~2MB of minified code to your bundles. At the time of this article, we’re still looking for a better solution to Twilio’s node package but found a way to limit its impact on our bundles.

No Minification

As this was is in direct conflict of the problem I was trying to solve, I needed to turn minification back on. I started by turning on webpack’s default minifier and running the bundle. Two minutes later, my system kernel panicked and crashed. I tried tweaking node’s memory allocation but that didn’t help either.

At this point, I noticed that we could try tweaking the minifier or switching out Terser with other alternatives. Playing with a few different configuration options, I was finding the same heap crashes were happening. At this point, I had exhausted the faster options and needed to delve into more extreme solutions.

Solution Attempt #3: ES6 Modules (FTW!)

Rather than just rewriting the entire codebase, I decided to identify high impact models that could definitively prove my hypothesis with the lowest effort. Based on the investigation so far, the UserModel was the best candidate as it was in most of our services and was the only file using the Twilio service that seemed to be causing our issues. I ran this test in two phases:

  1. Convert statics to export named functions
  2. Move functions to their own files for export

Phase 1

Running our analyzer, this yielded fantastic results as we saw our ~16MB become ~5.9MB unminified and saw webpack bundling again correctly. The new minified source was ~2.4MB. While this still isn’t fantastic, it was infinitely better than our ~16MB builds!

Image for post
Image for post
Image for post
Image for post
After removing statics from classes: Left — graphql server handler, Right — currentUser handler

Phase 2

To our surprise, we saw our bundle size for user functions to drop even further to ~2.9MB raw and ~1.4M minified. Clearly, this was the optimal solution. We didn’t see the same gains in our graphql service, but there are other optimization options now that we have this information.

Image for post
Image for post
Image for post
Image for post
After converting to full ES6 Modules: Left — graphql server handler, Right — currentUser handler

Solution

Conclusion

All code examples can be found at https://github.com/dustinsgoodman/serverless-webpack-optimization-article. It includes the HTML documents that contain the webpack visualizer tool results as well as stats.json files that can be used with WHYBUNDLED.

Credits

Sources

Senior Software Engineer at https://playerslounge.co

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store