Build memory issues since upgrade to Vite 4

[wkande]

Since the release of VitePress v1.0.0-alpha.31 (which moves to Vite 4) I have started to have build memory issues. Our repo (see here) holds a standard VitePress setup.

At first a local build started to need a little more memory. We start to increase memory as time progressed, not much but it needed more. This is fine in that our dev machines can give more memory to local builds. At this point we need to use about 9.5 GB to run a build. We were using Nodejs 14 but have moved to Nodejs 18. Both exhibit the same behavior.

So yes it builds but the problem is that we have a GitHub action that runs a build and pushes the distribution directory to Firebase. GitHub actions (using a standard runner) only allow a max or 7GB to run a build so it fails.

So my question:
Are we missing some Vite configuration to manage the memory use?

[brc-dd]

Refer the workarounds mentioned here: vitejs/vite#2433

[wkande]

So yes one can increase the memory which is what we did for local builds and that is fine. But when a build is run using GitHub actions (@github) it is going to fail because GitHub actions will limit you to 7 GB unless you start paying major money and use self-hosted runners for their forthcoming hosted runners, that gets messy.

I tried adding a vite.config.js file to the root of the VitePress project in hopes that the following might help. But it does not.

build: { sourcemap: false, rollupOptions: { cache: false, }, },

I truly wonder if vite.config.js gets used at all since running yarn docs:build executes vitePress run ruild and maybe the build is running somewhere inside the vitepress directory inside the node_modules directory.

"docs:build": "yarn vitepress build docs;"

This is a major concern. I have been DM'd by a few since starting this thread who have said they have abandoned VitePress over this issue. I find myself in that position if this cannot be fixed. I have been hesitant to open a GitHub issue inside the VitePress repo because: 1) I do not want to pollute its long list of issues: 2) it is hard to provide details other than the following.

---

1. Clone this repo: git@github.com:api3dao/vitepress-docs.git
2. yarn install
3. yarn docs:build

If you do not add more memory to the build it fails, out of memory.

4. Add memory export NODE_OPTIONS=--max-old-space-size=9500
   Note that: export NODE_OPTIONS=--max-old-space-size=7168 fails which is 7 GB.
5. Run yarn docs:build again

Remember that this is not a valid solution to use for standard GitHub action which has a limit of 7 GB. You can see the GitHub action here. You can change its memory but above 7 GB you will get a timeout or GitHub just cancels execution of the action.

Let's save VitePress for larger projects. It really is a nice environment. Should I move this to a GitHub issue?

[wkande]

There is so much chatter about VitePress (and Vite) needing lots of memory for builds. I spoke with some that have walked away from VitePress over this. The good news for us, we found out why VitePress was needing in excess of 10GB to build, at least in our case. We now have it down to a little less than 1024 MB.

I think this is related to comments others have made about using packages that have large files in them. For us it was a little different. Here goes:

The problem was using import (static or dynamic) for large files into a Vue component. We are using FlexSearch to generate an index JSON file. This file is 25 MB and was getting imported in our Search.vue component. So when VitePresss builds, it pulls in the file and the build memory usage goes from 1 GB to 10 GB, for a 25 MB file. The solution is not to import the file but rather access it at runtime.

1. To do this we moved the file into the /public directory.
2. Then the component uses Axios to get the file the first time the component is activated via a user action. The file is pulled automatically by VitePress from the server when this happens, so lazy load.

   let map = await axios.get('/indexes/latest/map.json');
   

3. Then map is used to build the index.

All this takes a few milliseconds, it's very fast. The VitePress build will not even touch the Axios call and try to pull it in.

% echo $NODE_OPTIONS                           
--max-old-space-size=1024

Our builds were taking a few minutes, now around seven seconds.

vitepress v1.0.0-alpha.35
✓ building client + server bundles...
✓ rendering pages...
build complete in 5.83s.
✨  Done in 6.90s.
```

[pavelhoral]

Just stumbled on this issue. In my use case I am generating a lot of dynamic pages with JSON data.

https://github.com/prekladyher/pillars2-website/actions/runs/5824224099/job/15792772897

Not sure how to optimize that. I wanted to go for Vitepress to generate content statically, but it seems that is not the way to go :(