You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Builds take too long -- can we use bazel cache? #30
A clean build takes a very long time, largely due to the need to build V8 and other dependencies. #8 introduces GitHub Actions for CI and those builds are now at 2 hours and counting. On my local 32-core EPYC I can build in 10 minutes.
Incremental builds are comparatively very fast (as long as you didn't modify V8).
We need to do something about this. Probably, the answer is "something something bazel cache something", but I don't know enough about bazel cache at the moment to say exactly how this should work.
Bazel's remote caching looks like it implements quite nicely with GCS. I would have quite liked to have a try at using R2 as the caching backend, but for ease of use GCS might be the best choice.
There's also the cache GitHub action - I assume it'll be the 'fastest' choice since you'll be fetching GitHub from a GitHub runner but I haven't checked if the build output will fit within their limits.
GitHub will remove any cache entries that have not been accessed in over 7 days. There is no limit on the number of caches you can store, but the total size of all caches in a repository is limited to 10 GB.
If you exceed the limit, GitHub will save the new cache but will begin evicting caches until the total size is less than the repository limit.
It looks like one of our builds is 7GB or so, which means that between the Linux and MacOS builds we'd exceed the 10GB GitHub cache limit. The docs don't seem to make clear if there's any way to, like, pay for more space. Remote caching may be the way to go.
Still working on this, but initial results with a worker proxying to R2 as a bazel remote cache are looking promising. The Linux build is down to ~1:30, and can hopefully be optimised further.
Activity
KianNH commentedon Sep 28, 2022
Bazel's remote caching looks like it implements quite nicely with GCS. I would have quite liked to have a try at using R2 as the caching backend, but for ease of use GCS might be the best choice.
There's also the cache GitHub action - I assume it'll be the 'fastest' choice since you'll be fetching GitHub from a GitHub runner but I haven't checked if the build output will fit within their limits.
kentonv commentedon Sep 28, 2022
It looks like one of our builds is 7GB or so, which means that between the Linux and MacOS builds we'd exceed the 10GB GitHub cache limit. The docs don't seem to make clear if there's any way to, like, pay for more space. Remote caching may be the way to go.
@penalosa offered to dig into this.
penalosa commentedon Sep 29, 2022
Still working on this, but initial results with a worker proxying to R2 as a bazel remote cache are looking promising. The Linux build is down to ~1:30, and can hopefully be optimised further.
fix(server): facet channel UaF after alarm timeout
fix(server): facet channel UaF after alarm timeout
Transfer byte-stream buffers to fixed length
Give every TS tee branch the same chunk when size() closes the stream