perf(build): skip unchanged links, write compile_commands.json, key the hot pieces by the profile's contents #2893

Open

github-sync wants to merge 1 commit from github/pr/2893 into main

github-sync commented

2026-11-12 19:01:04 +00:00

(Migrated from git.magicbean.dev)

@jordanella opened this pull request on GitHub · 2026-11-12 18:41 UTC · original

A build with nothing to compile still relinked the 440 MB executable and the renderer, a build from a second checkout of the same tree recompiled all eight hot pieces (~10-20 minutes) because their keys named the play profile by its path, and clangd had no compile database. Three small changes to nx2/build.py from the dev-loop proposal (#2636, A4 and A8); one PR because they touch the same lines, and I can split them if you prefer.

Change

nx2/build.py:

A4, no link when nothing changed: run_link skips a link whose inputs are the last successful one's. Its key is kept beside the output (<output>.linkkey) with the output's size:
- the command, the compiler and the work folder's system-headers stamp;
- each object's key (its .o.sha1; its size and time without one);
- the contents of the other files it names, inside options too (the hot symbol order, the libraries, the resources).
A failed link, or an output of another size or gone, links again. The executable's, the renderer's and the game DLL's links go through it.
A8, compile_commands.json: every compile as compile_one runs it, written to the build folder for clangd and clang-tidy (compile_commands.json at the checkout's top is git-ignored, for a copy there).
The hot pieces' keys name the profile by its contents: flag_text leaves -fprofile-instr-use=<path> out of the key, since -DHOOKSHOT_PROFILE_SHA1 beside it already says what the key needs of the profile. This changes their keys once.

Measured

Windows 11, Ryzen 9 3900X, a dev work folder:

A build with nothing to do: 16 s against 6 s, both links skipped.
One renderer edit: only the renderer's link runs.
A second checkout of the same tree building in the same work folder: the eight hot pieces from the cache, where main compiled them again.

In the game

Every build I ran in the game today used this build.py. The latest: Windows 11, RTX 3080 (NVIDIA 621.14), a dev-profile build, from a save, headless through the control bridge: in play in Kokiri Forest, 28-31 fps, nothing in the log about a lost device or a crash. A skipped link leaves the same binary in place.

Testing

tests/test_link_skip.py (new): a stand-in linker through the real run_link: the same inputs aren't linked again; a changed object, library or symbol order is; so is an output of another size or gone; a failed link forgets its key; the files inside options are inputs; the compile commands are the compiles; the build's links go through it.
tests/test_profile_path_key.py (new): two paths, one profile give one key; another profile, another key; every use of the profile's path names its contents too.
Every check that reads the changed files (37), through run_all.py, rebased on today's main, Windows: pass, apart from four that fail the same way on main here (test_driver_cache_folder, test_fpx_sticky, test_linux_update, nx2/test_dll.py). Rebasing found that main's build.py doesn't import re (#2706 adds it); this PR imports it too.
Not tested: Linux.

See #2636 (A4, A8).

🤖 Generated with Claude Code

> [@jordanella](https://github.com/jordanella) opened this pull request on GitHub · 2026-11-12 18:41 UTC · [original](https://github.com/NotRexed/Hookshot/pull/2893) A build with nothing to compile still relinked the 440 MB executable and the renderer, a build from a second checkout of the same tree recompiled all eight hot pieces (~10-20 minutes) because their keys named the play profile by its path, and clangd had no compile database. Three small changes to `nx2/build.py` from the dev-loop proposal (#2636, A4 and A8); one PR because they touch the same lines, and I can split them if you prefer. ## Change `nx2/build.py`: - **A4, no link when nothing changed:** `run_link` skips a link whose inputs are the last successful one's. Its key is kept beside the output (`<output>.linkkey`) with the output's size: - the command, the compiler and the work folder's system-headers stamp; - each object's key (its `.o.sha1`; its size and time without one); - the contents of the other files it names, inside options too (the hot symbol order, the libraries, the resources). A failed link, or an output of another size or gone, links again. The executable's, the renderer's and the game DLL's links go through it. - **A8, `compile_commands.json`:** every compile as `compile_one` runs it, written to the build folder for clangd and clang-tidy (`compile_commands.json` at the checkout's top is git-ignored, for a copy there). - **The hot pieces' keys name the profile by its contents:** `flag_text` leaves `-fprofile-instr-use=<path>` out of the key, since `-DHOOKSHOT_PROFILE_SHA1` beside it already says what the key needs of the profile. This changes their keys once. ## Measured Windows 11, Ryzen 9 3900X, a dev work folder: - **A build with nothing to do:** 16 s against 6 s, both links skipped. - **One renderer edit:** only the renderer's link runs. - **A second checkout of the same tree** building in the same work folder: the eight hot pieces from the cache, where `main` compiled them again. ## In the game Every build I ran in the game today used this `build.py`. The latest: Windows 11, RTX 3080 (NVIDIA 621.14), a dev-profile build, from a save, headless through the control bridge: in play in Kokiri Forest, 28-31 fps, nothing in the log about a lost device or a crash. A skipped link leaves the same binary in place. ## Testing - **`tests/test_link_skip.py` (new):** a stand-in linker through the real `run_link`: the same inputs aren't linked again; a changed object, library or symbol order is; so is an output of another size or gone; a failed link forgets its key; the files inside options are inputs; the compile commands are the compiles; the build's links go through it. - **`tests/test_profile_path_key.py` (new):** two paths, one profile give one key; another profile, another key; every use of the profile's path names its contents too. - **Every check that reads the changed files (37), through `run_all.py`, rebased on today's `main`, Windows:** pass, apart from four that fail the same way on `main` here (`test_driver_cache_folder`, `test_fpx_sticky`, `test_linux_update`, `nx2/test_dll.py`). Rebasing found that `main`'s `build.py` doesn't import `re` (#2706 adds it); this PR imports it too. - **Not tested:** Linux. See #2636 (A4, A8). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

notrexed referenced this pull request from a commit

2026-11-13 22:36:48 +00:00

perf(build): skip unchanged links, write compile_commands.json, key the hot pieces by the profile's contents (#2893)

This pull request can be merged automatically.

This branch is out-of-date with the base branch

The changes on this branch are already on the target branch. This will be an empty commit.

You are not authorized to merge this pull request.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin 'github/pr/2893:github/pr/2893'

git switch 'github/pr/2893'

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch 'main'

git merge --no-ff 'github/pr/2893'

git switch 'github/pr/2893'

git rebase 'main'

git switch 'main'

git merge --ff-only 'github/pr/2893'

git switch 'github/pr/2893'

git rebase 'main'

git switch 'main'

git merge --no-ff 'github/pr/2893'

git switch 'main'

git merge --squash 'github/pr/2893'