Repository navigation
Add TypeScript 7 external API benchmarks - #158
Merged
Merged
Conversation
Remap scenario and host indices independently when filtering a report. Serialize scenarioIndex under the correct key while accepting existing scenarioArgs data. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 958b45cd-e40a-46fc-9de9-3bfde150ace3
Andrew Branch (andrewbranch)
force-pushed
the
api-perf-runner
branch
from
October 19, 2026 16:35
30c7001 to
be9248f
Compare
Run declared Node scripts from prepared solution directories using existing argument expansion and measurement reporting. Require schema-v1 client and server timing dumps and record their scalar counters alongside whole-command wall time. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 958b45cd-e40a-46fc-9de9-3bfde150ace3
Build and retain the TypeScript API and native platform packages for API jobs. Add benchmark-local unstable export aliases and verify package gitHead and executable resolution without a separate artifact manifest. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 958b45cd-e40a-46fc-9de9-3bfde150ace3
Pin native typescript-eslint floating/recommended workloads and the vscode-languageserver metamodel generator. Prepare them with existing sandboxed setup scripts, inject the built API package, and check upstream command output. Wire the api preset to the existing benchmark pipeline with a pinned Node host and document adding scenarios. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 958b45cd-e40a-46fc-9de9-3bfde150ace3
Resolve workspace packages to source in Vitest so tests do not require a preceding build. Invoke the ESLint fixture generator through sh -c, matching the sandbox helper's non-root execution path. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 958b45cd-e40a-46fc-9de9-3bfde150ace3
Andrew Branch (andrewbranch)
force-pushed
the
api-perf-runner
branch
from
October 19, 2026 16:47
be9248f to
8d6f2c4
Compare
Andrew Branch (andrewbranch)
marked this pull request as ready for review
October 19, 2026 17:01
Andrew Branch (andrewbranch)
requested a review
from Jake Bailey (jakebailey)
as a code owner
October 19, 2026 17:01
Comment on lines
+6
to
+10
| resolve: { | ||
| alias: { | ||
| "@ts-perf/core": fileURLToPath(new URL("../core/src/index.ts", import.meta.url)), | ||
| }, | ||
| }, |
There was a problem hiding this comment.
How did this end up being needed?
Member
Author
There was a problem hiding this comment.
This is an alternative to setting up package.json exports with some kind of source condition, which I opted not to do because no packages currently define exports at all, but I can switch to that if you like
There was a problem hiding this comment.
Nah, it's fine, this is really not the codebase to be picky in
| import { configDefaults, defineConfig } from "vitest/config"; | ||
|
|
||
| export default defineConfig({ | ||
| resolve: { |
Jake Bailey (jakebailey)
approved these changes
Oct 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds API benchmarking, reading and summing client metrics exported to disk via microsoft/TypeScript#64846
Initial scenarios are
Copilot description
Summary
Add an opt-in
apipreset to the existing benchmark pipeline, with whole-command timing and scalar client/server API timing counters. Scenarios use the existingsetup.sh, sandbox, argument expansion, and report conventions.The initial corpus contains native typescript-eslint floating-promises and recommended-rule workloads (1,024 files each), plus vscode-languageserver-node's metamodel generator. Setup pins sources and dependencies; execution injects the matching built JS API/native package and checks the upstream command's output.
Commits
Validation
pnpm build; 124 tests across 10 files.typescript:build-mainandtypescript:build-platform, then packaged and ran all three consumers. Both lint workloads checked their expected results; the generated metamodel matched all 545 checked-in definitions. Saved measurements and comparison reports use the normal report path.Both compared compiler commits must include timing-dump support from microsoft/TypeScript#64846. A comparison of its first supporting commit against its parent therefore fails explicitly for missing timing data.