Add @rushstack/heft-zod-schema-plugin and pilot it on rush-lib's experiments.json#5788
Add @rushstack/heft-zod-schema-plugin and pilot it on rush-lib's experiments.json#5788
Conversation
…om zod validators Agent-Logs-Url: https://github.com/microsoft/rushstack/sessions/f6417dd3-99b5-4eb5-9343-ecf5de6c37c4 Co-authored-by: iclanton <5010588+iclanton@users.noreply.github.com>
Agent-Logs-Url: https://github.com/microsoft/rushstack/sessions/f6417dd3-99b5-4eb5-9343-ecf5de6c37c4 Co-authored-by: iclanton <5010588+iclanton@users.noreply.github.com>
Agent-Logs-Url: https://github.com/microsoft/rushstack/sessions/f6417dd3-99b5-4eb5-9343-ecf5de6c37c4 Co-authored-by: iclanton <5010588+iclanton@users.noreply.github.com>
Agent-Logs-Url: https://github.com/microsoft/rushstack/sessions/b7297c08-8212-4a0e-b5fa-f4d4ff39a8db Co-authored-by: iclanton <5010588+iclanton@users.noreply.github.com>
…c; fix shold→should typo Agent-Logs-Url: https://github.com/microsoft/rushstack/sessions/b7297c08-8212-4a0e-b5fa-f4d4ff39a8db Co-authored-by: iclanton <5010588+iclanton@users.noreply.github.com>
iclanton
left a comment
There was a problem hiding this comment.
@copilot - next steps:
- Make use of the other ported schemas in rush-lib
- Replace the .schema.json files in rush-lib's output with the ones from the schemas project
- Use zod to validate the loaded objects instead of
JsonSchema - Consider adding support for
zodschemas innode-core-library'sJsonFile.
| "name": "@reduxjs/toolkit", | ||
| "allowedCategories": [ "libraries", "vscode-extensions" ] | ||
| }, | ||
| { |
| // rush-mcp-server pins to zod 3 to remain compatible with @modelcontextprotocol/sdk; | ||
| // heft-zod-schema-plugin and the rush-lib pilot use zod 4 for its built-in | ||
| // z.toJSONSchema() API. | ||
| "~3.25.76" |
There was a problem hiding this comment.
@copilot - can we upgrade @modelcontextprotocol/sdk to pull in v4 of zod in the project that currently needs v3?
There was a problem hiding this comment.
Latest @modelcontextprotocol/sdk (1.25+/1.29) does support zod ^3.25 || ^4.0, so in principle yes. I tried the upgrade locally and it surfaces two unrelated breaks that need real source changes in rush-mcp-server:
apps/rush-mcp-server/src/tools/base.tool.tsusesz.objectOutputType<...>— that's a zod-3 internal type that was removed in zod 4; theBaseToolgeneric plumbing needs to be re-expressed.- The newer SDK pulls in a transitive
transport.d.tsthat references the DOMHeadersInitglobal, which doesn't resolve under the current@types/node/libsettings.
Neither is conceptually hard, but both are out-of-scope for this PR (which is the schemas package pivot) and want their own validation. Reverted the speculative upgrade in this branch and left a follow-up task in the PR description so the zod-3 alternative version stays scoped to rush-mcp-server for now.
| it('emits a JSON schema for a basic zod default export', async () => { | ||
| const generator: ZodSchemaGenerator = new ZodSchemaGenerator({ | ||
| buildFolderPath: projectFolder, | ||
| inputGlobs: [path.relative(projectFolder, path.join(compiledFixturesFolder, 'basic.zod.js'))], |
There was a problem hiding this comment.
@copilot - Seems like inputGlobs should be able to accept absolute paths. Also, just use template strings with /s.
Same in other places in this file.
There was a problem hiding this comment.
Switched the test to template strings with / and absolute paths in 60d69b0. Also hardened ZodSchemaGenerator itself: inputGlobs are now run through Path.convertToSlashes() (so absolute/Windows-style paths reach fast-glob in its required form), and outputFolder is resolved with path.resolve() instead of path.join() so an absolute value works as expected.
| "extends": "./node_modules/local-node-rig/profiles/default/tsconfig-base.json", | ||
|
|
||
| "compilerOptions": { | ||
| "target": "ES2019" |
There was a problem hiding this comment.
No good reason — leftover from copying another project's tsconfig. The rig already sets a sensible target. Removed the override in 60d69b0.
| "projectFolder": "libraries/rush-schemas", | ||
| "reviewCategory": "libraries", | ||
| "shouldPublish": true | ||
| }, |
There was a problem hiding this comment.
This should be in the Rush version policy.
There was a problem hiding this comment.
Switched to "versionPolicyName": "rush" in 60d69b0.
| */ | ||
| export type IRepoStateJson = Omit<z.infer<typeof repoStateSchema>, '$schema'>; | ||
|
|
||
| export default repoStateSchema; |
There was a problem hiding this comment.
Removed the IRepoStateJson type alias (and its index.ts re-export) in 60d69b0. rush-lib's RepoStateFile.ts already keeps its own internal interface, and nothing else consumes it.
| "pnpm-sync-lib": "0.3.3", | ||
| "read-package-tree": "~5.1.5", | ||
| "rxjs": "~6.6.7", | ||
| "@rushstack/rush-schemas": "workspace:*", |
| // See LICENSE in the project root for license information. | ||
|
|
||
| import { JsonFile, JsonSchema, FileSystem } from '@rushstack/node-core-library'; | ||
| import type { IExperimentsJson } from '@rushstack/rush-schemas/lib/experiments.zod'; |
There was a problem hiding this comment.
@copilot - import this from the entrypoint of the package.
There was a problem hiding this comment.
Switched to import type { IExperimentsJson } from '@rushstack/rush-schemas' in 60d69b0. The API report now records the entrypoint instead of the subpath. While verifying this I also surfaced that rush-sdk's bundled dist/rush-lib.d.ts re-exports the type but didn't list @rushstack/rush-schemas as a dep — added it in 08e3937 so downstream consumers of rush-sdk resolve cleanly.
…oStateJson, package entrypoint imports Agent-Logs-Url: https://github.com/microsoft/rushstack/sessions/83bb9e28-6062-4265-851a-ce82c95398ec Co-authored-by: iclanton <5010588+iclanton@users.noreply.github.com>
…ntsJson re-export resolves Agent-Logs-Url: https://github.com/microsoft/rushstack/sessions/83bb9e28-6062-4265-851a-ce82c95398ec Co-authored-by: iclanton <5010588+iclanton@users.noreply.github.com>
Acknowledged — these four are clearly the right next steps but each is a meaningful, independently-validatable change, so I'd like to take them as a follow-up PR (or a small series) rather than fold them into this one. Quick sketch of how I'd approach each:
Happy to open the first one of these when you're ready. |
| "phasesByName": { | ||
| "build": { | ||
| "tasksByName": { | ||
| "copy-json-schemas": { |
There was a problem hiding this comment.
It's the standard pattern every Heft plugin in this repo uses (e.g. heft-plugins/heft-lint-plugin/config/heft.json, heft-plugins/heft-typescript-plugin/config/heft.json, etc.) — the task hardlinks the plugin's optionsSchema (declared in heft-plugin.json) into temp/json-schemas/heft/v1/, and repo-toolbox collect-project-files --subfolder temp/json-schemas then aggregates those across all projects so the CI's "Collect JSON Schemas" step can publish them as a build artifact. See common/config/azure-pipelines/templates/post-publish.yaml and .github/workflows/ci.yml. Happy to remove it if we don't want this plugin's option schema published alongside the other Heft plugins, but it's there for parity with the rest of the Heft plugin family.
…as; add JsonFile.loadAndParse to node-core-library Agent-Logs-Url: https://github.com/microsoft/rushstack/sessions/8313d468-a395-4e21-9308-46ee2c100ee0 Co-authored-by: iclanton <5010588+iclanton@users.noreply.github.com>
First-draft of all four items pushed in 88fc2b6. Treating these as a starting point for design feedback rather than a finished form:
Validated:
Open design questions worth your eyes before treating this as more than a first draft:
|
| * | ||
| * @beta | ||
| */ | ||
| export interface IExperimentsJson { |
There was a problem hiding this comment.
@copilot - Why are we defining this here instead of getting it from experimentsSchema?
There was a problem hiding this comment.
Fair — switched in 429889d. IExperimentsJson is now Omit<z.infer<typeof experimentsSchema>, '$schema'> (same pattern as ICobuildJson), the drift-check assertion is gone, and the as IExperimentsJson cast in ExperimentsConfiguration.ts dropped out cleanly. Per-property docs now live on the schema's .describe() annotations as the single source of truth. Updated README.md and the now-obsolete cross-reference in cobuild.zod.ts to match.
Tradeoff worth flagging: TSDoc on individual experiment properties no longer renders in IDE tooltips for IExperimentsJson — only the schema's .describe() text is preserved (and that flows into the emitted experiments.schema.json). The interface-level TSDoc still shows. Tests: rush-lib 627/627 ✅.
…a via z.infer (drop hand-authored interface + drift check) Agent-Logs-Url: https://github.com/microsoft/rushstack/sessions/df3800f0-fee3-4aeb-a16a-04079be1662c Co-authored-by: iclanton <5010588+iclanton@users.noreply.github.com>
Address review feedback on the prior pivot to
@rushstack/rush-schemas, then take a first-draft pass at the four "next steps" follow-ups (use the other ported schemas in rush-lib, drop the duplicate.schema.jsonartifacts, validate via zod instead ofJsonSchema, and add zod support toJsonFile).Round 1 — review feedback
@rushstack/heft-zod-schema-pluginand@rushstack/rush-schemasfrombrowser-approved-packages.jsontononbrowser-approved-packages.jsonCHANGELOG.json/CHANGELOG.mdin both new packages (autogenerated at publish)target: "ES2019"override inlibraries/rush-schemas/tsconfig.json@rushstack/rush-schemasunder therushversion policy inrush.jsonIRepoStateJsontype alias fromrepo-state.zod.ts(rush-lib has its own internal interface)@rushstack/rush-schemasto its alphabetical position inlibraries/rush-lib/package.json's dependenciesIExperimentsJsonfrom the package entrypoint (@rushstack/rush-schemas)/s and pass absolute paths inZodSchemaGenerator.test.ts; harden the generator to accept absolute paths in bothinputGlobsandoutputFolder@rushstack/rush-schemasas a dependency of rush-sdk so its bundleddist/rush-lib.d.ts(which re-exportsIExperimentsJson) resolves for downstream consumersRound 2 — first-draft of the 4 next-steps items
node-core-libraryzod support onJsonFile— added a structuralIJsonFileTypeValidator<T>interface (any object withparse(input: unknown): T, which is the contract zod 3, zod 4, and most other validation libraries already match) and newJsonFile.loadAndParse/loadAndParseAsynchelpers. No runtime dependency on a specific zod major version.ExperimentsConfiguration,CobuildConfiguration, andRepoStateFilenow validate the loaded JSON againstexperimentsSchema/cobuildSchema/repoStateSchemafrom@rushstack/rush-schemas(replacing the legacyJsonSchema.fromLoadedObject(...)+JsonFile.loadAndValidate(...)plumbing).ICobuildJsonis now re-exported from@rushstack/rush-schemasinstead of being duplicated as a separate@betainterface in rush-lib..schema.jsonfiles in rush-lib's output — removed the now-unusedexperiments.schema.json,cobuild.schema.json, andrepo-state.schema.jsonfromlibraries/rush-lib/src/schemas/. They were only consumed via TS JSON imports inside the three migrated configuration classes; with the runtime path on zod, the bundled.schema.jsonartifact in rush-lib's output is no longer needed for these three.build-cachedeferred — intentionally left on the legacy path. As called out inlibraries/rush-schemas/README.md, the JSON Schema's discriminatedoneOfshape and the runtimeIBuildCacheJsonextension-point shape don't match structurally, and reconciling them needs a focused design discussion (drop the open[key: string]: JsonObjectindex? hand-author the runtime interface? add a relaxed runtime variant of the schema?).Round 3 — review feedback follow-ups
IExperimentsJsonfromexperimentsSchemaviaOmit<z.infer<typeof experimentsSchema>, '$schema'>instead of hand-authoring the interface and using a bidirectional drift-check assertion. Per-property documentation now lives on the schema's.describe()annotations as the single source of truth, and theas IExperimentsJsoncast inExperimentsConfiguration.tsdrops out cleanly. Updatedcobuild.zod.tscross-reference andlibraries/rush-schemas/README.mdto remove the now-obsolete reference to the hand-authored-interface pattern (the table collapses to the singlez.infer-based derivation, withbuild-cachecalled out as the structural-mismatch escape hatch).Validation
rush build --to @microsoft/rush-lib✅,--to @rushstack/rush-sdk✅,--to @rushstack/mcp-server✅,--to @rushstack/node-core-library✅,--to @rushstack/heft-zod-schema-plugin✅rush-libtests: 627/627 ✅node-core-librarytests: 267/267 ✅heft-zod-schema-plugintests: 9/9 ✅ICobuildJsonbecoming a re-export from@rushstack/rush-schemasandIExperimentsJsonswitching from a type-only to a value+type re-export. Interface shapes are unchanged for downstream consumers.Notes / known limitations of this first draft
z.infer-derived types, per-property TSDoc onIExperimentsJsonno longer renders in IDE tooltips; only the schema's.describe()text is preserved (and that flows into the emittedexperiments.schema.json). Interface-level TSDoc is still present.node-core-librarydoes not get a unit test specifically forloadAndParsein this commit — it's a thin wrapper aroundJsonFile.load+validator.parse, and is exercised end-to-end by the rush-lib test suite. Easy to add a focused unit test if preferred.libraries/rush-lib/src/schemas/(rush.schema.json, common-versions.schema.json, etc.) are still on the legacyJsonSchema-based path. The pattern established here (schemaName.zod.tsin@rushstack/rush-schemas+JsonFile.loadAndParse(...)in rush-lib + delete the correspondingsrc/schemas/*.schema.json) is now repeatable per-schema.