Spec Checks
Let DevStride's AI write an As Built for each finished spec-linked work item, compare it with the spec, check a spec's progress at any time, track every difference in one report until someone decides on it, and tell the spec's people, including the clients who wrote or approved it. Off until an admin switches it on.
On this page
What spec checks do#
A spec says what to build; a linked work item tracks the building. Spec checks close the loop once the work is done. When a work item linked to a spec finishes, DevStride's AI:
- reads what was actually built: the item's description, comments and history, and the code changes on its linked GitHub pull requests;
- writes an As Built for the item, a plain account of what was delivered, kept with every earlier version;
- compares that with the part of each linked spec the item covers, and records each meaningful difference;
- tells the spec's people, so someone decides whether the work or the spec is right.
When the last work item linked to a spec finishes, a whole-spec check also looks for requirements that nothing delivered. You can also check a spec's progress at any point in the build, before its work is finished.
The As Built and the detail of each difference are for your team only. A client who wrote or approved the spec is emailed that differences were found, and reads them on the spec's page in the portal, in wording written for clients. See What your clients see.
Switching spec checks on#
Spec checks are off in every workspace until an admin switches them on.
- Open Configure Settings → Knowledge Base → Specs.
- Switch on Check finished work against its spec.
- Under Model, choose the AI model the checks use.
Read the note under the switch before you turn it on. With checks on, each time a spec-linked item is finished, DevStride sends the linked spec's text, the item's description, comments and history, and the code changes on its linked GitHub pull requests to the chosen model's provider. Switching checks on is enough to let DevStride read the code on linked pull requests, even when the AI Assistant's GitHub context switch is off.
Every check counts toward your workspace's monthly AI budget. Paid models are listed once they are allowed under Settings > AI Assistants. If the model chosen earlier is no longer available, the page says so, and checks do not run until you choose another.
With checks on, your clients are told too. The clients who wrote or approved a spec are emailed when a check finds new differences on it, and anyone who can read the spec in the portal sees its differences there, in wording written for clients. See What your clients see.
You need Manage Knowledge Base Settings to open this page. See Who can do what.
Only work finished after checks are on is checked
A check runs when an item finishes. Items that finished before you switched checks on, or before they were linked to a spec, are not checked afterwards.
The As Built on a work item#
A work item linked to a spec has an As Built button beside Description above its description. Click it to see what was built; click Description to go back.
- Until the item is done, it says Written automatically when this item is done.
- While the AI is working, it says DevStride AI is writing the As Built. It appears by itself when it is ready.
- Each As Built says when it was written and what it was written from: from pull requests, or from comments and history only when the item has no readable pull request.
- When not everything could be read (a very long code change, say), it says Not everything could be read when this was written, so it may be incomplete.
Every version is kept. If an item is reopened, worked on and finished again, a new As Built is written and the earlier ones stay. Finishing it again with nothing new to read (no new pull request, comment or change) does not run another check. Pick one from the version list (Version 1 of 2), and click Compare with previous to see what changed between that version and the one before it.
If no As Built was written, the pane says why: for example, checks were switched off when the item finished, the monthly AI budget was reached, or the chosen model was no longer allowed. When the check did not run, could not be finished or was interrupted, a person with Create knowledge-base articles and edit access to a linked spec sees a Run again button, as long as spec checks are switched on. Deal with the cause first (for example, choose a model that is still allowed), then click it; the pane follows the item until the new As Built is written.
The comment on the item#
When what was built differs materially from the plan, or the compare found differences from a linked spec, DevStride leaves one internal comment on the item. It lists how the result differs from the plan and says how many spec differences were found, with a pointer to review them. It gives only the number of spec differences, never their words, because anyone who can read the item sees the comment and some of them may not be allowed to read the spec.
Checking progress at any time#
You do not have to wait for the work to finish. On a spec that has at least one linked work item, the Spec checks section of the editor's right-hand column has a Progress area with a Check progress now button. Click it and DevStride's AI reads the spec and what its linked work has delivered so far, then shows:
- a plain-language summary of where the build stands against the spec, and
- a score from 0 to 100: On spec (90 and above), Minor departures (70 to 89) or Major departures (below 70).
While it works, the area says Checking.... The latest result stays on the spec with its date, so anyone who opens the spec sees it.
A progress check only reports. It records no differences, writes no As Built, and tells nobody, because work that is not finished yet cannot be told apart from work that was dropped. Differences are still recorded when each item finishes, as described above.
If the check could not run, it says Not checked. and why (for example, the monthly AI budget was reached, or the chosen model was no longer allowed), or The check could not be finished. Deal with the cause, then click Run again. When the spec is very long, or has a great many linked items, the summary says which part was not read.
- With spec checks switched off, the area says Spec checks are switched off for this workspace. An admin with Manage Knowledge Base Settings also sees a link to the Specs settings.
- With no linked work items, it says Link a work item first.
Each progress check counts toward your workspace's monthly AI budget, so a person starts it from the spec; agents and the API can read a check's result but cannot start one.
Differences from the spec#
Each difference a check finds is recorded with:
- its kind: Left out, Control left out, Endpoint left out, Works differently, Not in the spec (something built that the spec never mentions), or Not delivered (from the whole-spec check);
- a short subject and an explanation;
- what the spec says: the passage it is about, quoted when it was found word for word in the spec. When it was not, it is shown as The spec says, roughly: followed by the AI's reading, with the note This passage was not found word for word in the spec.;
- any reason found in the work, when the item's comments or history explain why it was built that way;
- the date it was found.
Where differences show#
- On the work item, under Differences from the spec in its As Built pane. Each one links to its spec.
- On the spec, in the Spec checks section of the editor's right-hand column. Each one links to its work item. The same section lists every check run on the spec, with its date, a short summary and a score (85% as specified). A check that did not run, or only partly ran, says why. See all differences opens the differences report for that spec.
- In the differences report, which lists the differences across every spec. See The differences report.
- In an items table, the Spec Sync column shows Spec differences with the number still open, beside the as-built sync state.
Open differences are listed first. Resolved ones are folded away under a button that shows them.
The differences report#
The differences report lists every difference across your specs in one place, so you can work through them by project or for the whole organization. Open the Knowledge Base, go to the Specifications tab and click Differences, or click See all differences on a spec.
- Status: Open (the default), Accepted, Spec updated or All.
- Type: one kind of difference, or Any type.
- Project: pick a workstream or work item to see only the differences about work at or under it, including whole-spec differences on a spec with a linked item there. Clear it to see Any project.
- Spec: opening the report from a spec narrows it to that spec; clear the chip to see every spec.
Your filters are kept in the page address, so a copied link opens the same view. Each row links to its spec and its work item, and you can accept it or mark the spec updated from the row, just as on the spec. The report shows 50 rows at a time; click Load more for the next.
You see only differences on specs you can access. A difference about a work item you cannot read is left out, and the report says how many were hidden.
Deciding on a difference#
A difference stays Open until a person decides on it. If you can edit the spec, each open difference offers two buttons:
- Accept: the work is right as built and the spec does not need to change. You must answer Why is this difference acceptable?; your reason is kept with your name and the date.
- Spec updated: you have changed the spec to match what was built.
If you cannot edit the spec, the difference says An editor of this spec can accept it or mark the spec updated. A decision made on the item shows on the spec straight away, and the other way round. A client reading the spec in the portal sees the difference move to Resolved, labelled Accepted by the team or Specification updated, and never sees your reason.
A later check can also close a difference. If the item is finished again and the new check no longer finds it, it is marked No longer found, and it is no longer listed in the portal.
The whole-spec check#
When the last work item linked to a spec finishes, DevStride checks the spec as a whole against everything that was delivered. It catches requirements that no item delivered, and records each as Not delivered. A Cancelled item counts as finished for this purpose, but as not delivered: what it was meant to build is checked as missing.
Who is told#
When a check finds new differences on a spec, DevStride sends one in-app notification per spec: Finished work differs from a specification: open the spec to review the differences. It goes to:
- the spec's author,
- the people who approved or reviewed a version of it, and
- everyone watching the spec or its category.
Only people who can read the spec are told. A check that finds nothing new, or only finds again what was already open, tells nobody. Clicking the notification opens the spec.
The email your clients get#
When a client wrote the spec or approved it, each of those clients is emailed Differences found in "spec title". The email names your organization and the spec, and its View differences button links to the spec in their portal. It says nothing about what the differences are, how many there are, or which work items they came from.
- Each client gets one email for a check that finds new differences on the spec. A check that finds nothing new emails nobody.
- A client is emailed only if they are an active contact, the portal would let them sign in under their company's access mode, and their company can read the spec there.
- No email is sent while the help centre or the customer portal is closed, or while the organization is in headless mode.
What your clients see#
A client who can already read a spec in the portal sees its differences on the spec's page, in a Differences from this spec panel below the spec. The panel appears only when the spec has differences to show.
- Differences are split into Open and Resolved.
- Each one shows its kind (Functionality left out, Control left out, Endpoint left out, Behaviour changed, Added beyond the specification or Not delivered) and one sentence written for clients.
- A resolved difference is labelled Accepted by the team or Specification updated. The reason you gave is never shown, and a difference marked No longer found is not listed.
That client wording is all a client reads. They never see the As Built, the explanation your team reads, the quoted spec passage, the reason found in the work, who decided on a difference, work item numbers, code or file paths. If the sentence the AI wrote for clients contains anything that looks like code, a link, a file name or an item number, DevStride shows a plain sentence for that kind instead, such as Something works differently from how the specification describes it.
The item comment is internal, and the in-app notifications go to your team only. Clients are told by email, and only the clients who wrote or approved the spec. While the help centre or the customer portal is closed, clients cannot open the spec, so they see no differences.
Spec checks never change the spec's text, so the spec itself reads exactly as it did before. To change the spec itself, edit it, or record an as-built change, which does write into the spec.
Spec checks and as-built changes are different things
An as-built change is something a person records by hand: it replaces the item's description and writes a note into every linked spec, which clients can read. A spec check is DevStride's AI reading the finished work: it writes the As Built beside the description, never into the spec. Your team sees everything it finds; clients see each difference only in client wording, on the spec's page in the portal.
Who can do what#
| To | You need |
|---|---|
| Switch spec checks on or off, and choose the model | Manage Knowledge Base Settings |
| See a work item's As Built | Permission to read the work item |
| See a difference | Permission to read the work item and access to the spec. Differences on specs you cannot see are counted, not shown |
| See a spec's Spec checks section | Access to the spec. A check about a work item you cannot read shows only its date |
| Check a spec's progress, or run a check again | Create knowledge-base articles and edit access to the spec |
| Open the differences report | Any Knowledge Base permission. Rows need access to their spec; rows about work items you cannot read are only counted |
| Accept a difference or mark the spec updated | Edit access to the spec |
| Read a spec's differences in the portal, in client wording | A portal sign-in from a company that can read the spec |
See Knowledge Base permissions.
Related#
- Specs and work items: link a spec to its work items, and record as-built changes by hand.
- Watching articles and categories: get told when a spec you care about changes.
- Specs in the customer portal: where your clients read a spec and its differences.
- Knowledge Base permissions: who can write, publish and set up the Knowledge Base.