Which parts of a job should AI do?
It depends on what a mistake costs and who has to stand behind the result. Give AI the work that repeats, that has a clear standard, and that a person can check in minutes: first drafts, scoring against a fixed rubric, answers from a known set of facts. Keep with people the work where judgement is the product: reading a room, a decision someone will be held to, the advice a client pays for. If you cannot say who checks the AI's work, it is not ready to do it.
An example: an assessment centre for 271 candidates
Our client, an education assessment platform, runs developmental assessment centres for students and early-career candidates. Its old model put close to one assessor beside each candidate. The feedback was excellent. It also meant a cohort could only be as big as the number of assessors in the room.
When we scoped the platform together, the first question was not which model to use. It was which exercises a machine may score. We wrote the answer down before the build:
- AI scores the online situational judgement test (6 scenarios) and the structured online interview (9 questions, recorded as audio).
- AI drafts each candidate's feedback report, and every claim in it is traceable to the evidence it came from.
- Trained assessors score the live group exercise and the presentation. Always. The system has no way to score them.
- An admin can open and edit any report before it goes out.
The first production cohort ran on 11 June 2026: 271 candidates and 26 pairs of assessors, part online and part in person. The target, set before the day, was a report for every candidate within 5 working days. See what we build.
Why the line comes first
Build first and draw the line later, and the line ends up wherever the technology happened to be good. That is the wrong way round. The people who use the system need to know what they answer for on day one. The people it serves need to know when a person was involved.
Writing the line down early also makes the build smaller. Once the assessors owned the live exercises, we had no reason to build anything that scored them. Those weeks went into the reports instead.
Four questions for the scope workshop
- What does a mistake cost here, and who pays for it?
- Is there a standard to check the output against, such as a rubric, a policy or a set of facts?
- Who checks the AI's work, and how long does that check take?
- Does the person on the other end expect a human? Guests, candidates and clients have a view on this, and their view counts.
If the answers are clear, the AI side of the line is the easy part to build. If they are not, no model will fix that.
The line moves, and that is fine
We do not treat the first line as permanent. After a cohort or an event, the evidence shows where the AI was reliable and where people kept correcting it. Move the line with that evidence, one step at a time, and write each move down. What you must not do is let it drift because nobody was watching.
The companion app we built for a luxury hotel brand's event drew a different line. The AI answered questions about the agenda at any hour and wrote each guest's digest from their own notes. The brand set the agenda, the prompts and the tone of voice. Same principle, different job. See what we build.
If you are still choosing what to build, start with one real job. To talk it through with us, tell us what you have in mind.
