A first-year programming course. Three sections, 120 students, fourteen teaching weeks, one lab test a week plus a mid-term and an end-term.
That is 16 assessments and 1,920 submissions, from one course, in one term.
An earlier post looked at what a hiring rubric does to a single submission - the student who passes seven of eight test cases and is recorded as a zero. This one is about the other 1,919. The scoring problem is real but it is fixable with a setting. What follows is not, because it is a difference in what the tool is shaped like.
A campaign ends. A roster does not.
HackerRank and CoderPad are organised around a hiring campaign, and a campaign has a shape: you open a role, you invite a shortlist, each candidate produces one decision, and then the campaign closes. Every part of the product is built for that arc. The invitation is the unit of work, because inviting someone is the moment a stranger becomes a candidate.
A course has no strangers in it. The same 120 people are there in week one and week fourteen. Nobody needs to be invited; they need to be enrolled once and then present for sixteen assessments. Those are different primitives, and everything downstream inherits the difference.
| Hiring campaign | One course, one term | |
|---|---|---|
| Who takes part | A shortlist, different every time | The same 120, sixteen times |
| Unit of work | An invitation | A roster, set up once |
| Identity | An email address | A roll number |
| What it produces | One decision per candidate | 1,920 data points about 120 people |
| When it ends | When the role is filled | It does not |
| The useful output | A shortlist | A trend |
One first-year course: 3 sections, 120 students, 14 weeks, 16 assessments. Multiply by the number of courses a department runs.
Identity is the first thing to break
A hiring platform keys a candidate on an email address, and it is right to. There is nothing else it could use - a stranger has no employee number, and the address is how the offer gets sent.
An institution already has a better key and has had it since admission. Ananya is 21CS047. She is 21CS047 on the attendance register, on the internal assessment sheet, on the board result and in the conversation her parents will have in March.
Run her term on an email address and three things go wrong, all quietly:
- She uses a personal address for the first five weeks and the college one after that, and becomes two candidates with two partial histories, neither of which is her record.
- A typo in one bulk upload creates a third.
- Nothing in the term's data can be joined to anything else the institution holds about her, because the join key is an address the rest of the institution does not use.
None of that shows up as an error. It shows up in March as a student whose history looks thinner than it was.
What a term actually leaves behind
Sixteen assessments produce sixteen result sets. On a platform built around campaigns, that is sixteen exports, each one a snapshot of a test rather than a page in a record.
| Question somebody actually asks | Files to open | Why |
|---|---|---|
| How did the class do on Tuesday's test? | 1 | This is the question the tool was built for |
| How is Ananya doing this term? | 16 | Her history is one row in each of sixteen files |
| Is section B behind section A? | 16 | Sections are not a concept, so it is a manual tag on 1,920 rows |
| Is the class weak on recursion specifically? | 16 | Only if somebody recorded which question tested what |
| Which five students should I chase first? | 16 | There is no ranking across assessments to sort |
The first row is fine. Every row below it is a spreadsheet exercise that a lecturer does once in week four, resents in week seven, and stops doing in week nine. After that the term still has data and nobody is reading it, which is the same outcome as not collecting it.
Exam day is a room, not a window
A hiring assessment is taken alone, at a time the candidate chooses, inside a window of several days. That is deliberate and correct - a candidate has a job already, and pinning them to a Tuesday would cost you good applicants.
A lab test is forty students in one room at 11:00 for fifty minutes. The operational requirements are not the same ones:
- A start time, not a window. Everyone begins together or the first section tells the second what is on the paper.
- Resume after a disconnect. Somebody's laptop will drop at minute twelve. On a hiring platform that is a failed attempt and a support email; in a lab it is a student who needs to be back in their own session in ninety seconds with their code intact.
- One attempt, decided in advance. Retakes are an academic policy question, and the policy has to be enforced by the tool rather than by watching.
- A paper that closes. The room empties at 11:50 whether the submissions are in or not.
CoderPad is the sharper case here, because it is not really an exam tool at all. It is a shared editor for a live interview - one interviewer watching one candidate think. That is a genuinely good thing to have, and a viva is close enough to it that some departments use it that way. But it does not scale to forty simultaneous sessions with one invigilator, and it is not trying to.
A computer science paper is not only code
The last structural mismatch is the quietest. A first-year paper is rarely all programming. It is six multiple-choice questions on complexity, two written questions asking the student to explain an approach, and two programming questions.
Split that across a coding platform and whatever the institution uses for the rest, and you have not split a paper - you have split the record. Two mark sheets, two totals, two exports, and a per-student result that exists nowhere until somebody adds them up. The internal assessment mark that eventually goes to the board is then a hand-calculated number with no working behind it, which is the exact situation the platform was bought to end.
Six questions to ask, if the use is a course
- Can students be enrolled once, by roll number, and stay enrolled? If the answer involves inviting them per test, multiply that by sixteen.
- Is there a concept of a class or a section? Without one, comparing two sections is a manual tagging job on every export.
- Does one student have one page across the whole term? Sixteen result screens are not a record.
- Can a test have a fixed start time, and can a student resume after a disconnect? Both, not either.
- Can multiple-choice, written and coding questions sit on the same paper and produce one total? If not, the record is split before it is written.
- Do marks entered from a paper test land in the same record? Most courses still assess on paper at least once a term, and a record with a hole in it is not a record.
A hiring platform answers the first two with a workaround and has no reason to answer the rest. That is not a defect. Those are not its questions.
Use each for what it is
If the task is screening two hundred applicants for one role, a purpose-built hiring platform will do it better than any assessment system meant for a classroom - and the campaign model, the email key and the one-decision output that all fit a course so badly are exactly right when the output is a shortlist. If the task is a live technical interview, a shared editor is the correct tool and always will be.
The mistake is only in the substitution: reaching for a hiring tool because it also runs code, and finding in week nine that the term produced 1,920 submissions and no answer to any question worth asking.
Teemly enrols students once against a class and a roll number, so a term is one record per student rather than sixteen exports. Multiple-choice, written and coding questions sit on the same paper and produce one total, and marks from a paper test enter the same record as the online ones - so the half of the term that happened away from a screen is still in the picture. Tests carry a start time and a window, and a student who drops off can resume the session they were in. If you are working out what to ask a coding tool, our note on what a hiring coding test cannot tell a teacher covers the scoring half of this, and what makes a good coding assessment platform covers the evaluation in more depth. You can see how the execution and marking work on our coding assessment software page.