The Twelve Questions a Reports Section Should Answer

It is the second Monday of the month and a coordinator has one question: which class is behind, and on what?

There are three places to look. The test list gives averages per paper, so Class 9B is at 61% and Class 9A at 68%. The mark sheet gives totals per student, which says nothing about which class. Somebody's spreadsheet has a chart. None of the three answers the question, because the question has two halves and each source holds one of them.

Half an hour later the meeting concludes that 9B needs extra classes. Nobody in the room knows whether 9B sat a harder paper, whether the 9B numbers include a re-test, or whether the gap is one subject or all of them. The number was correct and the decision was a guess.

This is not a data problem. Every fact needed to answer it properly was already recorded. It is a reporting problem, and it has a specific shape: a number without the rule that produced it is not an answer, and a page of numbers with no rules on it is a dashboard rather than a report.

Twelve questions, not twelve charts

The useful test for a reports section is not how many charts it has. It is whether the things somebody actually arrives with are on it. There are about twelve of those in an assessment record, and they sort into six groups by how wide the question is.

Most assessment software answers the eighth row and stops. It is the easiest one - it needs no roster, no topics, no history, and no rules. It is also the row that produced the meeting above.

1. A report has to carry its own rule

"Students below their class average" sounds unambiguous until you try to build it, at which point it turns into four decisions:

• Below average on one test, or on most of them?
• Does an absence count as a zero, or is the student excluded from that paper?
• Is the average the class's own average, or the year group's?
• How many tests does a student need to have sat before the answer means anything?

Any set of answers can be defended. What cannot be defended is not stating them, because a list of names with a threshold behind it that nobody in the room can recite is a list that gets argued with rather than acted on - usually by the teacher whose class is on it, and usually correctly.

So the rule belongs beside the result, in words, on the same screen. It costs one line and it is the difference between a report somebody acts on and a report somebody disputes.

2. A report has to cover the half that happened on paper

This is where most assessment analytics quietly stop being about the student.

An institution assesses on paper and on screen. Unit tests are often written, the mid-term is almost always written, and the class test on Friday might be online. Analytics built on an online platform sees only the online half - and then reports a class average, a trend and a ranking as though they described the term.

They describe a fraction of the term, and not a random fraction: the online half skews towards the shorter, more frequent, lower-stakes assessments. Reading a student's year from it is like judging a season from the friendlies.

A performance breakdown that can be cut four ways - by class, by subject, by question type, and by paper against screen - is the only version of this report that is honest, and the fourth cut is the one that is usually missing. It is also the one that tells a coordinator something they did not know: a class that is fine online and weak on paper is a class with a writing-under-time problem, not a subject problem.

3. A report has to refuse to answer what it cannot know

The topic report is the sharpest example, and it is where the temptation to fudge is strongest.

Answering what should be re-taught requires knowing which question tested what, and whether each student got that question right. Online assessments carry that. A paper test, marked by hand and entered as a total, carries a number and nothing else - there is no question-level detail in "17 out of 25", and no amount of processing will produce it.

So a topic breakdown built from an institution's data can only describe the online portion, and it has to say so on the report itself. The alternative is a page headed "topic weakness" that silently omits the mid-term, which is worse than having no page: somebody plans a fortnight of revision around a gap that is an artefact of which assessments happened to be typed.

A report that names its own blind spot is more trustworthy than one that does not have one, because every report has one and only some of them admit it.

4. A report has to carry the action that clears it

There is a category of thing on this list that is not really analysis - it is work that has not been done yet, and it degrades the numbers above it while it waits.

• Written answers submitted and not yet marked. Until they are marked, every average that includes that paper is provisional and does not say so.
• Students who were assigned a test and did not sit it. Not an academic problem yet; an attendance one, and the first thing to chase.
• Class-and-subject pairs mapped in the timetable with nothing ever assessed against them - the syllabus areas that quietly go unmeasured for a whole term.
• Integrity flags nobody has looked at.

These are only useful next to the thing that clears them. "42 answers awaiting marks" is a statistic; "42 answers awaiting marks" with a link that opens them, grouped by question rather than by paper, is the afternoon's work. Marking one question across everyone who sat it is also markedly faster and more consistent than marking one paper at a time, which is a scheduling detail that turns out to be a fairness detail too.

The same numbers, wherever you read them

One last property, and it is the one that decides whether any of the above is believed.

If a summary on the front page and the full report on another page are computed by two different pieces of code, they will disagree eventually. Not dramatically - one will include re-tests and the other will not, or one will count an absence as a zero - and the first time somebody notices a two-point gap between two screens, the entire reporting section loses its authority. It never fully recovers, because the fix is invisible and the memory is not.

The defence is boring and structural: the summary and the in-depth version have to be the same calculation with a different row limit. Expanding a report is allowed to show more rows. It is not allowed to change a filter, a denominator or a rule - because the moment it can, the two views are answering different questions while appearing to answer one.

What this is actually for

An institution's commercial value comes from how its students do. Everything else it runs - the timetable, the fee collection, the admissions funnel - exists to support that one asset, and almost every institution measures the supporting machinery in far more detail than the asset.

The gap is not that students are not improving. It is that nobody can see it, and nobody can prove it. A parent asks whether their child is improving and is answered on feel. A student who was going to drop out was visible in the marks two months earlier and nobody was looking at that column. Board results arrive with enough detail to be useful and enough delay that the cohort has already left.

Reporting is not a feature that sits beside the assessment tool. On the evidence of what institutions actually ask for mid-term, it is the reason the assessment tool exists.

Teemly puts all twelve of these on one page, each with its rule stated beside its result, and marks from paper tests enter the same record as the online ones - so the class breakdown can be cut by paper against screen rather than reporting the online fraction as though it were the term. The topic breakdown says on the page that it covers online assessments only, because paper marks carry no question-level detail and pretending otherwise would make the most actionable report the least trustworthy one. If you want the worked example behind the topic report, finding the questions a batch keeps getting wrong goes through a five-question paper in detail, and keeping one student record across paper and online tests covers the half of the term most analytics never sees. You can see the rest on our assessment software for schools page.