Our intention in this first round of research was to answer the following questions:
- Can users navigate through the service successfully?
- Are we meeting the user needs for the assessment only route?
- Are we requesting the right data, relevant to the assessment only route?
Who we tested with
- University of Winchester
- University of East London
- University of Southampton
- Essex & Thames SCITT
- Surrey South Farnham SCITT
Research findings
Preparing data input
There is no unified approach to handling candidate data: Most send out a paper application or a Word document designed by them, following the DTTP data input structure. The data is typically stored in a digital (scanned original) and physical file as well as an Excel spreadsheet. HEIs do not use their student record systems for assessment only candidates.
“Sometimes this form comes back to me as a printed paper version filled out by hand, sometimes as an email - then I use two screens to fill this data in.”
Candidates are added throughout the year and at different stages in the assessment process: some upload candidates after the initial meeting, others at the end of the assessment process. Our participants currently do not create drafts, instead they gather all information and upload it at once.
“I would add the candidate once they had their final visit, at the end of the assessment process.”
“I would add the candidate once they had their final visit, at the end of the assessment process.”
Introduction screens
The guidance page is clear overall and its content correct, but the response to its usefulness was mixed.
“Is this information for candidates or for the university? It’s very simplified.”
Providers rely on what has been asked historically to know what data is needed for DTTP and therefore some find little use for our data requirements page:
“I know what data is needed through trial and error.”
Yet others saw the data requirements as a welcome support.
“Data requirements [page] is really good. As many times as you do it you can never actually remember what we need to put in so it’s very very good. Especially for new admins […], as I wouldn’t have to keep repeating myself.”
Registering a candidate
Our system gave users confidence and peace of mind to input the data at their own pace. We observed them create drafts and return to them for data input later.
“At the moment I’m keeping a log of missing info. Now we would just have to go into this system to check for gaps.”
“[The review page] is brilliant, I like that a lot because you can see any glaring mistakes, it’s very clear.”
Data requirements for international trainees need to be discussed with data consumers. By the time providers input data into DTTP they use equivalences, not original foreign grades and terms. International address, international degree institution and degree grade seem redundant.
“I have access to Naric, but I always request the candidate to provide me with a Naric equivalent certificate."
“I would never have to put an international address in, […] we don’t accept anyone who is not already employed by a school in the UK.”
Tracking TRN
Some users feel responsible to communicate the TRN to the trainees, others expect this to be done by the DfE. As a result users place different weight on the importance of tracking the TRN. Although at point of testing our content was incorrect (we stated that DfE does not send the TRNs to trainees, we now know that it does send it by post or email depending on the route they’re on), users responded positively to being shown content around next steps.
“Typically I have to keep going in every couple of days to check if it’s been generated. It would be nice to have an automated email, or for the TRN to go directly to the trainee that I’ve inputted.”
Recommending for and tracking QTS
Users were able to find the section easily, but there were some issues around terminology of data. “Assessment only end date” was interpreted either as the date of recommending a trainee for QTS (today’s date) or the final tutor visit. “Date of standards assessment” was by most interpreted as the date of the final tutor visit.
“What is ‘date of assessment’ - the last day of assessment or the first day of assessment? This is not very clear at all."
Passing the standards assessment is a requirement for receiving QTS. Therefore the options “Not taken” and “Not passed” are misleading, it should either not be there, or show an error message with a link to “Withdrawal”.
Most users were not interested in closely tracking the QTS outcome. Still we could make our “Recommended for QTS” confirmation page more useful by communicating how long it will take for the QTS outcome to be updated on the system, and what the users’ next steps should be.
The status “QTS pending” in the trainee records page was interpreted differently by different users. Some viewed it as “about to receive QTS”, while others believed these candidates were “about to be recommended for QTS”.
Managing trainee data
Users easily navigated through the trainee records screen and were able to complete the actions required to manage ITT data in DTTP, though some felt a little overwhelmed and would prefer a landing page before they got to this screen.
“This is very clear, very different to what we currently see. The system will update automatically which means you always know exactly what stage the applicant is at.”
“As someone who works in spreadsheets, this looks a bit confusing. I don’t like the view. I would like to have the first page I come to to be adding a trainee. Too much information in the visualisation.”
Some users had issues finding the export button, struggled to deselect filters, and would prefer to sort the trainees alphabetically as well as filter by subject.
“Not obvious what order the trainees are listed in.”
“Sometimes we would filter by subjects and routes, especially now we’re doing Secondary.”
What we’ve learned
- we’ve enabled users to complete all the tasks required to handle and input assessment only data
- we were able to serve most of their needs around clarity and efficiency
- the users navigated the service intuitively and much quicker than compared to DTTP
- users understood what data was requested and we were able to confirm that they would be able to provide it in a real-life scenario
“I think if this could be rolled out across everything [all routes], then it would make my job a lot easier.”
“This is very different to DTTP! It’s more friendly, a lot bigger and brighter. It’s a lot clearer.”