When candidates receive a rejection from a provider along with some feedback, we are now asking candidates whether this feedback is helpful or not.
This is an experiment, to see whether we get sufficient responses to help us make decisions about ways to try and improve the helpfulness of this feedback.
We started asking this question on 15 November 2022.
The issue
Despite changes to how we ask providers to give feedback to candidate when rejecting their application, reviewing the actual feedback given shows a lot of short, unhelpful answers.
Previous research with candidates has revealed that having an application rejected can make candidates feel dejected, and poor feedback can put them off from re-applying.
Deciding whether feedback is helpful or not is subjective, and so it is hard to assess whether any changes we make have been successful or not.
What we did
We added a question on the service which appears below any rejection feedback given.
There is no requirement for candidates to answer the question.
Once the question has been answered, the page is reloaded and the question is replaced with text to show how they answered.
There is no confirmation step, and no way to change their answer, as we wanted to keep it as simple as possible.
Initial results
As of 28 November 2022, almost 2 weeks after having first asked this question, there have been 379 responses. Of these:
- 191 (50%) said the feedback was helpful
- 188 (50%) said the feedback was not helpful
The response rate so far has been around 12%, for rejections given since the feedback launched.
Most candidates who have rated their feedback have done so within a day of receiving it.
Further considerations
Once we have received more responses, we can start to look for patterns in the data by seeing if the helpfulness of rejection feedback varies by:
- category of rejection received
- type of training provider
- individual training providers
- whether or not the candidate had an interview
- candidate characteristics
We can also then consider ways to improve the helpfulness of the feedback.