Kuldip (ESFA)

Have your questions answered about QARs (6 July to 16 July)

Edited

To coincide with the release of R10 in-year data for 2020 to 2021, we invite you to put your questions to the team behind the production of the QAR.

We especially encourage any providers that haven’t previously received QAR’s, but feel free to ask any questions that you may have, technical or otherwise. The team will respond to any questions and are keen to get constructive feedback on how we can make improvements.

Please remember that to post you need to be logged in. If you’ve not yet registered for Communities, use the top-right button ‘Sign in/Create an account’ to ask a question and subscribe to this post. If you need any help registering or using communities, please email customer.experience@education.gov.uk  

We’ve recently released the supporting documents for QAR’s and we welcome feedback on these as well. They can be found here:  Qualification achievement rates (QAR) 2020 to 2021 - GOV.UK (www.gov.uk)

Any queries on specific learners should not be raised in this public forum. Instead, please complete and return the QAR query form, which asks for the information we’ll need, and return that or any other specific QAR queries through our online enquiry form. It’s helpful if you selecting the query type ‘QAR’.

The team will be responding to questions or feedback that are posted during the period from now to Friday 16th July.

Replies

No one has replied to this post.


Ben James

Brace yourself..

Jessica De Melo

We're a new provider of apprenticeships (from July 2020) - are these relevant to us? Is there a guide anywhere on where we access these and how to follow them?

TIA

Martin Locock

The dashboard appears blank and I can't use the dropdowns for organisation or UKPRN.  

 

I suspect this is because none of our apprentices have reached their expected completion date yet, but it would be good if the screen said so.

 

Dave Jeffries

Same here. Other parts of the website (Management Information, Data Quality Assurance) do show my employer and information.

Achievement/performance doesn't pick up provider name or UKPRN. We only do E&T and have had hundreds of learners in scope, but the website is just blank.

Steveh

(I can't believe they're still doing this)

What is the point of you releasing data that is fundamentally broken? How can the EDUCATION AND TRAINING data be affected by an issue with AchDate* when AchDate shouldn't be anywhere near the E&T calculation? I'd rather have waited another week and had usable data than you scrambling to hit a 30 June deadline. With the exception of overdue continuers and overdue BILs, this data cannot be used for anything and, frankly, I'm not entirely sure I trust it to do those things either...

You've had 18 months since you last produced any data to work this stuff out, how can this possibly be acceptable?

 

*Quote from the Service Desk "We are aware that there is an issue with Hybrid End Year where Ach Date is blank (because the aim has withdrawn) and these are currently being marked as Hybrid End Year ‘2020.5’ when they should be marked as the Actual/Planned/Report Year depending on the latest of those three dates. The above will be addressed and resolved in the next release of QARs In_Year at R12."

Martin West

I think this is premature given that you have been unable to get the dashboard or underling data correct in this release of the QAR.

My only question would be how did you get it so wrong?

Gaynor Hunt

The tabs in Apprenticeships are for 16-18 and 19+.   Is there no overall achievement rate?

Ben Hankinson

I'll echo the comments about the withdrawals Hybrid End bug due to non-achieval date and lack of Overall achievement rate for all age groups.

Paul (ESFA)

Jessica De Melo thank you for your question (We're a new provider of apprenticeships (from July 2020) - are these relevant to us? Is there a guide anywhere on where we access these and how to follow them?)
Here's the response from QAR colleagues.

Qualification achievement rates will be relevant to you because they are one of a number of indicators that the ESFA and others, such as Ofsted, use to measure the performance of education and training providers.

At a very high level, achievement rates for apprentices are:

  • ‘overall pass rate’ - the number of achieved apprenticeships as a percentage of the total number of apprentices that have finished their training.

and

  • ‘overall retention rate’ - the number of apprentices that have completed all of the planned learning activities as a percentage of your total number of apprentices.

As your delivery started in July 2020, you may not have any information to view yet.

In terms of guidance documents, it’s worth reading how the ESFA measures the performance of FE providers and the more detailed QAR guidance

The QAR dashboards are in View your education data If you’re new to this, you will need to make sure you have the right login credentials by requesting the ‘View Your Education Data - QAR’ role from your organisation’s IDAMs super user.

The intention of sharing data in year (using your R10 ILR data return) is to give providers indicative QARs to their achievement data and identify, then resolve, potential data issues prior to the ILR R14 hard close on 21 October 2021. It is important that you use this in-year QAR to review your data, as you cannot change 2020 to 2021 ILR data after the final R14 window closes.

We hope that is helpful.

(Edited)

Paul Taylor

I echo the points above that Steve H has made regarding publishing data that is wrong which is incredibly frustrating. We spent half a day trying to unpick this before realising that it did not give us what we needed, which is the ability to cross check against our internally held data. Multiply that by however many providers and that is a lot of wasted hours.

Onto the other issues:

- When you publish R10 then do not only include learners with planned end dates up to the R10 submission. Give us the whole year as this is what we will be looking at.

- Please gives us 19/20. We want to check this also and redacting it means that we cannot unpick all our differences.

- A more fundamental issue; will the agency consider bringing in a planned EPA date to help more realistically plot when learners will complete within achievement rate cohorts? The planned end date for standards is likely to always be several months before the achievement date meaning that achievement rate cohorts are much more volatile than they have ever been.

- What is the process for updating an EPA outcome when a failure is recorded in one year, but a resit is achieved in a subsequent year? How does this get picked up within the QAR calc?

- can you tell us if and when an outcome related minimum standards replacement will be introduced.

Paul (ESFA)

@Martin Locock Thanks for your feedback and suggestion (The dashboard appears blank and I can't use the dropdowns for organisation or UKPRN. I suspect this is because none of our apprentices have reached their expected completion date yet, but it would be good if the screen said so.) @Dave Jeffries raised a similar issue. Here's the response from QAR colleagues.

You’re right, your dashboard will only have values if you have leavers in the 2020 hybrid end year. We will look to address the 'blank' issue ahead of the R12 QARs release, so that the dashboards confirm if there is no data to review.

We think there are a couple of reasons that many providers are not seeing the data they anticipated. This could be:

  • where a substantial proportion of E&T delivery fall under the 'Teacher Assessed Grade' rule noted in both the QAR Business rules and E&T Technical Specification which means they are out of scope for QAR. For the next in-year release, we are exploring options for making this clearer within the dashboards.
  • Because of two issues where we've incorrectly marked hybrid end year as 2020.5. They are records that have ended or planned to end between the R10 freeze date and 31/07/2021 and Apprenticeship standards that have a blank achievement date and have Withdrawn.

We can look at an R11 QAR release to address the hybrid end year issues if providers think that would be helpful?

Dave or Martin - if you the above doesn't apply to your data, please contact us directly with your UKPRN via our online enquiry form so that we can look at your data. Please select the ‘QAR query type’ when completing that form so it goes straight to the QAR team.

For new users to QAR or who didn’t see the message in ESFA Update, we said “recognising the ongoing impact of the pandemic and published guidance on the awarding of qualifications for summer 2021, those qualifications confirmed as subject to normal assessment will be included in institution level QARs and shared with Ofsted and providers. Those qualifications confirmed as teacher assessed and those which may be subject to a teacher assessed grade if they cannot safely access an assessment (and Access to HE courses regulated by QAA) will not be included in institution level QARs or shared with Ofsted or providers for 2020 to 2021. The spreadsheet within Ofqual’s Summer 2021 Qualification Explainer Tool confirms which categories apply for normal assessment/teacher assessed grades.”

For others who may be new to QARs, it’s worth us explaining ‘hybrid end years’. These are created by the ESFA to decide which year’s QAR data a learner’s data is included in. This is because training may end up taking longer than anticipated to finish, so we can’t just use the planned end date. The hybrid year also means that if learners are not rolled over from previous years, they will still be accounted for in QAR calculations.

The hybrid end year is the later of:

  • the achievement year (for apprenticeship standards on funding model 36 only)
  • the planned end year of the learning aim
  • the actual end year of the learning aim
  • the reporting year

Paul (ESFA)

Gaynor Hunt Thanks for your question (The tabs in Apprenticeships are for 16-18 and 19+. Is there no overall achievement rate?)

The QAR team are looking to add an 'all age' apprenticeship view as a future improvement.

Ben Hankinson

A R11 QAR release regarding the hybrid end issues would be very much appreciated, would potentially give us time to analyse the data before the R12 window closes if it is released quickly?

Kuldip (ESFA)

Thanks Ben Hankinson for you comment A R11 QAR release regarding the hybrid end issues would be very much appreciated, would potentially give us time to analyse the data before the R12 window closes if it is released quickly? QARs colleagues say:

We usually do an R12 release and this is published mid-august. In light of feedback, we're looking at the feasibility of an R11 release and we'll communicate this through ESFA communities as soon as we are able to, we're aiming to do this by the end of the month.