Provider support manual:

How do I send data to the ESFA?

Contents

How do I send data to the ESFA?

ILR data must be sent to the ESFA by uploading a file in XML (extensible mark-up language) format through the ‘Submit Learner Data’ service.

You file names must be unique (not used in the service before) and follow the naming convention.

All historic reports (pre 2019-2020) can be accessed through the old ‘Hub’.

It is best practice to send data to the ESFA regularly and not wait until a return date to upload a file. You should upload an ILR file to Submit Learner Data early in a return period to allow enough time to resolve any validation errors and warnings.

You should produce this XML file from your MI system and upload it to the ESFA via Submit Learner Data.

If you do not have an MI system capable of generating an XML file, the ESFA provide an ILR Learner Entry Tool which can be used to create an ILR file for upload to Submit learner Data.

If you experience problems uploading an ILR file, you should contact the ESFA service desk as soon as possible.

File compression

We recommend that ILR files are uploaded as compressed files; compressed files, sometimes referred to as archives or .zip files, contain a version of the ILR data that is significantly smaller than the uncompressed XML file. Upload times for compressed files are shorter, which benefits all providers and contributes to maximum reliability of the service.

Each compressed file should contain one ILR XML file only. The file must not be encrypted, or password protected. The name of the .zip file should be the same as the name of the XML file but with the extension .zip instead of .xml.

ILR file transmission

Your file transmissions must contain all Learner records, Learning Delivery records and Learner Destination and Progression records for all learners across all funding models for the year to date, for that ILR return.

Each file you submit overwrites all previously submitted records from your organisation. This means that you cannot split your data into separate files and transmit each file separately. You cannot send records for learners funded from different funding models in separate files as these will overwrite one another.

Destination and Progression data cannot be sent in a separate file to Learner and Learning Delivery data: doing so would cause the Learner and Learning Delivery records to be deleted from the ILR database.

You must submit a single file containing all of your learner, learning and destination records for all funding models, for the year to date.

Take care to ensure that data is not overwritten in error: reports such as the ‘Rule Violation Report’ are useful for checking that the expected number of learners and learning delivery records are in your file.

Once you have submitted a file it cannot be deleted, if the file contains incorrect data this can only be corrected by submitting another ILR file to overwrite the incorrect one.

The last file you submit prior to the close of an ILR return will be the one loaded into the national database for that particular return.

You may need to use the amalgamation tool in order to create a single ILR file containing all learner records for all funding models.

The impact of incomplete information

If you sent ILR data that contains incomplete, incorrect or omitted data, this may result in the learner record not being accepted onto the national ILR database. It is essential that ILR data is completed and returned promptly, fully and accurately.

Any data for individual ESFA funded learners that is not accepted by the national ILR database will mean that the Provider Funding Report (PFR) will not show the learner’s details and may result in non-payment or clawback of funding in that period for providers paid on actuals.

Funding calculations and provider payments for all ESFA funded learning aims are based directly on the data provided in the ILR. Inaccurate or late information may result in payments not being made.

You must ensure that all documentation relating to the enrolment of the learner and the record of learning activity is completed accurately and conforms to the eligibility rules of the appropriate type of learning. Funding compliance actions may be taken by the ESFA if your data and documentation do not support the eligibility of a funding claim.

The Funding Information System (FIS)

The FIS is a standalone piece of software produced by the ESFA.

The software contains the ILR Validation rules and allows you to check your ILR data before submitting it through Submit Learner Data. The FIS also contains most of the funding calculations for the ESFA and a number of funding reports which may be run and exported based on your data.

The funding reports generated by the FIS are indicative reports. Please note that, in some cases, the results of the funding calculation for Funding model 36 may be incorrect because not all data required to accurately calculate funding can be made available in the FIS. You should refer to the online service for your actual earnings and payments reports.

Some validation rule checks are not included in the FIS and only take place when data is submitted to us. These are rules that check external tables, such as the Unique Learner Number (ULN), Employer identifier and postcode validation.

Documentation about the FIS including installation/uninstallation instructions and user guidance is available on GOV.UK.

Combining ILR files: amalgamation

If you have multiple MI systems across your organisation you will need to combine the ILR data from these separate systems into a single ILR file before you submit it to the online service. For example, you may hold apprenticeship learners and learning aims in a separate system to Adult skills funded learners and learning aims or have multiple internal departments that hold data separately. If this is the case, you will need to amalgamate these separate files into a single ILR file for submission.

The amalgamation tool includes an amalgamation facility to combine multiple files to create a single XML file for all learners. You can also use your own software to create a single file.

The amalgamation tool joins together records from multiple ILR files for learner records that have the same Learner reference number. For the records to be successfully combined into a single learner record, all the data in the Learner fields on all of the records must be the same. Any conflicting learner data will result in an amalgamation error which will need to be corrected. For example, if the Date of birth has been incorrectly recorded on one record and does not match the Date of birth on the other record then an amalgamation error will be generated, and the records will not be combined.

Where data for the same learner is held in separate systems but with different Learner reference numbers, you should combine these into a single learner record. If you merge two or more records for the same learner, you may use any one of the learner’s previous Learner reference numbers.

Any changes to a learner’s Learner reference number from one ILR year to the next must be recorded using the Learner reference number in previous year field. This will enable data matching over multiple ILR years for a learner to be carried out for purposes such as achievement rate calculations. See the ILR Specification for further details and collection requirements for this field.

Data validation

ILR data is subject to validation checks to help ensure that the data received by the ESFA is accurate and complete. ILR files are validated at the point of transmission to the online service or through the FIS against both the XML schema definitions and validation rules.

If any data fails the validation checks, then the learner record and all associated records for that learner are rejected. Rejected records are not loaded into the national ILR database and do not generate funding; these records are reported on the rule violation report.

The FIS does not run all validation checks: rules that rely on external tables such as postcode or Unique Learner Number tables are not included in the FIS. These checks only take place when you submit your data.

The ILR validation rules document includes details about the error condition for each rule and the ILR data it checks. The validation rule document includes a column named “Online Only”, which identifies rules that are not included in FIS and are only run when data is submitted to the online service.

XML schema validation

XML schema validation ensures that your file is in the correct format and that the data complies with the definitions in the ILR specification. There are two stages to the XML schema validation

Stage 1: The ILR file is checked for the following:

• That the XML is well-formed. Well-formed means that the file adheres to XML’s strict syntactical rules for open and close tags and the nesting of data elements

• Elements are presented in the expected sequence, as defined in the schema

• An element conforms to its data type. Examples of this would include where a numeric item contains alpha characters, or where a date contains invalid values

If any part of the file fails any of these checks, then the whole file will be rejected and not processed further, the errors will be reported on the rule violation report.

Stage 2: The individual elements (fields) within the file are checked for the following:

• All mandatory fields have been returned

• Field lengths are adhered to

• Data patterns are adhered to

If a field fails any of these checks, then the learner record, and associated records for that learner only, will be rejected, reported on the rule violation report and not processed further.

The published XML schema contains all the definitions and requirements for both stages of schema validation.

Schema errors are reported on the rule violation report, however, the error messages generated are generally briefer than those for other validation errors.

Only learner records, and associated records for that learner, that have passed schema validation will be passed through to the validation rules.

Validation rules

The final data checks are made by applying the validation rules, which include business validation and funding eligibility checks.

Validation errors  are produced where data does not make sense or cannot be correct. You will need to correct any records that produce validation errors.

If any part of a learner or associated learning delivery record triggers a validation error, then the learner and all of their learning delivery records will be rejected and not processed further. Only valid learner records are loaded into the national ILR database.

Validation errors are reported on the validation error reports which are produced on the online service and in the FIS. You should check these reports after uploading an ILR file. Validation errors that start “FD_” indicate data that is breaking schema restrictions. You should check this data against the definition of the field in the ILR specification.

All learning aims returned for a learner must be valid, regardless of the type of funding. For example, if funded learning aims are included alongside non-funded aims in a return at the start of the year, then all of these aims must be valid in order for the learner records to be accepted.

Validation warnings  are produced where the data is unusual or where the data is allowed under certain circumstances. Records that produce warnings are loaded into the national database; however you should check these records to ensure the data is correct.

Postcode validation

It is important that postcodes are correctly formatted, this includes the space between the two parts of the postcode. Postcodes with two spaces will fail validation when the data is uploaded to the online service.

Most postcode fields in the ILR are mandatory fields and cannot be left blank. For mandatory postcode fields, if the postcode is unknown or does not apply then a value of ZZ99 9ZZ should be recorded.

For the Learning start date postcode field and if the learner's postcode is unknown or not yet updated on the ONS postcode table (i.e. a new build), providers must return a postcode of ZZ99 9ZZ.

For ESF funded learning (Funding model 70) the correct delivery location postcode must be used. ZZ99 9ZZ must not be used.

Monitoring the funding rules

As part of financial assurance processes, we review how the funding system and funding rules are operating to identify possible errors in funding claimed and areas requiring further investigation. The outcomes of this also allow us to confirm that policy specifications are working and achieving the desired outcome(s). As part of this funding monitoring process, we also aim to improve the overall quality of the data being reported to the ESFA.

The funding monitoring plan lists the current areas that are being monitored during the course of the funding year. You can access regular monitoring reports through the online service which will highlight any potential issues to you.

If we identify records in a monitoring area, you must review this data to investigate and determine whether you need to make any data corrections in your next ILR submission. Where there is an explicit statement of the action you must take, then you must comply with this guidance: actions to take can be found within the funding rules, ILR guidance or the monitoring reports user guide.

If you have data errors at the end of the funding year that have not been corrected by the R14 ILR hard close, the following actions may be taken:

• Funding recovered where funding has been claimed in error,

• Assurance visits conducted for specific issues before the end-of-year reconciliation statements are issued, and/or

• Issues used to inform the monitoring process for selecting providers for audit in future funding years.

Further details and guidance documents about funding monitoring can be found on the ESFA financial assurance webpage.

Provider data self-assessment toolkit

The provider data self-assessment toolkit (PDSAT) produces a suite of reports to test the integrity of ILR data in order to assist you with identifying potential anomalies or errors in your data.

Within the suite, the ESFA includes reports that enable providers to identify errors relating to its monitoring of the funding rules. The ESFA uses a similar set of reports to routinely monitor the ILR data, to identify potential errors, and for assurance that providers are complying with the funding rules.

We encourage you to use the PDSAT frequently throughout the year as part of your routine data cleansing, for identifying errors and preparing for financial assurance visits. PDSAT is designed to give you a toolkit to review your ILR data and to assist auditors in the audit of providers’ data.

The current PDSAT download and guidance can be found in section 16 ‘Related Links to Documents and Information’.

Earnings adjustment statement (EAS)

The Earnings Adjustment Statement (EAS) is a CSV file, submitted online, that providers can use to claim ESFA funding that is not reported on the ILR.

The ESFA funding rules give further information on categories within the EAS and their evidence requirements.

The EAS is available for collection periods corresponding to the 12 months of the funding year. Funding for period 1 is added to funding from the ILR from August on the Funding Summary Report, period 2 funding is added to funding from the ILR for September, and so on. The total amount of funding claimed in each of the collection periods’ EAS will be summarised to calculate the total for the funding year.

There are different expectations on how regularly EAS returns are submitted depending on whether the return is for a contract-funded provider or a grant-funded provider. See the EAS guidance for further details.