Case Study: Education
Main Content
Understanding educational achievement and differences between pupil residency and institutional data
What is the study about?
The study will first examine educational achievement at Key Stage 3 using educational institution based data, which are particularly relevant to government education targets. Finally the study will consider how differences in the reporting of a dataset can affect your analysis.
The study will investigate the data by considering those local authorities that have less than 50 per cent of pupils achieving level 5 or above at Key Stage 3 in English, Mathematics and Science.
The study will benefit anyone interested in educational attainment or someone keen to understand different ways of viewing data, and is available as a pdf document (260Kb).
The data
Data for this study are stored in the Education, Skills and Training domain of the Neighbourhood Statistics website. Data used for this study includes:
- Key Stage 3 Achievements of 14 Year Old Pupils in Maintained Schools (referenced by location of educational institution) 2001/2002 & 2002/2003
- National Curriculum Assessments of 14 Year Olds by Gender in England (referenced by location of pupil residence): 2001/2002
The Key Stage 3 data includes the educational achievements for 14 year old pupils in maintained schools in the 354 local authorities in England, using 2003 geographical boundaries. It shows the percentage of eligible pupils, boys and girls achieving level 5 or above in English, Mathematics and Science.
The National Curriculum Assessments is exactly the same data as the Key Stage 3 Achievements, however it reports on the pupils' residence rather than the locations of the educational institutions.
The technique
The study will cover how to present data and what it can show. This will include the use of line and bar charts. It will also show how the ways in which data are collected can affect the outcomes of the analysis.
Key Stage 3 national
We can first consider the Key Stage 3 results for the institution data at national level. This will enable us to show if there are any subjects where pupils are not achieving a 50 per cent pass rate at level 5 and above.
Figure 1: The percentage of achievements in 2001/2002 and 2002/2003 for England.
It can be seen that the achievements have increased in English (67 per cent - 69 per cent), Mathematics (67 per cent - 71 per cent) and Science (67 per cent - 68 per cent) over the two year period. Importantly no subject has less than a 50 per cent achievement.
We can next consider the nine regions within England.
Hint: The chart used here is a bar chart created in Excel: see Using charts to understand patterns in crime for an even more detailed understanding of bar charts and their uses.
Key Stage 3 government office region (GOR)
The data in the table below show that all nine GORs in England have higher than a 50 per cent achievement at level 5 or above in all three subjects for 2001/2002 and 2002/2003. All nine regions also show an increase from 2001/2002 - 2002/2003 in the three subjects.
Table 1: Table showing Level 5+ achievement in all government office regions
Both this and the national analysis have shown that achievements have increased in the two year period. We can now investigate at lower local authority levels to see if these patterns continue.
Key Stage 3 for local authorities
In this section the study will consider the percentage of achievements at level 5 or above for all the local authorities in England. This will show if there are any local authorities that do not follow the national and regional picture and have less than a 50 per cent achievement level.
Figure 2: Counts of local authorities with less than 50 per cent pass rates.
In 2001/2002 there were four local authorities that achieved less than 50 per cent in English, four in Mathematics and seven in Science. By 2002/2003 this number had fallen to two in English, zero in Mathematics and three in Science. This indicates that underlying the higher level data there are local authorities that are below the 50 per cent level, but which ones?
Key Stage 3 for each subject at local authority level
This section will expand figure 2 to show which local authorities had less than 50 per cent achievements in 2001/2002 and 2002/2003 for English, Mathematics and Science.
English
2001/2002 - The local authorities of Nottingham, Bolsover, Hackney and Sevenoaks all had lower than 50 per cent achievement at level 5 or above.
2002/2003 - The local authorities of Nottingham and Islington had lower than 50 per cent achievement at level 5 or above.
Figure 3: Change over time for Nottingham and Islington.
It can be seen that over the two years Islington has dropped from 51 per cent to 49 per cent. It also shows that Nottingham has increased from 46 per cent to 49 per cent.
Hint: Notice that the chart's scale does not start at zero (or the origin). This means it will only be comparable to other charts on the same scale. Some statisticians have suggested that using a 'y' axis that starts at zero for everything is correct, whilst others suggest that only some charts should. Neighbourhood Statistics uses charts that nearly always start at the origin as this aids understanding.
Mathematics
2001/2002 - The local authorities of Manchester, Hackney, Southwark and Tower Hamlets had lower than 50 per cent achievement at level 5 or above.
2002/2003 - No local authorities had below 50 per cent in 2002/2003
Science
2001/2002 - The local authorities of Manchester, Nottingham, Hackney, Haringey, Islington, Southwark and Tower Hamlets had lower than 50 per cent achievement at level 5 or above.
2002/2003 - The local authorities of Hackney, Southwalk and Tower Hamlets had lower than 50 per cent achievement at level 5 or above.
Figure 4: Those local authorities that had less than 50 per cent achievements in both 2001/2002 and 2002/2003 for science.
We have seen in this section that Nottingham, although still below 50 per cent for English, has increased its achievement level in 2002/2003. Islington however, shows a drop from 51 per cent to 49 per cent over the same period.
It is not appropriate to imply a trend from these two years of data as it is very difficult to predict what is happening with limited data. There may also be underlying issues that can cause fluctuations in the results, or changes in the method of calculation. These can affect any time-series.
We have seen that the overlying patterns that were presented at the higher geographical levels are not repeated at this lower geographical level for some local authorities. However, the analysis shouldn't stop here as there may be issues within these local authorities that can be explored.
We know that the Key Stage 3 data also shows the results split between genders so we can investigate this split to see if there is anything of note.
For this section of the study we will investigate Nottingham in depth. Nottingham has been chosen as it is one of the local authorities that are situated around the 50 per cent achievement level for all subjects in both years. However, this type of analysis could be repeated for any local authority of interest.
By gender and subject in Nottingham local authority
We will first consider all three subjects within this local authority and then see how the results vary by gender. Finally we will investigate the differences in Nottingham between the institutional and pupil residency data. This will highlight the effect of how data are reported.
Figure 5: Achievements at level 5 or over in the two years in Nottingham for the three subjects.
By viewing the data in this way we have a better understanding of what is happening in Nottingham for all three subjects over both years. We can further investigate how the gender of the pupils affects achievements.
Figure 6: Nottingham achievements for the 2001/2002 and 2002/2003 broken down by subject and gender.
This highlights the change that has occurred from 2001/2002 - 2002/2003. The chart shows that there are differences between genders for each of the subjects. Boys' achievements in English are lower than 50 per cent, but have risen from 36 per cent to 42 per cent. Also the girls' achievements in Science are still below 50 per cent.
Another interesting element of the chart is that it shows achievements for females in English have fallen. We know the data are rounded to the nearest percent. This means that the gap may not be as big as 1 percent but it does raise an issue that might be as a result of a targeting strategy and may warrant consideration.
By investigating the differences within Nottingham in this way we have seen how the pass levels are affected by the gender of the pupil. One further step of analysis is to investigate the differences between the data reported on the location of educational institution and on pupil residency.
Differences in Nottingham between datasets
We have shown, for Nottingham the changes that have occurred in educational achievements over a two year period. By extending the analysis to consider National Curriculum Assessments 2001/2002 we could also get a better understanding of how the data itself can affect the outcomes.
We know that the National Curriculum Assessments is the same data as the Key Stage 3 data with the only difference being how each of them reports. The difference is that the National Curriculum Assessments data are reported based on the postcode of the pupil while the Key Stage 3 data are reported based on the postcode of the school.
By exploring this we will be able to see the differences in achievements for all those pupils resident in Nottingham, as opposed to those that attend educational institutions there.
Figure 7: Difference between pupils resident and pupils educated in Nottingham for 2001/2002.
Figure 7 shows little difference in attainment for those pupils that are educated within Nottingham against those that reside in Nottingham. This might not always be the case and is important to consider.
Generating this analysis gives a more in depth view of education in Nottingham than just by considering the location of the institution.
Summary
We have seen that Nottingham along with some other local authorities, has patterns of achievement that are different from those found at higher levels of geography. Also there are differences in gender achievements that affect Nottingham itself. This highlights the need to investigate the data fully so a more detailed picture can be shown. It is very easy for an underlying issue to be hidden in an overall pattern or statement.
We have also shown that the method by which data are reported can influence your analysis. By comparing pupil residency with institution location we have highlighted one of the many factors that need to be considered when analysing data.
Using charts has enabled us to see a broader picture yet still uncover the detail of each subject and each gender. This detail would be more difficult to spot in a table, so a chart is a useful way of illustrating the patterns within the data.
Creating understanding with charts
Using charts is often the easiest way to understand more about data, and is suitable for the majority of datasets. Your knowledge of the data may be much greater than that of your audience, so charts can help by:
- conveying a concise message;
- making a complex issue easier to understand;
- highlighting something unexpected in the data.
However, you should ensure that your chart does not become so complex that your audience cannot understand it. In addition you should consider which form of chart to use, as well as good practice in chart production.
Chart selection
Choosing the right chart for your data can be difficult. Firstly you need to understand what each of the charts will show.
- Line chart - good for showing a time series of data
- Pie chart - Useful when trying to show the relative size of parts in a whole, but beware if there are a lot of 'segments'
- Bar chart - Size of related measurements when compared to national or other areas.
The most important thing to consider when choosing a chart is to ask 'does it show what I would have expected?'. You as a data user will know a lot about the data so does the chart seem realistic and will it be understood by someone who does not have your knowledge?
Producing charts
There are some key elements of a chart that can help the audience to understand the charts that you create, make sure to include:
- an appropriate title;
- appropriate labels on the axes;
- an appropriate scale for the axis / axes;
- information on what units are being used;
- information on the time period of the data;
- an indication of the source of the data.
Does the chart tell the right story?
Also key in presenting the data in a chart form is to make sure you fully understand the data so the true story is shown, and is not blurred by issues surrounding the data collection or processing. To aid this Neighbourhood Statistics provides a metadata document (see Limitations of data below) with every dataset. Metadata enables users of the data to gain knowledge of what the data are, how they are collected and any limitations. This means that more understanding can be added to the chart. The best way of doing this is through commentary.
What every good chart needs - commentary
Every good chart needs some commentary. Without fully explaining what the chart or chart shows is like telling a joke without a punch line. Even the most understandable charts need some interpretation. You wouldn't dream of not discussing your chart when it appears as a slide in a presentation. Commentary allows this discussion with every user.
Commentary should be straightforward and concise, and should reflect the information in the chart in one or two sentences. If it takes any more, then your chart is probably too complex.
Limitations of data
Since all the data used for this study are published on the Neighbourhood Statistics website there are also metadata about this data. Metadata are information about the data. In Neighbourhood Statistics the metadata explain amongst other things the source of the data, how they are collected, what the purpose of the collection was, the strengths and limitations of the data. It is important that we know about these limitations before we can make conclusions or recommendations.
Conclusion
The study has shown how and why these datasets can be analysed to aid the understanding of educational attainments in 14 year olds. However, this analysis should be seen as a starting point. More complex analysis of attainment may also consider the proximity of schools to their borders and the extent of the cross-border movement of pupils. This would give a clearer understanding of what is happening at the local level.
