It’s not easy to raise prior attainment, but universities could better contextualise applicants’ grades

Note: this piece was originally published here on LSE’s British Politics and Policy blog.

4images

The government has challenged the Higher Education sector to double the proportion of pupils from disadvantaged backgrounds and to raise by 20 per cent the number of undergraduates from Black and Minority Ethnic backgrounds. However, last month’s report by the Social Market Foundation (SMF) casts doubt on the achievability of either  goal.

Among the observations offered by the SMF is that the spread of disadvantaged students across UK universities is very patchy. While some institutions’ Widening Participation (WP) intake is pushing 30 per cent, proportions elsewhere barely top 2 per cent. No surprise there, perhaps. But what may come as more of a shock are differences in the rate of improvement. As the SMF graph below shows, progress since 2009 among Top 10 institutions (according to rankings in the Times Higher) is less than half that made by institutions ranked 11-20 and by those outside the Top 20. In other words, the rate at which the UK’s highest prestige universities are growing their WP intake is more sluggish than everywhere else.

smg1

Does that matter? Well, as the Social Mobility and Child Poverty commission has noted, the top professions tend to be dominated by alumni of the highest ranking universities. And according to the Sutton Trust, graduates from such universities enjoy the more substantial earnings premium. The risk is that the sector’s uneven distribution of WP students allows social hierarchies to be reproduced and causes social mobility to stall.

The response of selective universities invariably involves locating the problem further down the food chain by arguing the “real” barrier to access is the attainment gap: the difference in the grades with which young people from different socio-economic backgrounds leave school or college. This position is starkly reinforced by UCAS data reported in the SMF report: in 2015, the total number of young people from society’s most disadvantaged quintile holding entry qualifications that placed them in the top attainment bracket was 1,880; however, the total number of young people from the least disadvantaged backgrounds was 17,560. As the graph below shows, the ratio of high-attaining applicants to low-attaining applicants increases exponentially with socio-economic advantage.

smg2

One option suggested by the SMF is that “institutions themselves get much more involved in raising prior attainment.” Clearly, there are important ways in which universities could and should work more closely with lower-attaining state schools and colleges. We can ‘inspire’; we can do more to smooth school-to-university transitions; we can ensure that pupils apply to appropriate course and that our admissions processes treat them justly. Research continues to indicate that young people from low-participation backgrounds conceptualise higher-prestige universities as beyond their reach and worry about not fitting in. Selection practices may also disfavour them.

However, it’s another matter entirely to suggest that university staff have the expertise needed to close attainment differentials. The SMF suggests we offer tuition, provide summer courses and “directly take on responsibility for running schools”. However, the pedagogies favoured in higher education – those that develop critical thinking, independent scholarship and research-driven enquiry – are a far cry from the teach-to-the-test model to which schools are increasingly forced to submit.

If the problem is that the highest prestige universities are not pulling their weight in terms of progress with WP, an alternative approach would be for them to become more sensitive to the educational background in which applicants’ grades were achieved and more explicit about how this information is used in admissions processes. Contextual data is not a new idea, but the sector lacks a consistent, transparent policy on how, when and why it is applied. We even have the absurd situation of league tables using entry tariffs as an indicator of institutional quality, thereby incentivising the more elite end of the sector to continue fishing in familiar waters.

Some colleagues express concern that students admitted on the basis of contextual data might not have the skills needed to cope with higher education. But let’s not forget that state school applicants outperform their independent school peers at university on a like-for-like basis. It’s not so much social engineering as rational investment in talent that hasn’t yet had the opportunity to manifest as attainment.

The SMF doesn’t mention admissions. Instead, it turns to market-based solutions, speculating that some new providers may provide a boost to WP. However, as Andrew McGettigan and others remind us, newly-created private colleges have so far been associated more with empty classrooms and suspect business practices than with driving forward the nation’s social mobility agenda.

The job of improving attainment levels among society’s least advantaged groups is deeply specialised, and one that may be better left to trained, time-served professionals than to well-meaning university staff. However, the sector could seek to address social mobility in other ways. Our rankings could reward diversity and inclusivity, not penalise the use of contextual data. Our admissions processes could become more transparent and less gameable. Our teaching could compensate for previous educational shortcomings by offering targeted, sustained support. And we could fixate a little less on prior attainment and the league tables that peddle it.

Making A Statement

qLast week, the Sutton Trust published a Research Brief that I co-authored with the HE Access Network. The theme is a familiar one for me: the UCAS personal statement. I’ve blogged about it here and here, written a previous Sutton Trust report, and published findings in an academic journal and a book about global HE admissions practices.

Saint-Mary-s-School-oqLD3CThis study was a really interesting addition to the evidence because it was the first to compare how teachers at state schools and admission tutors at high-prestige universities read statements. The results were alarming: what teachers think make a good personal statement is a far cry from what universities are looking for.

The researjohnhumphMS2010_468x402ch attracted plenty of press attention, including an excellent opinion piece by Catherine Bennett for the Observer. Other print coverage included reports in The Sun, The Times, The Daily Telegraph and the Times Education Supplement. My interview on the BBC Radio Four’s Today programme is available here (listen from 52’45”) until February 26th 2016.

While I think personal statements offer a useful lens through which to view distributions of social capital and explore teenagers’ self-conceptualisations, I’m hoping this will be the last time I write about them. A review of their use in the application process – ideally as part of a wider review of the HE admissions in the UK – is long overdue.

 

What’s in a name?

Note: this piece was originally published as Anonymising UCAS forms is only a first step towards fair and discrimination-free university admissions on LSE’s Democratic Audit blog.

Glasgow University, Credit: Chor Ip, CC BY SA 2.0When pledging to make university applications “name-blind”, the Prime Minister yesterday cited research showing that top universities make offers to 55% of white applicants but only to 23% of black applicants. From 2017, universities will follow major employers that “recruit solely on merit” by offering anonymity to their applicants.

In many respects, this is a sensible move. Universities can hardly claim immunity from ‘unconscious bias’, and admissions processes could be seen to exemplify the “quieter and more subtle discrimination” that the Prime Minister wishes to address. However, those of us who have looked closely at the issue would argue that concealing candidates’ names does not go far enough.

In 2012, I authored a report for the Sutton Trust showing that the quality of UCAS personal statements could be predicted by applicants’ school type. For example, those from Sixth Form Colleges and Comprehensive Schools made several times more basic spelling and grammar errors than those from Grammar Schools and Independent Schools. Ethnicity was also a major factor, with British-Bangladeshi applicant making 2.29 errors per 1,000 words of statement, compared to white applicants’ 1.42 errors. All of the statements I examined were written young people who went on to achieve identical grades at A-level. The differences in their statements were not down to ability; they were down the amount of help and guidance available.

There are other ways in which our university application systems may reproduce existing forms of privilege. Candidates from the fee-paying sector are much more likely to mention the name of their school in their personal statement, even though this information is captured elsewhere in their application, perhaps as a means to accentuate their perceived fit for leading universities. Social capital is demonstrated through prestigious work placements, internships and job shadowing experiences; cultural capital through overseas trips and LAMDA examinations. Evidence suggests that interviews are no less discriminatory, with some candidates drilled extensively in how to perform under pressure while others remain intimidated by an unfamiliar, hostile environment.

So how should selective universities select when almost every indicator is potentially problematic and we cannot be trusted with even a candidate’s name? An extreme solution, favoured by some European countries, is to allocate places on over-subscribed courses based on a lottery for those who meet a minimum academic threshold. Other nations, notably the USA, ask for statements but offer greater reassurance to students from under-represented backgrounds that their application will be read in its appropriate context and the odd spelling mistake will not count against them. Few nations rely on the personal statement as much as the UK. However, with Independent Schools increasingly competing with one another on entry rates to leading universities, and with new markets emerging around the tutoring and coaching of applicants, the pressure to maintain the status quo is considerable.

The Prime Minister is right to say that the UK Higher Education sector needs to take a close look at why young people from some backgrounds can be disadvantaged in the application process. We also need to understand better why ethnicity predicts the likelihood of graduating with a higher degree award. But to stop at anonymised applications would be to pretend that the root of the problem is a handful of prejudiced admissions tutors. The candidate’s name is not the only issue. Indeed, this information may allow more sympathetic admissions tutors to make appropriate allowances. If the goal is to bring greater fairness to the process, we also need to think about more systemic issues, such as why offers are made on predicted rather than actual grades, how candidates’ attainment can be suitably contextualised, and why personal statements are given more prominence than any evidence suggests they are worth.

Have UCAS really revealed “the language tricks which will help you land that place at a top university”?

UCAS, the agency responsible for admissions to Higher Education in the UK, last week issued an analysis of the personal statements of 300,000 university applicants. They’d totted up the number of ‘passion-related’ words (such as  “love” and “explore”) and ‘career-oriented words’ (“salary”, “employable”, “job”, etc.) to see how frequencies differed according to the subjects for which candidates were applying.

1434994273-7fda9d77bd9580b55015eaf596aae1b3-600x450

Journalists seemed unsure what to make of the press release. “In all subjects, applicants used a mixture of career and passion-related words to set out their suitability,” reported The Telegraph.  “It’s not a huge surprise,” admitted another report, “that artsy students have cornered the passion market”. One reporter noted aimlessly that “despite the prominence of economics and economists over the last few years, students wanting to major in economics are among those least likely to mention either a ‘career’-related word”.

But none of the reports questioned whether such words are actually valued by admissions tutors. This gave the unfortunate impression that all applicants needed to do was use the right vocabulary and their place at university would be secure. The Mirror even promised to reveal “how to strike the balance between ‘passion and purpose’ to NAIL your written application”.UCAS_personal_stat_3350671a

UCAS, of course, can hardly be blamed for such misreporting. But it’s no secret that admissions tutors are rarely seduced by the language of love:

Dr Hilary Hinds, an admissions tutor from the English department at Lancaster University, finds clichés such as “passionate about literature” and “I’ve loved books for as long as I can remember” dull and predictable. “Demonstrate it rather than claim it,” she says. (The Guardian, 10.07.15)

My own research into the language of personal statements confirms that Dr Hinds isn’t alone. Words like “passion” and “love” are used more by applicants from state schools than those from independent schools, and they correlate negatively with the likelihood of acceptance by higher-prestige universities.

The press interest generated by the UCAS study was vacuous (at best) and specious (at worst). A far better use of the large, rich, not-publicly-available database would have been to identify patterns of use according to factors known to affect candidates’ chances of success, such as ethnicity and socio-economic status.

The Role of Ethnicity in Admissions to Russell Group Universities


(Note: I published this piece first on the British Educational Research Association’s Respecting Children & Young People blog on 17.03.15…)


a18178551_s

Here’s an excerpt from a UCAS personal statement written recently by an applicant to a Russell Group university:

“There are various times where I have been a team member such as in hockey, this is where we have to understand our team member’s strengths and weaknesses to evaluate best positions, it makes us understand that one’s ability may be skilful but can always be tackled by two. We had to quickly judge aspects; we also understood how goals and motivation can go through team members, as high motivation can motivate another.”

Within the excerpt, some details have been altered to protect the applicant’s identity. However, the writing style is unchanged and captures that of the whole statement.

A natural first response is that the applicant doesn’t belong at a high-prestige institution: the text is poorly punctuated, with muddled content, and reads as though it were thrown together at the last moment. Thank goodness for UCAS personal statements, one might conclude, for allowing universities additional evidence on which to make important selection decisions.

Except things aren’t quite so straightforward.

First, note that this applicant went on to receive A-level grades that were sufficient to gain entry to the courses for which she applied. This suggests that her personal statement difficulties were not caused by a lack of academic ability so much as confusion about what was required. Second, studies like this one, this one and this one, question whether personal statements are really of much value in predicting students’ subsequent performance at university anyway. Third, the applicant was educated in the state sector, and evidence suggests that she may therefore have had limited access to the kind of high quality information, advice and guidance available to many of her competitors. And fourth, the applicant is of British-Bangladeshi heritage, a group which fares poorly in admissions to high-prestige universities compared with White applicants of similar academic attainment.

In 2012, I undertook research for the Sutton Trust looking at how university applicants from different backgrounds set about the task of writing their personal statement. My primary focus was on school type, and I discovered that applicants from Sixth Form Colleges and Comprehensive Schools were much more likely to make basic language errors (spelling mistakes, apostrophe misuse, etc.) than those from Grammar schools and Independent Schools. Workplace experience could also be predicted by school type, with some applicants able to list up to a dozen placements at flash companies while others struggled to make a Saturday job sound relevant to their chosen course of study.

I’ve since returned to the data to find out whether personal statements also differ according to the ethnicity of the applicant. On average, I discovered, British-Bangladeshi applicants make 2.29 clear linguistic errors per 1,000 words of statement, compared to White applicants’ 1.42 errors. British-Bangladeshi applicants are also low on meaningful work-related activity, averaging 1.57 per statement compared to 2.32 for White applicants (where ‘meaningful’ means undertaken for genuine vocational experience rather than for cash – all such activities were blind coded by two text analysts). The total sample size is 327, and all of the statements were submitted by students who would go on to achieve identical A-level grades.

What prompted me to carry out these new counts was the graph below, based on UCAS data analysis by Durham University’s Vikki Boliver. This analysis showed that applicants’ chances of getting an offer from a Russell Group university differed markedly according to their ethnicity. British-Bangladeshi students have a 42.6% likelihood; White British students a 52.0% likelihood.

Table Russell Group Applications

Source: Boliver in Alexander and Arday (2015), via Economics of HE

As Parel and Boliver note, ethnicity actually trumps school type as a predictor of admission to leading UK universities. Figures obtained from Oxford University by the Guardian in 2013 under the Freedom of Information Act indicated that “43% of White students who went on to receive three or more A* grades at A-level got offers, compared with 22.1% of minority students”.

Such differentials can be explained in many ways. Even though Boliver’s data controls for ‘facilitating’ subjects (those the Russell Group claim are preferred by universities), it could be that Black, Asian and Minority Ethnic (BAME) applicants take inappropriate combinations for the degree courses to which they apply. It has also been implied that BAME applicants tend toward oversubscribed subjects, such as medicine or law. However, as Boliver continues to point out, the kind of individual-level data needed to develop a clear picture of why differentials arise is increasingly being restricted because, supposedly, it “presents a high risk of individuals’ personal details being disclosed.”

1astatementDoes the application process itself discriminate against some applicants? I’ve written before about how the UCAS personal statement is, in many respects, a flawed indicator, and I’ve also responded to specific arguments made in its defence. Other researchers have noted similar problems with university interviews (see Burke and McManus on would-be Art & Design students who make the mistake of citing hip-hop as an influence, or Zimdars on a tendency for admissions tutors at Oxford University to recruit in their own image).

Such studies raise awkward but crucial questions about what exactly non-academic indicators are supposed to indicate. Is the personal statement simply an “opportunity to tell us about yourself”, as UCAS benignly describes it, or is the real goal to flaunt one’s cultural and social capital, signalling what Bourdieu characterises as the “dispositions to be, and above all to become, ‘one of us’”?

The Observer’s Barbara Ellen notes that certain kinds of applicant are much more likely to “speak uni” and be able to “decode the foreign language of the admissions process.” And Pilkington reminds us that BAME applicants are “entitled to know that they will not be subject to potentially indirect − or indeed direct − discriminatory practices in an institution’s admissions processes.”

However, the problem may well go beyond the level of institution. A key structural barrier seems to be an admissions process that assumes all applicants are equally equipped to understand (and have sufficient support to meet) its veiled requirements. The personal statement purports to help university admission tutors make informed choices based on holistic evidence, but may actually reproduce White and other forms of privilege at the point of application.

The UCAS personal statement: “just enough rope for a hanging”?

(Note: I published this piece first in Research Fortnight on A-level results day, 2014…)

eco9791

For many students, A-level results day is the culmination of a lengthy and complex university application process involving open days, UCAS forms, grade predictions, school references and admissions interviews. For some, the game will have started even earlier, with awards, experiences and sporting endeavours accumulated throughout adolescence on the promise that “this’ll look good on your personal statement.”

But Britain’s approach to undergraduate admissions is the exception, not the norm. Elsewhere in the world, systems tend to be more sensitive to uneven distributions of social capital, extra-curricular opportunities and guidance from friends, family and school. At a recent international conference, delegates from Northern European countries began to chuckle when I explained that applicants to British universities are given 4,000 characters to write freely about themselves. They felt sure such an indicator would reveal less about the applicant than their social and cultural background. Similar suspicions were raised in the 2004 Schwartz Report, which warned that “some staff and parents advise to the extent that the personal statement cannot be seen as the applicant’s own work.”

The alternative for many nations is appropriately contextualised attainment data. Rather than turn to non-academic criteria to choose between eligible applicants, universities in both Holland and Greece have experimented with lotteries to distribute places on oversubscribed programmes. Meet your course’s entry requirements and your name goes into a hat with every other applicant who reaches that threshold. No gaming the system with interview coaching, LAMDA examinations or extravagant work placements.

Even in the largely decentralised US admissions system, applicants must respond to ‘prompts’, including those below, that are designed to make the process fairer. Such prompts, rather than encourage applicants to cash in on past opportunities, call for focused, candid and reflective responses. College websites in the US reassure applicants that statements will be read “in their true context” (Princeton), and justify the use of non-academic indicators in terms of a “long history of encouraging diversity” (Brown).

essaysca
My interest in the higher education admissions was sparked by Vikki Boliver’s 2012 finding that “applicants to Russell Group universities from state schools are less than two-thirds as likely to receive offers as privately educated applicants,” a differential that wasn’t attributable to ‘facilitating’ subjects. Last month, a study of 50,000 students by the University of Bristol confirmed similar concerns in relation to ethnic minority applicants. For example, for every 100 candidates of Pakistani ethnicity, seven fewer offers were made than for 100 equivalent white British candidates, a pattern that “could not be fully explained by differences in academic attainment or patterns of application.”

Last year, I published Sutton Trust funded research demonstrating that equal-attainment students submitted very different personal statements. Basic linguistic errors (such as spelling errors and apostrophe misuse) were almost three times more common in statements submitted by applicants attending sixth form colleges than by those attending independent schools. For some candidates, work experience meant school-facilitated day trips and paid Saturday jobs; for others, it involved shadowing public figures and undertaking high-prestige placements. My findings resonated with a 2010 survey by the Education and Employers Taskforce showing that 42% of young people from independent schools felt their work experience helped them get into university, as opposed to only 25% of those from comprehensive schools.

ucasapplications_600x300

Of course, it’s possible that admission tutors see through advantages of school type to the candidate beneath. “Does the Sutton Trust really think I’m taken in by slick expensive personal statements?” tweeted Prof Mary Beard when my report first appeared. But last year The Times reported that statements were now regarded as “worthless” by many tutors, and the recent Pearson Think Tank report, (Un)Informed Choices, concluded that “the use of personal statements should be ended.” Unless all students’ applications are judged by staff with the experience and skill to separate privilege from potential, it’s difficult to excuse the continued use of an indicator that, according to a 1996 paper by Karen Surman Paley, affords hopefuls “just enough rope for a hanging.”

Whether this year’s A-level results bring good news or bad for individual students, questions remain about whether the half a million or so personal statements written every year represent an efficient use of time, energy and resource, either for schools and colleges or for the higher education sector. A growing body of evidence suggests that non-academic indicators, rather than bringing equality to the selection process, further advantage those applicants already favoured by school type and socio-economic background.

Manchester Asks… Prof Les Ebdon

A couple of weeks ago, I hosted a University of Manchester public event at which the Director of the Office for Fair Access, Prof Les Ebdon, responded to pre-recorded questions from staff, students and alumni.

One of Prof Ebdon’s key points was about the under-performance of Black and Minority Ethnic (BME) students. According to Prof Ebdon, the issue is now “bigger than access into university” for such students.

Prof Ebdon was responding to a question asked by undergraduate student, Aasia Hanif, in which she cited HEFCE research showing that the likelihood of students from some minority ethnic backgrounds being awarded a good degree was lower than that for other students with the same entry qualifications.

“It happens at nearly every university,” said Prof Ebdon. “The expectation for those students is lower than the expectation for white students.”

Prof Ebdon described university as “the best investment you can make”. However, when pressed on the complexity of student loan model, he conceded that “the advantages of the system take a lot of explaining to people who just see the headline £9,000 per year”.

SJ_LE

In response to a question from Diana Khasa about the lack of encouragement received by some would-be applicants, Prof Ebdon urged universities to address the “myth” that young people from non-traditional backgrounds don’t fit in.

However, he also acknowledged important differences in the quality of advice, information and guidance received by students from different educational backgrounds.

“When I go into a fee-paying school, they’re usually very hot about university admissions,” said Prof Ebdon, before recalling his own difficulties navigating the university admissions system, which he described as “a complete lottery”.

“But lotteries are usually random,” I said.

“You’re absolutely right,” Prof Ebdon replied. “It isn’t a lottery. It’s a loaded dice.”

Prof Ebdon talked about “continued improvement” in the young participation rate of students from the most disadvantaged backgrounds. However, in response to a question about mature students from Student Union Campaigns Officer, Clifford Fleming, he accepted that participation rates for some other groups had fallen since the introduction of higher fees in 2012.

For mature students, Prof Ebdon advocated a “more flexible provision” noting that “ministers believe there are big opportunities in Distance Learning.”

“The picture is changing all the time,” added Prof Ebdon, pointing to “remarkable success” in admissions with minority ethnic groups, but noting that the increasing under-representation of ‘working class boys’ was “building up quite a significant social problem.”

When asked about access to postgraduate study by Clive Agnew, my University’s Associate Vice-President for Teaching and Learning, Prof Ebdon agreed that this was a growing area of concern. “Postgraduate admissions is the new glass ceiling for Widening Participation and we’ve got a problem with double glazing.”

Prof Ebdon also maintained that the Widening Participation agenda should not stop at the point of admissions, noting that non-traditional students “are likely to need extra support” once at university.

Finally, responding to a question about employability skills posed by Director of the Student Experience, Tim Westlake, Prof Ebdon said: “Students with professional parents very often have access to networks which enable them to understand what goes on in particular professions. They have a much wider range of professions that they know about. But students from non-traditional backgrounds may not have experienced that.”