The University Game

I’m looking forward to giving a Sarah Fielden seminar on May 11th at the University of Manchester. All welcome. Further details here.

333

Advertisements

What’s in a name?

Note: this piece was originally published as Anonymising UCAS forms is only a first step towards fair and discrimination-free university admissions on LSE’s Democratic Audit blog.

Glasgow University, Credit: Chor Ip, CC BY SA 2.0When pledging to make university applications “name-blind”, the Prime Minister yesterday cited research showing that top universities make offers to 55% of white applicants but only to 23% of black applicants. From 2017, universities will follow major employers that “recruit solely on merit” by offering anonymity to their applicants.

In many respects, this is a sensible move. Universities can hardly claim immunity from ‘unconscious bias’, and admissions processes could be seen to exemplify the “quieter and more subtle discrimination” that the Prime Minister wishes to address. However, those of us who have looked closely at the issue would argue that concealing candidates’ names does not go far enough.

In 2012, I authored a report for the Sutton Trust showing that the quality of UCAS personal statements could be predicted by applicants’ school type. For example, those from Sixth Form Colleges and Comprehensive Schools made several times more basic spelling and grammar errors than those from Grammar Schools and Independent Schools. Ethnicity was also a major factor, with British-Bangladeshi applicant making 2.29 errors per 1,000 words of statement, compared to white applicants’ 1.42 errors. All of the statements I examined were written young people who went on to achieve identical grades at A-level. The differences in their statements were not down to ability; they were down the amount of help and guidance available.

There are other ways in which our university application systems may reproduce existing forms of privilege. Candidates from the fee-paying sector are much more likely to mention the name of their school in their personal statement, even though this information is captured elsewhere in their application, perhaps as a means to accentuate their perceived fit for leading universities. Social capital is demonstrated through prestigious work placements, internships and job shadowing experiences; cultural capital through overseas trips and LAMDA examinations. Evidence suggests that interviews are no less discriminatory, with some candidates drilled extensively in how to perform under pressure while others remain intimidated by an unfamiliar, hostile environment.

So how should selective universities select when almost every indicator is potentially problematic and we cannot be trusted with even a candidate’s name? An extreme solution, favoured by some European countries, is to allocate places on over-subscribed courses based on a lottery for those who meet a minimum academic threshold. Other nations, notably the USA, ask for statements but offer greater reassurance to students from under-represented backgrounds that their application will be read in its appropriate context and the odd spelling mistake will not count against them. Few nations rely on the personal statement as much as the UK. However, with Independent Schools increasingly competing with one another on entry rates to leading universities, and with new markets emerging around the tutoring and coaching of applicants, the pressure to maintain the status quo is considerable.

The Prime Minister is right to say that the UK Higher Education sector needs to take a close look at why young people from some backgrounds can be disadvantaged in the application process. We also need to understand better why ethnicity predicts the likelihood of graduating with a higher degree award. But to stop at anonymised applications would be to pretend that the root of the problem is a handful of prejudiced admissions tutors. The candidate’s name is not the only issue. Indeed, this information may allow more sympathetic admissions tutors to make appropriate allowances. If the goal is to bring greater fairness to the process, we also need to think about more systemic issues, such as why offers are made on predicted rather than actual grades, how candidates’ attainment can be suitably contextualised, and why personal statements are given more prominence than any evidence suggests they are worth.

Have UCAS really revealed “the language tricks which will help you land that place at a top university”?

UCAS, the agency responsible for admissions to Higher Education in the UK, last week issued an analysis of the personal statements of 300,000 university applicants. They’d totted up the number of ‘passion-related’ words (such as  “love” and “explore”) and ‘career-oriented words’ (“salary”, “employable”, “job”, etc.) to see how frequencies differed according to the subjects for which candidates were applying.

1434994273-7fda9d77bd9580b55015eaf596aae1b3-600x450

Journalists seemed unsure what to make of the press release. “In all subjects, applicants used a mixture of career and passion-related words to set out their suitability,” reported The Telegraph.  “It’s not a huge surprise,” admitted another report, “that artsy students have cornered the passion market”. One reporter noted aimlessly that “despite the prominence of economics and economists over the last few years, students wanting to major in economics are among those least likely to mention either a ‘career’-related word”.

But none of the reports questioned whether such words are actually valued by admissions tutors. This gave the unfortunate impression that all applicants needed to do was use the right vocabulary and their place at university would be secure. The Mirror even promised to reveal “how to strike the balance between ‘passion and purpose’ to NAIL your written application”.UCAS_personal_stat_3350671a

UCAS, of course, can hardly be blamed for such misreporting. But it’s no secret that admissions tutors are rarely seduced by the language of love:

Dr Hilary Hinds, an admissions tutor from the English department at Lancaster University, finds clichés such as “passionate about literature” and “I’ve loved books for as long as I can remember” dull and predictable. “Demonstrate it rather than claim it,” she says. (The Guardian, 10.07.15)

My own research into the language of personal statements confirms that Dr Hinds isn’t alone. Words like “passion” and “love” are used more by applicants from state schools than those from independent schools, and they correlate negatively with the likelihood of acceptance by higher-prestige universities.

The press interest generated by the UCAS study was vacuous (at best) and specious (at worst). A far better use of the large, rich, not-publicly-available database would have been to identify patterns of use according to factors known to affect candidates’ chances of success, such as ethnicity and socio-economic status.

The Role of Ethnicity in Admissions to Russell Group Universities


(Note: I published this piece first on the British Educational Research Association’s Respecting Children & Young People blog on 17.03.15…)


a18178551_s

Here’s an excerpt from a UCAS personal statement written recently by an applicant to a Russell Group university:

“There are various times where I have been a team member such as in hockey, this is where we have to understand our team member’s strengths and weaknesses to evaluate best positions, it makes us understand that one’s ability may be skilful but can always be tackled by two. We had to quickly judge aspects; we also understood how goals and motivation can go through team members, as high motivation can motivate another.”

Within the excerpt, some details have been altered to protect the applicant’s identity. However, the writing style is unchanged and captures that of the whole statement.

A natural first response is that the applicant doesn’t belong at a high-prestige institution: the text is poorly punctuated, with muddled content, and reads as though it were thrown together at the last moment. Thank goodness for UCAS personal statements, one might conclude, for allowing universities additional evidence on which to make important selection decisions.

Except things aren’t quite so straightforward.

First, note that this applicant went on to receive A-level grades that were sufficient to gain entry to the courses for which she applied. This suggests that her personal statement difficulties were not caused by a lack of academic ability so much as confusion about what was required. Second, studies like this one, this one and this one, question whether personal statements are really of much value in predicting students’ subsequent performance at university anyway. Third, the applicant was educated in the state sector, and evidence suggests that she may therefore have had limited access to the kind of high quality information, advice and guidance available to many of her competitors. And fourth, the applicant is of British-Bangladeshi heritage, a group which fares poorly in admissions to high-prestige universities compared with White applicants of similar academic attainment.

In 2012, I undertook research for the Sutton Trust looking at how university applicants from different backgrounds set about the task of writing their personal statement. My primary focus was on school type, and I discovered that applicants from Sixth Form Colleges and Comprehensive Schools were much more likely to make basic language errors (spelling mistakes, apostrophe misuse, etc.) than those from Grammar schools and Independent Schools. Workplace experience could also be predicted by school type, with some applicants able to list up to a dozen placements at flash companies while others struggled to make a Saturday job sound relevant to their chosen course of study.

I’ve since returned to the data to find out whether personal statements also differ according to the ethnicity of the applicant. On average, I discovered, British-Bangladeshi applicants make 2.29 clear linguistic errors per 1,000 words of statement, compared to White applicants’ 1.42 errors. British-Bangladeshi applicants are also low on meaningful work-related activity, averaging 1.57 per statement compared to 2.32 for White applicants (where ‘meaningful’ means undertaken for genuine vocational experience rather than for cash – all such activities were blind coded by two text analysts). The total sample size is 327, and all of the statements were submitted by students who would go on to achieve identical A-level grades.

What prompted me to carry out these new counts was the graph below, based on UCAS data analysis by Durham University’s Vikki Boliver. This analysis showed that applicants’ chances of getting an offer from a Russell Group university differed markedly according to their ethnicity. British-Bangladeshi students have a 42.6% likelihood; White British students a 52.0% likelihood.

Table Russell Group Applications

Source: Boliver in Alexander and Arday (2015), via Economics of HE

As Parel and Boliver note, ethnicity actually trumps school type as a predictor of admission to leading UK universities. Figures obtained from Oxford University by the Guardian in 2013 under the Freedom of Information Act indicated that “43% of White students who went on to receive three or more A* grades at A-level got offers, compared with 22.1% of minority students”.

Such differentials can be explained in many ways. Even though Boliver’s data controls for ‘facilitating’ subjects (those the Russell Group claim are preferred by universities), it could be that Black, Asian and Minority Ethnic (BAME) applicants take inappropriate combinations for the degree courses to which they apply. It has also been implied that BAME applicants tend toward oversubscribed subjects, such as medicine or law. However, as Boliver continues to point out, the kind of individual-level data needed to develop a clear picture of why differentials arise is increasingly being restricted because, supposedly, it “presents a high risk of individuals’ personal details being disclosed.”

1astatementDoes the application process itself discriminate against some applicants? I’ve written before about how the UCAS personal statement is, in many respects, a flawed indicator, and I’ve also responded to specific arguments made in its defence. Other researchers have noted similar problems with university interviews (see Burke and McManus on would-be Art & Design students who make the mistake of citing hip-hop as an influence, or Zimdars on a tendency for admissions tutors at Oxford University to recruit in their own image).

Such studies raise awkward but crucial questions about what exactly non-academic indicators are supposed to indicate. Is the personal statement simply an “opportunity to tell us about yourself”, as UCAS benignly describes it, or is the real goal to flaunt one’s cultural and social capital, signalling what Bourdieu characterises as the “dispositions to be, and above all to become, ‘one of us’”?

The Observer’s Barbara Ellen notes that certain kinds of applicant are much more likely to “speak uni” and be able to “decode the foreign language of the admissions process.” And Pilkington reminds us that BAME applicants are “entitled to know that they will not be subject to potentially indirect − or indeed direct − discriminatory practices in an institution’s admissions processes.”

However, the problem may well go beyond the level of institution. A key structural barrier seems to be an admissions process that assumes all applicants are equally equipped to understand (and have sufficient support to meet) its veiled requirements. The personal statement purports to help university admission tutors make informed choices based on holistic evidence, but may actually reproduce White and other forms of privilege at the point of application.

The UCAS personal statement: “just enough rope for a hanging”?

(Note: I published this piece first in Research Fortnight on A-level results day, 2014…)

eco9791

For many students, A-level results day is the culmination of a lengthy and complex university application process involving open days, UCAS forms, grade predictions, school references and admissions interviews. For some, the game will have started even earlier, with awards, experiences and sporting endeavours accumulated throughout adolescence on the promise that “this’ll look good on your personal statement.”

But Britain’s approach to undergraduate admissions is the exception, not the norm. Elsewhere in the world, systems tend to be more sensitive to uneven distributions of social capital, extra-curricular opportunities and guidance from friends, family and school. At a recent international conference, delegates from Northern European countries began to chuckle when I explained that applicants to British universities are given 4,000 characters to write freely about themselves. They felt sure such an indicator would reveal less about the applicant than their social and cultural background. Similar suspicions were raised in the 2004 Schwartz Report, which warned that “some staff and parents advise to the extent that the personal statement cannot be seen as the applicant’s own work.”

The alternative for many nations is appropriately contextualised attainment data. Rather than turn to non-academic criteria to choose between eligible applicants, universities in both Holland and Greece have experimented with lotteries to distribute places on oversubscribed programmes. Meet your course’s entry requirements and your name goes into a hat with every other applicant who reaches that threshold. No gaming the system with interview coaching, LAMDA examinations or extravagant work placements.

Even in the largely decentralised US admissions system, applicants must respond to ‘prompts’, including those below, that are designed to make the process fairer. Such prompts, rather than encourage applicants to cash in on past opportunities, call for focused, candid and reflective responses. College websites in the US reassure applicants that statements will be read “in their true context” (Princeton), and justify the use of non-academic indicators in terms of a “long history of encouraging diversity” (Brown).

essaysca
My interest in the higher education admissions was sparked by Vikki Boliver’s 2012 finding that “applicants to Russell Group universities from state schools are less than two-thirds as likely to receive offers as privately educated applicants,” a differential that wasn’t attributable to ‘facilitating’ subjects. Last month, a study of 50,000 students by the University of Bristol confirmed similar concerns in relation to ethnic minority applicants. For example, for every 100 candidates of Pakistani ethnicity, seven fewer offers were made than for 100 equivalent white British candidates, a pattern that “could not be fully explained by differences in academic attainment or patterns of application.”

Last year, I published Sutton Trust funded research demonstrating that equal-attainment students submitted very different personal statements. Basic linguistic errors (such as spelling errors and apostrophe misuse) were almost three times more common in statements submitted by applicants attending sixth form colleges than by those attending independent schools. For some candidates, work experience meant school-facilitated day trips and paid Saturday jobs; for others, it involved shadowing public figures and undertaking high-prestige placements. My findings resonated with a 2010 survey by the Education and Employers Taskforce showing that 42% of young people from independent schools felt their work experience helped them get into university, as opposed to only 25% of those from comprehensive schools.

ucasapplications_600x300

Of course, it’s possible that admission tutors see through advantages of school type to the candidate beneath. “Does the Sutton Trust really think I’m taken in by slick expensive personal statements?” tweeted Prof Mary Beard when my report first appeared. But last year The Times reported that statements were now regarded as “worthless” by many tutors, and the recent Pearson Think Tank report, (Un)Informed Choices, concluded that “the use of personal statements should be ended.” Unless all students’ applications are judged by staff with the experience and skill to separate privilege from potential, it’s difficult to excuse the continued use of an indicator that, according to a 1996 paper by Karen Surman Paley, affords hopefuls “just enough rope for a hanging.”

Whether this year’s A-level results bring good news or bad for individual students, questions remain about whether the half a million or so personal statements written every year represent an efficient use of time, energy and resource, either for schools and colleges or for the higher education sector. A growing body of evidence suggests that non-academic indicators, rather than bringing equality to the selection process, further advantage those applicants already favoured by school type and socio-economic background.