Making A Statement

qLast week, the Sutton Trust published a Research Brief that I co-authored with the HE Access Network. The theme is a familiar one for me: the UCAS personal statement. I’ve blogged about it here and here, written a previous Sutton Trust report, and published findings in an academic journal and a book about global HE admissions practices.

Saint-Mary-s-School-oqLD3CThis study was a really interesting addition to the evidence because it was the first to compare how teachers at state schools and admission tutors at high-prestige universities read statements. The results were alarming: what teachers think make a good personal statement is a far cry from what universities are looking for.

The researjohnhumphMS2010_468x402ch attracted plenty of press attention, including an excellent opinion piece by Catherine Bennett for the Observer. Other print coverage included reports in The Sun, The Times, The Daily Telegraph and the Times Education Supplement. My interview on the BBC Radio Four’s Today programme is available here (listen from 52’45”) until February 26th 2016.

While I think personal statements offer a useful lens through which to view distributions of social capital and explore teenagers’ self-conceptualisations, I’m hoping this will be the last time I write about them. A review of their use in the application process – ideally as part of a wider review of the HE admissions in the UK – is long overdue.

 

Advertisements

What’s in a name?

Note: this piece was originally published as Anonymising UCAS forms is only a first step towards fair and discrimination-free university admissions on LSE’s Democratic Audit blog.

Glasgow University, Credit: Chor Ip, CC BY SA 2.0When pledging to make university applications “name-blind”, the Prime Minister yesterday cited research showing that top universities make offers to 55% of white applicants but only to 23% of black applicants. From 2017, universities will follow major employers that “recruit solely on merit” by offering anonymity to their applicants.

In many respects, this is a sensible move. Universities can hardly claim immunity from ‘unconscious bias’, and admissions processes could be seen to exemplify the “quieter and more subtle discrimination” that the Prime Minister wishes to address. However, those of us who have looked closely at the issue would argue that concealing candidates’ names does not go far enough.

In 2012, I authored a report for the Sutton Trust showing that the quality of UCAS personal statements could be predicted by applicants’ school type. For example, those from Sixth Form Colleges and Comprehensive Schools made several times more basic spelling and grammar errors than those from Grammar Schools and Independent Schools. Ethnicity was also a major factor, with British-Bangladeshi applicant making 2.29 errors per 1,000 words of statement, compared to white applicants’ 1.42 errors. All of the statements I examined were written young people who went on to achieve identical grades at A-level. The differences in their statements were not down to ability; they were down the amount of help and guidance available.

There are other ways in which our university application systems may reproduce existing forms of privilege. Candidates from the fee-paying sector are much more likely to mention the name of their school in their personal statement, even though this information is captured elsewhere in their application, perhaps as a means to accentuate their perceived fit for leading universities. Social capital is demonstrated through prestigious work placements, internships and job shadowing experiences; cultural capital through overseas trips and LAMDA examinations. Evidence suggests that interviews are no less discriminatory, with some candidates drilled extensively in how to perform under pressure while others remain intimidated by an unfamiliar, hostile environment.

So how should selective universities select when almost every indicator is potentially problematic and we cannot be trusted with even a candidate’s name? An extreme solution, favoured by some European countries, is to allocate places on over-subscribed courses based on a lottery for those who meet a minimum academic threshold. Other nations, notably the USA, ask for statements but offer greater reassurance to students from under-represented backgrounds that their application will be read in its appropriate context and the odd spelling mistake will not count against them. Few nations rely on the personal statement as much as the UK. However, with Independent Schools increasingly competing with one another on entry rates to leading universities, and with new markets emerging around the tutoring and coaching of applicants, the pressure to maintain the status quo is considerable.

The Prime Minister is right to say that the UK Higher Education sector needs to take a close look at why young people from some backgrounds can be disadvantaged in the application process. We also need to understand better why ethnicity predicts the likelihood of graduating with a higher degree award. But to stop at anonymised applications would be to pretend that the root of the problem is a handful of prejudiced admissions tutors. The candidate’s name is not the only issue. Indeed, this information may allow more sympathetic admissions tutors to make appropriate allowances. If the goal is to bring greater fairness to the process, we also need to think about more systemic issues, such as why offers are made on predicted rather than actual grades, how candidates’ attainment can be suitably contextualised, and why personal statements are given more prominence than any evidence suggests they are worth.

Have UCAS really revealed “the language tricks which will help you land that place at a top university”?

UCAS, the agency responsible for admissions to Higher Education in the UK, last week issued an analysis of the personal statements of 300,000 university applicants. They’d totted up the number of ‘passion-related’ words (such as  “love” and “explore”) and ‘career-oriented words’ (“salary”, “employable”, “job”, etc.) to see how frequencies differed according to the subjects for which candidates were applying.

1434994273-7fda9d77bd9580b55015eaf596aae1b3-600x450

Journalists seemed unsure what to make of the press release. “In all subjects, applicants used a mixture of career and passion-related words to set out their suitability,” reported The Telegraph.  “It’s not a huge surprise,” admitted another report, “that artsy students have cornered the passion market”. One reporter noted aimlessly that “despite the prominence of economics and economists over the last few years, students wanting to major in economics are among those least likely to mention either a ‘career’-related word”.

But none of the reports questioned whether such words are actually valued by admissions tutors. This gave the unfortunate impression that all applicants needed to do was use the right vocabulary and their place at university would be secure. The Mirror even promised to reveal “how to strike the balance between ‘passion and purpose’ to NAIL your written application”.UCAS_personal_stat_3350671a

UCAS, of course, can hardly be blamed for such misreporting. But it’s no secret that admissions tutors are rarely seduced by the language of love:

Dr Hilary Hinds, an admissions tutor from the English department at Lancaster University, finds clichés such as “passionate about literature” and “I’ve loved books for as long as I can remember” dull and predictable. “Demonstrate it rather than claim it,” she says. (The Guardian, 10.07.15)

My own research into the language of personal statements confirms that Dr Hinds isn’t alone. Words like “passion” and “love” are used more by applicants from state schools than those from independent schools, and they correlate negatively with the likelihood of acceptance by higher-prestige universities.

The press interest generated by the UCAS study was vacuous (at best) and specious (at worst). A far better use of the large, rich, not-publicly-available database would have been to identify patterns of use according to factors known to affect candidates’ chances of success, such as ethnicity and socio-economic status.

The Role of Ethnicity in Admissions to Russell Group Universities


(Note: I published this piece first on the British Educational Research Association’s Respecting Children & Young People blog on 17.03.15…)


a18178551_s

Here’s an excerpt from a UCAS personal statement written recently by an applicant to a Russell Group university:

“There are various times where I have been a team member such as in hockey, this is where we have to understand our team member’s strengths and weaknesses to evaluate best positions, it makes us understand that one’s ability may be skilful but can always be tackled by two. We had to quickly judge aspects; we also understood how goals and motivation can go through team members, as high motivation can motivate another.”

Within the excerpt, some details have been altered to protect the applicant’s identity. However, the writing style is unchanged and captures that of the whole statement.

A natural first response is that the applicant doesn’t belong at a high-prestige institution: the text is poorly punctuated, with muddled content, and reads as though it were thrown together at the last moment. Thank goodness for UCAS personal statements, one might conclude, for allowing universities additional evidence on which to make important selection decisions.

Except things aren’t quite so straightforward.

First, note that this applicant went on to receive A-level grades that were sufficient to gain entry to the courses for which she applied. This suggests that her personal statement difficulties were not caused by a lack of academic ability so much as confusion about what was required. Second, studies like this one, this one and this one, question whether personal statements are really of much value in predicting students’ subsequent performance at university anyway. Third, the applicant was educated in the state sector, and evidence suggests that she may therefore have had limited access to the kind of high quality information, advice and guidance available to many of her competitors. And fourth, the applicant is of British-Bangladeshi heritage, a group which fares poorly in admissions to high-prestige universities compared with White applicants of similar academic attainment.

In 2012, I undertook research for the Sutton Trust looking at how university applicants from different backgrounds set about the task of writing their personal statement. My primary focus was on school type, and I discovered that applicants from Sixth Form Colleges and Comprehensive Schools were much more likely to make basic language errors (spelling mistakes, apostrophe misuse, etc.) than those from Grammar schools and Independent Schools. Workplace experience could also be predicted by school type, with some applicants able to list up to a dozen placements at flash companies while others struggled to make a Saturday job sound relevant to their chosen course of study.

I’ve since returned to the data to find out whether personal statements also differ according to the ethnicity of the applicant. On average, I discovered, British-Bangladeshi applicants make 2.29 clear linguistic errors per 1,000 words of statement, compared to White applicants’ 1.42 errors. British-Bangladeshi applicants are also low on meaningful work-related activity, averaging 1.57 per statement compared to 2.32 for White applicants (where ‘meaningful’ means undertaken for genuine vocational experience rather than for cash – all such activities were blind coded by two text analysts). The total sample size is 327, and all of the statements were submitted by students who would go on to achieve identical A-level grades.

What prompted me to carry out these new counts was the graph below, based on UCAS data analysis by Durham University’s Vikki Boliver. This analysis showed that applicants’ chances of getting an offer from a Russell Group university differed markedly according to their ethnicity. British-Bangladeshi students have a 42.6% likelihood; White British students a 52.0% likelihood.

Table Russell Group Applications

Source: Boliver in Alexander and Arday (2015), via Economics of HE

As Parel and Boliver note, ethnicity actually trumps school type as a predictor of admission to leading UK universities. Figures obtained from Oxford University by the Guardian in 2013 under the Freedom of Information Act indicated that “43% of White students who went on to receive three or more A* grades at A-level got offers, compared with 22.1% of minority students”.

Such differentials can be explained in many ways. Even though Boliver’s data controls for ‘facilitating’ subjects (those the Russell Group claim are preferred by universities), it could be that Black, Asian and Minority Ethnic (BAME) applicants take inappropriate combinations for the degree courses to which they apply. It has also been implied that BAME applicants tend toward oversubscribed subjects, such as medicine or law. However, as Boliver continues to point out, the kind of individual-level data needed to develop a clear picture of why differentials arise is increasingly being restricted because, supposedly, it “presents a high risk of individuals’ personal details being disclosed.”

1astatementDoes the application process itself discriminate against some applicants? I’ve written before about how the UCAS personal statement is, in many respects, a flawed indicator, and I’ve also responded to specific arguments made in its defence. Other researchers have noted similar problems with university interviews (see Burke and McManus on would-be Art & Design students who make the mistake of citing hip-hop as an influence, or Zimdars on a tendency for admissions tutors at Oxford University to recruit in their own image).

Such studies raise awkward but crucial questions about what exactly non-academic indicators are supposed to indicate. Is the personal statement simply an “opportunity to tell us about yourself”, as UCAS benignly describes it, or is the real goal to flaunt one’s cultural and social capital, signalling what Bourdieu characterises as the “dispositions to be, and above all to become, ‘one of us’”?

The Observer’s Barbara Ellen notes that certain kinds of applicant are much more likely to “speak uni” and be able to “decode the foreign language of the admissions process.” And Pilkington reminds us that BAME applicants are “entitled to know that they will not be subject to potentially indirect − or indeed direct − discriminatory practices in an institution’s admissions processes.”

However, the problem may well go beyond the level of institution. A key structural barrier seems to be an admissions process that assumes all applicants are equally equipped to understand (and have sufficient support to meet) its veiled requirements. The personal statement purports to help university admission tutors make informed choices based on holistic evidence, but may actually reproduce White and other forms of privilege at the point of application.

The UCAS personal statement: “just enough rope for a hanging”?

(Note: I published this piece first in Research Fortnight on A-level results day, 2014…)

eco9791

For many students, A-level results day is the culmination of a lengthy and complex university application process involving open days, UCAS forms, grade predictions, school references and admissions interviews. For some, the game will have started even earlier, with awards, experiences and sporting endeavours accumulated throughout adolescence on the promise that “this’ll look good on your personal statement.”

But Britain’s approach to undergraduate admissions is the exception, not the norm. Elsewhere in the world, systems tend to be more sensitive to uneven distributions of social capital, extra-curricular opportunities and guidance from friends, family and school. At a recent international conference, delegates from Northern European countries began to chuckle when I explained that applicants to British universities are given 4,000 characters to write freely about themselves. They felt sure such an indicator would reveal less about the applicant than their social and cultural background. Similar suspicions were raised in the 2004 Schwartz Report, which warned that “some staff and parents advise to the extent that the personal statement cannot be seen as the applicant’s own work.”

The alternative for many nations is appropriately contextualised attainment data. Rather than turn to non-academic criteria to choose between eligible applicants, universities in both Holland and Greece have experimented with lotteries to distribute places on oversubscribed programmes. Meet your course’s entry requirements and your name goes into a hat with every other applicant who reaches that threshold. No gaming the system with interview coaching, LAMDA examinations or extravagant work placements.

Even in the largely decentralised US admissions system, applicants must respond to ‘prompts’, including those below, that are designed to make the process fairer. Such prompts, rather than encourage applicants to cash in on past opportunities, call for focused, candid and reflective responses. College websites in the US reassure applicants that statements will be read “in their true context” (Princeton), and justify the use of non-academic indicators in terms of a “long history of encouraging diversity” (Brown).

essaysca
My interest in the higher education admissions was sparked by Vikki Boliver’s 2012 finding that “applicants to Russell Group universities from state schools are less than two-thirds as likely to receive offers as privately educated applicants,” a differential that wasn’t attributable to ‘facilitating’ subjects. Last month, a study of 50,000 students by the University of Bristol confirmed similar concerns in relation to ethnic minority applicants. For example, for every 100 candidates of Pakistani ethnicity, seven fewer offers were made than for 100 equivalent white British candidates, a pattern that “could not be fully explained by differences in academic attainment or patterns of application.”

Last year, I published Sutton Trust funded research demonstrating that equal-attainment students submitted very different personal statements. Basic linguistic errors (such as spelling errors and apostrophe misuse) were almost three times more common in statements submitted by applicants attending sixth form colleges than by those attending independent schools. For some candidates, work experience meant school-facilitated day trips and paid Saturday jobs; for others, it involved shadowing public figures and undertaking high-prestige placements. My findings resonated with a 2010 survey by the Education and Employers Taskforce showing that 42% of young people from independent schools felt their work experience helped them get into university, as opposed to only 25% of those from comprehensive schools.

ucasapplications_600x300

Of course, it’s possible that admission tutors see through advantages of school type to the candidate beneath. “Does the Sutton Trust really think I’m taken in by slick expensive personal statements?” tweeted Prof Mary Beard when my report first appeared. But last year The Times reported that statements were now regarded as “worthless” by many tutors, and the recent Pearson Think Tank report, (Un)Informed Choices, concluded that “the use of personal statements should be ended.” Unless all students’ applications are judged by staff with the experience and skill to separate privilege from potential, it’s difficult to excuse the continued use of an indicator that, according to a 1996 paper by Karen Surman Paley, affords hopefuls “just enough rope for a hanging.”

Whether this year’s A-level results bring good news or bad for individual students, questions remain about whether the half a million or so personal statements written every year represent an efficient use of time, energy and resource, either for schools and colleges or for the higher education sector. A growing body of evidence suggests that non-academic indicators, rather than bringing equality to the selection process, further advantage those applicants already favoured by school type and socio-economic background.

Revisiting Debates about the UCAS Personal Statement

This post was first published on Oct 14th 2013 by “Manchester Policy Blogs“.

Revisiting debates about the UCAS personal statement

With the first UCAS deadline of the academic year looming, thousands of University hopefuls are putting the finishing touches to their personal statements. But growing evidence points towards the current process favouring some applicants more than others – and it may be time for a radical overhaul, according to Dr Steve Jones.

“The UCAS personal statement is academically irrelevant and biased against poorer students,” ran the headline of one Telegraph blog last month.

According to its author, paying a private company to write your statement now costs between £100 and £200, and the whole thing is little more than “an exercise in spin”. Meanwhile, The Times report that tutors “often ignore students’ personal statements,” describing the indicator as “worthless”.

Perhaps more significantly, last week saw the publication of a Pearson Think Tank report called “(Un)Informed Choices“. The executive summary was surprisingly frank in its recommendation: “the use of personal statements should be ended.”

uninformed-choices

The debate has interested me since I was commissioned by the Sutton Trust last year to collect new evidence about the personal statement.

My findings were stark. Basic writing errors (like misspelling and apostrophe misuse) were three times more common among applicants from state schools and sixth form colleges as those from independent schools. There were also big differences when it came to work experience: independent school applicants had lots more, and it tended to be high prestige.

All of the statements I looked at were written by students with the same A-level results, so I wondered whether the textual differences offered a partial explanation for the unfair outcomes reported in UK admissions processes more broadly.

For example, research at Durham University has shown that state school applicants are only 60% as likely to be made an offer by Russell Group universities as independent school applicants with the same grades in “facilitating” subjects.

So why hasn’t the personal statement been binned by UCAS already? In my experience, there are five main lines of defence:

1. “Admissions Tutors aren’t taken in by slick expensive personal statements”

This was the response of Cambridge University’s Prof. Mary Beard to my research, and I think it’s a very reasonable point. Any experienced reader of statements will have well-honed “crap detection” skills. Who’s to say our admissions tutors aren’t seeing right through the fancy work placements and LAMDA successes? The problem is, as a sector, we’re neither consistent nor transparent in how personal statements are read. Sometimes they’re given close, critical attention; sometimes not. Either way, we keep schtum about the criteria we use and the weight we attach to them.

2. “There’s never an excuse for spelling mistakes, is there?”

This point was made to me twice by a BBC TV newscaster. The answer is no, there’s never an excuse. However, if you have lots of people to proofread your statement and you’re repeatedly told it’s something you’ve got to get right, chances are you’ll take a bit more care. The sixth form college applicant who made twelve basic language errors in his statement wasn’t stupid – his attainment record proves that – he just didn’t understand how much those mistakes could count against him.

3. “We like to be holistic in the way we select our students”

It’s never easy to argue with the word ‘holistic‘, but there’s no advantage to using lots of indicators unless every one is bringing fairness to the selection process. Perhaps a small amount of appropriately contextualized attainment evidence is actually more equitable than a wide range of hazy non-academic indicators?

4. “We use the personal statement as a starting point for interview questions”

The Oxbridge colleges sometimes use this argument, but it isn’t a very strong one because most UK university applicants aren’t interviewed for any of the degree programmes to which they apply. And, for those that are, surely it’s not beyond interview panels to formulate their own questions? Besides, the most elite universities are often the sniffiest about statements: we don’t want “second-rate historians who happen to play the flute,” says Oxford’s head of admissions; “no tutor believes [the personal statement] to be the sole work of the applicant any more,” says his former counterpart at Cambridge.

5. “Actually, we know personal statements aren’t a reliable, and we don’t bother reading them”

This point is made regularly, but with half a million statements written every year, maybe it’s time someone mentioned it to the young people who stress and sweat over writing them?

There’s room for compromise, of course. In 2004, the Schwartz Report suggested redesigning the UCAS application form to include prompts that elicit more directly relevant information in a more concise fashion. Those applicants with the social and cultural capital to secure the best work experience and highest prestige extra-curricular experience would then have less opportunity to cash in on their good fortune.

ucasapplications_600x300

But for the last word on the subject, here’s a member of admissions staff (quoted anonymously in the Pearson report) on just how much difference school type can make to the personal statement:

“I’ve spoken to heads of private schools about the question of how much help they give students in writing statements. They say ‘well, they’re paying £7,000 a term, of course we give them a lot of help, that’s what they’re paying for’. And yet you see statements from what [are] potentially good students from schools which have not got a lot of experience of sending their students to HE, and they’re not very good because no-one knows what to do, how to do it.”