The University Game

I’m looking forward to giving a Sarah Fielden seminar on May 11th at the University of Manchester. All welcome. Further details here.

333

Making A Statement

qLast week, the Sutton Trust published a Research Brief that I co-authored with the HE Access Network. The theme is a familiar one for me: the UCAS personal statement. I’ve blogged about it here and here, written a previous Sutton Trust report, and published findings in an academic journal and a book about global HE admissions practices.

Saint-Mary-s-School-oqLD3CThis study was a really interesting addition to the evidence because it was the first to compare how teachers at state schools and admission tutors at high-prestige universities read statements. The results were alarming: what teachers think make a good personal statement is a far cry from what universities are looking for.

The researjohnhumphMS2010_468x402ch attracted plenty of press attention, including an excellent opinion piece by Catherine Bennett for the Observer. Other print coverage included reports in The Sun, The Times, The Daily Telegraph and the Times Education Supplement. My interview on the BBC Radio Four’s Today programme is available here (listen from 52’45”) until February 26th 2016.

While I think personal statements offer a useful lens through which to view distributions of social capital and explore teenagers’ self-conceptualisations, I’m hoping this will be the last time I write about them. A review of their use in the application process – ideally as part of a wider review of the HE admissions in the UK – is long overdue.

 

Why research and teaching need to be reintegrated as well as rebalanced

Note: this piece was originally published on the All-Party Parliamentary University Group website. Further details of the presentation I gave to the Group are here.

Among the stronger arguments made in the government’s Green Paper is that a ‘rebalancing’ of research and teaching in Higher Education is needed. As a sector, we’ve become accustomed to close scrutiny of our research while our teaching has largely remained unaudited, sometimes reliant on the dedication of personally committed academics. But there’s an equally strong case to be made for 4research and teaching to be reintegrated. What makes students’ learning at university different from earlier, more instrumental educational experiences is the opportunity to be immersed in a culture of scholarly enquiry and research advancement, to learn first-hand from those leading their field, and to conspire in the creation of new knowledge. In measuring teaching, we must take care not to set it further adrift from research.

For any teaching audit to benefit the sector, buy-in from both students and academics is vital. Attempts to frame the Teaching Excellence Framework (TEF) as siding with long-suffering undergraduates are undermined by ‘principled disengagement’ from the National Union of Students. The link with fees makes the TEF the hardest of sells to the ‘consumer’ it supposedly empowers, especially now maintenance grants have become loans and repayment thresholds are frozen.

For academics, the risk is that separate audits for research and teaching put the sector in a state of perpetual preparation and further fuel the kind of game-playing ‘industries’ that the Green Paper rightly chides. A better integrated, lighter-touch framework might allow more time for universities to do what matters, instead of just reporting it in the most favourable terms possible.

2015-LT-conference-37-1038x576

The questions a TEF might most usefully ask of the Higher Education sector are those that encourage us to make better use of our data, communicate more clearly with applicants, and draw on our own research to ensure that every student receives the teaching and support that’s best suited to their needs. For example, we know all about key outcome differentials, such as the relative under-attainment of Black and Minority Ethnic students compared to White students. But how do we address them? Part of the answer surely involves research. We need to understand better how cohort and staff diversity, curriculum design and campus culture affect performance.

201CKm_cEcUAAEB3arIndeed, one problem with relying on metrics is that some are such distant relations of teaching quality that they’d barely recognise one another. Graduates salaries, for example, are predicted much more by subject choice, university prestige and social capital than by how effective your lecturers were. Similarly, high satisfaction scores can be achieved by pleasing students rather than challenging them. In so diverse a sector, metrics can never tell the whole story.

Would-be students will benefit far more if universities – and then disciplines – created their own narratives. Many young people find their school-to-university transition difficult to negotiate and would benefit from clear, evidence-based guidance about the pedagogical approach and distinctiveness of individual courses.

2imagesThe Impact and Environment Statements used in the Research Excellence Framework (REF) offer useful potential templates. Teaching impact could be evidenced by localised measurements of learning gain; teaching environment by learning culture and staffing strategy, as well as by facilities and extra-curricular learning opportunities. Emerging narratives would be accompanied by relevant supporting evidence, such as student attendance at research seminars, the ratio of contact hours spent with senior academics relative to teaching assistants, the retention and performance of WP students relative to non-WP students, etc.

Eventually, any ‘excellence’ framework will get gamed. What’s arguably more important is the direction in which it nudges the sector and the behaviours it implicitly encourages. As universities grow more confident in their own research into Higher Education and articulate richer pedagogical narratives, the TEF’s role may develop into one of overseeing panel assessment rather than imposing metrics of its own. A low-maintenance REF and low-maintenance TEF could evolve and coalesce according to consistent underlying methodological principles, and in ways that allow research and teaching to complement, not compete with, one another.

Metaphors of HE Access: time to mind our language?

Like applying a bandage to lung cancer.”

That’s how Dr Martin Stephen last week described the idea of allowing disadvantaged students into top universities when they’re an A-level grade or two below the usual threshold.

Dr Martin Stephen is a former chairman of the Headmasters’ and Headmistresses’ Conference (HHC) and ex high master of St Paul’s School in London. He was responding to Bahram Bekhradnia expressing dismay that, in his time as director of the Higher Education Policy Institute, the top universities had remained “as socially exclusive as ever.

Mr Bekhradnia suggested that the UK should follow US institutions’ lead in seeking to create cohorts that “represent wider society as far as possible,” obsessing less about academic attainment at the point of entry.

qIuFEsn8W7GUR6h4Tali2Tl72eJkfbmt4t8yenImKBVvK0kTmF0xjctABnaLJIm9

For Dr Stephens, such a move would let low-achieving schools ‘off the hook’. It’s social engineering gone made, or whatever.

Our schools are not helping disadvantaged children to achieve respectable grades and these things don’t do anything about that problem,” he complained.

There are several problems with this position. First, a good deal of one-way evidence tells us that state schools pupils actually outperform independent school students once they reach university. Second, we know that state school applicants are less likely to be offered a place at Russell Group universities than independent school applicants with the same grades, even when ‘facilitating subjects’ are controlled for. Third, it is questionable whether low-achieving schools are incentivised by their students’ progression rates to top universities in anything like the way Dr Stephens implies.

But more disturbing than the views being represented are the metaphors increasingly being traded by those with vested interests.

Is academic under-performance, and the schooling system responsible for it, really like lung cancer? Or are such schools actually working hard to raise attainment among young people with multiple disadvantages, social problems and often chaotic home lives? The latest PISA findings suggest that socioeconomic background is the key determinant of educational success, not school type.

Note the similarly belligerent response to a recent report from the Institute for Fiscal Studies, who found that England’s grammar schools were now four times more likely to admit private school children than those on free school meals. This time it was the turn of Robert McCartney, chairman of the National Grammar School Association (NGSA), to return fire:

“Many, many parents from deprived areas, including what is generally called the dependency classes, are essentially not particularly interested in any form of academic education,” said Mr McCartney. “Their interests are directed towards pop culture, sports.”

indexer

Naturally, the HHC, NGSA and other such organisation are bound to defend their market edge. Many independent and selective schools actively recruit on promises of entry to prestigious universities.

But should this defence spill over into unsubstantiated slurs against those from less advantaged communities? Poorer parents share the same aspirations for their children as their wealthier counterparts. It helps no-one to liken low-attainment schools to horrible diseases.

Let’s debate the evidence and leave the name-calling in the playground.

Revisiting Debates about the UCAS Personal Statement

This post was first published on Oct 14th 2013 by “Manchester Policy Blogs“.

Revisiting debates about the UCAS personal statement

With the first UCAS deadline of the academic year looming, thousands of University hopefuls are putting the finishing touches to their personal statements. But growing evidence points towards the current process favouring some applicants more than others – and it may be time for a radical overhaul, according to Dr Steve Jones.

“The UCAS personal statement is academically irrelevant and biased against poorer students,” ran the headline of one Telegraph blog last month.

According to its author, paying a private company to write your statement now costs between £100 and £200, and the whole thing is little more than “an exercise in spin”. Meanwhile, The Times report that tutors “often ignore students’ personal statements,” describing the indicator as “worthless”.

Perhaps more significantly, last week saw the publication of a Pearson Think Tank report called “(Un)Informed Choices“. The executive summary was surprisingly frank in its recommendation: “the use of personal statements should be ended.”

uninformed-choices

The debate has interested me since I was commissioned by the Sutton Trust last year to collect new evidence about the personal statement.

My findings were stark. Basic writing errors (like misspelling and apostrophe misuse) were three times more common among applicants from state schools and sixth form colleges as those from independent schools. There were also big differences when it came to work experience: independent school applicants had lots more, and it tended to be high prestige.

All of the statements I looked at were written by students with the same A-level results, so I wondered whether the textual differences offered a partial explanation for the unfair outcomes reported in UK admissions processes more broadly.

For example, research at Durham University has shown that state school applicants are only 60% as likely to be made an offer by Russell Group universities as independent school applicants with the same grades in “facilitating” subjects.

So why hasn’t the personal statement been binned by UCAS already? In my experience, there are five main lines of defence:

1. “Admissions Tutors aren’t taken in by slick expensive personal statements”

This was the response of Cambridge University’s Prof. Mary Beard to my research, and I think it’s a very reasonable point. Any experienced reader of statements will have well-honed “crap detection” skills. Who’s to say our admissions tutors aren’t seeing right through the fancy work placements and LAMDA successes? The problem is, as a sector, we’re neither consistent nor transparent in how personal statements are read. Sometimes they’re given close, critical attention; sometimes not. Either way, we keep schtum about the criteria we use and the weight we attach to them.

2. “There’s never an excuse for spelling mistakes, is there?”

This point was made to me twice by a BBC TV newscaster. The answer is no, there’s never an excuse. However, if you have lots of people to proofread your statement and you’re repeatedly told it’s something you’ve got to get right, chances are you’ll take a bit more care. The sixth form college applicant who made twelve basic language errors in his statement wasn’t stupid – his attainment record proves that – he just didn’t understand how much those mistakes could count against him.

3. “We like to be holistic in the way we select our students”

It’s never easy to argue with the word ‘holistic‘, but there’s no advantage to using lots of indicators unless every one is bringing fairness to the selection process. Perhaps a small amount of appropriately contextualized attainment evidence is actually more equitable than a wide range of hazy non-academic indicators?

4. “We use the personal statement as a starting point for interview questions”

The Oxbridge colleges sometimes use this argument, but it isn’t a very strong one because most UK university applicants aren’t interviewed for any of the degree programmes to which they apply. And, for those that are, surely it’s not beyond interview panels to formulate their own questions? Besides, the most elite universities are often the sniffiest about statements: we don’t want “second-rate historians who happen to play the flute,” says Oxford’s head of admissions; “no tutor believes [the personal statement] to be the sole work of the applicant any more,” says his former counterpart at Cambridge.

5. “Actually, we know personal statements aren’t a reliable, and we don’t bother reading them”

This point is made regularly, but with half a million statements written every year, maybe it’s time someone mentioned it to the young people who stress and sweat over writing them?

There’s room for compromise, of course. In 2004, the Schwartz Report suggested redesigning the UCAS application form to include prompts that elicit more directly relevant information in a more concise fashion. Those applicants with the social and cultural capital to secure the best work experience and highest prestige extra-curricular experience would then have less opportunity to cash in on their good fortune.

ucasapplications_600x300

But for the last word on the subject, here’s a member of admissions staff (quoted anonymously in the Pearson report) on just how much difference school type can make to the personal statement:

“I’ve spoken to heads of private schools about the question of how much help they give students in writing statements. They say ‘well, they’re paying £7,000 a term, of course we give them a lot of help, that’s what they’re paying for’. And yet you see statements from what [are] potentially good students from schools which have not got a lot of experience of sending their students to HE, and they’re not very good because no-one knows what to do, how to do it.”

Why are American educators angrier than their British counterparts?

Last month, I was lucky enough to speak at the American Educational Research Association’s annual conference (AERA13), where the theme was Education and Poverty. Some of the research presented was utterly compelling: carefully-collected, long-term, large-scale empirical evidence, all pointing towards growing inequality of opportunity.  Young people are hungry for education, the argument went, but the US schooling system lets them down.

In the area I’m most interested in – access to higher education – several speakers talked compellingly about the problems faced by first generation applicants in accessing financial aid, getting appropriate advice, and negotiating the admissions process. The conference also screened a number of films, including this brilliant one about four ‘undocumented’ students and their attempts to reach college.

The visit of US Secretary of Education, Arne Duncan, was most controversial. Protests took place outside the venue and, inside, Not In My Name flyers were waved throughout. Duncan’s love of school testing does not sit well with AERA members. His defence was “Chicago-style nonsense,” according to one entertaining report.

The conference wasn’t all downbeat. Delegates clearly wanted to make education a fun and productive time for all young people. Social media was repeatedly cited as possible social leveler, gaming as a fresh way to engage young people on their own terms, and wrap-around policies as essential for less advantaged children.

At times, I wondered what the conference would make of educational policy in the UK, which seems not only to ignore empirical evidence but to purposely move in the opposite direction? Without wishing to generalise, AERA seems more politically aware (or maybe just political) than BERA.

I left thinking that if Obama’s Duncan gets this much stick, maybe us UK educationalists go a little too easy on Cameron’s Gove?

Does the ‘Attainment Gap’ get Russell Group universities off the hook?

In the UK, Russell Group universities are the posh ones: institutions with the highest entry grade requirements, the highest graduate salaries and the most prestige. There’s 24 of them, and the group take its name from the Hotel Russell, which currently ranks 455th of 1,079 hotels in London by Trip Advisor (“bathroom not hygienic,” says Jan from Ghent, “there was some brown substance in the corner of the window”).

The question of who gets into Russell Group university is, for obvious reasons, an important one. According the UCAS application figures for 2013/14, “18-year-olds from the most advantaged areas are three times more likely to apply to higher education than those from the most disadvantaged areas, and entry rates to institutions that require high grades are typically six to nine times greater for applicants from advantaged areas.”

‘Six to nine times greater’ sounds an awful lot. However, the Russell Group do have an explanation: “The main problem is that students who come from low-income backgrounds and/or who have attended comprehensive schools are much less likely to achieve the highest grades than those who are from more advantaged backgrounds and who have been to independent or grammar schools,” explains Wendy Piatt, Director General of the Group. And she’s right: go to a private school and you’re four times more likely to get AAA in your A-levels than you would be at a comprehensive. “Universities simply cannot solve these problems alone,” says Dr Piatt.

Among the research supporting the ‘attainment gap’ is a paper by Haroon Chowdry and colleagues at the Institute for Fiscal Studies. It’s  a fascinating study, following two groups of English pupils from the age of 11, and noting how their academic performance at each stage of school testing affects their likelihood of participation. Findings suggest that differences in participation rates across the social classes “are substantially reduced once prior achievement is included”. They add that:

“Poor achievement in secondary schools is more important in explaining lower HE participation rates among pupils from low socio-economic backgrounds than barriers arising at the point of entry to HE. These findings are consistent with the need for earlier policy intervention to raise HE participation rates among pupils from low socio-economic backgrounds.”

Few would argue with the second point. What’s most helpful for children of low socio-economic status is intervention at an early age. You can’t correct for years of educational disadvantage as a UCAS deadline is approaching. The first point is also true – low attainment is undoubtedly the UK’s biggest barrier to participation. However, there is a tendency for Chowdry and Co to gloss over the differences that still remain at the point of entry.

Take this finding: once all prior attainment is taken into account, girls from the lowest socioeconomic quintile are 5.3% less likely to enter HE than girls from the highest socioeconomic quintile. Boys are 4.1% less likely. If you want a place at a Russell Group university, your odds are reduced by 4.3% and 2.5% respectively. Similar findings were reported earlier this week by Vikki Boliver for equal-attainment applicants in a survey of UCAS applicants from 1996 to 2006. Applicants from lower class backgrounds and from state schools were less likely to be offered a place at Russell Group universities than their comparably qualified counterparts from higher class backgrounds and private schools (even when ‘facilitating’ subjects were controlled for, despite the spin put on the research by some).

For Chowdry and his fellows authors, the point-of-entry gap between applicants of different socioeconomic status is “modest”. Encouraging less well-off students to apply to university at the age of 18 is therefore “unlikely to have a major impact” on participation. In relative terms, of course, this is perfectly true – improving attainment for all young people of lower socioeconomic status would make a bigger difference than focusing on the small proportion who defy the odds and get good grades.

But doesn’t this line of thinking get HE off the hook a little too easily? What of the thousands of high-achieving young people who aren’t making it to a top university each year? Jonathan Portes makes the same points about Chris Cook’s interpretation of the Oxford University data. He also uses the graph below to show that, if these young people did participate, they’d probably outperform students from more privileged backgrounds.

We cannot offer places to those who do not apply,” says Dr Piatt. True. But there’s a growing body of research that suggests those who do apply to Russell Group universities are not always treated equally. The ‘attainment gap’ certainly isn’t to blame for that.