Could universities learn from the TEF’s advocates how better to influence public discourses?

Note: this piece was originally published here on the Sociological Review‘s website. It is co-authored by my Manchester Institution of Education colleagues, Steven Courtney & Ruth McGinity.9Public-Speaking

The Teaching Excellence Framework (TEF) is no easy sell. For a sector already awash with audits, metrics and league tables, the prospect of new measurements – especially ones underpinned by a brazenly market-driven ideology – is difficult to embrace. The ways in which the TEF is discursively framed therefore become crucial to its reception, and the strategies used offer a ready case study into how policymakers co-opt, cajole and (if all else fails) coerce their way to implementation. In an age where headlines matter more than procedural detail, and media messaging more than academic buy-in, the success of higher education policy can hinge on how convincingly it is spun. Wittgenstein’s notions of ‘language games’ are becoming as relevant to higher education research as Bourdieu’s theories of class distinction.

That’s not to implJo-Johnsony that the TEF is without any substantive arguments of its own. When the current Minister for Universities and Science, Jo Johnson, talks about “rebalancing” teaching and learning, few would argue that the scales are in need of no correction. When one of his predecessors, David Willetts, characterised teaching as “by far the weakest aspect of English higher education,” we grimaced, but we couldn’t deny that it has often been over-shadowed by research imperatives. Indeed, as TEF enthusiasts point out, only 37% of undergraduates now report that their degree represents good value-for-money, down from 53% just four years ago.

But such statistics should be treated with caution. First, because they assume commercial paradigms and implicitly deny any notion of university as a public good; in other words, once value-for-money becomes the currency, what counts is not society’s collective advancement but the individual’s net return on their financial investment. And second, because it’s wholly disingenuous to bash university teaching using value-for-money indicators; many students reporting poor value will be doing so because of the extraordinary hike in fees rather than any deterioration in their learning experience.

The reliance on metrics means that, all too frequently, universities are positioned as either reform averse (far too ivory tower’d to understand what their students want) or greedy and self-interested (seeking to preserve a bloated, over-protected sector from the market’s natural justice). The TEF, by contrast, is framed by its supporters as “strengthen[ing] the position of students and prospective students vis-à-vis these powerful institutions” (Emran Mian, 06.11.15). Jo Johnson goes further, claiming that “students are looking critically at what they get for their investment, and so must we, as a government, on behalf of taxpayers” (01.07.15). The government thus become plucky Davids slaying the Goliaths of an outmoded, authoritarian higher education sector. No matter that the National Union of Students passed a motion in favour of “principled disengagement” from the TEF, and has threatened to sabotage next year’s National Student Survey in protest.

9indexAnti-university discourses are legitimised through mass reiteration: the ingeniously named Office for Students sounds like it will champion and defend learners’ rights; Study UK emerges as a “national representative body for independent providers of higher education”; a methodologically flawed but widely reported survey of staff at independent schools finds they don’t much like the sound of how undergraduates are taught; “too many universities teach pointless degrees that offer nothing to their students,” runs a headline in The Telegraph (Fraser Nelson, 15.04.16). Space rarely opens up to question why one of the economy’s most consistently high-performing sectors (a “world leader, with four universities in the global top ten,” according to the government’s 2016 White Paper) should model itself, both commercially and pedagogically, on a private school system.

The co-opting is relentless, and stressed-out university staff eventually turn on the very undergraduates who should rightly be their allies. “My students have paid £9,000 and now they think they own me,” writes an anonymous academic in the Guardian’s Higher Education Network (18.12.15). Undergraduates become pawns in a very public game of chess, discursively courted by government and universities alike, but faced with the same unprecedented levels of debt regardless of allegiance.

9banksy-twitter-fight1On the day that the government’s White Paper was published, the Minister busied himself on Twitter, disseminating responses to the document from stakeholders such as the Confederation of British Industry (“it’s good that proposals have taken on board the business view”), the University of Buckingham’s Vice Chancellor (“full marks to the minister for not succumbing to pressure from university traditionalists”) and the editor of Conservative Home (“if more would-be students had better information about future earnings they might not go to University at all”). Some might claim that what’s important is the detail of the policy, not the social media clamour surrounding it. However, as quick-to-tweet ministers probably realise, to own the discourse is to the win the argument.

And so the TEF wheedles its way into the sector, despite the perverse incentive of inflationary fee rises and the likelihood of an already-stratified sector being divided further. The prospect of an “outstanding” rating (rather than merely an “excellent” one) will seduce those institutions best equipped to play the game. And despite Green Paper pledges to “address the ‘industries’ that some institutions create around the REF and the people who promote and encourage these behaviours,” similar activities are sure to emerge around the TEF, as numbers are crunched, metrics optimised and self-glorifying statements written.

9shutterstock_248785768

Meanwhile, so-called “challenger institutions”, summarily checked, enter the market. Public discourses frame them as high-quality food providers, and question why they must seek permission of their corporate competitors to compete (“akin to Byron Burgers having to ask permission of McDonald’s to open up a new restaurant,” Jo Johnson, 09.09.15). Their stakes are small: low start-up costs and minimal regulatory oversight. The bigger gamble is that taken by the UK higher education sector: centuries of hard-won reputational gain wagered on the untested principle that new providers will show a crusty establishment just how HE-level teaching should be done.

If the sector were better able to speak as a united profession, public opinion may be more inclined to lean in its direction. The best way to rebalance research and teaching is probably to obsess less about measuring the former rather than to obsess more about measuring the latter. But greater coordination and discursive agility is required to persuade those outside academia how damaging an unchecked marketisation agenda might ultimately prove. Students need winning over with evidence, not assurances, that their learning is our top priority; the role of research in pedagogy needs defending more stoutly; and the value of higher education to wider society needs articulating more forcefully and more often. Perhaps the sector could learn a thing or two from the TEF’s advocates about how to frame public discourses.

Advertisements

Why research and teaching need to be reintegrated as well as rebalanced

Note: this piece was originally published on the All-Party Parliamentary University Group website. Further details of the presentation I gave to the Group are here.

Among the stronger arguments made in the government’s Green Paper is that a ‘rebalancing’ of research and teaching in Higher Education is needed. As a sector, we’ve become accustomed to close scrutiny of our research while our teaching has largely remained unaudited, sometimes reliant on the dedication of personally committed academics. But there’s an equally strong case to be made for 4research and teaching to be reintegrated. What makes students’ learning at university different from earlier, more instrumental educational experiences is the opportunity to be immersed in a culture of scholarly enquiry and research advancement, to learn first-hand from those leading their field, and to conspire in the creation of new knowledge. In measuring teaching, we must take care not to set it further adrift from research.

For any teaching audit to benefit the sector, buy-in from both students and academics is vital. Attempts to frame the Teaching Excellence Framework (TEF) as siding with long-suffering undergraduates are undermined by ‘principled disengagement’ from the National Union of Students. The link with fees makes the TEF the hardest of sells to the ‘consumer’ it supposedly empowers, especially now maintenance grants have become loans and repayment thresholds are frozen.

For academics, the risk is that separate audits for research and teaching put the sector in a state of perpetual preparation and further fuel the kind of game-playing ‘industries’ that the Green Paper rightly chides. A better integrated, lighter-touch framework might allow more time for universities to do what matters, instead of just reporting it in the most favourable terms possible.

2015-LT-conference-37-1038x576

The questions a TEF might most usefully ask of the Higher Education sector are those that encourage us to make better use of our data, communicate more clearly with applicants, and draw on our own research to ensure that every student receives the teaching and support that’s best suited to their needs. For example, we know all about key outcome differentials, such as the relative under-attainment of Black and Minority Ethnic students compared to White students. But how do we address them? Part of the answer surely involves research. We need to understand better how cohort and staff diversity, curriculum design and campus culture affect performance.

201CKm_cEcUAAEB3arIndeed, one problem with relying on metrics is that some are such distant relations of teaching quality that they’d barely recognise one another. Graduates salaries, for example, are predicted much more by subject choice, university prestige and social capital than by how effective your lecturers were. Similarly, high satisfaction scores can be achieved by pleasing students rather than challenging them. In so diverse a sector, metrics can never tell the whole story.

Would-be students will benefit far more if universities – and then disciplines – created their own narratives. Many young people find their school-to-university transition difficult to negotiate and would benefit from clear, evidence-based guidance about the pedagogical approach and distinctiveness of individual courses.

2imagesThe Impact and Environment Statements used in the Research Excellence Framework (REF) offer useful potential templates. Teaching impact could be evidenced by localised measurements of learning gain; teaching environment by learning culture and staffing strategy, as well as by facilities and extra-curricular learning opportunities. Emerging narratives would be accompanied by relevant supporting evidence, such as student attendance at research seminars, the ratio of contact hours spent with senior academics relative to teaching assistants, the retention and performance of WP students relative to non-WP students, etc.

Eventually, any ‘excellence’ framework will get gamed. What’s arguably more important is the direction in which it nudges the sector and the behaviours it implicitly encourages. As universities grow more confident in their own research into Higher Education and articulate richer pedagogical narratives, the TEF’s role may develop into one of overseeing panel assessment rather than imposing metrics of its own. A low-maintenance REF and low-maintenance TEF could evolve and coalesce according to consistent underlying methodological principles, and in ways that allow research and teaching to complement, not compete with, one another.

“Fulfilling Our Potential”: what policymakers’ rhetoric reveals about the future of Higher Education

Note: this piece was originally published on LSE’s British Politics and Policy blog (September 29th 2015)

Jo-JohnsonSpeaking earlier this month at the Universities UK Annual Conference, the Minister for Universities and Science, Jo Johnson, offered few new pointers about the upcoming Teaching Excellence Framework (TEF) but may have revealed more – not necessarily intentionally – about the government’s broader view of the Higher Education sector.

Take the comment about new providers’ courses being validated by established universities. According to the Minister, it’s “akin to Byron [Burgers] having to ask permission of McDonald’s to open up a new restaurant.” The point being made is clearly about perceived anti-competitive practices, with eager young upstarts being denied market entry by larger, entrenched operators. However, the metaphor is a curious one, with the new providers framed as purveyors of posh hamburgers and their validating institutions as lower-end fast-food joints.

Byron-Burger-02-hop-1-interiorsNick Hillman cleverly traces the root of the analogy to Matthew Batstone, co-founder of the New College of the Humanities, whose favoured version was more confectionary: Batstone likened the validation process to new chocolate bar manufacturers needing approval from Mars Bar. However, as John Gill notes, “high-profile problems have dogged attempts to inject competition” into the Higher Education arena. And though some alternative providers have targeted Byron’s end of the market, not all private colleges have been made of the finest ingredients, as investigations by Andrew McGettigan and others have shown.

Also revealing was Johnson’s definition of inspiring academics as those “who go the extra mile, emailing feedback at weekends and giving much more of their time than duty demands”. Within the sector, eyebrows lifted at the expectation that university staff should work harder still, and the implication that their weekends aren’t already spent on the job. Some wondered whether excellence within all professions would now be judged on out-of-hours contributions, and questioned why academics’ work-life balance was being further eroded. Remember that 40% of university teaching staff are on temporary or zero-hours contracts, and that university management is plagued by gender imbalance.

But the definition was probably born more of frustration than disrespect. Finding TEF metrics that actually work has proved trickier than anticipated. Learning gain might (and probably should) be measurable at local levels for individual cohorts of students, but it doesn’t allow the kind of cross-institution and cross-discipline comparisons that the TEF craves. Employability and salary data tell you lots about students’ background characteristics but, as Graham Gibbs notes, they remain hopelessly distant proxies for the quality of teaching they received at university.

images3Elsewhere, the Minister’s speech did offer some optimism for the sector’s future. The goal of increasing by 20% the number of black and minority ethnic students going to university by 2020 is to be applauded. UCAS was ordered to publish more detailed breakdowns of candidates’ background characteristics and application patterns, as the Social Mobility Commission requested some time ago. There were even intimations of a lighter-touch Research Excellence Framework (REF), with welcome acknowledgement that many in the sector want an audit that is “less bureaucratic and burdensome” and which “takes up less of the time that could be spent more fruitfully on research and also, of course, on teaching”.

Keeping the sector on side remains the TEF’s biggest challenge. Mike Hamlyn rightly worries about higher education “being seen as a transactional good, rather than a transformational experience,” while Paul Martin Eve fears that the TEF heralds “a massive coming wave of shake-ups to Higher Education finance, both research and teaching”. Pleas to rebalance teaching and research may seem more reasonable to academics if excellence in the former was acknowledged to rely on excellence in the latter.

TEF-Briefing-August-2015Disappointingly, the student voice is fading from TEF debates, with the NUS executive electing for “principled disengagement” because of threatened links with an inflationary fee rise. This despite the NUS having previously issued an excellent briefing paper on the topic.

The full title of the Minister’s speech referred to “fulfilling our potential”. The challenge ahead is to ensure that “our” embraces the whole of the Higher Education sector, and that “potential” denotes opportunities for it to become more equitable, more pedagogically responsive and more transparent about what it does. Greater care should be taken with language. How exactly is “patchiness” in the student experience being differentiated from learner-appropriate pedagogical diversity? Who exactly is lamenting the “lamentable” teaching?

That “extraordinary teaching deserves greater recognition”, however, is incontrovertible. As is the claim that Higher Education is “the most powerful driver of social mobility we have”. The TEF will soon find friends in the sector if it nudges institutional cultures in this kind of direction.

But equally important is that the TEF skirts avoidable pitfalls, in terms of both policy and rhetoric. Such pitfalls include hierarchy-enshrining outcome indicators, student-alienating associations with a fee hike, and tortuous metrics that reward only the wiliest gamers. The sector may also have had its fill of burger metaphors.

What can the TEF can learn from the REF?

(Note: I published this piece first under the heading “Seven rules the Teaching Excellence Framework should followvia the Guardian’s Higher Education Network on 07.07.15…)

students-in-lecture-hall-010

The proposed TEF (Teaching Excellence Framework) is often characterised as a “REF (Research Excellence Framework) for teaching”. It’s a description that reminds us how much less attention university teaching has received than its showier sibling, university research. But to what extent is it a useful, or even meaningful, comparison? Can measurements of research quality be mapped straightforwardly on to teaching? Has the REF proved so successful at capturing research excellence that it’s now the gold standard for evaluating all university activity? Does HE teaching even need to be REF’d?

indexSuch questions defy simple answers. But if the TEF is the new kid on the block, perhaps there are ways in which it could learn from the REF’s triumphs and mistakes? Below are seven principles that haven’t always been synonymous with the sector’s research audits, but which might help the TEF off to a good start:

1. Be ungameable

Universities, like schools before them, have grown adept at gaming the best possible results from what’s available. The danger is that the game itself becomes the metric, with ‘winning’ institutions not necessarily those offering their students the best possible teaching, but those able to manipulate data, craft narratives and spin results in the shrewdest way. We can’t claim the TEF is for our students’ benefit if the final outcomes comprise an intricate series of metrics, league tables and seemingly contradictory information based on dubious formulae, such as ‘Teaching Power’ or ‘Teaching Intensity’. Of course, no auditing framework can ever be completely ungameable, but the TEF would benefit from a set of guidelines that keep the playing field level and the sector honest.

2. Be Collegial

Good teaching rarely occurs in isolation. Within good university departments, colleagues draw on one another’s strengths and compensate for one another’s weaknesses. Will the TEF place individual lecturers in competition, divide disciplines and create new hierarchies? Or will it facilitate constructive conversations, developmental feedback and candid evaluation?

3. Be long-term

Just as it’s difficult to measure the enduring quality of a research paper for serval years, sometimes decades, after its publication, so too can the value of good teaching not be immediately obvious. University degrees are three or more years in duration to give students the time and space to ruminate, reflect and ripen. Confident, socially-aware graduates are not the product of one bell-and-whistle-filled session (when the peer assessors happens to be around); good lecturers play the long game with their students.

4. Be cheap

The sector can ill afford to be spending millions more pounds measuring itself in increasingly incestuous ways. The TEF should be inexpensive to administer, both in terms of direct costs and the opportunity costs of taking lecturers away from their students.

5. Be inclusive

The expense and agony of deciding who gets entered can be avoided by making inclusion criteria as broad as possible. A TEF score should reflect the teaching strength of a whole discipline; it should not be the product of arbitrary, localised inclusion mechanisms. Some pedagogically-challenged staff would no doubt prefer to hide behind a research grant or admin role, but assigning values to one or two superstar lecturers in a department is of little value. Shouldn’t everyone involved in teaching, from GTAs to departmental heads, be eligible?

6. Be modest

Trying to distinguish between teaching that is “world-leading” and that which is merely “internationally excellent” is unlikely prove fruitful, especially if we’re not asking the opinion of anyone from outside the UK system to benchmark our judgements. Criteria should measure what matters, not just what lends itself to getting measured. The language of the TEF should be realistic and restrained, not lavish and conceited (“innovative, outcome-oriented delivery of outstanding learner experience” should be avoided at all cost!). What students want, broadly speaking, are enthusiastic lecturers who want them to flourish and make time for them accordingly.

7. Be open-minded

Might a TEF reward a safe, conservative approach to teaching over bolder, risk-taking methods? Might lecturers strive for the equivalent of four 3* journal articles rather than gambling on a major, paradigm-shifting pedagogies? Teaching thrives on experimentation, inventiveness and fresh thinking; if the framework encourages boxes to be ticked and learning outcomes to be dryly evidenced, the sector misses an opportunity to learn from its more visionary communicators. Universities are the place for young people to learn independent thinking skills, to become crack problem-solvers and to begin viewing the world through an informed, critical perspective.

Our teaching must reflect that ambition. indexUniversities Minister Jo Johnson, when introducing the TEF last week, quoted one of his predecessors, David Willetts, as saying that “teaching has been by far the weakest aspect of English higher education”. It’s a baseless claim, likely to further disaffect a sector with research long acknowledged to punch well above its weight and academic staff long pressured to excel at everything, always. But with the English HE system among the most expensive in the world, our students have every right to ask why university teaching has lagged behind university research, often relying more on the good will of committed staff rather than secure institutional scaffolding.

Identifying what good research looks and feels like, and then attributing a numeric value to it, has been a challenge for the sector. Teaching will pose no fewer problems. But perhaps by thinking carefully about the principles that we want to underpin a new framework, and learning from those upon which other frameworks sit, we increase the chances of coming up with something that’s genuinely helpful both for ourselves and for our students.