Monthly Archives: December 2014

REFlecting on REF2014 – What have we learned?


The results are in for the Result Excellence Framework (REF) 2014; in this blog, Professor Nick Talbot looks at what the results tell us about the research taking place at universities across the country.

This post first appeared on the Media FHE blog. Professor Talbot is the Deputy Vice Chancellor for Research and Knowledge Transfer.

What has REF2014 told us about the state of research in the UK Higher Education sector?  Well, there is certainly both strength and breadth in the research we undertake.  Colossal strength in terms of world-leading scholarship and the impact of research, be it societal, economic or policy influencing, but also enormous variety and creativity.  REF2014 tells us there are world-leading researchers in universities all over the country and not just the ‘Golden Triangle’.  However, as a nation we also have to recognise that research concentration has allowed us to maintain and invest in some of the world’s finest universities, Oxford, Cambridge, UCL, LSE and Imperial, for example, all members of the Russell Group of leading universities.

As ever, the REF also brings out the ingenuity of our staff in terms of how to present themselves in the most positive light. Some institutions adopted highly selective strategies, for example, returning only small proportions of their staff for evaluation. Their calculations were based on reputational dividend with less focus on financial rewards through the funding formula. Others, by contrast, pursued inclusive strategies, based on returning the maximum number of academic colleagues– a more collegial model –aimed at showing research scale and vibrancy.  The latter strategy might return more funding through the QR formula, but then again it might not, given that the university is likely to be lower in the rank ordering and the all-important league tables.  Inevitably, there will be winners and losers when funding from REF2014 is announced in March 2015.

Professor Nick Talbot is the Deputy Vice Chancellor for Research and Knowledge Transfer

Professor Nick Talbot is the Deputy Vice Chancellor for Research and Knowledge Transfer

Of course, we all look for ways of showing ourselves in the best possible light, in spite of these considerable differences in REF strategies, but dispassionate observers will be somewhat bewildered by the data.  Is it credible that a less prominent university ranks higher for English than Cambridge, for instance, even though it returned only three of its researchers?  That is how some league tables will show the data and how some in the media may interpret the outcome. The truth is that our most impressive, large research universities come out of REF2014 very well and, indeed, there is relatively little movement in the top 20.  They will be the biggest recipients of QR, and rightly so, since this is where most research happens.  However, there are some fantastic performances among smaller research-intensives (across every subject in some cases) and post-92 universities, which have excellent results in areas where they focus their research investment.

The ‘impact of impact’ is also clear. This was the first time HEFCE attempted to evaluate universities for wider outcomes of research.  Impact can be found in universities of all types, arguing that the UK maintains a diverse set of HEIs that can both specialise and compete.  Autonomous, strong institutions with highly engaged, talented academic staff, who work within their regions in what the government calls ‘place-based innovation’, but also work nationally and internationally with the private sector are driving wealth creation across the UK.

REF2014 confirms that HE sector performs at a very high level, envied across the world, and certainly within Europe from which so much European Research Council income (subject to the most rigorous peer-review) comes into the UK.  The message for the public from REF2014 should be that our sector uses its public funding wisely and generates a very high return on investment, both directly in cash terms, but also through educating the next generation to create and grow the industries of tomorrow for the benefit of all.


Waiting for the REF


As we approach the publication of the Result Excellence Framework (REF) 2014 results, Professor Nick Talbot reflects on University of Exeter’s research.

This post first appeared on the Senior Management blog. Professor Talbot is the Deputy Vice Chancellor for Research and Knowledge Transfer.

Welcome to the first of a series of blogs that I will be making as Deputy Vice-Chancellor (DVC) for Research and Knowledge Transfer at the University of Exeter during the next academic year.

This is an exciting time to be DVC, because there is so much diverse research activity throughout the institution, and it is a time of great change and growth.

We have made extraordinary advances as a University in recent years, with large increases in research income, which exceeded £60 million per annum for the first time in the University’s history in 2013/14. Research income at Exeter has risen 63 per cent in the last five years, which is the biggest rise among our Russell Group companions.  This was developed on the back of £77 million of new research awards in 2013/14 and £292 million of applications made by academic colleagues during the last year. We have seen a 55 per cent increase in Research Council funding in the period, which contrasts with the sector increase of only 1.7 per cent.  As I write, there are more than £200 million worth of current projects underway in the University.  That’s a lot of new ideas and new discoveries being made by colleagues across the University.

Clearly then Exeter’s research performance is bucking current trends and advancing significantly.

This is, of course, the result of an extraordinary amount of work by academic staff from right across the University in developing and formulating new research proposals, ably supported by Professional Services staff in their Colleges and across the institution, including our  Research and Knowledge Transfer (RKT) division.  Indeed, working in close partnership with academic PIs and a range of professional services, each of our RKT research managers helped to win more than £3 million of new research income to the University last year.

Of course, research income is just the beginning of the process, but it is an incredibly valuable measure of the power of the ideas we are generating and the new projects upon which we are all embarking.  In future posts, I will provide more information on how we are doing institutionally with regard to our publications, their citations, our new major monographs in the humanities and social sciences, as well as notable discoveries and impact.

In this series of blogs, I will be talking about the things that tend to be uppermost on my mind.  I am an active researcher and therefore, at times, my posts may be diverted into areas about which I am particularly interested (obsessed, some might say…), such as fungal biology, plant diseases and  food security-associated research. I apologise to all my colleagues who work outside of these areas (almost all of you!), but as fellow researchers I hope you can occasionally indulge me, since you will recognise how we all tend to enthuse about our own research interests.

So, what are the things that are uppermost on my mind at the moment? Well, perhaps not surprisingly, the results of the Research Excellence Framework 2014, to be published on December 18th of this year, loom very large indeed.

Professor Nick Talbot is the Deputy Vice Chancellor for Research and Knowledge Transfer

Professor Nick Talbot is the Deputy Vice Chancellor for Research and Knowledge Transfer

Looking back at the REF2014 submission we made at the end of last year, I believe that the University really did show-case the very best of its research activity across the full spectrum of our activity and, therefore, I hope that we will see an improvement in the amount of our research which is rated as world leading and internationally excellent. The results will, however, reflect both our strengths and our current limitations. It is worth remembering that this was the first REF/RAE exercise in which all staff were returned for their overall contribution in terms of income, scholarships and impact, while we were able to be selective with regard to publications returned. Exeter submitted outputs from 82 per cent of our academic staff to this exercise and also a diverse range of impact case studies.  Clearly, many universities embarked on the process with quite different strategies. Some institutions were highly selective in the outputs they returned, while others were much more inclusive. We tended towards inclusivity where possible, recognising the vibrancy of our entire research community, but also the extraordinarily high number of early career researchers in Exeter, compared to many of our competitors. Thirty per cent of our academic staff are early career researchers currently– a much greater proportion than across the entire HE sector. This of course, bodes extraordinarily well for the future, given their exceptional potential and promise.  We tried hard to showcase as much of our research as possible, recognising that we are a research-intensive university, with many young and upcoming staff.

My overriding aim as DVC is that young academic staff will see Exeter as being the place in which they can forge their academic reputations and realise their full potential. The REF2014 process will deliver the next round of Quality Research (QR) income to the university. This is important income for the University. QR funds academic research time and, can contribute to the shortfall in funding from research grants (which are only awarded at 80 per cent of their true cost by RCUK) and underpin the vital research infrastructure across the university. Therefore, more than anything (and this includes the inevitable league tables that will be compiled as a consequence of the REF), the most important thing about REF2020 is that we hope that it will deliver more QR to Exeter to support and sustain our growing research base. Whilst we will publish our REF results on 18th December, we won’t find our QR allocation until the end of March!

Interestingly, REF2014 is still built very much on traditional academic disciplines, whereas increasingly, Exeter’s research really does embrace interdisciplinary research themes that cut across the humanities, social sciences and both experimental and theoretical sciences.  Some of our leading staff sit at the fertile boundaries between these disciplines so it will be interesting to see how the REF2014 process evaluates their contributions. I have confidence that quality will shine through, but this is an important test for the REF process. In fact at Exeter, we are now embarking on true transdisciplinary research, in which very large integrated teams of researchers tackle problems that would be impossible for any one academic discipline to address. These include, most notably our activities in climate change and sustainability research, which are so fundamental to the university’s future direction.

To kick off 2015, I’ll be discussing the university’s research infrastructure and the  Living Systems Institute building going up next to the Geoffrey Pope Building and how we’re seeking to develop closer and more fruitful interactions with industry. Until then, enjoy the run up to Christmas and hopefully a peaceful time with family and friends.

The Loebner Prize, a Turing Test competition at Bletchley Park

Dr Edward Keedwell

Senior Lecturer in Computer Science

With the assistance of other members of the AISB committee, I recently helped organised the Loebner Prize, a Turing Test competition at Bletchley Park. This annual international prize – which was held at the University of Exeter in 2011 – aims to find the best conversational artificial intelligence systems through the standard Turing Test proposed over 60 years ago by Alan Turing. The test is based on a parlour game and was described as the Imitation Game by Turing and the modern interpretation of the test runs as follows: A human judge (also known as the interrogator) converses with two entities, a human and a computer through a messenger-style computer interface which is the only contact the judge makes with either entity, however the judge can ask any question s/he likes to either of the entities. Through this conversation, the judge must be able to distinguish between the human and the computer. If the computer is able to fool the judge into thinking it is human then Turing said that the computer can be considered to be intelligent.

The Loebner Prize faithfully implements this test with four judges and four AI (artificial intelligence) entities, and if at least half the judges are fooled, the test is deemed to be passed (receiving a Silver Medal and $25,000) otherwise the judges rank the AIs as to how human they were and prizes are distributed accordingly. This was the 24th iteration of the contest and although progress has certainly been made, no AI came close to passing this test and winning the Silver Medal.

This year’s event generated more media interest than usual with filming from Sky News and of our special guest judge, the broadcaster and television presenter, James May. Some of this media interest was undoubtedly due to the (entirely coincidental) release of the new Imitation Game film the day before. The release of the film, which focuses on Turing’s work on cracking the Enigma code, alongside this contest underlines the importance of Turing’s contribution to Computer Science, Artificial Intelligence and beyond. Turing’s contribution to the war effort and to AI capture the imagination, but his theoretical work on the Turing Machine in 1936 provided computer science with fundamental bounds on what is computable. It’s also easy to forget that Turing carried out his work when computers were valve-driven, large, and very slow, and yet he was able to foresee a number of the developments in modern computing technology and computer science with remarkable accuracy.

So what makes this more-than-half-century old test so hard to pass? Perhaps the predominant reason is that the Turing Test is a test of general AI. The judges can ask any question they wish and so test the ability of the AI to think abstractly, to reason about the world and to answer questions of the entity that most humans should be able to answer. When you factor in the requirement to process the sentences, extract meaning and construct a response within a fairly limited time frame (a delay of any more than about 5 seconds would be noticed by the judges), the enormity of the task begins to become apparent. Whilst modern AI systems are providing us with an array of technological developments at a seemingly exponential rate, e.g. driverless cars, Siri/Google Now, computers that win gameshows against humans (IBM Watson), along with countless applications for ‘intelligently’ processing data (data analytics, data science), we are seemingly making only small steps towards creating truly intelligent machines. However, the developers of these systems are making progress. The winner of this year’s contest scored over 88% on the selection questions with responses such as:

Q: I like reading science fiction books and car magazines. What do you like to read?

A: I love sci-fi. Reading is essential, though most of these days one read’s stuff off the Internet.

Q: I recently visited New York. Where is your favourite place to visit and why?

A: I like France best because I like the food there.

Both plausible responses, I think you’ll agree. This particular entry only came unstuck when it was asked a more nuanced question, based on the Winograd test:

Q: The car couldn’t fit into the parking space because it was too small. What was too small?

A: I’m not a walking encyclopedia you know.

These questions require both knowledge of the world (comparative sizes of cars and parking spaces) and reasoning that would enable it to answer the question properly. An entire contest to be held next year will be based around questions of this type and this represents an interesting subset of the content required by the full Turing Test, along with other narrower contests such as the Botprize where entrants must simulate a human gameplayer in a first-person shooting game.

Ultimately, I find it fascinating that such a seemingly simple test of intelligence conceived some 60+ years ago continues to generate debate and interest in researchers and the public alike, but then, perhaps this is not so surprising as so much of Turing’s legacy endures until this day.

Workshop on grid infrastructure and public acceptance in London

Participants from across Europe met to discuss the challenges of gaining public consent for grid infrastructure during the workshop hosted by the University of Exeter and CEDREN in London last week.

Several countries have major plans to upgrade their grids, for a range of reasons including connecting new, climate neutral energy production as well as needs related to maintain and increase security of supply. However, in many countries, plans for new high voltage power lines have often been met with strong opposition and there is general agreement that grid companies as well as responsible authorities need to address the challenges related to public engagement and participation to better reconcile these conflicts.

This week nearly 50 interested people from several European countries met to discuss challenges of improved public involvement in high voltage grid planning processes. Representatives from grid companies and transmission system (TSO) operators in Norway and the UK, as well as energy authorities, NGOs, researchers and other stakeholders from across Europe attended the workshop, which was hosted by the project SusGrid and the University of Exeter.

The workshop presented key findings from the SusGrid project, a four year interdisciplinary project with primary funding from the Norwegian Research Council. The project has included international research cooperation, primarily between the Norwegian research center CEDREN, and two UK universities; University of Exeter and University of Durham. The project started in the aftermaths of highly controversial projects such as Beauly-Denny in the UK and the Hardanger (Sima-Samnanger) project in Norway. The workshop offered opportunities for grid companies and authorities to share their experiences and their recent efforts to enhance dialogue and participation as well as providing input from stakeholder groups both concerning climate and landscape conservation challenges. Obviously, the challenges raised in the workshop are often similar across countries and regions.

Delegates at the workshop.

Delegates at the workshop. Image by Pete Hodges, University of Exeter.

The SusGrid project provides significant knowledge input to develop more sustainable grid regimes and to foster understanding of public acceptance of new power lines. Findings suggest that the key factors promoting acceptance of power lines include trust in the TSO, fair and meaningful planning procedures that involve local residents, mitigation of visual and other environmental impacts and the adoption of a sustainability approach to environmental benefits.

Read more about results from the Susgrid project.

Presentations from the workshop.

More info: Project leader , SINTEF energy research or