« October 2009 | Main | April 2010 »
February 10, 2010
Dept. for Business, Innovation and Skills' poor use of research in THES article
There is a good article in Times Higher Education Supplement this week: criticising some poor use of research by the UK's Department for Business, Innovation and Skills (BIS). I'm quoted saying that a report on 'future jobs'
promoted by many including Lord Drayson, the Science Minister, and Gordon Brown...is substandard...used "unjustified methodologies" to reach its conclusions and was "overly reliant" on weak online sources and media reports, with some sections referencing only Wikipedia.Although sources are referenced...the report lifted "significant passages of text" word for word.
You can read the report on the future of the job market - written by Rohit Talwar and Tim Hancock at Fast Future Research - online (PDF). The report was used for the Science: So What? So Everything? science communication campaign, and it really is strikingly bad. Problems range from the systemic (methodologies are unjustified: for example, the survey used doesn't really answer the research question or prove much beyond 'some people said some things') to tediously basic flaws (for example, relying on Wikipedia as a citation for some points or referring to 'future jobs' which have been in existence for many years). The report comes nowhere near the standards that one would expect from work commissioned for a department like BIS - and I would have serious objections to the use of such bad social science in the name of science communication.
My interest in this comes from some a couple of papers I'm working on - looking at science communications and at the role of impact in future UK research assessment, respectively. BIS has a role in both science communication and changes to research assessment - so I do feel that their mis/use of research is significant.
When I pointed out the obvious and serious problems with the report, I was expecting BIS to simply apologise and drop it. However, they chose to defend the report - after which, THES took an interest.
Even after the THES piece, BIS continue to stand by this report: telling THES that
the report was a "speculative look" into the future intended to spark public interest in science and did not inform policy or affect funding decisions.
Responding to THES, Rohit Talwar
maintained that citing websites, news reports and blogs was "accepted best practice in horizon scanning.
Except in very specific circumstances, even undergraduate students would be advised against this type of 'best practice'.
Of course, the UK government is free to publicise and commission whatever research they want. However, it is unfortunate if they choose to focus on such poor research - even after its quality is pointed out to them. BIS's role in here is especially worrying, given their major role in universities and research.
Further reading
I found Zoe Corbyn at THES very thorough, and am pleased with the story which has come out. It's worth noting, too, that a number of other people have looked at issues around BIS's Science: So What campaign. 2020 Science offers a good take on the science communication aspects of this, while Gimpy's Blog and 10minus9 both look at aspects of the science of Fast Future's report. Kylie Sturgess used TurnItIn to find lifted passages of text in the report. It will be interesting to see how this story develops - if BIS continue to use this report, there will be plenty of opportunities for others to add their voices to this debate.
Posted by jon_mendel at 10:27 PM | TrackBack
February 09, 2010
Peter Taylor's Generation Jihad: are we in a golden period of British security?
I've just watched the first episode of Taylor's documentary Generation Jihad. Some interesting discussion of radicalisation, but I almost spilled my coffee when the programme started by stating that
a small group of radicalised [Islamist] young men now constitute the single biggest threat to our national security
If this is correct, British security is remarkably little-threatened at the moment. Certainly, Islamist terrorism is a genuine and non-trivial threat: Islamist groups have killed too many British citizens, residents and visitors, and will very likely kill more. However, compared to previous threats (for example, the risk of nuclear war if the Cold War went 'hot') the risk posed by the tiny minority who are prepared to kill in the name of Islam seems strikingly mild.
It is therefore important, as Scheier argues (focussing on the US):
to convince the public to refuse to be terrorized. What frustrates me most about Abdulmutallab is that he caused terror even though his plot failed. I want us to be indomitable enough for the next attack to fail to cause terror, even if it succeeds. Remember: terrorism can't destroy our country's way of life; only our reaction to terrorism can.
London did not stop after the 7/7 attacks, horrible as they were. Whereas a Soviet nuclear attack might have destroyed most human life in Britain, future Islamist terrorist attacks will be on a much smaller scale than any such nuclear attack and will cause significantly fewer casualties. While preventing such attacks is important - and it is tragic when people die and are injured in terrorist attacks - if this is really the biggest threat to our security then I would view this as cause for celebration.
For what it's worth, I do think there are more significant threats to British security (at least in the medium-term) than small groups in the UK who are prepared to use violence in the name of Islam: climate change and geopolitical issues around British foreign policy both raise significant concerns, for example. It is also important to consider how 'security' is defined: according to some definitions, issues such as poverty and domestic violence would also be significant concerns. However, I very much hope that Taylor is proved right here: if these small groups are our most significant security threat, then these are remarkably safe times to live in.
