Mapping the people and citations in UK policy

We queried the Overton database for relevant documents and retrieved approximately 202k matches from UK government sources, ranging from documents on GOV.UK to committee reports from parliament, Hansard, and clinical guidelines from NICE

The Geddes study linked at the top of this page looked specifically at 2013-2014. We're looking at all of the policy documents available to us in Overton, which are typically but not always from 2015 onwards

Overton only knows about publicly available documents: it can't see internal Civil Service documents or when interactions haven't been recorded publicly

Overton tracks policy documents at the UK and devolved nations level, and from large city councils - London, Greater Manchester, Edinburgh, Leeds, Liverpool etc. - but not from smaller cities, so some local interactions will be missed

Overton works best for citations with DOIs, and many books and older papers especially in HSS don't have these: they may not be picked up and so won't be counted

Overton tracks research from think tanks and NGOs too, but in this dataset we're only looking at work in the scholarly record. That means we're ignoring any engagement academics might have through e.g. publishing a report via a think tank or foundation

Affiliation data comes from Microsoft Academic and while coverage is good it is not complete. Some academics won't be counted and there's no obvious pattern to this that we can compensate for

We're only including data from UK based institutions in GRiD classed as 'Education'. GRiD treats healthcare facilities separately so experts from e.g. university affiliated teaching hospitals are not counted

Overton only knows about academics who have been cited at least once somewhere in the policy literature, so the set is biased towards people whose work has already made it into policy

Overton can't robustly classify mention types: it can't tell if somebody is mentioned because they have given evidence, are being quoted, because they were commissioned to write a report or they just attended a workshop or discussion

Some policy documents use informal names or acronyms in affiliations e.g. Edinburgh University instead of University of Edinburgh. GRiD includes many name variants but not all, so some affiliation strings may not be mapped correctly, especially for smaller institutions

The first names of mentioned and cited academics were used to guess their gender using the genderize.io web service

See Stacy Konkiel's post on the Bibliomagician blog for a fuller overview of why

We're counting people mentioned or cited at least once in policy, rather than gendering each appearance: an academic may be mentioned in three different documents but her name would only be counted once

First names that were just initials - common in Overton's citation metadata - or fewer than three characters long were automatically marked "could not be determined" and we used 85% as the cutoff probability score for the guessed gender from genderize.io, following the methodology in Elsevier's Gender in the Global Research Landscape report

Thanks to @carlbaker on Twitter for pointing this out. An example is the University of East Anglia, which is placed in South Norfolk by GRiD but should actually be in Norwich and East Norfolk. We found about this after doing the data analysis, so it hasn't yet been fixed.

NUTS has been updated recently and a few regions have been added, removed or changed, mostly in Scotland and Northern Ireland. We've tried to map old NUTS codes to the new ones as best we can

Our map uses the December 2019 boundary data file from the Office for National Statistics