Opening up the Scottish Index of Multiple Deprivation


May 23, 2017 by No Comments

Maike Waldmann & Roman Popat

You have heard of the Scottish Index of Multiple Deprivation (SIMD) but wonder how it was calculated? You want to use SIMD but without all the health data in it? You want a rich dataset to play with in R? You want to calculate your own composite index but don’t know where to start? openSIMD is the solution to all your problems!

openSIMD makes the calculation steps between the indicator data and the final SIMD measure completely transparent and open to scrutiny – no black box any more.

openSIMD = R code and documentation

Apart from being the solution to all your problems, openSIMD is a bit of R code along with documentation and data, which lets you calculate SIMD16 for yourself while making any changes you want.

The R code consists of one script to calculate the SIMD domains, another script to calculate the overall SIMD, and a third script with some functions. The documentation explains how to run the code, what the functions do, and how well the code replicates the original code that was used to calculate the official SIMD16. The data consists of two datasets downloaded from the SIMD webpages: The SIMD16 indicator dataset, and the SIMD16 domain ranks dataset.

SIMD identifies Scotland’s most deprived areas

SIMD is the Scottish Government’s official tool for finding the most deprived areas in Scotland. SIMD is used by government, councils, charities and communities as evidence to help target their work to those areas that need it most. SIMD is best known for how it ranks each small area in Scotland by how deprived it is. But in addition to the rankings, all indicator datasets that go into SIMD are also published on a small area level. This data provides a wealth of detailed information about the underlying issues in deprived areas.

SIMD is made up of over 30 indicators which are grouped into seven domains of deprivation. Each domain summarises one aspect of deprivation by combining some of the indicators and using the resulting domain scores to rank each area in Scotland. The seven domain rankings are then combined into an overall, multiple-deprivation SIMD ranking.


Some technical details

We translated SIMD to openSIMD from SAS to R. You can find our documentation for this project here. If you want to fork and contribute to the project, we would be delighted. Please see the public GitHub repository here. If you click through to read the documentation, run the code or explore the results, you will notice that SIMD and openSIMD scores and ranks are not numerically identical. Our tests showed that this is due to the exact way that some algorithms are implemented between the two platforms.

The functions that we have defined in the project are designed to be very SIMD specific. This was to keep us on the straight tracks of the SIMD procedure and not to create new more general tools. Finally, we realise that the fundamental unit of repeatable analysis in R is the package. For practical reasons we decided against writing a package in the first instance, however we plan to convert the project into a package in due course. If you want to collaborate on this please get in touch.

Find openSIMD


Mas detalles:


Publicado en Enlaces | Etiquetado , , , | Deja un comentario

European Social Survey

Exploring the European Social Survey (ESS) – pipe-friendly workflow with sjmisc, part 2 #rstats #tid… –

| Deja un comentario

Web Scraping and Applied Clustering Global Happiness and Social Progress Index

(This article was first published on DataScience+, and kindly contributed to R-bloggers) Increasing amount of data is available on the web. Web scraping is a technique developed to extract data from web pages automatically and transforming it into a data format for further data analysis and insights. Applied clustering is an unsupervised learning technique that…

via Web Scraping and Applied Clustering Global Happiness and Social Progress Index — R-bloggers

Publicado en Enlaces | Etiquetado , | Deja un comentario

Informe sobre indicadores de calidad de vida en la UE.

El grup d’experts sobre indicadors de qualitat de vida creat l’any 2012 en el si del Sistema Estadístic Europeu (per part del Grup de Directors d’Estadístiques Socials) s’ha reunit en 8 ocasions en el període 2013-2015 i finalment a l’octubre de 2016 presentava els seu informe final. Per part espanyola hi han intervingut Antonio Argüeso […]

via Informe sobre indicadors de qualitat de vida de la UE — Bloc d’estadística oficial

Publicado en Libros | Etiquetado , , , | Deja un comentario

Pobreza y elecciones en Francia

Je suis tombé hier, sur le site de The Economist sur deux cartes, juxtaposées, une présentant le taux de chômage (en rouge, à gauche) et l’autre le vote FN (en bleu, à droite), L’idée serait ici de comparer les cartes, et de se dire que les deux variables “taux de chômage” (produit par l’insee) et…

via Élections, pièges à com (mentaires à la con) — Freakonometrics

Publicado en Opinión | Etiquetado , , | Deja un comentario

Indicadores de Pobreza Infantil en Cantabria

Publicado en Documentos de trabajo | Etiquetado , , , | 1 Comentario

Dos estudios metodológicos de Eurostat sobre la EU-SILC.

Eurostat publica dos estudis metodològics nous sobre l’enquesta d’ingressos i condicions de vida, EU-SILC: “How does attrition affect estimates of persistent poverty rates? The case of European Union statistics on income and living conditions (EU-SILC) S.P. Jenkins i P. Van Kerm. 2017 edition.” que es refereix a l’impacte de l’índex d’abandonament de ‘enquesta. “Among the […]

via Dos nous estudis metodològics sobre la EU-SILC — Bloc d’estadística oficial

Publicado en Documentos de trabajo, Libros | Etiquetado , , , | Deja un comentario