Working with Open Data

Information on economic data, population, lifestyle and demographics… all of which can be found saved within databases… can be consulted by anybody at anytime.

What can open data do for you?

For the most part, data is registered by governmental organisations, and more and more they are choosing to share that information with the public free of charge. In a previous publication we explored the world of open data, what exactly it is and what we know about it already.

To offer information freely, without restriction, could be considered an act of transparency on the part of institutions or companies. In addition, it allows institutions to form a link with the public; from the moment they are granted access to this information, they have the ability to expand their own knowledge, add social value and make improvements within society.

There are many countries that freely offer their data to anyone, but the United States stands out above all others; it was the first to publish a law in relation to freedom of information in 1996, dubbed the “Freedom of Information Act”, that allowed free access to a multitude of administrative documents.

This commitment to openness will be met with many challenges over coming decades. President Ronald Reagan, for example, limited access to data during the 1980s for reasons of  “national security”. It wasn’t until the arrival of the Internet that, thanks to the volume of easy access data available, this freedom returned, with more force than ever.

In 2007 the Sunlight Foundation (a non-profit entity founded in 2006, headquartered in Washington D.C), whose objective is to promote openness and transparency within the United State’s Government, defined the 10 basic principles to be referenced when evaluating government-held data:

1. Complete
2. Basic
3. Recent
4. Easily accessible, physically and electronically
5. Machine readable
6. Accessible without discrimination
7. In standard formats without using licenses  
8. Available in an open format
9. Permanently online
10. Free of charge

Thanks to this foundation, and everything achieved under Obama (remember, his was the first United States’ government to use executive orders to unblock and publish data systematically), the US has made great progress in their commitment to open data; collecting information and centralising it in the Government’s open data website.

The United Kingdom has been a leading reference point in public data openness ever since it launched its Open Data portal in 2010. The Open Data Institute, an entity funded by the British Government, whose objective is to create economic, social and environmental value through the promotion of  a culture of openness and reuse of public data, has played a key role in encouraging a similar culture of openness in other countries.

In 2012 the British Government published the Open Data White Paper, which looks in depth at the concept of an open government, establishes a series of open data goals, and advocates for the reuse and intelligent use of institutional information.

The Open Data White Paper asserts that it is the government’s duty to publish government-held data, that finally they invested to establish an openness of data by default.. Following on from this initiative, the Government worked with different departments and administrative organisations in order to guarantee the  reutilisation of public sector information. It aims to bring about transition from models of data collection to models of free reutilisation.

Open data is not only circulated throughout the governmental sphere; but also within those institutions conducting scientific investigations, within the stock-market, throughout the field of climate, transportation, and culture… application of this data is wide-ranging and varied. We just need imagination and the necessary knowledge to enable us to use this data to innovate, contribute socially or commercially, or to actively engage in participatory governance.

Let’s take a look at how the Madrid Metro station openings have developed over time.

The history of the Madrid Metro, through “Good Rebels”

At Good Rebels, we utilise different techniques of data representation dependant on what we want our client to focus on. In the past we’ve used maps, interactive graphs, network graphs, etc. A combination of different visualisations used to illustrate findings from the same project helps us construct a story that can illustrate the data.

We start with the facts. After an investigation process, we obtain the necessary information to create a database with information on all the Madrid Metro stations and their opening dates. We found the opening dates of the stations on the Madrid Metro website, and we sourced the coordinates of these stations on openstreetmap.  

The data is compared and revised and, through a process of refinement and clarification,  is adapted to fit the structure of the tool we use to represent that data: Carto.

We georeference each of the stations, that is, we find the latitude and longitude from one direction, in order to upload the data into Carto and create our Metro map of Madrid.

We can make the map interactive and add histograms or bar graphs that allow us to filter the information and present other types of data alongside our map.

You can play with the map, create filters and choose which data points you want to hone in on and see visualised:

Visualisation of the openings of the Madrid Metro throughout their history.

As you can see in the image above, and if you were to navigate the map yourself, by selecting a range of dates from the temporal scale at the base of the map, the data on the right is updated – for instance, the total number of stations, the distribution of openings each year, or the number of stations on each metro line. In the same manner, it is possible to filter some of these facts by selecting them.

We can add new characters to our story and incorporate data on the population of the city of Madrid, available in the open source INE, and the Gross Domestic Product – GDP – from the 60s until the present (in Millions of Euros).

The population of Madrid has been growing at an exponential rate ever since the end of the civil war. This population increase is especially striking in the years between 1940 and 1970, caused mainly by the huge volume of internal immigration consequent of rural exodus.

Representation of the Madrid Metro openings since 1970. Demographic and economic facts are also shown for these years.

This increase in metro stations formed around irregular settlements, without thought to urban planning, , explains the number of residential zones marginal to the centre, mainly in southern districts. As you can see, public services arrived years later.

Since the 70s, there has been a deceleration in population growth within Madrid, in favour of municipalities within the metropolitan area, that, with a greater urban supply, began to absorb the migrant flow facing a capital without any plan to expand.

Once again, at the start of the 21st Century we see a rise in the population. This time caused as much by external immigration as by urban development bound to the new Urban Action Programmes (PAUs), one which we saw partially altered by the shattering of the real-estate bubble in 2008. From the crisis of 1993 until 2008, plans for a proliferation of stations were produced (and later opened with a disparity of time of work) with equal aims to implement new urban project services and, through a strategy that began as a socio-political standpoint repositioned itself as a key player within the  Community of Madrid’s government, to connect peripheral zones without previous access to the suburban network.

Representation of the Madrid Metro openings after 1970

We can see clearly all these shifts in population reflected in the map.

It is also possible to acquire information on GDP during this time period, by analysing  the relationship between changes in demographics and economic developments within the state.

We have worked with external and public data, but the possibilities are endless, and we can incorporate other elements into this map such as the number of residents or shops in each area, levels of communication, number of businesses, purchasing power, etc.

We are living in the so-called “age of information” and are lucky to have a such a wealth of open data available to us.

Advances in technology and the evolution of technological processes, along with the availability of easy access, free flowing data, will help to improve levels of  productivity and scientific creativity.

A culture of openness around data should be default within any study financed by public funds or any governmental organisation, and the results should be promoted and utilised by all local educational bodies. The dissemination, creation and utilisation of open data marks the difference of a truly advanced society.