so who's been polling whom? by Achim Tack

The election of the German Bundestag takes place later this year and, of course, journalists and data-nerds are slowly beginning to think about data-driven stories. For this reason I have refurbished an old scraper of mine, which I wanted to use for quite some time. It scrapes meta information of state level election polls from the great website, which is the go-to place for polling data in Germany. 

Using the scraper I aggregated data on 1885 individual polls beginning in 1999 and including state, release date, pollster and commissioning organisation. Obviously news organisations* concentrate on the polling results but I would like to focus on polling intensity and the question of who is conducting and paying for the polls. This is a little just for fun side-project with a pretty low level of detail.

Polling intensity by federal state

It is clear from the scraped dataset that Berlin has a much higher polling intesity than the other federal states. It is certainly worth pointing out that Berlin as a capital has a higher public interest than say the small city-state of Bremen. Still I consider it hardly remarkable that the intensity ranking does not exactly reflect the population ranking of the states: The data should be normalized against election individual years of the states.

Polling intensity over time

The polling intensity varies between 50 and 150 polls a year with a high in 2011 and a low in 2000 (Of course, the current year 2017 can not yet be fully counted). In 2011, a total of 7 state parliaments were newly elected, so that the peak can be well explained.

Top pollsters and commissioning organizations

Polls in Germany as a rule are conducted by four large pollsters, which in my sample account for a market share of about 72%. The value would be even higher if I had adjusted the data for the merger of Infratest and Dimap which are still included as two different institutions in the older part of the dataset.

Let us take a look at the commissioning organizations involved: Most of them are media outlets - both private and public service media. The strong presence of the Berliner Zeitung is striking, which commissioned a poll at Forsa nearly every month .

Pollster preferences by organisation

In the following table you will find the combinations of commissioning organization and pollster. It turns out that most organizations stick to a single pollster. Use the search mask and find out which media house uses which pollster.

So what?

Only few surprises emerged from this little project: Polls are increasingly being held before elections and most of the customers remain with a pollster. However, I find it interesting to see how many pollsters are in the market and who is commissioning them. There is certainly a "so what factor" to this analysis - but it was a nice project for a rainy sunday morning :)

* Disclaimer: I work part-time as a data journalist for Spiegel Online.

my #airrohr - a solar powered fine dust sensor by Achim Tack

I have already adressed the topic of fine dust in this blog several times. But since the #Dieselgate debate, air quality is not just a topic for New Year's Eve. So I was excited by the project, which was created in the Code for Germany Context in Stuttgart. The aim of the project is to build fine dust sensors from inexpensive components. Citizens can thus build their own sensors and form a large measuring network.

It's based on a NodeMCU ESP8266, CPU / WLAN chip and an SDS011 fine dust sensor connected to it. The kit is supplemented by a DHT22 for the measurement of temperature & humidity. Detailed instructions including the (open-sourced) firmware can be found on the project website.

During the assembly of the sensor I had the idea to operate it using solar cells or a small wind turbine. I went for for a relatively cheap 14 Watt Lidl solar cell solution with a connected 12.000 mAh buffer battery. Some first tests with the sensor connected to the solar cell with direct sun exposure showed the sensor to be performing quite well. The cell is weather-proofed with transparent plastic and mounted on our carport. I hope the performance of the setup will be sufficient to power the sensor continuously (although I doubt it given the expected losses of the battery and attached cables). 
UPDATE (26.06.2017): The sensor has been continously running on solar power for over a month now.
UPDATE (16.07.2017): First half-hour blackout of the sensor at night (following a rainy day)

The sensor data is accessible via the sensor's URL, but is also published to the central server of, where it can be viewed on a map. Detailed data for each active sensor is available on a separate website. A nice addition is the amazon Alexa skill, with which the currently measured value is available by voice query.

Happy New Year! by Achim Tack

As in recent years I had a look on the fine-particle concentrations of the Hamburg air quality measurement network during the fireworks. Especially with the ongoing discussion about the Diesel scandal I guess we might want to reconsider this habit... Keep in mind that the graph shows a moving average over 24 hours so the actual concentrations were much higher.

Update: There appears to be an issue with embedding Tableau-Charts in Squarspace. You can find the interactive Version here.