Author Archives | David A. Shamma

Tweeting Minarets: A personal perspective of joining methodologies

David Ayman Shamma

Editor’s note: In the last post of the Ethnomining‘ edition, David Ayman Shamma @ayman gives a personal perspective on mixed methods. Based on the example of data produced by people of Egypt who stood up against then Egyptian president and his party in 2011, he advocates for a comprehensive approach for data analysis beyond the “Big Data vs the World” situation we seem to have reached. In doing so, his perspective complements the previous posts by showing the richness of ethnographic data in order to deepen quantitative findings.
David Ayman Shamma is a research scientist in the Internet Experiences group at Yahoo! Research for which he designs and evaluate systems for multimedia-mediated communication.


There’s a problem we face now; the so called Big Data world created an overshadowing world of numerical data analysis leaving everyone else to try to find a coined niche like “small data” or “long data” or “sideways data” or the like. The silos and fragmentation is overwhelming. But really, it’s just all data. Regardless of the its form or flavor, there are people who are experts at number crunching data and people who are experts at field work data. Unfortunately, the speed at which data science moves is attractive and that’s part of the problem; we don’t get the full picture at speed and everyone is racing to produce answers first.

A few months ago, in a conversation with a colleague, he told me “you don’t know what you don’t know, especially when it’s not there.” We were looking for a way to automatically surface a community of photographers on Flickr who didn’t annotate their photos. They didn’t use any titles or tags or any annotations what so ever. But they were clearly a strong and prolific community. If there was some way to automatically identify them, then we could help connect them.

Now, finding metrics for social engagement in unannotated data is not an impossible task when provided with some signal in the data that has some correlation, statistical or otherwise, to the effect you’re trying to surface. But in some cases, it’s just not possible. What you need is just not there; therein is a problem. In other cases, it’s much harder to surface features when you don’t know what they look like.

When you have a lot of data, finding that unexplainable prediction through algorithmic statistics becomes easier. It doesn’t explain why and it doesn’t always work.

Enter Ethnography to answer the why and find out what things might look like—surfacing findings in the age of big data. When I was invited to write a post on Ethnography Matters, I decided to illustrate this through a personally motivated example.

In the late January of 2011, the people of Egypt stood up against then President Hosni Mubarak and his National Democratic Party. They wanted employment, a fair government, and an end to the 30 year long emergency law which had removed most of their civilian rights. Undoubtedly, you read about it somewhere. At the time, my mother was in Cairo visiting her 100+ year old mother. So this left me glued to the only source of news I could find—a rather buggy Al Jazeera video stream. U.S. news agencies were slow to start some sparse coverage. Somewhere in-between, it was burning up on Twitter.

Tharir tweets

A visualization of Twitter activity directed towards Tahrir by aymanshamma

Read More…