You are here

How and why the Government is Stalking you

The spring of 2011--what is now known as ‘The Arab Spring’--saw a new phenomenon emerge on the global political stage. A new type of revolution upended one Middle Eastern regime after another. In almost every country the story was the same: protestors and activists used social media to coordinate and communicate their activities. This mixture of activism and open communication proved to be a potent combination.


Egypt's social network.


"We Use Facebook to Schedule the protests, Twitter to coordinate, and YouTube to tell the world," said a protester.

While you might think this scares governments into cracking down on the Internet and social networking, it has produced the opposite effect. Governments with the ability to monitor online networking have instead looked at it as a valuable reservoir of information that they can mine to stay one step ahead of online activists.

Social activity on the Internet generates mountains of data on social sentiment and public opinion. A government with the power to monitor this data flow can predict the direction of social sentiment and public opinion, and use this information to preemptively shape the online discussion and, presumably, its outcome.

What we know about government social media monitoring comes from a combination of what they say they do, what those of us in the industry know is possible, and common sense. The government response to most direct queries is “that’s classified”. But in an AP article about government use of social networks, the CIA admitted to monitoring twitter, facebook, and blogs. This shouldn’t surprise anyone, but the tools they’re using might be enough to get you to rethink posting anything person personal or ever checking in.

Big Brother

In 2009, the venture arm of the CIA, called In-Q-Tel, put money into a company called Visible Technologies. Visible developed technology, which--last we knew--crawled over half a million web 2.0 sites a day, scraping more than a million posts and conversations taking place on blogs, online forums, Flickr, YouTube, Twitter and Amazon. A Visible spokesman said that the CIA wanted Visible to keep track of foreign social media, and give spooks “early-warning detection on how issues are playing internationally.”

A defense contractor named Raytheon has developed a deep analytics tool called Riot (Rapid Information Overlay Technology) that gathers all your social data and reports on who you are, what you say, how you feel about different topics, who you know, and where you go.

The most interesting and disturbing part of this software is its ability to use social data to figure out where you are likely going to be and when. Using social networks like Twitter, Facebook, and Google+, these tools can develop a sophisticated psychological profile of users. Because the majority of social networking is now done from mobile devices, they can also determine where you typically are on a current day, and where you will probably go next (this type of technology is already publicly available in products such as Google Now personal assistant). Layer on top of this what we already know about the close relationship between mobile providers and the government, and it’s a safe bet that if the government wants to know where you and what you’re up to right now, they know.

Taking it Further: Controlling the Conversation and Astroturfing

Beyond simple monitoring, the government is actively using their capabilities to control the conversation online and direct public opinion. It’s well known that humans value social proof highly, and that a herd behavior often develops around popularity of any kind. And we now know that the government is capitalizing on this behavior: the US government is running a large ring of puppet accounts as part of Operation Earnest Voice (OEV)--a practice commonly called “astroturfing”.

The little declassified information we have about OEV comes from a contract awarded by Central Command (Centcom), which oversees military operations, to a California corporation called Ntrepid. The stated goal of this technology is to support, “classified blogging activities on foreign-language websites to enable Centcom to counter violent extremist and enemy propaganda outside the US." But Isaac R. Porche, a researcher at the RAND corporation, claims it would not be easy to exclude US audiences when dealing with internet communications.[4]

Operation Earnest voice is built to allow 50 real users to manage 10 fake accounts each. These 500 accounts appear to be from anywhere in the world they user would like, “replete with background , history, supporting details, and cyber presences that are technically, culturally and geographically consistent.” What it does, effectively, is create a network of online personas that, when working in a coordinated fashion, can control the tone and direction of just about any online conversation.

These accounts could have some very useful applications outside of the stated goal; they could be used to help monitor social networking sites, and they would be quite powerful in influencing social movements through hashtags and what’s hot features on twitter and Google+. Also, the personas could quite effectively drown out viewpoints that the government would like to suppress. By dominating forums and engaging in some basic SEO, government propaganda would be hard to distinguish from the truth.  

So what can you do?

The explosion of Big Data in our society gives us tools and opportunities we couldn’t have imagined 20 years ago, we can now freely promote what we believe in and even bring about real change. But this comes at a cost, in the process we’ve given up dramatic amounts of data about ourselves, we’re telling the world who we care about, what really matters to us, and where we are going to be. Social Networks are great, but they offer a trade off. So before you join foursquare or start geotagging your images, take a moment to think: Do I want everyone to know this about me?

If you’re interested in learning more about how this type of data mining is done, we just finished a book on this very topic that we're offering you for free! Data Science Secrets explores in greater detail the process's that enable companies to do this and other amazing things with their data. Free for for a limited time, get your copy today by signing up below:







Data Stories

How to increase sales by looking at your customer data

Are you a B2C company interested in increasing sales? If you have some form of customer data, you may be in luck.

IBM's Watson and the future of Healthcare Analytics

What would it be like to have a doctor who’s always up on the latest research and has learned about treatments from over 1.5 million previous cases? It would look alot like Watson, IBM’s Jeopardy! playing supercomputer that’s getting ready to roll out with an all new look and a Memorial Sloan Kettering Cancer Center education in oncology.

How Facebook's Graph Search will affect Google, Technology, and Privacy

What has been both feared and expected is finally on its way: Facebook is building a better search; they're opening their vast stores of user data and giving us the ability to discover what’s inside. Lars Rasmussen, the mind that brought us Google Maps, is now hard at work creating Facebook’s new Graph Search, and from the looks of it, it’s going to put unprecedented power in the hands of its users.

How Big Data and Analytics will Change Society.

The mission statement of most police departments includes something like this, “our goal is to increase public safety, prevent crime, and protect human life.” With sufficient records of criminal behavior and analytics tools like Crimespotting, Cities can have the ability to predict when and where crimes are likely to take place and dispatch accordingly.

The Big Data Revolution

Want to learn more about how big data is changing business and how you can take advantage? Pick up our latest book: The Big Data Revolution.

"With everyone talking about Big Data and Data Science, its tough to know whom to listen to and how to make sense of it all. This book cuts through the noise and presents the reader with a roadmap for success. Whether you've been in this space for a while or are just coming up to speed, Secrets of the Big Data Revolution is a must read."
-Chris Crosby, CEO of Inflection Point Global

Trend Watch

What Higher Education Teaches us about Data-Driven Customer Retention

Rio Salado Community College is currently optimizing their student retention through focused testing and they are finding some truly telling results. We can learn a lot about customer retention and segmenting by studying what they've done.

The Growing Collaborative Consumption Market

Remember the Big Data mantra? how Big Data will enable us to better understand everything, reduce waste, and improve efficiency? Well honestly, without concrete examples it fades into the mass of voices shouting about how great the world is going to be. So lets take a moment to talk about collaborative consumption and it’s implications.

Andreas Weigend on the Future of Social Data

Dr. Weigend, Stanford Professor and former Chief Data Scientist at Amazon, tells us about social data and it's implications.

Big Data and Government Transparency

A wealth of government data is available to us today on .gov sites and private sites across the web. If we analyzed this data properly, we could build a rich understanding of how our government works and how it could be improved. But as the big data challenge dictates, the chokepoint is consumption.


Gartner splits the 2014 Business Intelligence Magic Quadrant in two.

In an interesting turn, Gartner decided this year to split the annual Magic Quadrant for Business Intelligence and Analytics Platforms.

New Gartner Magic Quadrant: Advanced Analytics Platforms

The big story this year is how Gartner split the Business

How RoboCharm is using data to optimize customer interactions

Let’s face it, the ultimate goal for any use of data is to drive profits, and more often than not that comes back to learning how to enga