Fast Company/Daniel Terdiman: How AI Helps The Intelligence Community Find Needles In The Haystack

Fast Company

    10.24.174:30 am robot revolution

How AI Helps The Intelligence Community Find Needles In The Haystack

Technology from the startup Primer helps analysts find even the most obscure events they need to know about in a sea of data.
[Animation: Primer]

By Daniel Terdiman5 minute Read

Last July, more than two dozen people were hurt during a shooting at a nightclub in Little Rock, Arkansas, an incident that was gang-related, according to local and national media coverage.
advertisement

If you were an analyst in the American intelligence community covering terrorism issues related to Russia, that conclusion wouldn’t have interested you. What would have stopped you in your tracks, though, was that the next day the Russian consulate tweeted that the attack may have been terrorism involving Russians.

In the flood of news coverage of the attack and the countless other developments in the world, you might have missed the terrorism theory. After all, humans can only process so much information manually. But if you’d been using a new visualization tool from the artificial intelligence startup Primer, you would seen the intense interest in Russian media about the Little Rock attack. Though that theory eventually proved to be incorrect, you’d at least have been aware of the possibility, and been able to make some decisions about what you wanted to report to your superiors.

Today, Primer is coming out of stealth. The 35-person startup, which recently closed a $14.7 million Series A round of funding, has developed a machine learning system that is able to quickly search through tens of millions of data sources–news articles, academic papers, social media posts, and so on–to surface the kinds of information that is essential to both intelligence analysts and corporate analysts alike. The system is also capable of delivering the most salient data points in natural language that approaches the level of what a human analyst might write.

Primer’s initial customers are In-Q-Tel, a nonprofit venture capital firm that invests in companies developing technologies useful to the CIA and other intelligence agencies; Walmart, and Singapore’s $100 billion sovereign wealth fund, GIC.

“For us, the big goal is to build a technology that can read and write, and help us understand the world,” said Primer CEO Sean Gourley, “as it becomes increasingly volatile, uncertain, and complex.”

Primer’s pitch to corporate and government clients is its technology’s ability to sift through immense quantities of data. According to IDC, the total amount of data produced globally will grow from 16.1 zettabytes (a trillion gigabytes) in 2016 to 163 zettabytes by 2025. Of that, 5.2 zettabytes will be subject to data analysis by 2025, 50 times more than last year. Artificial intelligence systems are expected to touch about 1.4 zettabytes by 2025, 100 times more than last year.
advertisement

Those are big numbers, and Primer believes its technology can help its customers find the meaningful information that lives at the far end of the long tail, things that might be hidden in, as Gourley puts it, the seventh paragraph on page 163 of an obscure report. A human analyst might never have the time in her day to uncover something so deep in the stack, but Primer’s system is meant to highlight it if matches the analyst’s interests–and do so in a way that’s easy to digest and pass on to others.

Part of what Primer is unveiling today is its visualization tool–built by a former New York Times infographic artist. As shown, the tool shows the evolution, over time, of tens of millions of Russian- and English-language articles on a map of the world, with color-coded geographical hotspots indicating media interest in topics related to terrorism. That’s how an American intelligence analyst might have first noticed Russian interest in the Little Rock attack.

Similarly, the tool can easily show how, in September 2016, there were a significant number of articles written in Beslan, Russia, about a new documentary film that reports on a massive hostage crisis that took place there in 2004. Or how last January, Russian media was obsessed with coverage of a court case in The Hague brought by the Ukrainian government alleging support by Moscow for terrorism in Crimea.

The visualization tool–which Primer clients can use to search for numerous topics of interest–is meant to be advanced day by day, which allows analysts to see, at a glance, where the daily hotspots of interest were in both Russian and English media. In essence, it’s about being able to quickly triage the most important pieces of information in a very messy mountain of unstructured data.
advertisement

“Primer’s technology is poised to revolutionize the way our Intelligence Community Partners consume and prioritize information, helping them to identify emerging areas of interest from real-time data,” said George Hoyem, the managing partner of Investments at In-Q-Tel, in a statement provided to Fast Company by Primer. “Many enterprise companies claim they’re leveraging artificial intelligence to generate insights from unstructured content—but Primer is one of the few actually delivering on that promise.”

The same strategy applies to corporations, Gourley says. Although he wouldn’t spell out exactly how Walmart is using Primer’s tools, he explained that, for example, the retail giant might have an internal analyst covering the snacks or beverages businesses. That person would likely be spending part of every day searching for data on organic beef, or looking for consumer trends, new behaviors, or health studies that could help produce internal reports.

As with those in the intelligence world, these corporate analysts face an overwhelming amount of data on a daily basis, and Primer believes its tools can give its customers a leg up on those who rely on more manual approaches to sifting through the innumerable sources of information.

The system is also meant to be quick. Depending on how much computing power a customer wants to devote to it, Primer’s tools can generate reports in between five to 30 minutes.

Ultimately, Gourley argues, customers using Primer’s technology will find themselves with more time to devote to following hunches–something they may not have been able to do previously because they’d have had to be heads-down trying to digest the never-ending flood of data.

“As we automate, it starts to save time,” he says, “and then allows humans to say, ‘What happens if we look at this?’ We get curious, and we do that in ways that are cheap for machines, but expensive for humans. We like humans telling us stories, but we also like machines telling us stories at scale we can’t.”
About the author

Daniel Terdiman is a San Francisco-based technology journalist with nearly 20 years of experience. A veteran of CNET and VentureBeat, Daniel has also written for Wired, The New York Times, Time, and many other publications.

More
You Might Also Like:

    Facebook’s New Lab Bolsters Montreal’s Bragging Rights As An AI Hub
    How Facebook Used AI To Make The Trippy Effects In This VR Film
    Don’t Fear Super-Intelligent Robots. Fear Dumb, Unpredictable Ones

Technology Newsletter
Receive special Fast Company offers.
See All Newsletters
Video
I Took Levi's And Google's First Smart Jacket Out For A Spin
Ideas
Ideas
Average Americans Are Giving Away Less Money And It’s A Big Problem
Ideas
Google’s New Office Will Be Heated And Cooled By The Ground Underneath
Ideas
Why Impact Investing Funds Need To Invest In Creativity
Entertainment
Entertainment
A Sea Shepherd Documentary Created By Two Unlikely Partners
Entertainment
Andy Cohen And Cecile Richards On Activism And Pop Culture
Entertainment
How CoverGirl Built An Ad Campaign Around Multicultural Badassness
Co.Design
Products
A Peek At The Design Method Snohetta Uses To Get Community Buy-In
Products
For Foursquare, Location Is About Way More Than GPS
Products
Snøhetta’s Underwater Restaurant Looks Like A Stranded Spaceship
Fast Company
Technology
Gmail add-ons put tools like Trello and QuickBooks in your inbox
Entertainment
A Sea Shepherd Documentary Created By Two Unlikely Partners
Leadership
Slow Day At Work? Try One Of These Energizers

    AdvertisePrivacy PolicyTermsContactAbout UsSite MapFast Company & Inc © 2017 Mansueto Ventures, LLCDigital Advertising Alliance (DAA) Self-Regulatory Program