• Home
  • Latest
  • Fortune 500
  • Finance
  • Tech
  • Leadership
  • Lifestyle
  • Rankings
  • Multimedia

Trendingnow

1

MacKenzie Scott alone accounted for one-third of America's $19.2 billion in megagifts last year

2

Philanthropy leader at Warren Buffett and Bill Gates’ Giving Pledge says children of billionaires are pushing them to give their wealth away faster

3

Ex-Google engineer says Larry Page, Sergey Brin and Sundar Pichai share the same trait—it's the lesson he swears by as a $7.2 billion AI CEO

1

MacKenzie Scott alone accounted for one-third of America's $19.2 billion in megagifts last year

2

Philanthropy leader at Warren Buffett and Bill Gates’ Giving Pledge says children of billionaires are pushing them to give their wealth away faster

3

Ex-Google engineer says Larry Page, Sergey Brin and Sundar Pichai share the same trait—it's the lesson he swears by as a $7.2 billion AI CEO
Big Data

The Big Apple’s Big Data advantage

By
Anne VanderMey
Anne VanderMey
Down Arrow Button Icon
By
Anne VanderMey
Anne VanderMey
Down Arrow Button Icon
August 20, 2012, 6:11 AM ET
Add Fortune on Google for similar content.



The new Microsoft (MSFT) research lab in Manhattan is the latest addition to New York City’s increasingly buzzy tech scene. Headed by mathematical physicist Jennifer Chayes, the lab will focus on Big Data analysis, parsing the vast troves of information created by the world’s digital denizens. Chayes spoke with Fortune about the lab, the world-changing potential of Big Data, and why New York City has an edge when it comes to tech’s next big thing.

You have a new lab in New York City, what are you working on?
We’re pretty much centered on Big Data questions — how Big Data can help answer social science questions, how it can help answer economic questions, and what it means for the interaction of the social sciences with technology. If you’re in that field, these are very, very exciting times.

What are some of the projects the lab is taking on?
Well, one example is we have researchers in economics and prediction engines looking at the bets people make. Generally speaking, if people place bets on certain things, they are often much more invested in that thing. So, when large numbers of people place bets on who’s going to win, say, a primary election, that’s a very good way of getting data. We have some researchers who have come up with really amazing precisions on who’s going to be winning the primaries based on the bets that people are making. You can also do this with sports events. You can do this with all kinds of events. The future of interactive entertainment will have a prediction engine component to it.

At the lab we also have leaders in machine learning. John Langford is building a machine learning platform that people all over the world are using. It’s called VW, which stands for Vowpal Wabbit. [Editor’s note: The name is Elmer Fudd speak for “Vorpal Rabbit.”] Which gives us an incredibly fast way to analyze huge data sets. And a few of our guys are setting up a large-scale experimental platform, using Amazon’s (AMZN) Mechanical Turk to find participants online. In the past, a lot of experiments would use 30 or 40 student volunteers. This is a much larger scale.

In general, the magnitude of the data that we are able to take in now dwarfs anything that we were able to do in the past, and so you need a new scientific method to answer new kinds of questions. It’s really a new age of social science.

MORE: Sony: Down and out in Tokyo

How is the lab settling in so far?
The lab opened on May 3 and now has 15 researches, and we’re actually hiring more. We expect to have very strong relationships to the universities in the city and to the startups. A lot of the guys in our lab have close colleagues in the startup scene. They’re very involved because, you know, all these scientists know each other. They go to the same conferences, they went to the same graduate school. It’s a very tight community.

We also have strong relationships with all the major universities in the area. New York Univeristy’s CUSP program, the new Center for Urban Science & Progress, is hiring 50 faculty and researchers, and is being headed by Steve Koonin, who was the undersecretary of energy for science. We’re going to be pursuing joint projects in mining urban data with them.

Some of our computational social scientists are teaching Columbia University courses. And we’re in touch with the Cornell Technion campus planners. The new campus opens on Roosevelt Island in about 5 years and will have a couple thousand graduate students and some 250 faculty. It will have three themes, all centered around Big Data. So it’s actually a really, really exciting time in the city right now.

What’s your take on New York City’s tech scene at the moment?
Silicon Alley is really becoming a hub for these data-intensive startups in Web 2.0 and beyond Web 2.0. New universities and new companies are coming in. Mayor Michael Bloomberg has really taken the lead here with the Applied Sciences Initiative, which has played a part in the NYU CUSP program, the Cornell Technion campus, and others. The initiative is really helping to provide the training for the people who are going to populate these companies.

Can New York compete with Silicon Valley in Big Data?
There are certain things that give New York an edge. The ad agencies are centered in the city and Madison Avenue, and a lot of the Big Data questions are being driven by the fact that the a lot of high-tech business models have to do with ads. There’s tons of creativity around this, but there’s also always a question of what’s going to fund all of these new products and all of these new initiatives. A lot of that is centered on ads. So it’s very natural that it’s gotten to be so strong in the city.

There’s an intersection of the design world, the ad industry, and Big Data that’s really happening in the city. There are obviously other companies that have that, like Apple (AAPL), but you really aren’t seeing this nexus that New York has anywhere else.

MORE: Hulu’s network drama

So are Silicon Valley and New York neck-and-neck in this field?
Well, there aren’t yet as many people in high-tech in the city as in Silicon Valley, but the rate of growth is certainly higher. And if you look at the boundary of social sciences and technology, some of the most interesting things there are happening in the city. It’s a different cut of high-tech due to the design influence and the ad agency influence, and there are different kinds of advances being made.

So, Big Data is really the next big thing?
I think that’s certainly true. Both scientifically, and in terms of where our advances are going to come from. In the past, we have had to build models for how we thought people interacted, and now we just let the machine tell us what the answer is by finding patters in these huge amounts of data. This research will power everything from online purchase suggestions, to the ads on social networks, to search engines taking your terms and looking at a trillion websites and coming up with the right ones to give back to you.

When you look a the new exciting companies out there, so many of them are really data-driven businesses. And of course cloud computing is going to be our infrastructure for dealing with Big Data. Everything from social media to physical infrastructure to biotech will be data-driven. Look at the Genome Project, it relies on these same algorithms.I would say this is the age of Big Data, certainly.

Sounds like a shift is underway in New York.
You’ve got to understand that it’s the age of big data and it’s the age of the geek. Years ago, nobody wanted to say that they were a geek and now it’s kind of cool. It’s New York City, and it’s filled with geeks and they’re giving geeky names to their projects.

About the Author
By Anne VanderMey
See full bioRight Arrow Button Icon
Add Fortune on Google for similar content.

Latest in

Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025

Most Popular

Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Finance
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam
By Fortune Editors
October 20, 2025
Fortune Secondary Logo
Rankings
  • 100 Best Companies
  • Fortune 500
  • Global 500
  • Fortune 500 Europe
  • Most Powerful Women
  • World's Most Admired Companies
  • See All Rankings
  • Lists Calendar
Sections
  • Finance
  • Fortune Crypto
  • Features
  • Leadership
  • Health
  • Commentary
  • Success
  • Retail
  • Mpw
  • Tech
  • Lifestyle
  • CEO Initiative
  • Asia
  • Politics
  • Conferences
  • Europe
  • Newsletters
  • Personal Finance
  • Environment
  • Magazine
  • Education
Customer Support
  • Frequently Asked Questions
  • Customer Service Portal
  • Privacy Policy
  • Terms Of Use
  • Single Issues For Purchase
  • International Print
Commercial Services
  • Advertising
  • Fortune Brand Studio
  • Fortune Analytics
  • Fortune Conferences
  • Business Development
  • Group Subscriptions
About Us
  • About Us
  • Press Center
  • Work At Fortune
  • Terms And Conditions
  • Site Map
  • About Us
  • Press Center
  • Work At Fortune
  • Terms And Conditions
  • Site Map
  • Facebook icon
  • Twitter icon
  • LinkedIn icon
  • Instagram icon
  • Pinterest icon

Latest in

As JPMorgan’s CEO race heats up, the case for a two-person succession contest is put to the test
C-SuiteNext to Lead
As JPMorgan’s CEO race heats up, the case for a two-person succession contest is put to the test
By Ruth UmohJune 29, 2026
2 hours ago
wendy
Commentary250 Years of Innovation
Wendy Schmidt: Three centuries of science is something to celebrate
By Wendy SchmidtJune 29, 2026
2 hours ago
a
Commentary250 Years of Innovation
Atomic Industries CEO: America spent 60 years retreating from manufacturing. The next 100 are about building it back
By Aaron SlodovJune 29, 2026
2 hours ago
Today’s top high-yield savings rates: Up to 5.00% on June 29, 2026
Personal FinanceSavings accounts
Today’s top high-yield savings rates: Up to 5.00% on June 29, 2026
By Glen Luke FlanaganJune 29, 2026
3 hours ago
Top CD rates today, June 29, 2026: Lock in up to up to 4.40%
Personal FinanceBanks
Top CD rates today, June 29, 2026: Lock in up to up to 4.40%
By Glen Luke FlanaganJune 29, 2026
3 hours ago
Photo: Kevin Warsh
EconomyMarkets
President Trump will not get what he wants from Kevin Warsh, a source tells us, as inflation will force the Fed upwards
By Jim EdwardsJune 29, 2026
3 hours ago

Most Popular

MacKenzie Scott alone accounted for one-third of America's $19.2 billion in megagifts last year
Success
MacKenzie Scott alone accounted for one-third of America's $19.2 billion in megagifts last year
By Sydney LakeJune 25, 2026
4 days ago
Philanthropy leader at Warren Buffett and Bill Gates’ Giving Pledge says children of billionaires are pushing them to give their wealth away faster
Success
Philanthropy leader at Warren Buffett and Bill Gates’ Giving Pledge says children of billionaires are pushing them to give their wealth away faster
By Preston ForeJune 27, 2026
2 days ago
Ex-Google engineer says Larry Page, Sergey Brin and Sundar Pichai share the same trait—it's the lesson he swears by as a $7.2 billion AI CEO
Success
Ex-Google engineer says Larry Page, Sergey Brin and Sundar Pichai share the same trait—it's the lesson he swears by as a $7.2 billion AI CEO
By Orianna Rosa RoyleJune 28, 2026
1 day ago
Cristiano Ronaldo is soccer's first-ever billionaire: He went from begging for burgers outside McDonald's to landing a $400 million contract
Success
Cristiano Ronaldo is soccer's first-ever billionaire: He went from begging for burgers outside McDonald's to landing a $400 million contract
By Preston ForeJune 28, 2026
1 day ago
The retired college professor fighting a $313 trespassing ticket in Wisconsin thinks he's part of a national struggle
Environment
The retired college professor fighting a $313 trespassing ticket in Wisconsin thinks he's part of a national struggle
By Catherina GioinoJune 28, 2026
1 day ago
Iran is forcing the U.S. into an escalation trap as a 'shadow war' over the Strait of Hormuz heats up that could kill the tenuous ceasefire
Politics
Iran is forcing the U.S. into an escalation trap as a 'shadow war' over the Strait of Hormuz heats up that could kill the tenuous ceasefire
By Jason MaJune 28, 2026
19 hours ago

© 2026 Fortune Media IP Limited. All Rights Reserved. Use of this site constitutes acceptance of our Terms of Use and Privacy Policy | CA Notice at Collection and Privacy Notice | Do Not Sell/Share My Personal Information
FORTUNE is a trademark of Fortune Media IP Limited, registered in the U.S. and other countries. FORTUNE may receive compensation for some links to products and services on this website. Offers may be subject to change without notice.