Solve These Tough Data Problems and Watch Job Offers Roll In

Late in 2015, Gilberto Titericz, an electrical engineer at Brazil’s state oil company Petrobras, told his boss he planned to resign, after seven years maintaining sensors and other hardware in oil plants. By devoting hundreds of hours of leisure time to the obscure world of competitive data analysis, Titericz had recently become the world’s top-ranked data scientist, by one reckoning. Silicon Valley was calling. “Only when I wanted to quit did they realize they had the number-one data scientist,” he says.

Petrobras held on to its champ for a time by moving Titericz into a position that used his data skills. But since topping the rankings that October he’d received a stream of emails from recruiters around the globe, including representatives of Tesla and Google. This past February, another well-known tech company hired him, and moved his family to the Bay Area this summer. Titericz described his unlikely journey recently over colorful plates of Nigerian food at the headquarters of his new employer, Airbnb.

Titericz earned, and holds, his number-one rank on a website called Kaggle that has turned data analysis into a kind of sport, and transformed the lives of some competitors. Companies, government agencies, and researchers post datasets on the platform and invite Kaggle’s more than one million members to discern patterns and solve problems. Winners get glory, points toward Kaggle’s rankings of its top 66,000 data scientists, and sometimes cash prizes.

Ryan Young for Wired

Alone and in small teams with fellow Kagglers, Titericz estimates he has won around $ 100,000 in contests that included predicting seizures from brainwaves for the National Institutes of Health, the price of metal tubes for Caterpillar, and rental property values for Deloitte. The TSA and real-estate site Zillow are each running competitions offering prize money in excess of $ 1 million.

Veteran Kagglers say the opportunities that flow from a good ranking are generally more bankable than the prizes. Participants say they learn new data-analysis and machine-learning skills. Plus, the best performers like the 95 “grandmasters” that top Kaggle’s rankings are highly sought talents in an occupation crucial to today’s data-centric economy. Glassdoor has declared data scientist the best job in America for the past two years, based on the thousands of vacancies, good salaries, and high job satisfaction. Companies large and small recruit from Kaggle’s fertile field of problem solvers.

In March, Google came calling and acquired Kaggle itself. It has been integrated into the company’s cloud-computing division, and begun to emphasize features that let people and companies share and test data and code outside of competitions, too. Google hopes other companies will come to Kaggle for the people, code, and data they need for new projects involving machine learning—and run them in Google’s cloud.

Kaggle grandmasters say they’re driven as much by a compulsion to learn as to win. The best take extreme lengths to do both. Marios Michailidis, a previous number one now ranked third, got the data-science bug after hearing a talk on entrepreneurship from a man who got rich analyzing trends in horseraces. To Michailidis, the money was not the most interesting part. “This ability to explore and predict the future seemed like a superpower to me,” he says. Michailidis taught himself to code, joined Kaggle, and before long was spending what he estimates was 60 hours a week on contests—in addition to a day job. “It was very enjoyable because I was learning a lot,” he says.

Michailidis has since cut back to roughly 30 hours a week, in part due to the toll on his body. Titericz says his own push to top the Kaggle rankings, made not long after the birth of his second daughter, caused some friction with his wife. “She’d get mad with me every time I touched the computer,” he says.

Entrepreneur SriSatish Ambati has made Kagglers a core strategy of his startup, H2O, which makes data-science tools for customers including eBay and Capital One. Ambati hired Michailidis and three other grandmasters after he noticed a surge in downloads when H2O’s software was used to win a Kaggle contest. Victors typically share their methods in the site’s busy forums to help others improve their technique.

Related Stories

H2O’s data celebrities work on the company’s products, providing both expertise and a marketing boost akin to a sports star endorsing a sneaker. “When we send a grandmaster to a customer call their entire data-science team wants to be there,” Ambati says. “Steve Jobs had a gut feel for products; grandmasters have that for data.” Jeremy Achin, cofounder of startup DataRobot, which competes with H2O and also has hired grandmasters, says high Kaggle rankings also help weed out poseurs trying to exploit the data-skills shortage. “There are many people calling themselves data scientists who are not capable of delivering actual work,” he says.

Competition between people like Ambati and Achin helps make it lucrative to earn the rank of grandmaster. Michailidis, who works for Mountain View, California-based H2O from his home in London, says his salary has tripled in three years. Before joining H2O, he worked for customer analytics company Dunnhumby, a subsidiary of supermarket Tesco.

Large companies like Kaggle champs, too. An Intel job ad posted this month seeking a machine-learning researcher lists experience winning Kaggle contests as a requirement. Yelp and Facebook have run Kaggle contests that dangle a chance to interview for a job as a prize for a good finish. The winner of Facebook’s most recent contest last summer was Tom Van de Wiele, an engineer for Eastman Chemical in Ghent, Belgium, who was seeking a career change. Six months later, he started a job at Alphabet’s artificial-intelligence research group DeepMind.

H2O is trying to bottle some of the lightning that sparks from Kaggle grandmasters. Select customers are testing a service called Driverless AI that automates some of a data scientist’s work, probing a dataset and developing models to predict trends. More than 6,000 companies and people are on the waitlist to try Driverless. Ambati says that reflects the demand for data-science skills, as information piles up faster than companies can analyze it. But no one at H2O expects Driverless to challenge Titericz or other Kaggle leaders anytime soon. For all the data-crunching power of computers, they lack the creative spark that makes a true grandmaster.

“If you work on a data problem in a company you need to talk with managers, and clients,” says Stanislav Semenov, a grandmaster and former number one in Moscow, who is now ranked second. He likes to celebrate Kaggle wins with a good steak. “Competitions are only about building the best models, it’s pure and I love it.” On Kaggle, data analysis is not just a sport, but an art.


Data Sheet—Amazon, Google, and Microsoft Have Plenty to Celebrate Right Now

There have been times in years past that Amazon, Alphabet (read: Google), and Microsoft have had rough patches. Amazon has suffered quarters-long profit droughts. Alphabet has given its investors agita over profligate spending on non-core products. Microsoft’s growth—if not its profit engine—stalled for years, causing its stock to idle too.

The middle months of 2017 have not been one of those times for any of these companies.

Each, for their own reasons, reported anywhere from solid to spectacular earnings results Thursday. Amazon didn’t make any more money than it did the year before, but its growth resembles a startup more than the grownup it is. Alphabet’s ad machine is humming. Microsoft, a cloud computing convert, has completed the most impressive tech turnaround since Lou Gerstner taught an elephant to dance.

That these giants are charging simultaneously—throw in Twitter and Intel too—is no coincidence. Their success is the digital transformation explained. Where the action once was in high-trafficked shopping malls, print and TV advertising, and client-server software, businesses that deliver digitally rule the roost today.

This is the point in any account of Big Tech’s success where journalists insert the “to be sure” paragraph, as in, to be sure, this garish performance will draw the attention of regulators, legislators, and other haters that could ruin the fun. The New York Times and The Wall Street Journal both turned in laudable examples of the art form.

Those concerns only matter, of course, when they matter. For now, they don’t. It is party time. And what a party it is.


The cinema business is thinking about experimenting with dynamic pricing, as Bloomberg reported this week. Dynamic pricing—charging more when goods and services are in high demand and short supply and less when the opposite is true—isn’t new. Gasoline retailers, hoteliers, and airlines have been deploying the technique for years. Uber did it too—to great success but also with dollops of controversy largely attributed to its own stubbornness and insensitivity.

According to Bloomberg, Regal Cinemas will charge more for hits and less for flops, an intriguing concept given that the real money is in egregiously overpriced popcorn and soda, not movies.

Long live capitalism. And see you at the movies.


Big capital gains. As Adam noted, it was one of the busiest days ever for people who follow big tech company earnings.

If you want a few more interesting tidbits below the headlines, Microsoft said its annual run rate for its Azure cloud unit exceeded $ 20 billion for the first time. Google said pre-orders for its new Pixel 2 phones were double the number from last year and that YouTube watching on old-fashioned TV sets jumped 70% to 100 million hours a day. CFO Ruth Porat also disclosed massive cuts in spending on Google Fiber. With the Whole Foods chain now in hand, Amazon broke out sales in physical locations for the first time: $ 1.3 billion, or 5% of total sales. Intel surprised everyone with flat revenue in its PC chips unit. Stocks of all four companies rose in premarket trading on Friday. Microsoft was up 6%, Google parent Alphabet 4%, Amazon 8%, and Intel 3%.

Pre-panic mode. Fresh off its attack on the grocery business, Amazon appears closer than ever to shaking up the prescription drug market next. The e-commerce giant has obtained wholesale pharmacy licenses in at least 12 states, including Louisiana, New Jersey, and Michigan, the St Louis Post-Dispatch reported on the basis of public records searches. The fallout is already here, as CVS’s bid for Aetna is said to be motivated in part by the looming specter of Amazon’s entry.

Munch all you want, we’ll make more. With pre-orders for the iPhone X starting at midnight Pacific Time on Friday, Apple quickly ran out of phones for immediate delivery on November 3. Delays were out to six weeks on Friday morning, though according to some reports, wireless carrier Sprint still had units available for next week for Sprint customers.

Horse, meet barn door. After Russia used social networks to meddle in the 2016 election, Twitter said it would ban advertisements from any accounts owned by Russian news outlets Russia Today and Sputnik. The bans are part of Twitter’s “ongoing commitment to help protect the integrity of the user experience,” the company said.

Wrist slap. HR services startup Zenefits and co-founder Parker Conrad will pay almost $ 1 million, but won’t have to make an admission of guilt to settle charges they misled investors. Federal regulators said the company’s failure to disclose it was not in compliance with state insurance laws constituted “false and misleading statements and omissions.”

Momentum play. The tech IPO train is still steaming forward. Cloud security startup Zscaler has made a confidential filing to go public, TechCrunch reported. The leak comes just as cybersecurity firm ForeScout Technologies priced its IPO at $ 22. It starts trading on Friday under the symbol “FSCT.”

Conspiracy theories. The National Archives released almost 3,000 documents connected to the assassination of President Kennedy and put them all online. Spoiler alert: the Russians were just as surprised as everyone else by Lee Harvey Oswald’s shooting of the president.


We are, unfortunately, beset by news of the continuing problems of discrimination, bias, and harassment in the workplace. One related problem feeding the culture of abuse is the lack of diversity, particularly in leadership roles, at many companies. Consultants Jack Zenger and Joseph Folkman have studied the problem and put a lot of the blame at the top: leaders who aren’t good self-critics of their own biases.

In fact, their surveys found that leaders who were found to be lowest rated at valuing diversity were more likely to think they were doing a great job at inclusiveness:

The implications of this data are: leaders are not good judges of their own effectiveness on valuing diversity; and those leaders who are poorest fail to see the problem, while those who are the best don’t realize their skill and capability. This phenomenon is not limited to inclusiveness — the Dunning-Kruger effect, for example, explains that unskilled people are particularly prone to thinking they are more skilled than they are. Conversely, our research has found that many of the most skilled leaders are too humble and modest in assessing their strengths. Nonetheless, we find this result particularly disturbing when we see it in the context of inclusivity. While a person’s effectiveness with any skill always needs to be based on the evaluations of others, rather than self-perception, it seems especially true in this case. Inclusivity is particularly in the eye of the beholder. You might intend to be inclusive, and even think you are inclusive, but your impact on others might be very different.


A few interesting longer reads I came across that are suitable for your weekend reading pleasure.

Love in the Time of Robots
In a secluded room at IRL, a collection of androids is stored and maintained: his hardest workers. Arranged in this space today, with its blackout curtains, thin corporate carpeting, and shelves cluttered with cables and monitors and an array of wigs, is a pair of his replicas of grown women. They are models of the Geminoid F series. The name is a play off geminus (Latin for “twin”), a reminder that their human counterparts exist somewhere in the world.

Three MacBook Mistakes: Will Apple Correct Course?
Apple’s not a company that backtracks easily. It’s got a lot of pride and a reputation for moving forward. And yet every so often the company makes a decision that it thinks is right and is ultimately proven to be completely wrong. In 2008, Apple removed FireWire from the MacBook, only to put it back in 2009. In 2009 the iPod Shuffle went buttonless, only to revert to its previous buttony design in 2010. The third-generation iPod, with its row of touch-sensitive controls, was a similar design cul-de-sac. And the most recent example is the Mac Pro, which Apple introduced to fanfare, but ultimately admitted was a mistake.

The Scientists Persuading Terrorists to Spill Their Secrets
The interviewer wanted him to provide an account of his plan, and to reveal with whom, if anyone, he has been conspiring. But the detainee – we will call him Diola – refused to divulge any information. Instead, he expounded grandiloquently on the evils of the British state for 42 minutes, with little interruption. When the interviewer attempted questions, Diola responded with scornful, finger-jabbing accusations of ignorance, naivety and moral weakness: “You don’t know how corrupt your own government is – and if you don’t care, then a curse upon you.”

When the Revolution Came for Amy Cuddy
Cuddy became famous in her field for a 2010 study about the effects of “power poses.” The study found that subjects who were directed to stand or sit in certain positions — legs astride, or feet up on a desk — reported stronger “feelings of power” after posing than they did before. Even more compelling than that, to many of her peers, was that the research measured actual physiological change as a result of the poses: The subjects’ testosterone levels went up, and their cortisol levels, which are associated with stress, went down.


They’re saying it looks like a Pokemon monster, but Sinosauropteryx was a real dinosaur with feathers and the “bandit mask” shading of a raccoon or badger, researchers studying its 120-million-year-old fossils say. Somebody tell Wes Anderson. Maybe the sequel to Fantastic Mr. Fox can feature dinosaurs, too?


Social Capital Will Let Data Decide Where It Invests

The firm launched a data-focused investment platform, called “capital-as-a-service.”

This article originally ran in Term Sheet, Fortune’s newsletter about deals and dealmakers. Sign up here.

On Monday, we talked about how SoftBank’s Masayoshi Son is capitalizing on the “information revolution” with his monster investment fund.“Those who rule data will rule the entire world,” he said. On Tuesday, MasterCard’s CEO Ajay Banga said at a conference in Saudi Arabia that “data is the new oil.” The theme continues nicely for the third day in a row after Social Capital’s launch of its new data-focused investment platform.

On Wednesday, the firm announced an operating system for early-stage investing called “capital-as-a-service.” Put simply, Social Capital will invest in startups without having to go through the process of a traditional pitch. (Which means fewer humans have to listen to pitches that start with: “Imagine a jacket, but for your legs.” Really.)

“No hoops, no $ 7 artisanal coffee chats, no designer pitch decks, no bias, no politics, no bullshit,” Ashley Carroll, the partner in charge of overseeing the project, explains in a Medium post.

Here’s how the self-serve platform works: Entrepreneurs fill out a questionnaire, submit relevant figures such as revenue and raw engagement data, and/or grant the firm access to its cloud services. Social Capital will then evaluate the company and write a check or pass and deliver feedback.

Social Capital evaluated nearly 3,000 companies during its private beta and committed to funding several dozen across 12 countries. An interesting byproduct of the data-oriented approach was that CEO demographics skewed 42% female and majority non-white. (For context, female founders received 2.19% of venture capital funding in 2016.) In an email to Term Sheet, Social Capital CEO Chamath Palihapitiya called the 42% data point “simply fucking awesome.”

It’s no surprise that the firm is taking this route. For years, Palihapitiya’s vision has been pretty clear — operational experience coupled with a focus on data. But as Social Capital begins to expand and veer toward being stage-agnostic, some people aren’t on board with the direction the firm is taking. Social Capital co-founder Mamoon Hamid abruptly departed in August to join Kleiner Perkins Caufield & Byers. The company’s third co-founder Ted Maidenberg won’t be making new investments or participating in any future funds, although he’s currently still supporting the portfolio.

In spite of the high-profile turnover, Palihapitiya seems to be hyper-focused on this data-driven approach, and he reiterated his plan to make Social Capital a full-service capital partner to the businesses it invests in throughout their lifecycles. He added:

“CaaS is designed for entrepreneurs who are either over-served or under-served by today’s venture status quo. In that first bucket: founders who prefer a low-touch, highly efficient funding process, or don’t want to give up a large chunk of ownership in their company. In the latter: founders outside Silicon Valley and the US more generally who often don’t have access to Silicon Valley-based firms, nor the networks necessary to get the right warm intro.”


EU-U.S. data transfer pact passes first annual review

BRUSSELS (Reuters) – A year-old pact underpinning billions of dollars of transatlantic data transfers won a green light from the European Union on Wednesday after a first review to ensure Washington protects Europeans’ data stored on U.S. servers.

The EU-U.S. Privacy Shield was agreed last year after everyday cross-border data transfers were plunged into limbo when the EU’s top court struck down a previous data transfer pact in 2015 because it allowed U.S. spies excessive access to people’s data.

The European Commission last month conducted its first annual review of the framework as it seeks to ensure the United States lives up to its promises to better protect Europeans’ data when they are transferred across the Atlantic – failing which it could suspend the Privacy Shield.

The EU executive said it was satisfied that the framework continues to ensure adequate protection for Europeans’ personal data although it asked Washington to improve the way it works, including by strengthening the privacy protections contained in a controversial portion of the U.S. Foreign Intelligence Surveillance Act (FISA).

The conclusion will come as a relief to the more than 2,400 companies signed up to the scheme, including Alphabet Inc’s Google, Facebook and Microsoft, especially since the Privacy Shield is already being challenged in court by privacy activists.

The Commission said the U.S. Department of Commerce should be more pro-active in monitoring companies’ compliance with the privacy obligations in the framework.

“Transatlantic data transfers are essential for our economy, but the fundamental right to data protection must be ensured also when personal data leaves the EU. Our first review shows that the Privacy Shield works well, but there is some room for improving its implementation,” EU Justice Commissioner Vera Jourova said.

Companies wanting to transfer Europeans’ personal data outside the bloc have to comply with tough EU data protection rules which forbid them from transferring personal data to countries deemed to have inadequate privacy protections unless they have special legal contracts in place.

The Privacy Shield allows firms to move data across the Atlantic without relying on such contracts, known as model clauses, which are more cumbersome and expensive.

The Commission urged the United States to appoint a permanent Privacy Shield Ombudsperson – a new office that was created to deal with complaints from EU citizens about U.S. spying, but which is currently only filled on an “acting” basis.

It also urged Washington to fill empty posts on the Privacy and Civil Liberties Oversight Board.

In addition, the Commission said it would welcome privacy protections for foreigners contained in a Presidential Policy Directive issued by former U.S. President Barack Obama being enshrined in FISA.

Section 702 of FISA – that allows the U.S. National Security Agency (NSA) to collect and analyze emails and other digital communications of foreigners living overseas – will expire at the end of the year unless it is re-authorized by the U.S. Congress.

Reporting by Julia Fioretti; Editing by Adrian Croft


8 universities at the forefront of big data

Universities at the forefront of big data

Image by Thinkstock

Big data has exploded in a way that has left companies unable to find enough qualified candidates to hire, and schools can’t churn out skilled data scientists fast enough. Up until now, boot camps have helped fill the data skills gap and plenty of colleges and universities have created master’s programs in this burgeoning field. And, while it’s true that boot camps are a great way to learn new skills fast and a master’s degree is a great way to move your career in another direction, there has been a glaring lack of undergraduate programs for young people looking to get right into the industry after graduation.

To read this article in full or to leave a comment, please click here

All articles

Week in Review: Come to Our DX Summit + Risky Data Management

Grab your calendars and mark the dates: We kicked off preregistration for our second annual DX Summit this week. We’d love to have you join us to learn the strategies, technologies and thought leadership that deliver game-changing digital experiences — and we’re offering an incentive for early registration.

Continue reading…
Cloud Computing


All articles