Opendoor discusses the secret sauce: ‘A deeper mechanism to the world’ | ZDNet

Opendoor, the seven-year-old startup that went public in February via merger with a special-purpose acquisition company, or SPAC, believes it has the artificial intelligence capability to optimize what is currently a very messy task of selling your home.

The company will give you cash money, after answering a few questions in the app, so you can be on your way to unloading your home without the tedium of weeks or months of dealing with brokers and buyers. It’s all about increasing the liquidity of homes, and bringing the residential real estate market online. The market is only 1% “penetrated” as far as online sales, according to the company.

Achieving all that involves a healthy dose of machine learning forms of AI, according to the company. 

But just what kind of ML? The details are somewhat obscure. In the company’s prospectus for IPO, Opendoor describes using ML on a unique dataset of home sales to derive the pricing predictions that will help it decide what price to pay a homeowner:

We have conducted over 150,000 home assessments during which we collect over 100 data points on each home and its surroundings. Utilizing this base of unique offline data, our algorithms use machine learning to drive pricing decisions through demand forecasting, outlier detection, risk pricing, and inventory management.

To try to uncover the nature of those algorithms — the secret sauce, as it were — ZDNet spoke to the company’s head of data science, Kushal Chakrabarti, via Zoom. Chakrabarti signed on to Opendoor just a year ago, after serving previously in an advisory capacity. 

Chakrabarti came to machine learning via a circuitous path. A decade ago, as a research scientist at the University of California at Berkeley, where he got his degree, Chakrabarti worked on the Human Genome Project, the quintessential Big Data application of our time. “I’ve been working on data science since before it had a name,” he remarked. 

After Berkeley, Chakrabarti had numerous posts, both in established companies, such as Amazon, where he served as engineering lead, and startups he helped create, such as micro-loan company Vittana. 

In many of these instances, he worked with the technologies of machine learning, such  as recommendation systems at Amazon back in 2005. 

Also: Amazon AWS’s AI team seeks the profound in the industrial

Despite that fact, “I was a skeptic of a lot of the deep learning work that was coming out in those early days,” recalled Chakrabarti. Eventually, he came around. “It’s unquestionable at this point that it’s clearly better at a number of things,” he said.

Deep learning is a “really, really good tool to understand how lots of different bits of information interact with each other, and dramatically better at teasing out those interactions than humans have ever been.”

The scale of the problem at Opendoor, namely right-sizing the entire housing market, is one of the big things that attracted Chakrabarti

“There aren’t that many trillion-dollar problems out there,” observed Chakrabarti. “This is one of the most challenging problems I’ve ever come across.”

Deep learning, Chakrabarti said, is “one tool we’re using” to plumb the depths of that trillion-dollar problem.

What’s hard about solving the home sales question is the nature of the problem. It consists of numerous variables whose mutual dependencies have to be  inferred from data.

“There are tens of millions of homes we have to consider,” explained Chakrabarti. The common mis-conception, he said, is that it’s all “location, location, location,” in real estate. In fact, there isn’t just one point at any moment in time, there are a plethora. 

“There’s location, but then there’s three and four and five bedrooms, and there’s whether the view has a northern exposure, etc., etc.” explained Chakrabarti. 

“If every home is unique, then how do you really say anything about it?” becomes the problem. 

“All of which is to say, these interactions are really, really, complex,” he said. “There are hundreds and thousands and millions of interactions between any one of these homes, and what kinds of information are relevant or not.”

“What we fundamentally do is use deep learning to understand better what those linkages between one home versus another home might be in terms of what might be the best price for it,” he said.

The secret sauce, presumably, is precisely how to turn that multi-variate, combinatorial problem into an algorithm that yields a price with confidence. 

As to what that algorithm is, “It’s hard for me to get into the full details for reasons that you can imagine around our intellectual property,” said Chakrabarti.

Also: AI in sixty seconds

The secret sauce, in other words, remains a secret, for the most part. In an attempt to tease out some details, ZDNet asked what the “objective function” is for the deep learning that Opendoor uses. The objective function, or the loss function, as it’s sometimes called, refers to the score that any deep learning algorithm seeks to optimize.

It stands to reason that discovering the right price is part of the objective function. But it’s also conceivable the objective function is something non-intuitive, something less obvious of which the right price could be a byproduct. 

So, just what is that objective function? 

“It’s a great question,” said Chakrabarti. “I suspect I can’t get into that,” he said, explaining that, again, such detail is part of the protected intellectual property of the company. 

What Chakrabarti is able to describe is the goal, in broad terms, of the entire endeavor. The over-arching goal is for a homeowner to “get a price that they trusted,” meaning, could feel comfortable was the best deal that could be obtained, a price “that made sense for them.”

Given tight profit margins, one would imagine that  optimizing the spread could be a part of the objective function, meaning, the difference between Opendoor’s cost to acquire a home, including interest and holding period costs, and the price for which the company can turn around and sell it. 

Opendoor had gross profit equal to just 13.4% of revenue in the most recent quarter, and “contribution profit,” where holding and selling costs are deducted, of just 10.8%. Hence, a percentage or two spread can be a big difference in the profit picture for Opendoor. 

“It is definitely not about maximizing the spread,” said Chakrabarti, referring to the objective function. But it does involve understanding how the spread can be optimized, he said, which could include reducing the spread. 

“That’s exactly what a lot of the tech investments go into, to understand better, to reduce the fees that we charge — as we’ve done over time — to provide an even more trustworthy price to home-sellers,” is how Chakrabarti described it. 

“I think what a lot of my team focuses on, and what the company focuses on, is how do we minimize that so that we can reach the most amount of people over time,” while, of course, balancing the interests of the company’s shareholders for profit.  

One of the most challenging questions that Chakrabarti  faces in helping Opendoor is knowing how to measure success. On one level, progress is an indicator of success. The company has opened operations in 18 new residential markets in the last five months, noted Chakrabarti, so that it now operates in a total of 39 markets. 

And Opendoor sold almost four thousand homes in the latest quarter, up 19% from the year-earlier quarter, while the average price it got rose by 35%. 

“The proof is in the pudding,” said Chakrabarti of the company’s progress. “We are outperforming the market.”

That’s not the whole answer, however. When Opendoor starts business in a new market, it must spend perhaps a year refining its pricing models. Success is not certain at the outset, and must be proven over time. 

Moreover, the ultimate payoff for getting deep learning right won’t come in one single quarter’s financial results. Instead, it will be measured over years, in the total financial returns, and the shareholder returns, of Opendoor.  

Hence, the payoff is really what’s known in machine learning as a sparse reward, something that sends a signal back in time, providing very narrow information about what action to take now.

How does one know what that future signal is that will guide Opendoor’s actions today?

“That’s a great question, but I can’t answer it,” said Chakrabarti. “That is one of the trillion-dollar questions we think about,” he said, adding, “That is exactly the problem I think about every day.”

“Those signals are key areas of active work and active research,” added Chakrabarti. “The challenge of exactly the sparse signals in deep learning, in machine learning, is a fascinating challenge.”

The challenge of sparsity tells Chakrabarti he’s onto something important. “There’s this quote that Paul Erdős, the famous mathematician, has,” recalled Chakrabarti, “that problems that are worth solving show their worth by fighting back.”

“This is a problem that has such a rich landscape, it is a problem you have to think about incessantly, otherwise it just doesn’t work,” he said. For those who like to obsess over such worthy problems that fight back, “this would be an amazing place to start working on some of those problems,” he said, meaning, at Opendoor.

Sparsity, and other aspects of deep learning have made Chakrabarti something more than a convert. He believes he is not only solving a particular problem, he is at the threshold of something more significant. 

There is an underlying structure being revealed, he said, a set of patterns that to Chakrabarti are proof that “there is a deeper mechanism to the world, there is a way the world works.”

“It’s all about uncovering that mechanism,” he said.

That mechanism can work to the company’s advantage, he suggested. Opendoor’s technology, if trained on houses in one market, can extend to houses in another market fairly well — the algorithms “generalize,” to use the technical term in machine learning. That is because, said Chakrabarti, “there is more shared information about what’s happening in one market versus another.” 

Explained Chakrabarti, “This is a fundamental thing in deep learning […] It turns out that there is so much that’s fundamental to how humans behave, how prices behave, etc., that the shared, underlying knowledge is a massively powerful thing that, if you then just tweak with some specific data, you get this massive acceleration in how your algorithms perform.”

“To me, data science, operations research, machine learning — all of these things, are really about the art and science of reconstructing that mechanism, given the data and the stories that mechanism throws off,” said Chakrabarti. “There is something truly beautiful about being able to get that glimpse of what that underlying thing really is.”

What is that glimpse?

“I thought at some point, well, this is the closest I’m going to get to the mind of God,” said Chakrabarti. “It is actually, truly a spiritual thing for me.”