The Creepy TikTok Algorithm Doesn’t Know You

It’s a truth universally acknowledged that “the algorithm” knows you better than you know yourself. A computer can supposedly predict whether you’ll quit your job or break up with your partner. With 1,000 words of your writing, it can determine your age within four years. And no algorithm seems closer to omniscience than TikTok’s, which is reportedly helping users discover their sexuality and unpack their childhood trauma. Whereas Facebook asks you to set up a profile, and hand over a treasure trove of personal information in the process, TikTok simply notices—or seems to. The results can feel magical, writes Jess Joho in Mashable, as if TikTok is “reading your soul like some sort of divine digital oracle, prying open layers of your being never before known to your own conscious mind.”

As ever, there’s a dark side. Most users will be sucked into WitchTok and gardening how-to’s, but others will end up in an infinite scroll about ADHD, Tourette’s syndrome, or autism (#mentalhealth, for example, has garnered almost 21 billion views—a fraction of #dance, which has 341 billion views, but nonetheless a significant showing). This information can be extremely liberating, especially for conditions usually shrouded in stigma. But some are concerned about the deference given to the platform’s pseudo-psychiatric content. “Once [the algorithm] puts you on a side, it keeps you there,” says psychology professor Inna Kanevsky, better known on TikTok as the debunking @dr_inna, “and it starts seeming like you’re diagnosed.” That’s a potentially life-altering takeaway proffered up by a for-profit algorithm and content creators of varying reliability.

But the debate over self-diagnosis is just one small part of the ways in which we are positioning TikTok and similar algorithms not as mere prediction-generating machines but insight-producing ones. While recommendation algorithms from Amazon to Netflix are designed to guess what you’d like to see next, TikTok can feel as if it’s showing you who you’ve always been. In the process, we are opening ourselves to the peril and promise of “outsourcing self-awareness to AI.” While this new relationship to computing may offer new opportunities for personal growth, it’s also started to sort us into ever more rigid identities.

The human mind has tried to make sense of itself through technological metaphors for thousands of years, writes artificial intelligence expert George Zarkadakis in his 2016 book In Our Own Image. In the Roman empire, for example, the success of hydraulic engineering gave rise to the understanding of human intelligence as the flow of four “humours.” Now we describe the brain as a computer—just one of so many machines that “stores” and “retrieves” memories and “processes information.” While all of these metaphors remain deeply flawed, the close collaboration of human neuroscience and computer science has revealed some intriguing similarities between our minds and our machines.

Like any algorithm, TikTok’s divinatory properties are just the end result of a series of repeated steps. When someone creates a new account, the algorithm targets them with a variety of popular videos designed to test their response to broad categories of content, from viral dances to home repairs, according to a recent Wall Street Journal investigation. When the newspaper set 100-plus bots loose on TikTok, the platform’s “rabbit holing” keyed in on every bot’s preprogrammed interests in less than two hours.

It’s pretty simple math, but it’s also an uncanny mimic of statistical learning, says John Bargh, a professor of psychology and cognitive science at Yale University. That’s one way the human mind unconsciously acquires new knowledge—simply by noticing patterns in the world around it. Like the TikTok algorithm, people are learning all the time, often without even realizing it.

In other words, it’s a process that plays out mostly without a person’s conscious awareness, and yet is entirely determined by what pulls your attention.