TikTok’s Secret Sauce | Knight First Amendment Institute:

Recommender systems are extremely well studied in computer science, and relatively simple to understand, but public comprehension of how they work is poor. That has led to these algorithms being viewed as magic, demonized, or mythologized. (I hope to play a small role in changing this through my ongoing project on algorithmic amplification and society.) TikTok’s recommender system is not its secret: rather, it’s the design, which, of course, isn’t secret at all. More generally, in AI applications, the sophistication of the algorithm is rarely the limiting factor. The quality of the design, the data, and the people that make up the system all tend to matter more.

I feel like this explanation also makes a pretty good case for why conspiracy theories are so popular.

We want to think that we are all very special and hard to predict; in order to be manipulated, we must be subjected to these obscure and super-complex advanced machine-learning algorithms. In reality, we all tend to be fairly predictable, and it doesn’t take much more than some basic calculations and sleight-of-hand to predict what we are going to choose.

Likewise, you don’t need so posit some vast and all-consuming evil conspiracy to explain most bad things that happen. They can mostly be explained by simple disaggregated greed and stupidity.