Stuck in a Local Optimum, Predictive Analytics Should Not Stop Here

Original Title: Two Kites Dancing In A Hurricane

Original Author: 0xsmac

Original Translation: SpecialistXBT

Editor's Note: This article takes a sharp look at the current prosperity facade of the prediction market. The author astutely points out that today's prediction market is falling into a "local optimum solution" trap akin to the days of BlackBerry and Yahoo. The binary option model adopted by mainstream prediction markets has seen a huge influx in the short term but is plagued by structural issues of liquidity scarcity and inefficiency. The article proposes the concept of the prediction market evolving towards a "perpetual contract" model, providing constructive deep thinking to achieve a true "market of all things."

Why do businesses find themselves chasing the wrong target? Can we fix the prediction market before it's too late?

"Success is like a strong tonic, intoxicating. Harnessing the subsequent fame and praise is by no means easy. It will corrode your mind, make you start to believe that everyone around you reveres you, everyone desires you, everyone's thoughts revolve around you at all times." — Ajith Kumar

"The roar of the crowd has always been the sweetest music." — Vin Scully

Early success is intoxicating. Particularly when everyone tells you that you won't succeed, this feeling becomes even more intense. Screw the naysayers, you were right, they were wrong!

However, early success carries a unique danger: you may have won the wrong reward. While we often joke about "playing a stupid game, winning a stupid prize," in reality, the games we engage in are often evolving in real-time. Therefore, the factors that led you to win in the initial stages may ironically become stumbling blocks for you to win a greater reward as the game matures.

One manifestation of this outcome is: a company falling into a "local optimum solution" without realizing it. The feeling of winning is too good, so good that it not only loses your direction, but even blocks self-awareness, making it impossible to see the true situation you are in.

In many cases, this may just be a mirage, an illusion propped up by external factors (such as economic prosperity leading to an abundance of disposable income in consumers' hands). Or, the product or service you have built indeed functions well, but only within a specific scope or under specific conditions, unable to expand to a broader market.

The core conflict here is that, to pursue the true ultimate prize (i.e., the global optimum solution), you need to come down from the current peak. This requires immense humility. It means making tough decisions: giving up a core feature, completely refactoring the tech stack, or personally overturning that once-thought-effective pattern. Making all of this more challenging is...

Most of the time, you have to make this decision when people (mainly investors and the media) are telling you how "great" you are! Many who previously said you were wrong are now scrambling to validate your success. This is an extremely dangerous place to be, as it can foster complacency at the very moment you need to make radical changes.

This is exactly the predicament the prediction markets find themselves in today. In their current form, they can never achieve mass-market adoption. I won't waste ink here arguing whether they have achieved this status (after all, there is a huge gap between knowing something exists and actually having a demand to use it). Maybe you disagree with this premise, and you're now ready to close the page or begrudgingly read the rest. That's your prerogative. But I will reiterate why this model is broken today and what I believe such platforms should look like.

I don't want to sound too much like a tech industry person, I won't rehash the "innovator's dilemma," but the classic examples of Kodak and Blockbuster belong in this realm. These companies (and many others) achieved great success, leading to an inertia against change. We all know how the story ends, but merely throwing our hands up and saying "do better" is not constructive. So, what specifically led to these outcomes? Do we see these signs in today's prediction market?

Sometimes, the barrier is at a technical level. Startups often build products in a particular subjective way that may work well in the initial stages (just getting to that point as a startup is a feat in itself!), but soon solidifies into future architectural shackles. Wanting to scale post-initial traction or tweak product design means threatening certain seemingly effective core components. People naturally tend to patch problems incrementally, but this quickly turns the product into a patchwork monster. And, this only delays the time to face the harsh truth: what's really needed is a complete rebuild or reimagining of the product.

Early social networks faced this when hitting performance ceilings. Friendster, a 2002 social network pioneer, allowed millions of users to connect online as "friends of friends." But trouble brewed when a specific feature (viewing friends within three degrees of connection) caused the platform to crash under the load of exponentially calculated connections.

The team refused to trim this feature and instead focused on new ideas and flashy partnerships, even as existing users threatened to defect to MySpace. Friendster reached a local peak of popularity but couldn't cross it because of flaws in its core architecture, which the team refused to acknowledge, break down, and fix. (By the way, MySpace later fell into its own kind of "local optimum" trap: it was built around a unique user experience of highly customizable user profiles and focused on the music/pop culture crowd. The platform was primarily ad-driven, eventually becoming overly reliant on its ad portal model. Meanwhile, Facebook emerged as a cleaner, faster, and identity-based network. Facebook attracted some early MySpace users but undoubtedly appealed more to the next wave of massive-scale social media users.)

The continued existence of such behavior is not surprising. We are all human. Attaining some superficial success, especially as a high-failure-rate startup, naturally inflates the ego. Founders and investors start to believe in the performance they boast about and double down on the formula that got them here, even as warning signs flash brighter and brighter. People are quick to ignore new information and even refuse to confront the current and different reality. The human brain is fascinating that way—if there is enough motivation, we can rationalize a lot of things.

The Stagnation of "Research In Motion"

Before the iPhone came into the picture, Research In Motion (RIM) and its BlackBerry devices were the kings of smartphones, commanding over 40% of the U.S. smartphone market share. It was built around a specific idea of what a smartphone should be: a better PDA (Personal Digital Assistant) optimized for enterprise users, especially focusing on email, battery life, and that addictive physical keyboard. However...

Times change in the blink of an eye.

One underestimated aspect today is how well BlackBerry served its customers. It was precisely this excellence that left RIM unable to change when the world shifted around them.

It is widely known that its leadership team initially dismissed the iPhone.

"It's not secure. The battery life is too poor, and it has a terrible keyboard." — Larry Conlee, COO of RIM

They quickly became defensive thereafter.

RIM arrogantly believed that this new phone would never attract its enterprise customer base, and not without reason. But this completely missed the groundbreaking shift of smartphones evolving from a "messaging device for everyone" to a "universal device for everyone." The company suffered from severe "technical debt" and "platform debt," common symptoms of companies that achieve early success. Their operating system and infrastructure were designed for secure messaging and battery efficiency. By the time they acknowledged the reality, it was too late.

There's a view that companies in this predicament (the greater the initial success, the harder the evolution, which is one reason why Zuckerberg is the "GOAT/Greatest of All Time") should operate with an almost schizophrenic mindset: one team dedicated to leveraging the current success, another team dedicated to disrupting it. Apple might be a prime example of this, as they let the iPhone cannibalize the iPod market, then let the iPad cannibalize the Mac market. But if it were easy, everyone would do it.

Yahoo

This might be a presidential mountain-level "missed opportunity." Once upon a time, Yahoo was the homepage of millions. It was the internet's portal (you could even call it the original "super app") — news, email, finance, games, you name it. It saw search as just one of many features, to the point that in the early 2000s, Yahoo didn't even use its own search technology (it outsourced search to third-party engines, even briefly using Google).

Now, it's well known that its leadership team passed on multiple opportunities to deepen search capabilities, most notably the chance to acquire Google for $5 billion in 2002. Hindsight is 20/20, but Yahoo failed to grasp what Google understood: search is the foundation of the digital experience. Whoever owns search will own internet traffic, and consequently, ad revenue. Yahoo overly relied on its brand strength and display ads but catastrophically underestimated the significant shift to a "search-centric" navigation and later to social networks with personalized content feeds.

Remember this guy?

Forgive the cliché, but in a frothy market, "a rising tide lifts all boats." The cryptocurrency space knows this well (see Opensea and many other examples). It's hard to discern if your startup has genuine traction or is merely riding an unsustainable wave of momentum. Further complicating matters, these periods often overlap with a surge in venture capital and speculative consumer behavior, masking underlying fundamental issues. The comical rise and fall of WeWork exemplify this well: easily accessible capital led to massive expansion, masking a fundamentally broken business model.

Stripping away all the branding and fancy lingo, WeWork's core business model is very simple:

Long-term lease office space → Spend money on renovations → Mark up and sublease short-term.

If you're not familiar with the story, you might think, well, this sounds a lot like a short-term landlord. And that's exactly what it is. A real estate arbitrage play disguised as a software platform.

But WeWork wasn't necessarily interested in building a lasting business; what they optimized for was something entirely different: explosive growth and a valuation narrative. This worked in the short term because of Adam Neumann's personal charisma and ability to sell a vision. Investors bought into it hook, line, and sinker, fueling a totally detached-from-reality kind of growth (in WeWork's case, meaning opening as many office buildings in as many cities as profitably ignored, with a disregard for unit economics as they believed "we will grow into profitability"). Many outsiders (analysts) saw through it—this was a risk-conditioned real estate company with unstable clients and structural losses baked into the business.

Most of the above is a retrospective analysis of a failed company. In a sense, it's "hindsight bias." But it reflects three different failure insights: failing to advance technologically, failing to identify and address competition, or failing to adjust the business model.

I believe we are witnessing a similar scene play out in the prediction market right now.

The Promise of Prediction Markets

The theoretical promise of prediction markets is compelling:

Utilize the wisdom of crowds = Better information = Turning speculation into collective insight = Unlimited markets

But the current top platforms have hit a local peak. They have discovered a design that generates some traction and volume, but it falls short of the true vision of "everything is predictable and liquid.".

Superficially, both show signs of success, no one doubts that. The Kalshi report suggests that the industry's annualized trading volume will reach approximately $30 billion this year (we will discuss later on how much of this is organic growth). The industry saw a surge of renewed interest in 2024-25, especially as narratives of DeFi combined with the gamification of trading further penetrated the zeitgeist. Polymarket and Kalshi's aggressive marketing may also play a role (in some cases, aggressive marketing does work).

But peeling the onion back one layer reveals some warning signs that growth and PMF may not be as straightforward as they seem. The elephant in the room is liquidity.

For these markets to function, they need deep liquidity, meaning a significant number of people willing to take bets on one side of the market so that prices make sense and reveal true price discovery.

Both Kalshi and Polymarket, aside from a few very high-profile markets, are struggling with this.

Huge trading volumes are concentrated around major events (US elections, highly anticipated Fed decisions), but most markets exhibit wide bid-ask spreads and almost no activity. In many cases, market makers don't even want to transact (a Kalshi co-founder recently admitted that even their in-house market maker isn't profitable).

This indicates that these platforms have yet to crack the puzzle of expanding market breadth and depth. They are stuck at a level: performing decently in a few dozen popular markets, but the vision of a long-tail "market for everything" remains unfulfilled.

To mask these issues, both companies have resorted to incentive schemes and unsustainable behaviors (sound familiar?), which are typical signs of touching a local optimum and insufficient organic growth. (As a side note here in this specific market dynamic, I have a feeling that most people think these two are the only major players in competition.

I don't think this necessarily matters at this stage, but if both teams believe this, then if one is seen as "ahead" in this assumed "two-horse race," it poses a survival threat to the other's company. That is a particularly unstable position, based on a mistaken assumption).

Polymarket has launched a liquidity mining program in an attempt to narrow spreads (in theory, if you place an order near the current price, you receive a reward). This helps make the order book appear tighter and does indeed provide traders with a better experience by partially reducing slippage. But it is still a subsidy. Similarly, Kalshi has introduced a trading volume incentive program, essentially offering cashback based on users' trading volumes. They are paying people to use the product.

Now I can almost hear some of you shouting, "Uber also subsidized for a long time!!!" Yes, incentive schemes themselves are not bad. But that doesn’t mean they are good! (I also find it interesting how everyone always likes to point to the exceptions to the rule and not look at the bodies. Especially considering the dynamics of the current prediction market, this will quickly turn into a hamster wheel that can't stop until it's too late.)

Another fact we need to be aware of is that a considerable portion of the trading volume consists of fake trades. I believe spending time arguing about the exact proportion is pointless, but evidently, fake trades make the market appear more liquid than it actually is, with only a few participants frequently trading to gain profits or create market hype. This means that the natural demand is actually weaker than it seems on the surface.

「Final Trader Pricing」

In a healthy, well-functioning market, you should be able to place a bet close to the current market odds, and price swings should not be too significant. However, that is not the case on these platforms today. Even moderate-sized orders significantly impact the odds, clearly indicating a lack of trading volume. These markets often only reflect the movements of the final traders, which is at the core of the liquidity issue I mentioned earlier. This situation suggests that while a small group of core users sustains some market operations, these markets are neither reliable nor liquid overall.

But why is this the case?

The market structure of pure binary trading cannot compete with perpetual contracts. It is a cumbersome approach that leads to liquidity fragmentation, and even though these teams attempt to address this issue through workarounds, the results are at best clumsy. In many of these markets, you also encounter a peculiar structure where there is an "other" option representing an unknown factor, introducing the problem of splitting emerging competitors from that basket into separate markets.

The binary nature also means that you cannot provide true leverage in the way users desire, which, in turn, means you cannot generate valuable trading volume as with perpetual contracts. I see people endlessly debating this on Twitter, but what still astounds me is that they fail to recognize that betting $100 on a 1-cent outcome in a prediction market is different from opening a 100x leveraged position with $100 in a perpetual contract exchange.

The unspeakable secret here is that to address this fundamental issue, you need to redesign the underlying protocol to allow for generalization and treat dynamic events as first-class citizens. You must create an experience similar to perpetual contracts, which means you have to address the jump risk present in binary outcome markets. This is obvious to anyone actively using perpetual contract exchanges and prediction markets—and these are precisely the users you need to attract, unaware as these teams may be.

Addressing jump risk means redesigning the system to ensure asset prices move continuously, meaning they do not arbitrarily jump from, for instance, 45% probability to 100% (we have seen how frequent and blatant these events are manipulated/insider traded, but that is another topic I do not want to delve into at the moment. Please stop the crime.).

If this core constraint is not addressed, you will never be able to introduce the kind of leverage needed to make the product appealing to users (those who can bring real value to your platform). This leverage relies on sustained price fluctuation to safely liquidate positions before the loss exceeds the collateral, avoiding sudden swings (e.g., jumping from 45% to 100% in an instant) that would wipe out one side of the order book. Without this, you cannot promptly add margin or settle, eventually leading the platform to bankruptcy.

Another core reason these markets do not function in the current structure is the lack of a native multi-outcome hedging mechanism. Firstly, there is no natural hedging available as these markets resolve to YES/NO, with the "underlying" being the outcome itself. In contrast, if I go long on a BTC perpetual contract, I can hedge by shorting BTC elsewhere. This concept does not exist in today's prediction market structure, so if a market maker is forced to bear direct event risk, providing deep liquidity (or leverage) becomes extremely challenging. This further underscores why I believe the argument that "prediction markets are a nascent thing, and we are in a high-growth stage" is naive.

Prediction markets will ultimately settle (i.e., they actually close at resolution), whereas perpetual futures obviously do not. They are open-ended. Designs akin to perpetual contracts can alter the market by incentivizing active trading, making it more continuous to alleviate some common behaviors that make prediction markets unattractive (many participants simply hold until resolution, rather than actively trading probabilities). Additionally, since the prediction outcome is a one-time discrete result, and while the oracle feed also has its issues, but at least it is continuously updated, the oracle problem in prediction markets becomes more pronounced.

Behind these design issues is a capital efficiency problem, but that seems to be well understood at this point. I personally do not believe that "earning stablecoin yield on idle funds" brings about significant change. Especially considering exchanges will offer such yields anyway. So, what is the trade-off being made here? Pre-funding each trade in full is certainly a good thing for mitigating counterparty risk! And it can attract some users.

However, it is disastrous for the broader user base you need. From a capital perspective, this model is highly inefficient and only significantly raises the barrier to entry. This is particularly bad when these markets require different types of users to operate at scale because these choices mean a worse experience for each user group. Market makers need a significant amount of capital to provide liquidity, while retail traders face substantial opportunity costs.

There is certainly more to unpack here, especially around how to attempt to address some of these fundamental challenges. A more complex and dynamic margin system will be necessary, especially considering factors like "proximity to event time" (when the event resolution is near and the odds are close to 50/50, the risk is highest). Introducing concepts like leverage decay nearing resolution and early-stage tiered liquidation levels will also be helpful.

Borrowing from the traditional financial brokerage model to achieve instant collateralization is another step in the right direction. This will unlock capital for more efficient utilization, enabling cross-market atomicity and updating the order book post-trade. Introducing these mechanisms first in scalar markets and then expanding to binary markets seems like the most logical sequence.

The key point is that there is still a lot of unexplored design space here, partially because people believe today's model is already in its final form. I just haven't seen enough people willing to acknowledge the existence of these constraints upfront. Perhaps not surprisingly, those who do tend to be the type of users these platforms should want to attract (aka perpetual contract traders).

What I have seen, though, is their criticisms of prediction markets mostly being waved off by enthusiasts and told to look at the trading volumes and growth numbers of these two platforms (absolutely real and organic numbers, ahem). I hope prediction markets evolve. I hope they see mainstream adoption. I personally believe that "everything is a market" is a good thing. Most of my frustrations stem from people seemingly accepting today's version as the best version, which I don't think is true.

“Original Article Link”