I think the a lot of the value of Brainstorm comes from points that RavebornMuse raised
here.
While that is a totally separate topic, the key takeaways are that you want to provide your players with a strong sense of agency, a sense that the decisions that they are making matter. It allows your players to feel as through they have solved a puzzle in order to derive the maximum value out of their cards. Brainstorm provides a lot of space in which players can feel smart for 'finding a line', since the interactions that it has with other cards are wildly varied, and are hardly spelled out explicitly. Whether it is a better card (in the power-in-a-vacuum sense) than
Ponder is not really a meaningful question, since due to the wealth of interactions, the power of Brainstorm is pretty context dependant.
I suspect that the reason Brainstorm is held in such high regard here is because a large part of the Riptide mantra is about trying to design such that games are decided far more by a series of good decisions, both in drafting and in play, than by external factors. Sure, sometimes there will be no lands in the top 6 cards of a deck, but in the majority of scenarios I want my drafters to feel that they have earned their wins, and learned from their losses, rather than feeling like they simply lost and there was nothing they could have done about it. You can see this attitude reflected in everything, from cutting uninteractive cards; the approach to mana-bases to minimize colour screw and add decisions in fetching; minimizing 'boring' finishers; even lowering mana curves to create more early game decisions.
Brainstorm fits very neatly into this approach, as it is a very decision dense card, and creates its own sets of sequencing decisions and board assessments that cards like Ponder or Preordain do not. While I feel that Ponder or Preordain can fit into this environment, as they also reward good assessments and help to minimize non-games, I suspect that Brainstorm's increased complexity and potential interactions makes it more desirable to us designers as a way to better create these scenarios in which our players can feel satisfied that they have found a hidden line to maximise the card's value. Ponder and Preordain just don't have the same ceiling, from a both a card power and player psychology perspective.
Obviously these points echo well outside the 'which combination of Blue cantrips do I run?' discussion, but it is rarely called out in these terms. You can see it in RavebornMuse's post that I linked earlier, and also in a lot of Grillo's posts and articles about his attempts and thoughts on adding meaningful decisions to aggressive decks. As cube designers, a core component of balancing our lists is making our players feel like they were responsible for their win or loss, not that they simply won this week's Sol Ring Lottery.