We even have a court criticizing Elon for taking an options package where he made no money unless he saved the company from bankruptcy. Meanwhile, the CEO of his arch rival, who created no shareholder value over a period of five years, is making tens of millions of dollars a year in our shoes.

Hey, man, great to see you. Good to see you. Good to have you back. Glad to be here. Man, I raced up here from NVIDIA GTC. How was that? I mean, it's a, you know, I texted Lincoln yesterday when I was down there for the keynote and he said, you know, I said, how'd your day go at school?

And he said, good. He's like, hey, Pops, what are you up to? And I said, you know, I'm just watching the future. And he said, he asked a great question. He said, well, why is this any different than all the other computers in the world, right? And I said, you know, like we've talked about before, we're going from this era where computers were highly efficient calculators.

And I said, you know, we're not far off from computers being able to reason and think and being smarter than every human who's ever lived on the planet. So it's, I mean, it's an exciting time, disruptive time. We're going to talk about that more later in the show. But we thought for episode five, we'd mix it up a little bit.

I mean, one of the things you and I talked about when we started the pod was not only do we want to talk about the topics of the day, but we also wanted to do some deep dives on topics that you and I have talked about over the years.

You know, I think about the topics that you've covered on Above the Crowd, whether it's network effects or whether it's a rake too far. And so, you know, one of those things we're going to talk about today. After that, we're going to get into some of these Google Apple rumors.

We're going to talk a little bit more about the TikTok fallout, et cetera. But one of the, we've been pretty passionate and talked a lot, argued a lot over the years about stock-based compensation. And it's so important to Silicon Valley because it's all about incentives and alignment. It's such a special part of what makes this place great, but it also has been a source of a lot of consternation from Warren Buffett down the line.

So we knew we wanted to talk about this, but in order to talk about it, we had to invite one of our old friends. Correct. Who may be, you know, the best expert on the topic. The legendary CFO of Priceline, Bob Mylod, who's served on tons of VC boards.

And he's truly got, gets this from both the startup perspective and the public company perspective. So Bob, thanks for joining us. Gentlemen, thanks for having me. It's great to be here. Yeah. If any of my CEOs or CFOs are of the mindset, they want to get this right. There's no one on the planet I'd rather, you know, introduce them to and have them talk to them, Bob.

Bob, I think you joined Priceline in 1999. You know, we must have met around 2000 or 2001. You were the CFO of the company from when I believe it was a billion dollar company all the way to it being worth, you know, upwards of a hundred billion, certainly on the board for that entire period of time, you're still on the board of booking and you know, it's become extremely valuable in the public market.

There was a little bit of a, there was a big rollercoaster along that way, but that's generally the and I wasn't CFO the entire time, but yes, that's the general math of it. Bill, why don't you frame for us, why is this topic so important and importantly, a little bit about how we got here with stock-based compensation?

Yeah. And I think everyone in Silicon Valley knows that equity participation is part of the model, always has been. Right. And in general, when it was started, venture capitalists felt strongly that it created alignment, right? You wanted managers and shareholders owning the same equity and having the same interest.

And if you create one of these monster outsize, you know, wins, everyone's a winner. And so it just made sense. Right. And I think it's been since certainly since I got here, but way before I got to Silicon Valley, I think it's been a part of what people do.

In the late 90s, you know, we went through the dot-com boom and bust. And prior to that time, I think the primary vehicle that was used to incent executives and employees were stock options, employee stock options. And they would be priced at the current price of the company. And you know, while a company's private, you had this, you know, evolving argument of how you value the options prior to being public, they were typically a discount to the preferred.

Likpref and things could make that argument stand. Anyway, coming out of 1999, you know, you had a bunch of companies boom bust. You had a bunch of companies go out of business that people said were never a business. And you had a lot of skepticism coming in from, you know, people like writers for the Wall Street Journal, but even ISS, Institutional Shareholder Services.

You may want to tell people quickly what ISS is. Yeah. I mean, you know, ISS is basically, you know, a body that helps advise owners, shareholders of public companies on how to vote on a variety of topics, including compensation, quite controversial. But they have a voice. But they have influence.

That matters in this world. And people also were upset about repricing, backdating. You know, there were options that were backdated and then re-granting. So the stocks collapsed and then they just issued new options. And people didn't like that. And people begin to argue that options created too much risk.

And so there was this idea, you know, that maybe we needed a different instrument. Along that same time, a very large investor in a very prominent internet company was worried about dilution. He was worried about the dilution from options. And for those that want to just use a rough framework, you know, when a startup would get started in early years, you might have as much as, say, 10% dilution from options.

You might hire three executives during that year and it's a big grant. But then as you move towards being public, people would typically, and these are very gross rules of thumb, steer towards, you know, a 3% type dilution rate. This particular investor in this very large public internet company felt that if they moved to zero-based RSUs from options, and this gets into a technical detail around Black Scholes, that, you know, there's an argument from Black Scholes that a single RSU is worth about two and a half or three options.

So if you could switch from these options to an RSU, this investor thought dilution might drop from, say, 4% to 1.2% or something like that. And there would be, quote, less dilution. And so that change happened at this one company, but then these zero-based RSUs became prevalent everywhere. Okay.

So let's just pause there for a second because I think there's a potential we've lost half of the audience because it's a very technical and detailed topic. So if I just step back for a second, I want to invite you into the conversation, Bob. The reason, obviously, we care in the case of this large internet company, they're granting options.

They want employees to be incented and aligned with them. They may not have a lot of cash in order to pay cash, but you said something critical. As they get toward the public market, right, there is—these things are obviously expensive to the potential shareholders. So folks who are going to buy it, Fidelity, Wellington, others in the public market, they care and are sensitive to that cost.

Bob, Bill's describing the world that existed about the time, right, that you guys were going through this at Priceline. So tell us a little bit about kind of your experience in those early years with options and then kind of this transition that we struggle with to RSUs. Yeah, I think Bill does a very good job of describing what the world looked like in venture capital and private equity for that matter.

That's what I did for 10 years before I joined Priceline. It was a very simple, straightforward model of investor makes investment at a certain price per share and sets aside an option pool to be granted to management at the same price. There wasn't some—there was no exercise of contorting oneself into figuring out how to make the strike price lower.

It was just simply as investors reap rewards from the outcome of the company growing, employees share in those rewards. So by the way, I became kind of battlefield promoted during the dot-com bust. So I became CFO of Priceline in the fall of 2000 and this was—Brad, as you mentioned, I joined the company just after it was founded, less than a year or almost exactly on the one year anniversary of the founding, the company went public.

A few months after that, we had a market value of about $35 billion. We were a tiny little company but this was the absolute heyday of the dot-com bubble and then the dot-com bubble burst and essentially our stock went to zero. If you go back and look at the stock price run of Priceline, we were all the way down to $5 a share which is a post-reverse split adjusted value.

I became CFO at that point and I wasn't too much worried about stock, stock options. Our stock had just plummeted so much that all of the stock options that were sitting in the hands of whether it was myself or every single employee, they were under water. So the only possible way for us to compensate people at that point was in cash which really sowed the seeds for some of the ultimate successes.

When you start to think of your business in the form of you're paying people based upon the cash that you have in your bank and you have no more ability to raise money, it's very, very clarifying about what you can afford to pay people. That was the model that we operated for a couple of years, operating Priceline and then what I will define as one of the great unintended consequences of an accounting change occurred right around the 2002 to 2004 timeframe and there were two things.

First, there was the adoption of Sarbanes-Oxley where people like me and Jeff Boyd are – this CEO, we had to sign our names on the financial statements under penalty of criminal prosecution and there was the adoption of FAS 123 which required that historically, any granting of stock options to employees did not come with a charge.

The theory historically was it's hard to value the stock options because they really only have value if a stock goes up and therefore, we won't try and do that even though there is such a thing as a Black-Scholes model and the things do have value. Historically, a stock option granted resulted in no charge.

The SEC and the FASB were trying to get to a place where companies had to recognize that granting of a stock option did represent the transfer of economic value from the employer to the employee and that needed to be accounted for. So FAS 123 came along and every public company had to expense the granting of a stock option.

Well, as you can imagine for somebody like me with a hugely volatile – being the CFO of a company with a hugely volatile stock and I'm a University of Chicago grad and I studied the Black-Scholes model and had a very clear understanding that there is a lot of inputs into the calculation of a Black-Scholes calculation that there's some subjectivity to it and that made CFOs like me very uncomfortable as to signing your name to financial statements with the stock option charges in it.

Not to mention the fact obviously that when we're sitting down talking to employees about their compensation and you say to them, "Hey, here is your stock option and by the way, Fisher Black and Myron Scholes says it's worth – they say it's worth $100,000," and you're looking at it and saying, "Well, the stock is trading for $20 a share and you're telling me my strike price is $20 per share.

So in my mind, it's worthless." In fact, it is Bob, unless the stock goes up in value. But the reason it has some intrinsic value according to Black-Scholes is they look at the volatility and other measures related to the stock and it's effectively a probability weighted estimation of what this thing is worth today, right?

So from an – go ahead, Bill. I believe – and Bob could correct me because I didn't go to the University of Chicago. But I believe Black-Scholes was backed into by looking at – you can create it once synthetically with the short and a call option and these kind of things and – Explain that to an employee.

But I understand – Yeah. But you said it didn't have value. At the money call option has value. No, for sure. They trade on the public markets today. That's my only point. Yeah. Absolutely. And they do have value. Again, which is why for the dawn of the venture capital era, stock options were granted and they did have value and they created huge value.

But again, only in the event that value was created for shareholders. As the 2000s moved along, there were more and more companies – and by the way, this includes Priceline. We moved away from stock option grants to RSU grants. And let's just explain again to folks what some of these things are.

So stock option, I get a security. It says the strike price is $20 that you referenced. Our stock today is trading at $20. According to Black-Scholes, that would have some intrinsic value today. You would have to recognize and expense that over time. But for me as the employee, I'm just looking at this and saying this is only going to be valuable to me if this stock goes up over time.

In fact, if I quit the company three years later and it's still at $20 a share, then I don't walk away with anything. The move to RSU's restricted stock units means that I'm going to own a security and zero basis RSU's. I'm going to own – this is just like get somebody handing you a share of stock and when I leave three years later, whatever I vested in that share of stock, I can cash that out and I can walk away with that cash value.

So it works very differently from an incentive and alignment perspective than options were working. Sorry to interrupt you, Bob. Why don't we – Bob Gourley Yeah. No, that's exactly right. So if you give one share of a restricted stock unit at a value of 20, then the recipient has $20 of value.

Now, if the stock happens to go down by 50 percent, the employee still has $10 of value, whereas in the case of the stock option, if it goes down one percent, let alone 50, it has zero value. Here's where we get into questions of alignment with shareholders because you could theoretically have a management team that leads a company over a year to a 20 percent decline and pick up 80 percent of the value of their equity award, whereas shareholders are down 20.

You as a large public owner of shares, you would be down 20 and management gets 80 percent of their equity value. Trevor Burrus So Bob, why did you go from options to RSUs? Bob Gourley Well, again, I will just say that before the adoption of FAS 123, if you were to grant an RSU, the full cost of that RSU would have to be expensed in the P&L, in the income statement of a public company.

Trevor Burrus So treat it like cash, same as cash. Bob Gourley It was treated as cash and – by the way, there were some companies that did it, but there was – it was very clearly understood by all people that that was essentially no different than cash. It would be no different than issuing a share of stock to the public for $20 and then turning around and taking that $20 and handing it to the employee.

So that was always expensed. It was just when the stock option also got expensed that issuers looked at that and said, "OK. Well, if I now have to expense stock options and I always had to expense RSUs, I might as well just do RSUs." It was at this time also that companies got into the practice of adding up all the charges related to this stock-based compensation and proforming the effects of those out of the income statement so that they could effectively compare current results to what the results would have looked like before FAS 123 was adopted.

Trevor Burrus OK. So hold on a second here. So you shifted from stock options, which as a shareholder, I like because nobody makes money unless I make money, to RSUs where people made money even if I lost, right? And then because that was all now in the P&L, you had to expense that like cash, which Warren Buffett and many others had been pushing for since '97, '98.

But in order to get around that, everybody started adjusting what they – their earnings, EBITDA. They started adjusting the stock-based compensation expense out of EBITDA, OK? So as though the expense no longer counted. Did you start that? Warren Buffett I don't know that I started it. But we did – so we did do the proforming.

But the reason that we did the proforming wasn't because we didn't view that every single dollar of RSUs that we were handing out had real, very significant value. It was more that we were going through this transition period where some of the compensation we had handed out historically were stock options and now we were moving to a world of RSUs.

So there was a multiyear period where some of the stock-based compensation was option-based. Some of the stock-based compensation was RSU-based. So that would have created even more confusion around how to think about core earnings. So we did proforma them and obviously almost all companies proforma them. I think the big difference is that we, as a company, we always viewed one dollar of an RSU as fully being worth one dollar.

In fact, if you really are bullish on your company, which we were, we viewed that a dollar of RSUs was actually worth more than a dollar because we certainly had a view hopefully that if we did our jobs, that the stock was going to go up. So it was our most valuable form of currency that we used to pay employees and we certainly thought about that all along the way.

We never – it never occurred to us that it shouldn't count or that it didn't count and we always looked at it in terms of the dollars associated with employee compensation, period. We never looked at it in terms of what dilution – what do these share grants represent as a percentage of the total outstanding shares or "dilution"?

I assure you, as somebody who sat across the table paying people their compensation, I wouldn't sit across the table at year end and say, "Congratulations, we've had a good year. Your salary is X and your bonus is Y and we're giving you a stock grant of percentage points of dilution." Percentage points of dilution do not show up in a W2.

Percentage points of dilution are not what you pay taxes on. People on the other side of the table are looking at dollars. So we always looked at dollars and I think where some other companies have strayed is they've forgotten that those dollars are dollars. For a tech company, oftentimes stock-based compensation happens to be the single largest expense component of a tech company and yet somehow, inexplicably, even though it's the largest single expense component of that company, it is not considered to be even an expense in some companies and in some boardrooms.

The result of that is that you've seen runaway stock-based compensation on an almost uninterrupted basis for the better part of 10 to 15 years. As this happened, exactly what Bob talked about, and as it became way more prevalent, we were entering that ZERP period and so there was spend at all costs.

There wasn't a lot of attention to profitability or even kind of paying attention on the expense side because that window happened. Coming out of the ZERP window, people like yourselves are finally raising their hand, I'm not blaming this on you, and saying, "Hey, wait a minute. This isn't right." Not only is there a non-alignment problem, but these numbers are massive and this drumbeat has just started to grow.

If we boil it way down, because I think we all have benefited from the magic of stock options as incentive and alignment for all the value creation that's occurred in Silicon Valley over the course of the last 20 years, I think what we're trying to point to and I want to drive toward is kind of what are those best practices if you're a founder, if you're on the comp committee on a board, if you're an investor.

To me, when you pro forma something out, when you stop treating it like cash, it leads to a misallocation of resources because you're effectively saying this doesn't exist. What happens when the cost of something disappears? You get a lot more of it. Let me give you a very specific example that I heard the other day.

Take for example the recent competition for AI engineers in places like OpenAI and Google because venture-backed private companies have to compete in an ecosystem against public companies. There are rumors that lead engineers are offered millions of dollars a year. But instead of cash, for example, Google's offering $500,000 in cash plus a million and a half dollars in RSUs vesting monthly to these engineers.

Every single month, those RSUs are immediately available for sale. And then as a shareholder, you have to ask the question, how much of this expense hits the book at the end of the year? If you're adjusting out the cost of those RSUs, then you're pretending that the cost of that engineer is only $500,000.

That to me is what leads to this spiral in inflation. Now, you saw this in the early 2000s and you said when you had to treat it as cash, you got a lot more sober about what everybody was paid. Now the difference in that period of time is the whole world deflated.

Everybody kind of came down together. But today, we have public companies that are pro-forming this out. They're still spending a lot on these engineers. For me, the problem with this is when we – it leads to excess in the public markets which then misallocates resources away from startups and I think leads to less innovation.

It's – we're now – whatever, we're 20 years since the adoption of FAS 123. I especially thought when Zerp disappeared that people were going to wake up and say, you know, we're actually not counting profits correctly here. These tech companies are far less profitable than we actually think they are.

I've been on – I've been involved in so many companies now as a venture investor that have gone public. I've sat on multiple compensation committees of both private and public companies and Brad, exactly to your point, I can't tell you the number of conversations I've had with the leadership of a company who – I can think of a specific conversation where CEO called me and said, "I'm trying to hire this really talented software engineer.

But that person wants $400,000 annual salary which really screws up our compensation here because that will have that person making $100,000 more per year than anybody else at our company. What do you think? Should we stretch for this person?" I said, "Well, what's the RSU grant?" The answer was $10 million.

I'm sitting here like, "Wait, I just got a phone call from somebody worrying about the $100,000 salary." Of course, the CEO should worry about the $100,000 salary because if he's playing the game where that is the number that counts and the other one doesn't, then they're going to focus on the $100,000.

Of course, I'm looking at it going, "No, you really want to focus on the $10 million." Absolutely. It seems to me, just back to why I care so much, obviously, when I think about companies getting fit, companies dealing with the post-SERP hangover where we had excess hiring, excess pay, they got out of shape, let's make it very concrete.

If a company has $20 a share in earnings but has 10% dilution every year, then my claim on that earnings as a shareholder is going down. Absolutely. Right? If they produce $20 in earnings with no dilution, they're not issuing stock at all. They're paying it all in cash, then I have a claim on that $20 in earnings.

I think the same thing in venture-backed companies. I think what some people find as shocking is they get to the end of the journey, Bob. They start off owning – you're a Series A investor, a Series B investor in a company. You think you own a certain percentage of the company.

But if you're not paying attention to that 10 million share grant, by the end of the time, like your ownership claim – by the way, founders should care because their ownership claim as founder is going down. But unfortunately, they've had to compete in this arms race where the public companies are all pretending this isn't an expense.

And look, I think competition is real. I don't want to be seen as this investor type that's saying, "Oh, the owner should keep everything." You're out there competing for employees and talent every day, and the price has gone up, and you are competing with large companies. Listen to these numbers.

Big companies have an advantage from my point of view. Microsoft's SBC is $2.8 billion a quarter, so that's over $11 billion a year. And this morning, we woke up and read about them taking some employees from a hot AI company in the Valley. And when you're handing out $11 billion a year in RSUs, you've got a lot to play with.

Now, that $11.2 billion happens to only be 10% of free cash flow and 35 basis points of dilution. It's one-third of one percent because they're worth, what, $3 trillion? You can't be a big company unless you sort of – in my opinion, unless you establish much better hygiene when you're the smaller company or even before you're a public company.

And I want to be very clear, like booking holdings – I'm chair of the board of booking holdings and I'm on the comp committee. And I want to be clear, like I view that our employees – I know that our employees have been very, very generously rewarded over a multi-decade period, partly because we live in a competitive marketplace and we have to make competitive stock grants.

But the biggest reason is because our stock has appreciated over that period so rapidly. Has gone from $10 a share to $3,400 a share. That is the appreciation that has driven returns for not only the employees, but driven returns for all the shareholders of Booking.com. Correct. And at least from our perspective, our culture has been, hey, the way to get rich, the way to make a lot of money is for – to create huge shareholder value.

And the senior management and the board of directors along the way is going to try to put our thumb on the scale in the form of smartly using our capital to further reward shareholders. Well, again, like you're exactly right. If you look at the top – certainly the NASDAQ 100 companies, but absolutely tech companies below that market cap, what you will generally see is a year-over-year increase in share count, literally the expansion of the number of slices of pie because of these stock granting practices.

You will see some companies that do share buybacks and they will literally state, "I'm doing the share buyback to offset dilution associated with employee stock-based compensation," and thereby doing that, taking a huge cash expense in the form of stock buybacks and putting it below the line so that when people look at operating cash flow or free cash flow, it looks like one number.

But yet below the line, there's all this money being spent just to buy back the stock that employees are selling when their stock vests. We have taken the view that we want to be very, very fair. But again, back to the thumb on the scale, we take our excess cash flow or we have tried to take our excess cash flow and buy our stock back.

So if you were to look at our share count, say 10 years ago, I think we probably had 50, 52 million shares outstanding. Today we have 35 million shares outstanding, meaning that during the last 10 years, if the value of our company, if the enterprise value has doubled, the share price has tripled.

The biggest beneficiaries of that are the people who have gotten the grants that we gave in the prior periods. So to me, creating that virtuous cycle where you really are counting that stock as an expense, that's the way to do it. A long, long time ago, we adopted the practice of not pro-forming out the cost of stock-based compensation because it's a very, very real cost.

We do reference cash flow and we do reference free cash flow. But in our most recent proxy statement, we actually pointed out that the thing that we really look at is how much of our free cash flow is being spent on stock-based compensation. Maybe that's a segue, Brett, to what are the solutions to get this under control?

But we have always had booking. And certainly with all the companies that I'm involved with, I try to get everybody to focus on the fact that a dollar of stock-based compensation is actually, not only should it not be ignored, it's the most single valuable expense. You had a chart and in the chart you compared booking holdings, and we'll show this, the percentage of operating cash flow that various tech companies spent on stock-based comp.

And so again, I've been on comp committees on public companies. So if I'm on the comp committee of a public company board, generally what I do at the start of every year when I'm negotiating the options or the RSU packages for the employees, we bring in a comp consultant, right?

And this comp consultant makes some recommendations to us based upon the peer groups that look like this particular company. Not surprisingly, because everybody has adopted this practice, you know, it's a little bit of **** in, **** out. And so if, you know, if everybody's, you know, following a dumb comp practice, that is your peer group, you know, compare.

But tell us a little bit about this chart and, you know, if I'm, we're on a public company comp board, and I'm going to, Bill, ask you the same thing as it relates to venture. But if I'm on that public company comp board, what is my takeaway in terms of what the gold standard is?

What, is this the way that I should be measuring it? Or should I be paying attention to Compensia and these other comp consultants that are coming my way? I don't know that there's a gold standard. To me, the gold standard is to just count it. Again, as I said, like you would never think of not counting expenses that are denominated in euros if you were, if you had a European subsidiary.

Just the dollars count. I guess what I'm saying is, is like, it shouldn't be excluded to begin with. So, one easy way of looking at the practices of various companies is to look at, okay, what is the dollar value of stock-based compensation expressed as a percentage of OCF or FCF?

That is a blunt way of looking at it. And by the way, it will shock you when you look at just how much of a company's free cash flow goes directly back to stock-based compensation. I think we've done that analysis, Bob. You've done that analysis. You know, I think on a chart of maybe 40 or 50 names, we have maybe 10 names that are under 20 percent of their free cash flow as a percentage of stock-based compensation.

So it is – like the thing that jumps off the page at you is just for tech companies, which is what we're here talking about, what we care about, it's a massive, if not the largest single expense for them. And it's the one thing that way too many of them are not paying attention to.

I think to give some credit where credit is due, I do think some of the adjustments, for example, that you've seen Meta make both in terms of headcount and now how they're talking about SBC as a real expense, Uber is talking about SBC as a real expense. Like there has – partly due to your leadership, partly due to this move to GetFit, people are paying attention and are talking about it.

I think from our perspective, it's not that you should stop giving – using stock. But to your point, you should discount the damn thing and treat it as cash and we should have an honest conversation on an all-in basis what these various companies are earning. Yeah, and look, let me talk – first of all, I do think shareholders are starting to care.

Like it's – the conversations are happening. I think most of them oddly happen in one-on-one conversations and as was said earlier by Bob, the industry seems to kind of accept this adjusted EBITDA thing. It shows up in the sell-side research. It's in the earnings releases from the companies. There are many companies who only talk about that on the earnings call and never talk about the net income.

So it is – but I think the shareholders are paying attention. I think they're very unhappy with where things are. A couple of things Bob hit on. You know, if you're in a boardroom where people are saying we should buy back the exact number of shares to offset the SBC, this is going to sound like I'm really – but you're dealing with financially ignorant people and they just don't understand and those same people I think are being widely disrespectful to people in your practice, Brad, of owning shares.

You're not that stupid to where you would say, "Oh, if you buy it back, then it never happened." Like – I've had this argument many times. Right. Right. You're taking my earnings that we could either use to invest in new projects, invest in new countries – Do a dividend.

Or just hand – give back by way of dividends and instead you're buying back the shares, which effectively just acknowledges that this in fact was the cash that we said it was in the first place. By the way, when the RSUs invest, they sell – so you're selling and buying at the same time.

Right. What's the point of that? But anyway, I also agree with the comment that was made that I think management teams treat cash – I thought that conversation about the hiring example you mentioned. But management gets very concerned about spending a dollar of cash and they're – and many of these companies, they almost don't care about spending the equity, which is a weird, bizarre thing because they're actually the same thing in these cases.

Well, it's not weird and bizarre if at the end of the year, remember, a lot of their bonuses are based on adjusted EBITDA. Okay? So again, to Bob's point, if we adjust this out, then it gets back to Charlie Munger, show me the incentives and I'll show you the behavior.

Let me jump off of that. So having been in many public comp committees, here's what happens. There's no one in that committee that's thinking about things from a shareholder alignment perspective. They're all thinking about what are the common practices. So you go hire a comp consultant and the comp consultant is going to tell you what your peers are doing.

In fact, you create a peer list, you study what the peers are doing and you build your program based on what the peers are doing. And if enough people choose the 75th percentile, then you move – what happens there? You move even higher. But nowhere does anyone – I don't think the comp consultant or anyone stand up and say, "Hey, is this a good program for the shareholders?" And when I've tried to do that in the board meetings, I get pushback.

And there's a great – Ironic given that there's a fiduciary responsibility to the shareholders, but I agree with you. No doubt. And I love this quote. Warren Buffett said, "I'd rather stick a viper down my shirt than hire a comp consultant." And my big takeaway from that is the best and the brightest companies have to have the courage to lead and go in their own direction and create a compensation program that is unique to them and that is financially rigorous and understands these things.

We've heard a great example of booking. I think another one is Netflix. So Netflix today has a remarkably unique compensation structure. No one does anything like them. But today, if you're an employee, you get to choose either options or cash. And there's a slider and you get to pick where you want to be on that slider.

They don't have RSUs, none. And in a very recent comp committee consultant conversation, I said, "I'd rather use options." They said, "I wouldn't recommend it. No one else is doing it." Yeah, no one else is doing it. No one else is doing it. Unbelievable. And so anyway – But this gets back to the conversation you and I had last week, right?

We even have a court criticizing Elon for taking an options package where he made no money unless he saved the company from bankruptcy. Meanwhile, the CEO of his arch rival, who created no shareholder value over a period of five years, is making tens of millions of dollars a year in RSUs.

And by the way, I'll say it again, I said it before, I would offer any CEO I work with the Elon package and I'm certain none of them would take it. And two, if Delaware doesn't get this right, they really need to understand that shareholder alignment and executive compensation is a very, very important piece of making these markets work.

But I'll just – again, Bill, I'll agree with you that – because I've been on both sides of it obviously when I was on the management side putting together these compensation plans and presenting them to our compensation committee and to our compensation consultant. And of course, as a director of many public companies and many comp committees, the dynamic is kind of almost always the same.

A company establishes its peer group and no board ever wants to think of their management team as being anything other than above the mean. And you have 100 percent of all those companies that are thinking of their management teams as being above the mean, so you have this constantly upward pressure on these – on the granting practices.

It takes a mindset of like how do we as executives and how do we as leaders of the company want all of us to make money? And I think one of the – back to sort of what are solutions. Consultants, comp committees, venture capitalists, even investors, they're very far too focused on the term annual dilution percentage.

Hey, this is a reasonable granting practice because it's only 2 percent or it's only 3 percent or whatever. But really, all anyone is saying when they say that is they're making a commentary on the valuation of the company because if I'm a company making a billion dollars of free cash flow but I have a 100 billion dollar market cap, right, and I've got a 1 percent dilution rate, well, I theoretically look like I'm managing the company well, it's 1 percent even though I just wiped out all my free cash flow.

You just have to focus on dollars, ignore dilution percentage and if you want, yeah, you could look at stock based compensation expressed as a percentage of free cash flow. But again, Brad, to your point on that slide, it's a shocking number. The percentage of free cash flow that goes right back in to stock is a huge number and when you do that over and over and over again, if it's 25, 30, 35 percent every single year which that's kind of the math of it for many, many years.

The dilution is actually – if you dilute free cash flow by 30 to 35 percent every single year for 20 years, guess what? Your dilution is 35 percent. It's not a half of – it's not 50 basis points. It's not 1 percent. It's not 2 percent. It's literally a third of the company.

I think you make such a good point, Bob. I go back to this Buffett interview he did on CNBC because this gets pretty esoteric. You sit in these comp committee meetings. I see the eyes glaze over. There's a social dynamic at play. You sit on the board. You're in the room.

You often know these people, right? You want to treat them fairly. But Buffett said this and we'll show this quote. He said, "I think the best way to compensate people is with cash. If you want to give them a bonus for exceptional performance, give them cash. It's simple. It's straightforward.

It's clean." He said, "I don't believe in using stock options as a form of compensation. I think it's just a way to get around paying out cash. I think it's a way to sort of camouflage what's really going on." Now, I disagree with Buffett on the idea of whether or not we should use options.

I actually think there's a tremendous amount of alignment and incentives that are created through that. But if you ask me, Bob, what do you think is the best thing if you're sitting on a comp committee, the conversation that I would force is let's have a discussion about cash, right?

What is the value of what we're giving up, right, in order to achieve our objectives? Does it make sense? Because if not, then it obscures bad business models. It obscures sloppy performance, right? To me, it comes back to this question of getting fit. You want to know, "Am I really fit?" If you're going to work out and you're not measuring your weight or your body fat or anything else, you're kidding yourself.

To me, the measure of whether or not a company is really performing is the cash equivalent focus. Bill, on this, the thing I wanted to ask you is what has been the rule of thumb in Silicon Valley? If you're a founder, you used to invest almost exclusively in Series A companies, first institutional money in.

I don't know. The rule of thumb is there would be a 20% or a 30% stock option pool in addition to what the founders owned in the business. That would go down over time, I think. As you march to the public market, you would have to focus on, again, treating this as cash and getting profitable.

What are the rules of thumb today in Series B, Series C, Series D? Where do you think you're seeing some of the sloppiness? Well, look, the thing I'd start with in answering that question is mostly startups never get to this place that we're talking about. A lot of the commentary in Silicon Valley is always about the 10 or 20 or 50 companies at the top, and the vast majority of these companies never get to that place.

The general rules of thumb, as I started with, are like 10% headed down to three. It is done as a dilution perspective. Profitability first, I think, is if you have a company, especially one that burns a lot of cash, and if new fundraising and capital is expensive, I think one of the reasons people start with this adjusted EBITDA number is it does represent a point in time at which you no longer have to raise capital from the markets.

You could argue over whether that matters or not if you're issuing stock, but it does. It may matter, especially if capital is hard to come by. Then I would just say some of the same things we've been saying. Don't fool yourself. Don't assume cash is cheaper or more expensive than equity.

They're really the same thing, and eventually it's going to matter. I do think Wall Street figures it out. I don't think there are any secrets. I think alignment is really, really important. I encourage companies to, and I do this when I'm recruiting, to look at scenarios of the stock being up 20%, 40%, 80%, 100%, and look at the payouts that people are going to make on that, and see if you have alignment.

I think if you're only doing zero basis RSUs, you've got a horrible alignment problem. Don't be mad. When you think you're EBITDA profitable, and your stock doesn't have the same multiple as the other company, and I guarantee if you look under the hood, they've got a lower SBC expense, and they're not pissing away their free cash flow.

I don't think there are any secrets. I think competition is a real issue. One of the things I wanted to go to with Bob, I saw this really interesting presentation from Gary Steele, who turned around Splunk in a similar way to what happened at Meta. Gary crushed it at proof point, and a huge value creator.

He said, in a very short window, he said he took stock participation from 87% to 15%. I saw so many CEOs in the audience, their face just went into shock. It has become almost ritual in Silicon Valley to assume you have to give everybody stock, including the chef. It literally is part of, "Oh, we're cool like everybody else." I think that's being re-examined.

The other thing he said was, they moved a lot of employees out of Silicon Valley. That's an interesting thing, and I'm hearing that from more and more CEOs. They say, "Silicon Valley is a wonderful place to start a company. It's a horrible place to scale a company." My goal isn't to come on this podcast and say, "Hey, everybody is getting overpaid." Far from it.

It's more just how I think people – we need to have a better model by which people get paid. At least at Priceline Booking, we always had the mantra of a dollar has to be worth a dollar, whether it's in cash or comp. If we can't convince somebody that a dollar worth of stock is worth a dollar, then you actually shouldn't grant it.

You should just give a dollar of cash because they will actually value it greater. I think over time, because we've been so careful about this and because we have been able to create shareholder value through many ways, mostly through the incredible work of the employee base, but also, as I said, through smart capital management that has allowed us to reduce our share count, that's an incredible benefit that employees have gotten over the past many years.

When the share count is getting reduced, theoretically, the stock price is supposed to go up. That benefits not only shareholders, but the RSU recipient. Ryan: Hey, Bob, do you have any sense how many employee millionaires that Booking Priceline has made over the last 20 years? Bob: No, I don't.

But it's – it is certainly – Ryan: Extraordinary number. Bob: Yeah. Ryan: Extraordinary number. Maybe this is a good place to wrap. I think your leadership on this issue, right, with this fundamental thing – this is not about undermining founders' and employees' opportunity to make an extraordinary return, make millions of dollars in these startups.

In fact, it's about protecting that. And it starts in the public markets. Because if the public markets all pretend that this doesn't exist, then they're going to pay excessive amounts, which deprives startups and the venture capital community of being able to hire those employees and compete. That's the example you just gave us just now of Microsoft.

And so I think that accountability, treating stock as cash, as public shareholders – and the good news is, like I said, there's a lot of evidence that this is starting to be a real matter of conversation on Twitter, on CNBC, in annual letters, folks like Bob leading the way.

And then I think we need to begin telling the stories again in Silicon Valley. The reason the Michael Dells of the world own as much of their companies at the end of the day as they do is because from day one, they were concerned about giving shares away where they didn't need to give shares away.

And so that discipline cascades down. I think all companies are better when their DNA from the get-go is just honest. They treat all of these things as expenses. It forces you to build a better business model. We were lucky to be partnered with Bob and investors from the early days.

And they were forced to really reboot and think about the business model that they were going to focus on. If they kept pretending that this didn't matter, I think it would have been a very different outcome for that business. I think you would have produced far fewer millionaires than the number that you've produced.

And again, in Silicon Valley today, I do see this also cascading down into the companies that we're investing in. Certainly the companies that were on the comp committees of the boardrooms that we're in, this is a matter of conversation. I would say during the heyday of Zerp, I was seeing some really crazy stuff in terms of annual dilution.

And now it's a tough conversation at each of those boards because I think a lot of venture capital firms and founders woke up and said, "Wow, I just gave away a lot of the business. I thought that was okay because the valuations had gone from $500 million to $5 billion.

But now that I know I'm really only worth a billion dollars, the consequence of that dilution was way more dramatic than I thought." And I would just close by once again going back to this notion of independent leadership. The only companies I see kind of getting out of this box are ones where you have people that are thinking from a first principles standpoint, like Bob did at booking.

In the Netflix example I gave, they're only diluting 1.5% a year, which is pretty left rail kind of thing if you look at them on one of these charts we're going to post. And so it takes someone that really understands this stuff at a fundamental level to be willing to step outside the box because once again, the comp consultants, maybe even ISS are going to tell you to do what everybody else is doing.

And I don't think that's going to get you to the finish line. Right. Thanks for being with us, Bob. Well, Brad, I thought maybe we'd hit two or three more topics today. And wrapping it up, I just thank you once again to Bob. And I wanted everyone to know that we're going to put a bunch of resources in the show notes.

We've taken snapshots of 50, 100 different companies, looked at SBC as a percentage of free cash flow, which Brad and his team believe is the most important, but also revenue and market cap. I think those are all useful ways to kind of get a sense of how companies are doing on this issue.

And then lastly, my good friend, Mike Mobison, who many people in the internet finance world know about, he put out a really great piece on stock-based comp and reading it put a lot of thoughts in my head as we went through this. So I'll put a pointer out there to that as well.

OK, so there's a lot of new stuff happening, Brad, since we talked last. One of them that's kind of interesting, and I think I saw a photo on the internet of Sundar and Tim Cook at a restaurant that may have kicked this off. But there's a rumor that Apple's talking to Google about leveraging Gemini, I believe.

And tell me what you think of this. Yeah, I mean, I think the news broke yesterday. Both our stocks were up a bunch on it. This idea that-- I think the implication of the rumor or the tweet was that Apple was somehow maybe outsourcing or partnering with Google on Gemini.

And I tweeted in response to this that I put the probability of them, quote unquote, outsourcing AI to Gemini at next to zero. We talked about this on the pod a few weeks ago. And I think that we-- I go back to that analysis, which is this. Apple is in this enviable position to be able to build the killer personal assistant that has memory, that has persistent, that can do transactional thing for us, whether it's make a phone call, send a text, book a restaurant reservation, book a hotel, et cetera.

So we suggested that they're going to work on their own native small language model that really doesn't need to be Einstein. It just needs to serve me really well as my personal assistant. I think what's likely being commingled here is I presuppose that Apple is also talking to Google, to Meta, to OpenAI, and others about bringing those models in as part of a generative search application that will sit on the deck as well.

So think of that maybe similar to perplexity. And then, of course, they have their search deal with Google. So to me, I don't think anything really changes in my view about Apple. I think they continue to be in this really interesting position. I'm sure they're going to talk about it in June at their developers conference.

I think it's unlikely that you're going to see some transformative thing get shipped this fall. I think you may very well see a generative search application that gets shipped this fall that may, in fact, have multiple of these answer engines embedded inside it, including Gemini. But the final thing I would just say on this is I've been, over the course of the last couple months, pretty openly critical in my critique that Google needs to face up to this innovators dilemma, that needs to make some real changes in terms of how they're going to market.

And you and I have those debates all the time. And I think there is a group of people on Twitter and otherwise who assume that I must somehow-- that must mean that I'm short Google. And we've said many times, this show is not investment advice. Don't presuppose you know where we stand on a particular stock.

I'm often asking tough questions of stocks that we own. I mean, go back to Meta. And in fact, we own Google. We weren't shorted this week when this news came out. And so I think it's an important reminder, much like Stan Druckenmiller. At this moment in time, when you're in the middle of these phase shifts, mental flexibility is incredibly important.

And we are going to follow the facts, and we may change our opinions on a particular stock. I think ultimately for Google, the challenge remains the same. The challenge for Apple remains the same. You should imagine that everybody is talking to everybody right now as they try to sort out where they're going to compete.

But I don't know. What was your reaction to it, Bill? Well, it's funny, because I think you can have two reactions that are exactly opposed to one another. Right. So you could view it-- you said both stocks were up. Yes. So that's the interpretation, that this is positive for Apple, that they're maybe getting their AI game ready to go.

You could view it as weakness on Apple, that in order to get AI right, they need to go talk to Google, and they can't do it on their own. As we have said before, I think the handset is a remarkable asset in the long-term AI race. I mean, you know, one issue that could come out of this is regulatory.

I mean, if you combine the two of them, that's nearly every handset. So if the same AI code were running on both, I don't think that'd be great. I do believe that the handset is so valuable here. If I were Apple and wanted to prove to the world that they had their AI game together, and I know this is a bit of a broken record with people, but fix Siri.

Like get Siri involved and release-- But that is the small language model that I'm talking about, right? That is going to be the new back end for Siri. Like there's no way they're outsourcing that to Google. And by the way, on that topic, I think voice AI may be more interesting than, say, Sora or the video stuff that OpenAI released.

I think more and more people are realizing, because LLMs are language models, that people would love to just talk to them. You brought up the point that Meta may be more focused on the sunglasses with an earpiece, right? Right. And you want to be able to talk to that.

And so for me, the irony of this is, I think voice AI may be a new frontier that you may see a bunch of these people race after. And getting that to work appropriately so that you feel comfortable-- I always like to reference the movie Her-- just talking to this thing and not having any errors, I think that's super interesting.

So we'll see how this all plays out. Let's move on to the next topic. So there's been a lot of conversation in the past few weeks about TikTok and ByteDance. And obviously, it very quickly moved to the center of almost all discussion, because it involved the House and the Senate.

And it's on the Sunday morning talk shows. It's in every newspaper. And it's been discussed quite a bit. But I'd love to hear your thoughts, Brad, on this situation. Yeah, well, I've said publicly before, we were early shareholders in ByteDance. I think we've been involved almost a decade in the company.

So take everything I share with that grain of salt. First, I think everybody knows ByteDance is a super large company. There's lots of numbers that are rumored out there. But the reporting suggests that over 90% of its revenues are outside the United States, and over 100% of its profits are outside the United States.

What's that mean? Well, I think I've seen other numbers out there that they're doing somewhere between $8 billion and $10 billion in TikTok US revenue, and that they're losing money on TikTok US. So I think that it's important just to understand where that sits in the context of the entire business.

Secondly, with respect to myself, I'm a dad, and certainly an American first. And my kids use TikTok. They create on TikTok, et cetera. But I read all the same things everybody else reads. And I think it's impossible for me to prove or disprove. And I think there's legitimate debate as to what's going on.

I think there's also lots of legitimate debate about the House bill itself and whether or not that's narrowly tailored enough, whether the language needs to be improved, et cetera. But clearly, it passed the House by a wide majority. I think there's a real question as to whether or not that will pass the Senate.

But I think that misses the point a little bit. Set all of that aside, what's become clear to me over the course of the past 12 months, and certainly over the course of the past 30 days, is irrespective of the merits of the matter, right? This is now in the sovereign domain, right?

AI, over the course of the last 12 months, has gone from an interesting technology to literally the leading edge of where we're battling with global sovereigns over trade, national security, national economic interest. So given that, I think it's almost inconceivable for me now to see how TikTok US has a path forward under kind of the status quo.

So where does that leave us? That leaves us, on the one hand, with the proposal that I think President Trump had made toward the end of his term, and which I hear bantered about by lots of senators and House members, which is some sort of spin to US shareholders, where maybe ByteDance swaps some of its ownership or sells some of its ownership to somebody like a Microsoft or an Oracle, and the app can continue running in the United States, OK?

I think there are a lot of US regulators that would find that an acceptable outcome. But I'm increasingly of the mind that the Chinese regulators would not find that an acceptable outcome, OK? And the US regulators won't allow the app to continue as is. So I think there's a real chance that when we're looking at this 12 to 18 months from now, that it's an absolute no-go, that they can't find a middle ground, and that the app is shut down, which, of course, we saw President Trump tweet, like, the implications of what that makes Meta more valuable or more powerful, as he suggests, or perhaps these other US companies.

But the thing, I guess, that interests me the most is what is—you know, that's just the first step, Bill, right? Like that's the first move on the AI chessboard between these two sovereigns, if you will. It's hard for me to see a world where TikTok US is shut down or banned, where there's not some response by China.

And I think we just had this conversation about Apple putting Gemini AI on the Apple phone. I'm certain, you know, Apple China is a monstrous business, is a huge business in China. And I imagine that now that AI is going to be embedded in these phones, that there's probably a little bit different conversation that's being had in China about, you know, about that there as well.

So I think we're seeing just the early innings, unfortunately, right? I tend to think that global trade is a good thing for human prosperity. I think it's driven a lot of our global GDP and prosperity over the course of the last 20 years. So I don't celebrate that we're entering what appears to be a global AI Cold War on the economic front.

And I'm certainly not critical of the fact that US regulators have these concerns. But unfortunately, I don't see much of a way out here. I hope I'm wrong. I hope that, you know, when President Xi was here a few months ago, he said, I don't want a Cold War, I don't want a hot war.

I want a partnership. But that's not what it looks like from my vantage point today. Yeah. I'll give you a couple of my thoughts. First of all, I thought our friends over at All In did a really great job of tearing this apart. And even where they disagreed, I think every single one of them agreed that the data probably should not be going to the CCP.

Like I haven't seen anyone said, oh, I'm completely cool with data going to the CCP. So if everyone's in agreement on that, and I do understand the arguments that maybe the House bill is too flexible or too vague or too broad, and you could tighten it up and achieve the same things.

As I've told you, I feel like that the company, and the company I'll say is either US TikTok or ByteDance, could have done a better job of proving data independence. And I think they could have hired an auditor, you know, KPMG or Pricewaterhouse or whatever, rather than simply saying, oh, it's in Oracle in Texas.

To me, there's not much certainty in that comment. I would also say relative to your Apple comment, I do believe Apple has to keep all their data locally over there and prove it to the CCP. So I think- But that may not be enough. No, it may not. In this new context.

But I agree with you. But I'm just saying from a prosody standpoint, I think they're already doing what is being asked here. And as we all know, and we've said, the social networks here aren't allowed to operate there. So you could have a, I think you have a pro-China mindset and just still hold the line and say fairness is fairness.

So we will see how it plays out. I'm hopeful that there's a way to get it together. But who knows? We will see. Anything else on that? No, I mean, I think that it's, once you start going down this slippery slope, right, you know, we have a lot of US companies that have a big footprint in China.

And so- But you're already down the slippery slope. Well, I mean, we haven't banned it yet. So I think if and when this effectively, you know, if this were to get banned in the United States, again, I would like to think that that would be, you know, where this ends.

All I'm suggesting is that ChatGPT only came out 18 months ago. And you can't use it in China. The wheels of international relations, I think, turn slowly. And when we look at, it's first started with silicon, right, with restraints on China's ability to get Nvidia chips. And again, I don't disagree with, you know, our ability to keep our best stuff to ourselves.

But it's hard to look at the facts on the field and say that this is not, we're not entering a new phase of global international trade relations, you know, vis-a-vis China. Now, one might say that we're just, you know, creating a more fair and balanced playing field, right, relative to the restrictions that have been imposed on U.S.

companies. But I think it would be naive of us to think that there's not going to be, you know, counter responses out of China. I hope not, but- I take the other side of it simply because we're not even getting to the point of equality here. So anyway, we'll leave it at that.

And then this morning, just this morning, we rarely have news that broke just this morning, a very unusual thing has played out with Microsoft and inflection. Why don't you tell people what went down? Well, inflection, you know, is, you know, just as a reminder, was one of the early LLMs to be trained on H100s.

They have a consumer product called PI, which stands for personal intelligence, founded by Mustafa Suleyman, one of the co-founders of DeepMind, you know, and a friend. And you know, what they did was, you know, as one of the- And by the way, just to put in, I think there's another way to frame this.

There were three LLM companies in Silicon Valley that raised over a billion dollars, although some of that was with credits, but there were three, Anthropic, OpenAI, and this one. Correct. And, you know, this is a large startup, but what they were going after was really think the chat GPT consumer market with PI.

And, you know, today we heard news that Microsoft has effectively hired the employees out of inflection. We don't know all the details, you know, of the deal here. But you know, we looked at investing inflection, as many others did, and, you know, one of the reasons that strategic investors have displaced a lot of venture capital investors in the LLM game, and we talked about this, I think, on, you know, episode two, is the quantum of capital it takes to compete in the LLM space is very high.

And on the other side, understanding the durability of the business model, like in this case, getting consumers to pay you, you know, for PI is far from clear, particularly in a world where you got to go compete against Apple, you got to compete against Meta and others who are going to build a personal assistant.

And so, I think it was a very difficult venture bet to underwrite. I think it was a logical bet for folks like Microsoft, you know, to underwrite because, you know, they want to be in the consumer answer engine space, the consumer chatbot space, if you will. I think the most interesting thing out of this, to me, and if you read Satya's tweet that he sent out this afternoon, is he said he's excited to have them build consumer AI, which, you know, he said, like the copilot, that is loved and benefits people from around the world.

So, just step back from a second here. He owns 49% of OpenAI, ChatGPT is charging really hard to be, you know, kind of my personal copilot, if you will. And now they have another, you know, another team in the hunt. So, you know, I think this is smart by Satya and Microsoft.

Some might view it as hedging their bets a little bit against ChatGPT, but I think it's great for the consumer. They're going to be a lot of at-scale players that are vying to be your personal assistant, that are vying to answer your questions. We just talked about Apple, we talked about Meta AI, Google's certainly going to be in that hunt.

And now it looks like Microsoft is doubling down on the consumer copilot, which will bring even more competition to the table. What was your takeaway? Well, look, I mean, I got to tell you, I was floored a bit when I saw this, mainly because of the peculiarity of it all.

So to have a company that is one of the three, let's just call it biggest backed AI companies in Silicon Valley, not get bought, like if they got bought, that means, "Oh, Microsoft thought this was strategic." Microsoft, you remember when Facebook brought WhatsApp, it was 10% of market cap.

So 10% of Microsoft's market cap is $300 billion. So they could have paid $10 billion, and the last round here was at $4 billion. And you'd go, "Oh, great, you know, everyone loves AI, investors are getting paid, everything worked out." But this isn't everything worked out. This is employees, all the key employees leaving the company, and the blog post suggests the company goes on with a new CEO.

And to me, I just had to look this up because I don't watch racing, but the yellow flag comes out when there's a hazard on the track and everyone needs to slow down. And you look at this Series A here, a well-known venture firm, Greylock, 260 or 250 at a billion pre.

These are these AI deals that have remained like Zerp-like, right? And this is a warning sign that you can't just pay any price for any deal in the AI sector and get paid. And because I don't, you know, I don't see this now, I don't see the current status as a company that's going to have a venture-like return from here.

There's a lot of details, there's a lot of these credit things that are all tied up, there's stuff we don't know. But it's unusual, and I view it as a yellow flag that we should slow down on the track, on the AI track, and maybe be a little more careful.

Well, one of the things that it brings up too is a conversation you and I were having, you know, just about LLMs generally, you know. I spent the last two days down at NVIDIA, you know, GTC, their Global Developers Conference. It's just extraordinary. The ecosystem of AI that they are consuming, right?

This is not a chips company. This is not just a systems company or a supercomputer company. They are now expanding on the CUDA stack with a totally new inference software cluster called NIMS. And in NIMS, you know, they have embedded, you know, open-source models that are, you know, highly and tightly integrated with their overall software stack.

And one of the things we've talked about and debated the last 12 months is just the durability. Like, what's the business model for the LLM? Is it going to, you know, you have these open-source models by well-financed companies like Meta that don't seem intent on charging for them. And when you have companies that are valued at $10, $20, $90 billion, right, around that, the real question was about the durability of revenue.

And you know, we have a vibrant debate within Altimeter on this, right? So the acronym we all know when it comes to software companies like these LLMs is ARR. So everybody wants to know what's their revenue. ARR stands for Annual Recurring Revenue. And recurring revenue, like a monopoly cable subscription, gets a really high multiple.

That's why softwares have gotten high multiples. They're very predictable. But when we look at what's going on in AI, I've banned the use of the term ARR within Altimeter when we talk about AI revenues, because they're certainly not recurring like a monopoly. They're consumption-based. And they have some amount of cogs tied to them.

Right. So we call them ERR, Experimental Run Rate Revenues, right? And so when I look at these, you know, one of the things that inflection and these other models that it's been very challenging for us to get our arms around backing, even though they have extraordinary founders and are building extraordinary things, is just if you have to pay a high price, then you need to have confidence in the durability of those revenues and profits on a go-forward basis.

And I think you have some thoughts on this. Yeah, look, there's nothing that I've spent more time on recently, just trying to learn and ask questions, but, you know, both trying to understand how these things work, but also how they're being used. And one thing you constantly hear from developers on LLM, and let's separate this maybe from the consumer game that I guess—and then there were two or three, right?

You got Microsoft. I guess you got Google perplexing OpenAI, $20 a month. That's a whole separate conversation. But on this B2B side, the model that just seems so prevalent with all the companies I talk to, you might test your thesis against one of the premier models, but then the minute you go to run it, you then run it on four or five others and see which one—at different price points and see which one is cost-effective for you.

And the promiscuity is unbelievably high. Another way of saying that, one of the things that leads to high valuation multiples is switching costs. And today, you know, as it stands, I would say if you were to rate these LLMs on their switching costs with zero maybe being none and 100 being perfect, I'd say they're at two.

Like, it's remarkably low. And there's another huge irony. Some people believe—and I won't try and make this argument, I'll just say I know some smart people have said it—that as the context window gets bigger, there'll be less training and even less fine-tuning, which means you're just dumping data into the prompt.

And this data is not being dumped in in a structured way. It's just PDFs or CSV files or whatever. There is no lock-in. You just dump it in the next one. So I think one thing that's imperative in any of these LLM models, if they want to be successful in the long run in terms of creating equity value, they've got to find a way to get to switching costs.

And I don't know what that is exactly right now. I think some element of fine-tuning or embedding data or memory like we've talked about. But I don't think they have them today. And maybe this is a sign of that. I don't know. It'll be interesting to watch. Well, I think along those lines, I had dinner this week with one of the heads of data and AI at one of the largest banks in the United States.

And they did what I think a lot of other people did after they saw ChatGPT. So remember, ChatGPT comes out in Q4 of '22. The CEOs of all these banks and every other enterprise went home, and that's what the conversation was around the holiday table. So the CEOs or the board came back in Q1 and they said, "Hey, what are we doing in AI?" And they turned to their tech team and they said, "Let's get something done." So in Q2 of 2023, you saw all of these new customers sign up the Azure OpenAI API, right?

Or do it directly with OpenAI because it was the only game in town. But quickly, all of the other data platforms, whether you were AWS, whether you're Google, Snowflake, Databricks, et cetera, they all spun up their own AI capabilities. Now they all embed free models, whether it's Lama, Mistral, et cetera.

Even Microsoft now has competitive open models. And my hunch is that OpenAI will also be found on some of these data platforms, not just on Microsoft. And so it seems to me the switching cost is not with the LLM. I agree with you on this. The switching cost is really around where your data resides, right?

We've heard this talked about, data gravity. So this particular bank, which had spun up the OpenAI API in Q2, has been playing with it on a big rag over the course of the last year. But I asked them what their plans were going forward. They said, "Oh, we're moving.

We're going to run all of our AI in AWS, right, on open models, on our production data," right? So it seems to me that the AI is finding its way back to the data. And the only way, I think, for Azure or anybody else to keep this experimental revenue when people run to use these models is you have to deliver something sufficiently differentiated that it gets somebody to pull all of their data to you.

And that's a hard road to hoe. And I'm reminded – I don't know if I've used this metaphor before, but there was a time before digital music where you just couldn't – there was so much music in the world, you couldn't imagine it. And I remember one day back in the Napster days where someone tapped me on the shoulder and he said, "Do you see this?" And he was holding a hard drive and he said, "This is all the music I've ever recorded," and like he just had it.

And to a certain extent, you know, the LLM, it may be – and I've heard this argument from some academics – it may be that most of the data, think Wikipedia, everything we know is already in there. And like it's gotten in, you know, small very quickly. And what – the reason I bring that up relative to what you said is it may be that the open source LLMs are good enough and that putting them proximate to your data is a smarter play than trying to put your data proximate to the leading proprietary LLM.

And if that plays out, that'll be super interesting. And one thing's for sure, the open source LLM horse is out of the barn, right? It's running wild and free. I mean, and you've got to love the competition. And you know, again, it's a – you and I have talked about some of the concerns about regulatory capture as, you know, in preventing this competition.

You know, going back to the conversation about China, it's so important that we protect this competition among these models because it's driving down the cost to all enterprises. It's driving down the cost to all inference, to consumers. And you know, all I know is it's going to be an exciting few months ahead.

Always. Always. So fun to have you here. Fun to do this again. All right, man. And until next time. Take care. Bye-bye. Bye-bye. Bye-bye. Bye-bye. Bye-bye. Bye-bye. Bye-bye. Bye-bye. Bye-bye. (dramatic music) you