- Hi everybody and welcome to the literally just launched Queensland AI Hub. There's the rock in the hoodie. Queensland AI Hub is in Queensland. So I actually was only wearing this for the advertising. I actually don't need it. All right. So welcome to sunny Queensland. My name's Jeremy Howard.

I'm originally from Australia, grew up in Melbourne and then spent 10 years over in the San Francisco Bay area. What I always used to think of as Silicon Valley, but then I got there, was staying in San Francisco when somebody said, let's meet up in Silicon Valley and an hour and a half later, I still hadn't got there.

And I thought, oh my God, okay, it's actually quite a long way, especially with the traffic. So San Francisco Bay area, I was there for about a decade and returned back here to Australia two months ago and have made the move from Melbourne to Queensland, which I'm very, very happy about.

So this is a really lovely place to be. Having said that, overwhelmingly the reaction that Rachel, my wife and fast AI co-founder and I get when we tell somebody, when they come up and they'll say, oh, welcome to Australia, welcome to Queensland. How long are you here for? Oh, we've moved here.

You've moved here, why? And there's this kind of sense of like, why would anybody wanna move to Australia? Why would anybody wanna move to Queensland? You were there, you were in Silicon Valley. Not really, San Francisco, but what are you doing? And to be fair, it is a reasonable question because, so to be fair, this is not exactly the global hub of AI and AI investment.

In fact, we're way down here in terms of investment in AI at a massive 0.29% of global investment. And this data is from Andrew Lai from Boab AI. Thank you very much to Andrew, who's actually given me quite a lot of cool data that I'll be sharing. So yeah, I definitely feel that.

I gotta say it's 0.29% more than when I left, so that's good. But I wanna kind of make the argument today that actually this is a really great place to start a tech startup and actually a really great place to do AI research or AI implementations despite the obvious issues.

So let me tell you about this insight through the lens of kind of describing my journey, I guess, to get here. So my journey, as I said, kind of started in Australia, right, that's a bit of a thick one, isn't it? Let's try making that a bit thinner. Okay, so I started out in Australia and 25 or so years ago, I thought, you know, it'd be really cool to start a startup.

I mean, I can only think of the startup sense, start a company, you know, make a company. And then I thought, well, there's a problem. Jeremy, you don't know anything about business. So, you know, initially it was like, oh, let's do a startup or a company. And it's like, nah, you don't know anything about business.

You don't know what you're doing. So let's learn about that. So I actually went into consulting. So I thought, okay, let's go to McKinsey and company. They know about business and spend a couple of years there. And I went to a couple of different consulting firms along that journey.

And what I discovered along the way is there's no such thing as business. There's such a thing as like making things that people want and then selling it to them. And that's about the end of it. So I did certainly learn some valuable skills from our time in consulting, particularly the skills around how to influence people, how to influence organizations, but the actual explicit feedback I got about my ideas were on the whole terrible.

For example, I was very proud of myself when one day I came in to work with a CD-ROM that I had bought that contained this really cool thing. Somebody had like got lots of data about what movies people like. And it's like, this person likes these movies and this person likes these movies.

And through some kind of magic I didn't understand, which I now know is called collaborative filtering, you could type in some movies you like and it would tell you other movies you might like. And so I went into and I talked to one of the directors at the consulting firm and I said, imagine building a company based on this.

Like you could even have like a website that wasn't static, but you go to their homepage and it could like tell you what things you might want to buy and wouldn't that be awesome? And the consulting director was like, you have no idea how companies work. This isn't a company.

Companies are about that competition, about market forces. This is nerdy technology. Similar reaction when somebody was talking about creating a new web search engine, which was gonna be just like Yahoo, but as a Java applet. And so, and it would also have the power of these like big brands behind it.

And I kind of said to them, I don't know, I wondered about like, what if we instead of having like lots of humans finding websites and putting them into a hierarchy, could we use like an algorithm that would automatically find interesting websites based on like what you typed in or something?

Similar reaction. No, no, no, no, you don't understand. Humans need other humans to help them find things. You can't like get some computer to like do this very human job. And so overall, this was kind of my experience of learning business. And this is the first piece of advice I have for potential people doing tech startups here is don't listen to old people.

'Cause us old people don't know what we're talking about, unless it's explicitly about the actual thing that you wanna do. And they actually have years of experience in that thing, doing it in the new way that you're thinking of doing it. Because otherwise, all you get is these kind of biases about business as usual about the status quo.

So somehow, you know, and I mean, in my 20s, I didn't know that and I thought there's something wrong with me that I didn't understand business, that I didn't understand why these ideas were bad ideas. So I actually ended up doing consulting for 10 years, which was eight years longer than I had planned, still trying to figure out what's wrong with me.

Eventually, I decided to do it anyway. So that was the end of consulting. And I thought, okay, I'll start a company. Now, the problem is that I'd read that statistically speaking, new small businesses generally fail. So I actually had a genius move. I decided to start two new small businesses.

'Cause I thought, you know, probabilistically speaking, better chance of success. So I started two companies. I started Fastmail. And literally within like a month of each other, I started Optimal Decisions Group. Now, aren't you drawing Optimal Decisions Group? So Fastmail was an interesting startup. It was basically the first one to provide synchronized email, whether email you got in your phone or on your laptop or in your workplace, you're to see the same email.

It's something that actually everybody in business already had 'cause they used MS Exchange or they used Lotus Notes, but normal people didn't. And I wanted to have that. So I built this company and it's still going great. And then Optimal Decisions was a insurance pricing, algorithms company. So very, very different.

Fastmail sold to millions of customers around the world. And Optimal Decisions sold to huge insurance companies. So it's basically only three or four insurance companies in Australia big enough to use our product. And then, you know, a couple of dozen in America, some in South Africa and so forth.

So very different kind of things. I didn't know anything about, you know, the Australian startup scene. So I didn't get any government grants. I didn't get any funding 'cause like for a consultant, you don't know about this stuff. You just build things and sell them to people. And so these were not Australian startups.

They were startups that happened to be in Australia. But like, for example, Fastmail at the time, this is really weird. I called up IBM and I ordered servers and I had them shipped to somewhere in New York that I'd never been. And they plugged them in for me. And so my servers were in there because like, why wouldn't you do that?

The cost of bandwidth in America was about a hundred times cheaper than Australia. And the number of customers I had access to in America was orders of magnitude higher. And so it never occurred to me to have my servers in Australia because Australia is far away and it's small and it's expensive.

And kind of similar with ODG, you know, the focus, I mean, I certainly had some Australian clients but my focus was on American clients 'cause there's a lot more big insurance companies in America. And so this turned out great because living in Australia, I didn't quite have a sense of how far away we are and how much no one gives a shit about us other than maybe like cricket.

But they don't. And but the fact that then we were just companies, not Australian companies, it didn't matter. It didn't matter we were a long way away. It didn't matter we were somewhere with crappy, expensive internet, you know? It just, you know, we were competing on a global stage without any constraints caused by our location.

And so that turned out to be great. We ended up selling fast mail to Opera, which is a Norwegian company. We sold ODG to Lexus Nexus, which eventually is a UK company. And, you know, that turned out great. And so the kind of advice I guess I found, I feel like from that I got out of that was, in Australia don't try to be an Australian company, you know?

Yes, there's lots of agriculture. Yes, there's lots of mining, but that is tiny compared to all the world out there. And furthermore, Australian companies are very, very hard to sell to. They're very conservative. They're very slow moving. If you create something like fast mail, right? Where anybody can go on the internet and give you money for your thing, that tends to work out great.

So like, for example, you come across this company called Octopus Deploy, which was a guy in Queensland who thought, "Oh, I could create a better kind of "continuous integration system for .NET." Created an open source software, chucked it up on GitHub, made a better version that you could buy if you wanted like 10 copies of it.

Like it was, again, it's similar idea. It wasn't an Australian company. It was a company that happened to be in Australia. And a few years later, now a few months ago, they got, I think it was $185 million of funding. And none of that funding was from Australian investors.

That was all from American investors. So it kind of bypassed the whole Australian thing and just focused on saying like, "You know what? "I'm a pretty good .NET developer. "I pretty much understand quite well deployment. "You know, why don't I make something "that anybody can just come along and use?" And so it's a similar thing now for Rachel and I with FastAI.

We started FastAI, which we'll come back to later in the US. We're now moving to Australia. It doesn't matter. Like no one thinks of FastAI as being an American AI company, and we can do it just as well here as there. And so, you know, we have access to the global marketplace.

Having said that, the next startup and some of these I co-founded. So ODGI co-founded, and obviously the next one, which is Kaggle co-founded. With Kaggle, we decided to try a different approach which was to get VC funding. Now, a similar thing, you know. I said to Anthony who we're doing this with, let's not even try to get funding in Australia because Australia doesn't fund tech startups.

Like it's basically so little as you could just ignore it, it's tiny. In fact, the amount of funding of startups in Australia in a year is less than the amount of funding of startups in the US in a day. So when I say it's different, it's very, very different.

So we went to San Francisco to try and get funding. And we were pre-revenue. And honestly, we didn't tell this to the VCs. We were kind of pre-business model. We were pretty enamored with the idea, but didn't quite know how to make money out of it. And so we thought we were being very bold by asking for $500,000.

It's like, okay, that's crazy. But we did, you know. And I will never forget the time we went into Andreessen Horowitz and Mark Andreessen said, how much money you're looking for? And we said, $500,000? And Mark was like, hmm, what would you do with $5 million? And we were like, make a better company.

But this was actually a start of a theme in the Bay Area, which was every time we'd say we want to do X, people would say like, well, okay, that's great. What if you could make an even bigger X? Or like, what if you could make an even better X?

So then the Node Coaster came to our little co-working space in San Francisco. And this is the other thing to know. If you ever go fundraising in the Bay Area, everybody knows everybody. And they all know everything about what's going on. So the Node was like, oh, I heard Mark Andreessen is looking at giving you $5 million.

They're like, oh, yes. What would you do if Coaster Ventures gave you another $5 million? And they were like, wow, it just kept pushing. And it was a very different experience 'cause I found doing my little startups in Australia, it was always like, oh, I'm trying to create an email company that does synchronized email and I'm trying to sell it on the internet.

And almost everybody would say like, why? Microsoft already has an email service. Yahoo already has an email service. They're bigger than you. They've got more developers than you. There's like, honestly, is there any chance that, obviously there's no chance you can beat them. So why are you doing this?

Is there something smaller you could do? Is there something more targeted you could do? Is there something focused on the Australian market you could do? I was like, everybody, best friends, colleagues, acquaintances. And it's very difficult because you end up constantly doubting your sanity. And the truth is, to be a tech founder requires a whole lot of arrogance.

You need the arrogance to believe that you can actually build something that other people are gonna wanna buy and that then other people who come along and try to compete with you won't do as well as you and you'll do better. You have to have the arrogance to believe you can win.

Which is a lot of arrogance. But you also need the humility to recognise that other people come along and they actually have some better ideas than you. And so sometimes you should borrow those ideas or sometimes you should try and find ways to do it better. So it requires this weird combination of great humility and great arrogance.

And in Australia, I found people mainly noticed the arrogance. But yeah, in the Bay Area, everybody was just like, oh, this is really cool that you're trying to do this thing. How can we help? Can we help you make it bigger? The other thing that I got a lot in Australia was this kind of sense of like, why are you trying to create that when they're already perfectly good things?

It's almost like you're a whinger or a complainer. It's like, things aren't good enough. You know, why aren't you okay with what's there? Whereas the other thing is there's this nice sense in the Bay Area of like, oh, it's really cool that you're trying to do something better. And so there are some cultural things that I felt Australia's kind of needs to get over to build a great tech entrepreneur ecosystem.

'Cause it doesn't have to be Australia-wide, but you want people in your community who are cheering you on and who are believing in you. Anyway, we didn't actually end up taking money from Andres and Horowitz. I can't quite remember. Oh, that's right, I remember why. They hadn't done any machine learning investments before.

And so what actually happens with these VCs is the VCs you speak to don't do any of the tech stuff themselves, they hand it off to, mainly to academics, which is something we don't say to have a great ecosystem for here either, is like, you don't see this strong connection between investors and academics in Australia.

In the US, you know, Bernard would ring up one of the professors at Stanford or Berkeley and say, can you please meet with Jeremy and Anthony? You know, this is what they're building. Can you check this, this and this? So with Andres and Horowitz, I mean, to their credit, they, through their DD, they kind of came to the point where they said, okay, we're just not convinced about the size of the machine learning marketplace.

We haven't done machine learning before. We're not comfortable with this. So we got out, we ended up getting our $5 million from somebody else. And one of the really interesting things in the VC world over there is the whole thing is so driven by fear of missing out, by FOMO.

So then suddenly people that we hadn't heard from suddenly started emailing us with like, can you come here today? You know, we really wanna see you guys. We're really excited about what you're doing. These are people who not replied to emails for weeks. And I'll never forget one of them.

I'm not gonna say who. We went down to their office. We're like, we kind of had a promise between Anthony and I had a promise between ourselves. We'd never say no, right? We'd take every opportunity. We're like, we were sick of talking to VCs. We're like, okay, we've said, we always say yes.

I'm so glad we did. Otherwise we would have missed out on this amazing situation. The people who said they were dying to see us left us waiting, I can't remember, for like half an hour in their giant board room. And then this guy finally does come in. He charges in, no introduction.

I hear you're gonna take money from fucking Mark fucking Andreessen. Is that right? And I think Anthony was about to reply and the guy doesn't let it and he goes, well, let me tell you something. If Mark fucking Andreessen was here right now, I'd throw him out the fucking window.

I'd break his arm. I'd take him to Stanford hospital. It's just down the road, you know? And then I'd fucking break it again. (audience laughing) This was his introduction, and Anthony goes, we're not taking money from Mark Andreessen. Well, that's fucking all right then, 'cause I fucking hate Mark fucking Andreessen.

(audience laughing) It's like, it was so much like this over there. The place is crazy. If you've ever seen Silicon Valley, the TV show, it's all real, but it's crazier than that, but they couldn't put that in the real thing. Do you guys remember the hot dog detector in that show?

Did you notice there was a real hot dog detector? They actually built for it on the app store. That was built by a fast AI student, by the way. He used to come in every week to class and he'd always ask these weird asked questions. He'd be like, I can't tell you what I'm doing, but let's say somebody was trying to find microphones and then they got lots of pictures of microphones and then some of them weren't microphones, but they looked like microphones.

(audience laughing) And then eventually, the show comes out and he's like, okay, that's what I was building. (laughs) That was so great. That was definitely one of our star students. Anywho, so. Yeah. Okay, so with Kaggle, what happened? Was I actually didn't expect us to raise any money, honestly.

So I just kind of was humoring Anthony. He was always the one with gumption, you know? And I was like, yeah, okay, I'll pitch and I'll build the financial models and I'll build the deck, but don't have high expectations. So then we raised over $10 million and yeah, the Node Coaster kind of looked at us and was like, so when are you guys moving here?

I was like, oh. (laughs) And obviously at that point, I can't not 'cause I've been in every pitch and whatever. So that's how I moved to San Francisco and I got to call my mom and was like, oh, this is what just happened. So yeah, I mean, moving to San Francisco was interesting.

It was like, all right, so let's do that. Australia, US, what is going on with this? US, there you go. It was interesting, like I was really starstruck. I was like, oh, there's Google, you know? There's Facebook, you know, meetups would be at Google or Facebook and I'd be like talking to a Google product manager and I was definitely like, wow, this is very exciting.

I felt quite starstruck. But the other thing I really noticed was like, I was talking to like these legends, but then I was like, they're actually really normal. You know, I kind of expected them to be on another level. I felt like as a little Australian nobody, I would just be dominated by these people.

But no, I mean, when I compared them to my mates back in Australia, they weren't all that. I mean, they were fine, you know? They were smart enough, they were passionate. But they weren't on another level at all. And I kind of realized that actually the Australian kind of talent pool is just fantastic, you know?

But there's this huge difference in opportunity and belief, you know? Like everybody I spoke to, you know, in San Francisco, like literally that I'd staying in Airbnb for the first few months. The Airbnb, people that ran the Airbnb, I was at like, "Oh, are you here doing tech startup?" 'Cause like everybody is there doing tech startup.

Yeah, yeah. Oh yeah, me too. You know, I'm a photographer. And I've got this idea that's gonna revolutionize how photography is done, you know, in product development settings. Like everybody you talk to has not just got an idea, but they want to tell you about it. They believe it's the best idea.

They believe it's gonna succeed. Which I don't get that, or at least at that time in Australia as I was kind of in Australia, I didn't get that nearly as much, you know? So I think that was a really interesting difference. And it gave me a lot of confidence in myself as an Australian to see that like actually, Aussies are not way behind.

We're actually pretty damn good, you know? So that was kind of interesting to me. But there was other differences there. I guess it's part of this kind of boldness, right? So I felt like folks there were on the home or bold. But interestingly, even though they were in the center of the world's biggest marketplace, they were still actually more global.

You know, none of them were trying to build American startups through American audiences, American companies. There was always this assumption that we're gonna chuck stuff up on the internet and everybody's gonna go and buy it. And you know, in terms of like who really needs that attitude, it's us in Australia.

Now one of the really cool things about being at Kaggle was that I got to see, you know, I was the chief scientist there as well as the president. So I actually got to kind of validate and check out the winning solutions. And so I was always like really seeing what are the actual best ways to do things right now.

And around 2012, I started noticing deep learning, starting to win things or at least do pretty well. And I had last used neural nets like 20 years earlier. They kind of put them aside as being like, probably gonna change the world one day, but not yet. And then 2012, it's like, oh, I think the day is coming.

And that really became very clear during 2013. So one of my real concerns was, which I shared with my wife, Rachel, was that the people using these neural nets were like, they were like all the same person. They were from one of five universities that were all very exclusive.

They were all white, they were all male, and they were all solving stupid problems, like trying to find their cats in their photos or whatever. I mean, look, okay, it's nice to find your cats in your photos and people make a lot of money from that. But where were the people trying to deal with global water shortages or access to education or dealing with huge economic inequity?

It wasn't on the radar. And we knew that that was because you only get a kind of a diversity of problems solved if you have a diversity of people solving them. So we actually started getting pretty concerned about that. But at the same time, I also felt like maybe there's some low-hanging fruit.

There's something I could do right now that would make a really big difference. So to give you a sense of this, I wonder if I've got any slides about this thing, let me have a little look. So to give you a sense of how I feel about deep learning now, and I felt the same way about it then, is it's a fundamental kind of like, it's a fundamental technology that I think is as important as electricity.

And it's literally like electricity and steam engine kind of said, okay, you don't really need to generally put human or animal energy inputs in anymore once it was eventually really sorted. And kind of deep learning is on the way to doing the same thing for intellectual inputs. It's kind of this fast, extraordinary thing.

And there are people who, there are people who kind of have this sense of like, oh, neural nets are some hypey, fatty thing. It's, I don't know, it's just another in a long line of AI and ML technologies. I just don't agree with that at all. Like if you just look at what it can do, right?

So here's an example of DALI, which is an open AI algorithm. You type in an illustration of a baby daikon radish in a tutu walking a dog. And these are not cherry picked. These are the first things that it does. It's not finding these, it's drawing them from scratch 'cause nobody's asked for that before, right?

You type in an armchair in the shape of an avocado, it draws these for you. Like this is not something an SVM does. This is not something a random forest does. This is not something a logistic regression does. This is, you know, to somebody who doesn't know what's going on, it just feels magical, you know?

DeepMind created this thing called alpha fold, which blew away decades of research in protein folding from a bunch of people who had basically never worked on protein folding before. I mean, the closest, you know, really close example of this from kind of what I've seen is early in the days of my medical startup in Lytic, we were bringing in everybody we could to tell us from the pathology world, from the radiology world and so forth to tell us about their research.

And so we had this guy come in and tell us about his PhD in histopathology segmentation. And he spent 45 minutes telling us about his, you know, new approach involving a graph cut algorithm and waterfall and blah, blah, blah. And he was getting like new state of the art results on this particular kind of histopathology segmentation.

And we were like, oh, that sounds pretty cool. He was like, yeah, I used to think that too yesterday. But I saw you guys are doing some stuff with deep learning and I kind of got curious. So I thought I'd try this with deep learning yesterday and I ran a model overnight and it beat my last five years of work.

So now I'm not so sure. And like, this is like a really common story. Like every time I try just about anything with deep learning, I'm like beating everything I've done before, beating other people, what other people have done before. And the interesting thing about this is if you haven't done any deep learning yourself, you might not realize that there really is kind of just one algorithm.

Like there's very, very little changes that go between kind of one model and another. So for example, I looked at the source code for the AlphaGo Zero model, which was the thing which absolutely smashed all previous Go playing approaches. And the model was almost identical to the computer vision object recognition models that I used.

It's a base of basically a bunch of residual layers with convolutions and relu's and batch norms and stacked up. And it's just an extraordinarily powerful general approach. And so it's really cool kind of as a researcher because you can read papers from proteomics or chemo informatics or natural language or game playing or whatever.

And like 90% of it you get because it's just the same stuff rejigged in a slightly different way. So that was kind of how I felt and how I feel about deep learning. And actually, I realized that there really was some low hanging fruit at that time in deep learning and specifically as medicine.

No one literally was doing deep learning in medicine. And it turns out that there's such a shortage globally of medical specialists, of doctors that according to the World Economic Forum, it's gonna take 300 years to fill in the gap to basically allow the developing world to have access to the same medical expertise as the developed world.

And I thought this is totally unacceptable. I wonder if we could help make doctors more productive by adding some deep learning stuff to what they're doing. Let's try and do some kind of proof of concept. And so we spent four weeks, me and three other people spent four weeks just training a model on some lung CT scans.

And again, like literally none of us knew anything about radiology or whatever. And we discovered much throughout kind of shock that this thing we trained had much lower false negatives and much lower false positives at recognizing malignant lung tumors than a panel of four top Stanford radiologists. So that turned into my next startup, which was called Enlidic.

And yeah, again, for Enlidic, I went the VC route, raised over $10 million. So this time, this was actually started from the start in the US. And it was kind of a lot easier 'cause I knew people. And yeah, I mean, this was both great and disappointing. It was great in the sense that I really hoped that this startup would help put medical deep learning on the map and it absolutely did.

It got a huge amount of publicity. And within a couple of years, particularly in radiology, deep learning was everywhere. On the other hand, it always felt like I'm just doing this one little thing, when there's so many great people around the world solving important problems and disaster resilience or access to food or whatever, and they don't have a way to tap into this incredibly powerful tool.

And so between this and this kind of concern about inequality and the kind of exclusivity and the kind of homogenous group of people working on deep learning, Rachel and I actually decided to start something new, which was fast.ai. And so fast.ai is all about helping everybody do what Enlidik is doing, but not having a bunch of deep learning people do it, but to have disaster resilience built by disaster resilience people and have ecology stuff built by ecology people.

Because it's much easier, this is our hypothesis, it'd be much easier for a domain expert in ecology to become an effective deep learning practitioner than from a deep learning practitioner to actually fully immerse themselves in the world of ecology to the point that they would know what problems to solve and where to get the data from and what the constraints are and how to operationalize things and understand the legal frameworks and make the connections in the networks, blah, blah, blah.

So at the time we started fast.ai, this was quite at the extreme end of kind of ludicrous ideas because there was just this total knowledge that everybody said to do deep learning, you need a PhD, you probably need a postdoc. It's something that only a few people in the world could ever be smart enough to do.

You'd need very, very deep math. And you need, increasingly you're gonna need more computers than anybody can afford. And it was really lots and lots of gatekeeping. And thankfully it turned out our hypothesis was actually correct. And in the intervening years, we've trained through our courses, hundreds of thousands of people.

And every few days we get lovely, lovely emails from people telling us how they've just published a paper in a top journal, or they've got a new job, or they've bought deep learning to their startup. And increasingly they're using also the software that we're building, the fast.ai library to do this more quickly and better.

And so that's been really great. And one of the important things here, which I guess is something I did learn from consulting is that the world's smartest people are not all at universities. What universities do have the people who stay in the same place their whole life. If you're an academic at a university, you've literally spent your whole life in educational institutions.

And so these are not generally, not always, but they're not generally the most bold and grounded group of people, as you may have noticed. And in fact, in industry, there's a lot of brilliant people doing brilliant research. And so this has been one of the interesting things in fast.ai is a lot of the really powerful examples we hear about are actually coming from industry.

Unfortunately, the problem with America is, well, you know. So we realized we couldn't stay there and we certainly couldn't bring up our child there, particularly after 2020 because, you know. So we tried really hard to get back and eventually the government here let us in. And coming back to Australia was just amazing because having lived here my whole life, I kind of had this vague sense that Australia had a really nice culture and kind of this, like something about going to America that was a bit off.

But then coming back here, it just really hit me that like Australia is such a bloody good country. Like, and the people, like there's this kind of like, you know, sense of this kind of fair go and this kind of sense of helping people out and this kind of informality.

And it's just after spending 10 years in America, it was just this huge breath of fresh air to be back here and that fresh air, you know how when you're really hot and there's a cool breeze and you've really, that feels great, it was like that. You know, it was like, it felt like I'd been stifling humidity for 10 years and I kind of came back to sanity.

So that was amazing, but at the same time, I was also shocked by how little have changed here. Yes, a whole lot of accelerators and incubators and angel networks had sprung up, none of which existed when I was here. But when it actually came to the rubber hitting the road, I was trying to find people like doing like really world-class deep learning research or building startups which had, you know, huge global impact or venture capitalist investing in the biggest, boldest ideas.

And I can't really find it, you know. And actually, Michael Evans was kind enough to let me share some stuff that he has been working on, kind of looking at this from a data point of view. And you can kind of see it in the data, right? From an investing point of view, seed and angel investment in Australia is like, per capita, is like an order of magnitude behind the US.

And this is like, this is where things get going, right? If you've got 10 times less money per person going into like getting things going, that's gonna be really hard for entrepreneurs, right? Investment activity, Australia is not even on the chart. So our investment activity in AI is averaging around $20 million a year.

And here's something that Michael told me that shocked me. Last year it decreased by 80%. Now you might think, oh, fair enough, COVID, guess what? The rest of the world, it grew by 20%. So on the rest of the world, investors went like, oh, this is creating new opportunities.

In Australia, which is like not even hit that much by COVID, investors, but they went home. So this is kind of lack of risk-taking, that's a real concern. There's a lack of investment in research. So, you know, this is the OECD average, not only are we worse, but we're getting worse, right?

And again, this is the fundamental stuff, seed investment, angels, research. So in general, tech, our share of the global value added, it's the amount of stuff, value that we're adding to the economy. This is the Australian tech share of that. It's plummeting and it's near the very bottom of the OECD.

We're behind Chile, Turkey. So, and these are like data points that reflect something that I was already seeing. So like I kind of caught up Michael and I was like, this is something I'm seeing. Am I mad? And it's like, no, you're not mad. I've got the data to show you what you're seeing.

This is actually the one that kind of resonated the most with me. In terms of talking with enterprises, this is a Deloitte study, talking with big enterprises. They asked, okay, why are you interested in AI? Half of Aussie enterprises said, oh, we want to catch up or keep up.

22% said, 'cause we want to get ahead. And this is a worse, this is worse than every other country that they spoke to. Aussie customers are so conservative. I really noticed this, like if you want to sell to enterprises in Australia, you have to tell them that their competitors already bought it.

If you want to say you could use this to power ahead of your field and become a global success story, they don't care. I don't exactly know why this is, but it's true in the data and it's kind of absolutely true from all of my experience. Having said that, in the OECD, Australia ranks right at the top in terms of like our use of tech, right?

And this is what I was saying earlier, like Aussies are awesome. You know, we're smart, we're technical, you know? And yet we're nearly at the bottom in terms of our investment in tech. So it's kind of this weird thing. And this is actually why I think Australia is a great place to build a startup.

The reason I think this is because if you can get past all this stuff pulling you down, all this like, why bother? You'll just get beaten. Can you take less money than you want? Blah, blah, blah. You're in a place where you're surrounded by brilliant people. They don't have other cool tech startups to go to on the whole.

I mean, it's not that there's none, right? But there's relatively very few, you know? And so when, one of the things that was fascinating in San Francisco was that people would say like, oh, we've got such an edge because our R&D hub is in Melbourne. And so we're paying, you know, I think it was like on average one quarter to one fifth of the salaries of being paying in San Francisco and they could actually get people like straight out of university.

And in Lytic, to get people straight out of undergrad, I had to pay them at least 200 grand US, right? Which by the way, if you're a student not working on deep learning, right? This is the technology where like people who understand it and can wield it well can get paid 200 grand straight out of undergrad, you know?

So it's not a bad thing to have in your toolbox even from a job market point of view. So it's actually, sadly, it's kind of like this hidden gem. It's like this diamond in the rough. And so I've often noticed when kind of VCs come and visit or top researchers come and visit, they're often really surprised at how many brilliant people are here.

Because let me tell you, in San Francisco, even though I'm Australian, I'm looking out for it, you don't hear about that, you know? It's like, you know, even looking at like academic papers, I'd always be like looking out for really influential academic papers that helped me with my work in deep learning.

Do they have any Aussie authors? And invariably, if the answer was yes, it was because they've moved to the Bay Area, you know? And I think that's, yeah, I think that's such a waste. You know, we have all these brilliant people. We have this kind of fantastic system. We've got, you know, technically competent people, you know, in the workplace.

I think there are big opportunities here, but I'd say for building a tech startup, and obviously for me, I particularly think building an AI startup, you know, where deep learning is some key component, you know, why wouldn't you be like being at the start of the steam age and trying to create a new kind of loom that doesn't use steam, you know?

It doesn't make any sense to me. Anyway, so you create startups here. It's like, do it in as un-Australian a way as possible, right? It's like, you don't have to have Australian investors. You don't have to have Australian customers. Like just believe that you can put something up on the internet that people are gonna buy, you know?

And, you know, don't worry about whether it's mining or whether it's agricultural, whether it's something your PhD advisor, who's never built trained a deep learning model, thinks is interesting or whatever, you know? To me, that's kind of the secret to how, you know, we can have some great startups here.

And I will say, as that happens, things will change, right? And things are already starting to change. So like something really interesting is what's happening in Adelaide, right? So Adelaide has this fantastic AI and machine learning center and they're doing something which is almost unheard of in universities, which is that they're forging really great partnerships with the tech community to the point where Amazon is now there too, right?

And so Amazon has gone and said, okay, we're gonna partner with Adelaide University of Adelaide. And so there's now kind of the two centers next door, very closely related. And of course, what's now happening, I can't tell you the details, but it happened to know lots more big tech companies are now planning to head to Adelaide as well.

And so you can imagine what's gonna happen, right? Now, lots of people are gonna like go to those and then they'll leave and they'll create startups and then other startups who wanna go there and then other big companies who wanna go there. And so, and then of course, what's gonna happen in all the other capitals, they'll be like, oh my God, it looks like happening in Adelaide, we have to do that as well.

And this is very, very different to how things are currently done. 'Cause universities like here are in many ways incredibly anti-entrepreneur, anti-tech entrepreneur. So for example, you know, a lot of brilliant work gets done out of UQ and QUT. They're sponsoring this AI hub, that's fantastic. But if an academic there wants to start a startup, they have to give QUT 70% to start.

And let me tell you, that's literally impossible. So there's zero successes 'cause that's like no one will invest in that company and the founder can't even be invested in that company. Like, and it's not just Queensland, this is basically every university in Australia. Adelaide made a huge step of going from 70% to 49%.

Compare this to like Stanford or Berkeley, where like every academic I know there in engineering or computer science has four or five startups, that they have a 5% equity stake in. You know, half of their students go to those startups. Then those students find interesting research directions from the work that they're doing, which they then go back and then they fund a new group of people at the university.

I mean, if you look at the relationship, for example, between Stanford and Google, it's like constant back and forth research, huge amounts of funding from Google to Stanford, lots of job opportunities for standard people at Google. The idea that the way you leverage your academic talent is by forcing them to give you 70% of their company is absolute insanity and it's totally not working.

And I personally know of many academics in Australia who have decided not to start startups because of this reason. And also because most universities will tell you, you're not allowed to keep working here if you're working at a startup, which of course it should be the opposite. It should be like, oh, wow, you're getting industry experience, you're learning about actual applied problems, we'll pay you a bonus, you know?

So there's a lot of kind of issues with how the kind of tech sector is working here and how entrepreneurialism is working here, but the most important thing is the kind of the raw foundation that we have, which I think is one of the best in the world. And so that's one of the reasons that we came here is because we wanna help any way we can change Australia from a diamond in the rough to a glowing diamond that everybody around the world knows.

So that's what we wanna do, thank you. (audience clapping) - That's awesome to get an insight into your experiences over the last, well, since you started your first startup. From the beginning when you first started to when you went to US and now when you had your first couple of months back in Australia, what's harder, getting an idea, getting money or getting good data to make it all happen?

- I think if getting good data is the thing you find hard, then you're doing the wrong thing, right? So the thing you're doing should be something which you're deeply in that field, right? So like if you're somebody in the legal industry, you should be doing a legal startup.

If you're in the HR industry, do an HR startup. If you're in the medical field, do a medical startup because then getting data is easy because you're surrounded by it. Or your friends working companies with it, you personally worked in companies with it. So I'd say like start working on a problem that you're deep into.

And then coming up with an idea, that shouldn't really be hard because everything's broken. If you noticed, nothing quite works properly. Everything's finicky and frustrating and has stupid bits. So just particularly stuff at your workplace, do you know all the stuff that takes longer than it should or problems that have never been solved properly?

So really the key thing is execution and tenacity. Like one thing I really noticed with fast mail was when we started fast mail, it was actually pretty hard to start an email company 'cause there was very little open source software around and very few examples of how to build this kind of thing.

But very quickly there was kind of like all kinds of open source software appeared. It became pretty easy and we got new competitors monthly and they'd stick around for like six months and then they'd disappear because it'd give up 'cause it was hard. And I will say like in most startups I've been involved in every month, it feels like there's a problem so dire that we're definitely gonna die.

But you kind of have to keep going anyway. So I think it's your execution and tenacity. - Thank you Jeremy. The Dolly model is very impressive. When I was young it was obvious what a computer model didn't understand, it couldn't recognize a car for example. When you look at that model, it's not clear to me what it does and doesn't understand anymore.

I wondered if you had a comment about that. - Only to say I actually don't care about understanding or not. Like I'm kind of philosophically interested and I am a philosophy major, but as a big learning practitioner, all I care about is like what it can do. So yeah, I mean, it's a fascinating question.

I don't think there's any way to ever answer that. I actually don't know what you understand. You could tell me, but I don't know if you're telling the truth. You know, it's just a fundamentally impossible question to answer I think. And but it's not one we need to answer.

We just need to know what can it do, what kind of do. Any new courses planned for 2021? Under some vague definition of planned, yes. We need to do a part two of our deep learning for coders course. So that's planned in the sense of like, yeah, I should write that sometime.

And the other course, which I'm really excited about is I'm planning to do a course which is a kind of full stack startup creation course involving everything from like creating a Linux server and a system administration of Linux through to how the domain name system works through to investment, through to getting product market fit, through to collecting data and so forth.

There is a course a bit like that, that Balaji Srinivasan did on Coursera could store up startup engineering, but it's not quite available anymore 'cause of Coursera and it's also getting a bit dated and it doesn't really have such an AI thing. So that's, I don't know if that'll be 2021.

It might be 2022, but there's a couple of courses I'm looking at. Okay, so that's that one already. Are you getting some check days? Since I had a five year old, I'm suddenly less interested in motorcycling, I'm sad to say. So yes, those courses I described will probably be in person at whatever university feels like having us.

So that's what, so yeah, what's next? I'm gonna keep doing what I'm doing. But what I wanna do is, I wanna do fast AI with awesome Australians. It's from a purely selfish point of view. I'd like this to be like a real global hub of brilliance because I want people around me to be awesome.

So I would love it if people were flying here in order to be part of this amazing community. And I actually think that's totally, totally doable. Particularly 'cause you're so beautiful. Like I think we've got a lot of benefits, particularly in Queensland. Like who wouldn't wanna come to Queensland?

Yeah, thank you for getting graph data. Sure, that was a great question. What's your recommended way of marketing? Okay, so how to market an early stage company? The first thing is make it very, very easy to use your product and to buy it, right? So I don't wanna see, like, okay, so there's gotta be a pricing section, right?

I don't wanna see a section that says like, email asks for sales inquiries. That's insane. Like, no, I'm not gonna, who does that, right? If it says it's $5 a month, it's like, fine, here's the credit card, right? I need to be able to use the damn thing. So like have an open source version or at least a limited demo or something.

Have screenshots. Like I wanna be able to go to your site and immediately know what are you selling? Is it any good? What does it look like? Can I give it a go? And then pay you for it. So that's kind of like the first is to avoid anti-marketing, where you make life difficult for your customers.

And then the best kind of marketing is the media, right? So like you will get far, far, far more awareness of what you're doing if you can get something written about it in Wired or the Washington Post or BBC than any amount of advertising. And that is all about personal outreach from you, the CEO, to journalists who you have carefully researched and confirmed would definitely be interested in what you're doing and then telling them about it.

And that actually doesn't happen very often. Most people go through like PR firms who journalists can't stand dealing with. And so like I've basically never paid for any advertising of any sort. But if you do a Google news search, you'll see that we've got a shitload of media, right?

And last year in particular, I wanted to like go take that to another level because I co-founded Masks for All globally. And so I literally wanted every single person in the world to know they should wear a mask. And so this is like my media campaign. So I just wrote to everybody, I talked to everybody and ended up on everything from Laura Ingraham on Fox News, through to BBC News and wrote in the Washington Post and USA Today.

And nowadays, thank God people actually wear masks. So yeah, media is your magic marketing tool. Last one? Okay, last one. - Thanks so much Jeremy and Rachel and your team for the Fast AI course. It's amazing. - Thanks. - And accessible. In the era of global warming, how concerned should we be with the energy usage of deep learning models?

And yeah, your thoughts or ideas on how we can master this challenge. - So it's a great question. I would, the way I think of it, and I'm not an expert on this, but the way I think of it is from a general resource constraint point of view, we should not be using no more resources than necessary to solve the problem, including energy.

Unfortunately, a lot of companies like Google, to pick one out at random, have huge research departments that are very explicitly in center to create research that shows the results of using huge amounts of energy. Specifically huge amounts of Google Compute Hours. And this is very, very effective marketing because if you can, like journalists love writing about big engineering solutions.

And they'll always say like, this used 10,000 TPU hours or whatever. Now, so the thing is, this is what we focus on, the vast majority of problems that we see solved in practice, useful pragmatic solutions are solved on a single GPU in a few hours and you can buy a GPU for a few hundred bucks.

And there's all kinds of resources like this, as the resource of just like the amount of education that you need or the resource of the amount of data that you need or whatever, but like overall, people dramatically overestimate the amount of resources you need to get good results out of deep learning.

This is very explicitly because that's what a lot of people want you to believe. They want you to believe that you have to hire their consulting firm, that you have to use their compute hours, that you have to use their special software, that you have to buy lots of their cards or whatever.

But yeah, overall, there's a massive over emphasis on, you know, using vast amounts of stuff in deep learning. Sure, I have to mention Dawnbench. So in fact, I have a slide about Dawnbench, if I remember correctly, 'cause I kind of skipped over it. Yeah, so this is something that Rachel and I are passionate about and we went crazy when TPUs came out because Google was like, oh, these are these magic special things.

And the media was like, okay, everybody else is screwed now 'cause they don't have TPUs. So only Google can now do deep learning. And so there was a competition at that time that had just come out just shortly after TPUs got marketed to hell called Dawnbench, which was basically who can train ImageNet the fastest.

And at this time, the fastest people were solving it in about 12 hours. When I say solve it, that means getting it to an accuracy, like I'm at a top five accuracy of something percent. And yeah, not surprisingly, Google put in their pitch and I think they got like three hours or something.

And Intel put on a huge TPU pod or whatever. Intel competed and they of course put in an entry with 1024 Intel servers operating in parallel. And we thought, okay, if these guys win, we're so screwed because it's gonna be like, okay, to be good at this, you really do need to be Google or Intel.

So some of our students and me spent basically a week saying if we could do better and we won. And we did it in 18 minutes. And it was just by using like common sense, and just like, yeah, just keeping things simple. And so like, and we kind of like, we've done similar things a few times because these big tech BMS are always trying to convince you that you're not smart enough, that your software is not good enough, that your computers are not big enough, but it's always been bullshit so far and it always will be.

- Thank you, Jeremy. I think we'll call it there. If anyone else has any further questions, feel free to try and have a chat to Jeremy, depending on when he chooses to leave. I think from everyone here at the Meetup, we just wanna say thank you for sharing your time.

Rachel as well, we'll hopefully have you down here in the next few months. And really looking forward to having you involved in the local community. For everyone who is keen to be involved