back to index

Using OSS models to build AI apps with millions of users — Hassan El Mghari


Whisper Transcript | Transcript Only Page

00:00:00.000 | .
00:00:15.680 | Hello, everybody.
00:00:16.560 | Thank you for coming.
00:00:18.600 | My name is Hassan.
00:00:19.560 | I'm super excited to be here today to talk to you
00:00:22.560 | about how to use open source models to build AI apps
00:00:25.680 | with millions of users.
00:00:27.400 | I've been building AI apps for several years now,
00:00:29.960 | and I just want to come tell you a little bit
00:00:31.840 | about my journey, show off some of the apps,
00:00:34.040 | and then tell you about kind of my whole process
00:00:37.080 | of building these apps from idea to having the finished app,
00:00:40.520 | and how I market them as well, and then some advice overall.
00:00:45.600 | So I'll start off with some intros about who I am,
00:00:48.160 | Together AI, the company I work for,
00:00:49.880 | why it's one of the best times to build right now,
00:00:51.900 | and then we'll get into demos.
00:00:54.440 | I'll talk about my tech stack and architecture,
00:00:56.320 | and then I'll end with my process for building these apps,
00:00:59.920 | the advice I have for other people building these apps.
00:01:03.960 | So I lead developer relations for an AI and for a company called Together AI,
00:01:09.600 | but probably the most relevant part for this talk is the fact that I build a lot of these AI apps,
00:01:14.320 | and I've probably averaged one a month for the last four years,
00:01:18.520 | and I've been fortunate enough that some of them have done pretty well
00:01:21.920 | and have gotten a good amount of users and GitHub stars as well.
00:01:25.960 | One thing I do also is open source, so I'm going to show off some demos, but also you can
00:01:30.000 | check those out in your own time and you can dig into the code as well.
00:01:35.640 | Like I said, I lead developer relations for an AI cloud platform called Together AI.
00:01:39.680 | We're a platform for open source models. We give you an inference API to be able to query
00:01:45.320 | really any open source models probably that are out there that are good.
00:01:49.000 | So we have chat models like QEN3. We have reasoning models like DeepSeq R1.
00:01:53.560 | We have image models like the flux context model that came out recently.
00:01:57.000 | We have vision models, audio models, you know, almost all modalities.
00:02:01.080 | We also let you run these models on dedicated instances or fine tune them on your own data.
00:02:07.320 | And we have a GPU cluster product as well if you want a training cluster.
00:02:13.080 | So before I get into the demos, I just want to talk about the fact that it's really a historic
00:02:18.040 | time right now for building. And I think this is because of two things.
00:02:21.720 | Like one, I think the barrier for building has lowered dramatically with a lot of these AI tools,
00:02:27.720 | right? You can use Cursor and Windsurf with your IDE to auto-complete stuff.
00:02:31.080 | You could use these AI builders like Bolt and V0 and Lovable to prototype
00:02:36.840 | apps really quickly. You can use chat apps to kind of brainstorm ideas or go through the process of
00:02:43.080 | building an app. You know, you can go to a chat app and say, "I'm building a Swift app for the first
00:02:46.920 | time. Can you guide me through how to do it?" Right? So I think like the the barrier has lowered
00:02:51.480 | on one side and on the other side we're having these like groundbreaking models come out every single
00:02:56.360 | week that are enabling completely, enabling builders to build completely new things. So it's like
00:03:02.520 | these incredible AI models are coming out that you can build with and it's gotten a little bit easier.
00:03:06.360 | So I just think it's an incredible time to be building apps right now.
00:03:10.600 | I'm going to dig into a couple of demos. These are some of the apps I've built that I'll spend like
00:03:17.560 | 30 seconds on each one demoing really quickly. But one thing to note really quick is the fact that some
00:03:23.160 | of these apps have a couple thousand people that have used them and then some of them have like a
00:03:27.000 | million plus. And I think like at a certain point it is a numbers game and you kind of have to try out a
00:03:32.040 | lot of different stuff. And I'll talk about kind of how I approach that. So we'll switch into some
00:03:38.040 | demos. So in the beginning when I was when I started a building kind of a few years ago,
00:03:41.320 | I wouldn't know what to build. So I'd go on Twitter and I'd look up, you know, people that tweeted like,
00:03:46.600 | "Oh, I need an app to do this." And so here Samina said, "Can someone build an AI app to help me pick some
00:03:51.000 | glasses?" And I was like, "Sure, I got you." And, you know, built this little tool that takes a bunch of
00:03:59.240 | different requirements, turns them into a query and then actually uses the Amazon API to find products
00:04:06.200 | that you can actually, you know, click on and buy. And another one of these was one of my friends Theo
00:04:11.640 | tweeted out like, "You know, I just want an AI app that writes my commit messages for me." And my CTO at the
00:04:16.760 | time was like, "I love that idea." And so I just decided to build it. And all it does is when you
00:04:22.200 | do git add, it takes the git diff, it sends it to an AI model, and it writes a commit message for you,
00:04:27.400 | and it kind of shows it to you and you can kind of accept it. So this is one of my first apps that
00:04:31.560 | I built that ended up doing really well and got, I think I got about 40,000 people that have installed
00:04:38.040 | it and downloaded it. And it's also open source. And I had a bunch of people contribute, which has been
00:04:43.720 | great. Another app I built was a like text-to-app builder. There's a bunch of these where you can
00:04:51.080 | type in an app that you want to build. Here I'm going to say like a quiz app about American history.
00:04:55.000 | And the way this works is it takes this prompt, it sends it to an AI model to come up with a project
00:04:59.160 | plan, and then it sends it to another model to actually write React code that we can show. So in a
00:05:05.560 | second this should populate, but obviously also open source. I've had about 5 million requests go
00:05:12.520 | through this app and about a million apps built just through this. And I have a little over a million
00:05:18.200 | people that have ended up using it. So this is an app that kind of just generates images as you go
00:05:24.120 | through it. And also has about over a million people and has about 48 million images that were actually
00:05:30.840 | generated with this app. I have an app called napkins where it takes a screenshot of, the idea is it takes
00:05:39.160 | a screenshot of a napkin. The idea is like you draw a little web app on a napkin, it takes a screenshot of
00:05:42.840 | it and it can actually build it for you. So yeah, that's, that's another one of my apps. That one has
00:05:50.840 | about 40,000 users. I have another one that lets people kind of upload a resume and then builds them
00:05:56.440 | like a personal site like this for people to check out. And you know, there's, there's a bunch of others,
00:06:01.720 | ones that visualize menus. I built like an AI chat app, like a tutor app where you can, you know, put in a
00:06:09.960 | topic that you're interested in, like personal finance, and it can explain it in the level of
00:06:13.800 | like an elementary school, a person, and then some OCR stuff as well. So I'll get back to the slides.
00:06:22.040 | A lot of these apps have a very similar architecture that I end up using. And so this similar architecture
00:06:31.560 | starts with a user coming in and typing something or uploading an image usually. So it's some sort of
00:06:37.080 | user input, right? And I take that and I generally will send it to an AI model on Together AI. So it
00:06:42.280 | could be like an image to image app where I take an image and I send it to an AI model to make another
00:06:47.560 | image and then show that to the user. So step two is I send it to an AI model. Step three, a lot of the
00:06:52.440 | time is I store that image or that text in the database to be able to show it to the user. And then
00:06:58.040 | step four is I just show it to the user. So it's a very, very simple architecture. A lot of the time,
00:07:03.560 | there's one single API call happening. The user does something, I send one API call to an AI model,
00:07:08.840 | I get the response back and I show it to the user. And I think it's this simplicity that is really,
00:07:14.920 | really important, both for moving very quickly, but also for validating these ideas. The more simple
00:07:20.760 | you can drill down an idea, the faster you can kind of build. And then this is my tech stack that I
00:07:26.360 | usually use. I use Together AI for all of my AI models. I use Next.js and TypeScript as my full
00:07:31.560 | stack framework for building things. I use Neon as my database. It's like a really good serverless
00:07:38.040 | Postgres host. I use Clark for authentication. I use Prisma as a way to talk to my database and TypeScript.
00:07:43.880 | I use Shatzian and Tailwind for styling. S3 for uploading images. Plausible for website analytics. And I
00:07:51.880 | kind of showed you guys this in a couple different places. So it's cool that I can see the number of
00:07:56.840 | unique visitors. I can also see where they're coming from and what countries and what device they're
00:08:01.000 | using it on. And one thing I always get very surprised by is the fact that a lot of people use most of my
00:08:07.960 | apps on mobile. So it just goes to show that the mobile experience is really, really important.
00:08:12.840 | So plausible for website analytics. Helicone for LLM analytics so I can dig into my LLM requests and
00:08:17.880 | like troubleshoot things. And then Vercel for hosting these apps. Cool. So I'm going to talk
00:08:26.200 | about my process for building these apps. And a question I get a lot is like, how do you come up with
00:08:31.320 | these ideas? Uh, and, and I think like the, the biggest thing I did for ideas that, that helped
00:08:38.520 | me out was just to keep a list of running ideas. Um, I think we all get great ideas at random times and
00:08:45.000 | most of us don't write them down. And I think like, uh, that makes a lot of the difference. And so I think
00:08:49.560 | keeping this list of ideas, uh, trying to write down anytime you see an interesting idea or see an
00:08:53.640 | interesting product and you say, oh wow, that's actually really cool. Maybe I can use a similar methodology
00:08:59.320 | to and apply it to something else. Um, so ideation is like, uh, a big thing. And, uh, the list of ideas
00:09:05.720 | I have, I usually always have this short list of like the top five. And so I like, I know right now
00:09:10.200 | kind of the top five apps I want to build next. Um, and then if anything kind of drops in between,
00:09:15.560 | then I'll, I'll build it. You know, if a new image model comes out next week, that's like really,
00:09:19.560 | really incredible and open source. Um, I'm probably going to be scrambling to build an app with, with that.
00:09:23.720 | Uh, naming is another big one. You know, you want a short memorable name. I, you know,
00:09:28.200 | you can use AI tools like domains GPT to kind of, uh, check names that also have the domain name
00:09:33.080 | available, uh, to, to use them. Uh, number three is design, you know, thinking through how the app
00:09:38.120 | will work. Like, you know, I'm going to have a landing page and a user is going to click like
00:09:41.240 | enter and then enter and get this page where they upload something and then they see an image, you
00:09:44.840 | know? And so it's like these two screens and, uh, you can either kind of sketch it out on a piece
00:09:49.160 | of paper or you can use Figma, uh, or you can use a lot of these prototyping tools, uh, to, to try to help you
00:09:53.640 | think through, uh, what, what the app is. Uh, and then I go about building the actual app.
00:09:58.440 | So trying to make the simplest possible working version of a specific app. Um, and like I said,
00:10:05.400 | I, I always try to shoot for like one API endpoint, you know, like one, like just very, very simple.
00:10:10.920 | Um, step five is like, you know, I have a working prototype now. I, I start to think through, uh,
00:10:16.120 | authentication and limits of like, okay, like how expensive, uh, is this? Like how many, um,
00:10:20.920 | how many uses uses do I want to give each person per day? Do I want to want to add authentication?
00:10:26.600 | Do I want to add, bring your own API key? Uh, so it really just depends on, on the app and the AI
00:10:30.840 | model that I'm using. Uh, and then step six is usually kind of prepping for the launch, getting a
00:10:34.840 | nice OG image, getting a domain, adding analytics, uh, writing a nice read me. Cause like I said,
00:10:39.720 | everything I kind of do is open source. So I also want to make sure the code is really easy to use and really
00:10:44.200 | easy to, uh, to clone. Um, and then the last step is actually launching. And so I usually use LinkedIn
00:10:51.160 | or X to, to launch and, uh, you know, kind of, uh, just see what people say at that point.
00:10:57.160 | Cool. Final section. Uh, and then I think we'll get into some Q and A. We have a little bit more time
00:11:03.400 | because the demos didn't work. Um, so advice for building apps. I have these like seven tips that
00:11:08.360 | I'm going to go through. Uh, one is thinking of an idea that excites you, but is very, very simple,
00:11:13.800 | uh, that you should be able to describe to anyone in five words, right? Like blink shot,
00:11:17.880 | generate real time images, llama coder, go from some text to an app, right? Like, and I think this
00:11:23.080 | is one of the biggest mistakes people use as they try to, they try to think like, okay, you know what?
00:11:26.760 | I like, I want to build this like personal CRM software that has this dashboard that will email
00:11:32.440 | me every week and do this and that. And you know, they come up with this like grandiose version and
00:11:36.520 | spend like six months building it. And then they realize, oh wow, it's like, it's, it's, um, this isn't,
00:11:40.920 | this wasn't the right thing to build or, um, this is just like really hard and nobody cares about it
00:11:45.080 | now. And so it's all about, uh, thinking of an idea that excites you, but it's also a really,
00:11:49.240 | really simple number two. And this is very underrated is making sure the UI looks good.
00:11:54.760 | A lot of the apps I showed you are AI apps, but I actually spend 80% of the time on the UI for most
00:12:01.320 | of my apps. Um, and that should tell you like how, how important this thing is. I,
00:12:05.720 | I've like the first few apps I've built kind of looks really, really bad. And that was a big part
00:12:09.720 | of why nobody kind of used them. And I've started learning that even if you take a really simple
00:12:14.200 | idea or something that's like so simple, like summarizing a PDF, right? You can go to ChatGPT
00:12:18.920 | and summarize a PDF right now, but I built a PDF summary app and I spent a lot of time making it look
00:12:23.480 | really, really, really good. And then I had tens of thousands of people that ended up using it.
00:12:27.080 | Um, so it's all about making it look really good, making it really easy to use, and making it really,
00:12:32.440 | really straightforward. Um, the third tip is just keeping the app simple. I talked about this a bunch,
00:12:37.960 | but most of my apps have only one or two API calls. Um, tip number four is trying to incorporate the
00:12:44.120 | latest AI models. A lot of these apps, um, that, that I showed you have like used some of the latest
00:12:50.520 | AI models like BlinkShot, the real-time image generation app, um, that used, uh, a model called
00:12:56.200 | FluxChannel. And I, and I launched this app, I think two days after this model came out.
00:13:00.200 | Um, and so it was one of the first apps that leveraged kind of this new technology or this new,
00:13:04.200 | really, really fast model that was good. Um, and so doing that kind of increases, uh, the potential
00:13:09.320 | for, uh, virality. Uh, tip five is launching early and then iterating. Um, for a lot of these apps,
00:13:15.960 | like still to this day, you know, like 40 apps later, I still have no idea what will do really,
00:13:20.600 | really well versus what won't. Um, and the only way to de-risk this is to simplify your project,
00:13:25.320 | to launch early. And then if nobody really cares about it or not a lot of people use it,
00:13:30.120 | then at least, you know, you didn't spend six months building, you spent a week building and
00:13:33.400 | you can kind of move on to another idea. Cool. Uh, another tip that I, that I've kind of done is
00:13:39.000 | trying to make it free and open source. Um, so folks can kind of learn from you and are also very
00:13:43.880 | incentivized to share it. Um, and so, uh, the, the, this is something that that's worked well, uh, for me.
00:13:49.160 | And then the last tip I have is, uh, just keep shipping. Um, a lot of AI apps, like I said,
00:13:56.200 | don't do very well. It's a numbers game. You have to kind of keep building and building and building.
00:14:00.120 | And the more you build, the more you realize, you know, kind of what resonates and what doesn't,
00:14:03.480 | the faster you get at building, uh, and the better you get at like picking, picking ideas. So,
00:14:08.520 | um, a lot of it just comes down to, you know, putting in the hours, building a lot of stuff,
00:14:13.240 | and then, uh, seeing what happens.
00:14:16.680 | And that's all I have. We actually have five minutes for Q and A if anybody has any questions.
00:14:21.880 | Um, and yeah, I think you can line up at these, uh, podiums and we can, uh, take a few questions,
00:14:28.200 | but just, uh, before that, uh, you can find me on Twitter at nutlope or, uh, my email hassan@together.ai.
00:14:34.840 | Uh, we also have a Together AI booth here at S25. Um, I'll be there, uh, for a few hours after,
00:14:40.120 | after the talk. So, uh, come and talk to us. Yes.
00:14:42.520 | Yeah, I'll just yell. Uh, oh, it's on. Okay. Uh, thanks for sharing. This is great. I'm just
00:14:50.520 | curious. Uh, you have a lot of traction success more than many, most YC startups. Why don't,
00:14:56.280 | why don't you just start a company? It seems like the perfect formula. I'm just really curious.
00:15:00.360 | Yeah. Um, that's a good question. I mean, a part of it is I, I really enjoy, um, I really enjoy teaching
00:15:06.120 | and a lot of these apps, like I can, a lot of these apps, first of all, do well because they're free and
00:15:10.760 | open source. Right. And so, because I'm, I'm strictly not trying to monetize them. I'm, I'm
00:15:14.840 | strictly like trying to make them free and I get companies to kind of sponsor it and, and, uh,
00:15:19.320 | and, and keep it open source. Um, so I just kind of enjoy doing that and launching these apps. And I,
00:15:24.520 | I will also like make videos and blog posts about different things. Uh, but yeah, maybe eventually
00:15:29.240 | I'll, I'll come out with an idea that I really, really love working on and, uh, and, and do that.
00:15:34.040 | But, um, for now it's just been really fun getting to build a lot of the stuff and experience.
00:15:37.800 | If you ever do start a startup, let me know, uh, how Andrew invest. Thank you.
00:15:41.960 | Yes. So what are some common trends with ideas that maybe you thought were good,
00:15:49.560 | but didn't resonate with people and the reverse? What are the surprises been?
00:15:52.360 | Yeah, that's a really good question. Um, where have the surprises been? It's, it's been a little
00:15:58.280 | bit random, honestly. Um, I think, I think one insight I got is like building apps where people
00:16:07.400 | can very easily share what they create tend to do way, way better. Um, and so now I try to think
00:16:12.600 | about that, like building in that viral loop into my apps on like, if you can generate an image, I want
00:16:17.080 | to make it really easy to share that image or and share it with like a really nice OG image. Um, and so
00:16:22.920 | that's one insight I've had is like, you know, apps that have that viral loop of, of like, Oh, this is
00:16:27.800 | really cool. And I can share this with a friend and they can go on it and try it out and they can go
00:16:31.080 | on it. Um, and so that's something that I've incorporated a little bit more. Yeah. Yes.
00:16:36.520 | I love that you're making all of this, like, you know, it's all free so far. It's not monetized,
00:16:41.000 | but how are you paying for like all the compute and the model calls yourself? Like, are you funding this?
00:16:45.400 | Yeah, great question. So, um, first off, you know, I, I work at together AI, we sponsor all of the
00:16:51.000 | compute in terms of the AI models, uh, but then I'll also partner with other AI companies like neon
00:16:55.480 | gives me a free database to use their database and, and like, you know, all like Clark gives me a free
00:17:00.200 | account for authentication. And, uh, uh, and the reason for that is because it's, it's, it's all open
00:17:04.200 | source. So it's, it's in their best interest to be in that open source project and, and for people to use
00:17:08.440 | it. Um, and so that, that's also, uh, like a good piece of feedback I have for people is like,
00:17:13.480 | if you want to just tinker with apps and build them, a lot of people are like, oh, but I can't
00:17:16.840 | launch an app because AI models are too expensive or this thing's too expensive. Um, more often than
00:17:21.160 | not, if you launch something and you reach out to these companies and you say, hey, like I'm building
00:17:24.840 | this, this open source app, can you please give me some credits? More often than not, they will.
00:17:29.400 | Uh, and so that's generally how I do it. Awesome. Thank you. Yeah. Yes. Yeah. Um, I mean, thank you
00:17:34.120 | for this talk. It was, it was really great. Um, I just had a question because you talk about how frequently
00:17:39.480 | you build these apps, like every, every week you said for, for a decent amount of time. Um,
00:17:44.360 | and you also talked about, you know, using the latest and greatest in AI models with them changing so
00:17:49.480 | quick and with you having so many apps out there, do you ever struggle with like going back through and,
00:17:55.560 | and like changing the models that you've used for some of these apps and how do you deal with that?
00:17:59.480 | Yeah, that's a great question. Um, I, I think like a really cool, um, a really cool thing is
00:18:06.600 | the fact that you can do that. You can just build an app with an AI model and then a better AI model
00:18:10.520 | comes out three months later and you can go. And a lot of the time it's like a one line change of like,
00:18:14.680 | let me update this model and the app just gets way better or it just unlocks new things. Uh, and so
00:18:18.840 | that's something I do frequently where I'll, I'll go back and I'll like, even like relaunch an existing
00:18:23.240 | app with a new AI model or add a, a tiny feature to it. Um, and so, um, yeah, I, I think that's kind
00:18:29.800 | of the superpower of like building with AI is the fact that you can just kind of replace these AI models.
00:18:33.960 | Thank you. Yeah. All right. Awesome. Thank you all so much for coming. I appreciate it.