The Accidental AI Canvas - with Steve Ruiz of tldraw

00:00:00.720 | Welcome back to Latent Space.

00:00:03.040 | I'm very excited to have my good friend, Steve Ruiz,

00:00:06.480 | how are you this morning?

00:00:08.040 | Hey, how's it going?

00:00:09.360 | Evening for me, but yeah.

00:00:11.200 | Evening for you.

00:00:12.200 | Well, thanks for staying up.

00:00:13.760 | I have had the good fortune of knowing you before you got famous

00:00:17.400 | and actually hanging out in the precise office and studio

00:00:22.720 | that you're recording from right now.

00:00:24.080 | Congrats on Make It Real.

00:00:25.280 | Congrats on TL Draw.

00:00:26.560 | I think it's been something that's sort of years in the making,

00:00:29.320 | but it's probably looks like overnight success to a lot of people.

00:00:32.880 | Yeah, thank you.

00:00:36.200 | It's kind of a funny story.

00:00:39.400 | I don't know, where should we jump into it?

00:00:41.960 | Well, I like to people give you a little background on the person.

00:00:45.120 | You don't have a lot of detail on LinkedIn, just like myself.

00:00:49.000 | I just found out just before recording that you're also a late entrance into tech.

00:00:54.440 | So maybe just like, you know, what's your background coming into something

00:00:57.600 | like TL Draw?

00:00:59.320 | What makes you so unique at doing sort of creative, collaborative experiences?

00:01:04.520 | Like, you know, I know you and I've actually used TL Draw.

00:01:09.240 | So I have some appreciation for how hard this thing is.

00:01:12.320 | Like you said, I kind of came into this a little late

00:01:15.200 | and kind of came into it from a weird angle.

00:01:17.280 | My background is actually in like fine art and studio art.

00:01:20.680 | So I have my master's from University of Chicago in visual art

00:01:25.000 | and would write about contemporary art and put together exhibitions

00:01:29.800 | and do my own paintings and drawings.

00:01:32.400 | And that was back when I was living in Chicago.

00:01:35.160 | And then when I moved over to the UK,

00:01:39.560 | kind of, you know, got a new studio, kept that going.

00:01:42.120 | But when I turned 30, I kind of decided I should probably make some

00:01:45.840 | make some money and kind of work with other people closer than I was at the time.

00:01:51.760 | Studio art is primarily a solo thing.

00:01:55.160 | And I always had kind of like an analytical background or kind of side to me.

00:02:00.440 | My day jobs were, you know, I was working for lawyers.

00:02:02.720 | I was doing this writing, like magazines and stuff.

00:02:05.800 | And so when I when I did that, kind of that switch back

00:02:11.720 | eventually to to design and product design, it.

00:02:17.040 | I was also able to use a little bit of like a tiny little bit of technical skill

00:02:22.040 | that I had had just building like WordPress websites for myself

00:02:25.120 | and other artists as portfolios

00:02:27.880 | kind of take that and just some natural curiosity

00:02:30.840 | around the way that the products work and kind of create a career direction

00:02:36.320 | that was more around prototyping and like technical design and doing that

00:02:40.320 | kind of like doing the design on the bits of a product

00:02:44.160 | that really couldn't be designed otherwise.

00:02:46.400 | So the interactive bits, the bits which are

00:02:51.120 | maybe more there's more questions about them.

00:02:56.520 | There's no clear answer to terms of like, how should this work?

00:02:59.680 | You know, in all those places, you kind of have to build something

00:03:02.680 | in order to to figure out what you want to build.

00:03:05.600 | It turns out, you know, to skip right to the end for a moment, like

00:03:10.080 | canvas is just full of those types of problems.

00:03:13.040 | So it's no surprise that I ended up there.

00:03:14.400 | It's like kind of an extreme form of the same problem.

00:03:17.440 | But yeah, so I was working.

00:03:19.280 | This was back in like 2017, 2018.

00:03:21.840 | And I used at the time a product called Framer.

00:03:26.600 | That was back when it was more of like a more of like a code product

00:03:31.560 | than than what it is now, which is more of like a visual

00:03:34.440 | visual builder that is kind of backed by code.

00:03:37.000 | And so I sort of just drilled into that.

00:03:41.400 | It was cool. Uber was using it.

00:03:43.280 | No one knew how it worked. No one could use it.

00:03:45.120 | So I got good at it and got a lot of advancement, early traction,

00:03:48.560 | whatever, in my career based on that.

00:03:50.800 | But it also, you know, taught me to code, taught me to think about,

00:03:54.520 | you know, building things that other people are going to use.

00:03:56.800 | Taught me about

00:04:00.320 | kind of like the type of code that you write when you're in an exploratory phase

00:04:04.520 | rather than like in a like execution, like production phase.

00:04:08.240 | And that. Yeah.

00:04:11.480 | Kerry, I actually ended up working for Framer.

00:04:13.720 | I did their education for a year,

00:04:15.880 | which is very different than the type of product design that I was doing before that.

00:04:18.680 | I did a lot of video tutorials and writing and tweeting,

00:04:25.520 | trying to figure out some way to make technical design content interesting,

00:04:31.240 | you know, in little chunks that people could could consume.

00:04:34.920 | And I joke that like they probably got less out of me

00:04:41.040 | in that job than I got out of the job itself.

00:04:43.400 | Like because, yeah, I walked away from that.

00:04:46.760 | Not sure if I'd helped anyone really learn how to use Framer,

00:04:50.000 | but I certainly learned how to how to tweet

00:04:52.040 | and learn how to record a good gif and learn how to talk into a microphone

00:04:56.760 | and all that type of stuff.

00:04:59.760 | And so in the next roles that I had, which were

00:05:02.720 | I worked for a company called Play out in New York, who is also doing design tools.

00:05:06.680 | And I really wanted to work in design tools after that.

00:05:09.480 | Play's doing like a mobile, I guess, or now it's like just general iOS, Mac OS

00:05:14.680 | platform specific design tools where you're you're using actual elements

00:05:19.240 | from the kind of widgets from that that component or component collection.

00:05:24.400 | I don't know in your designs and kind of bringing that a lot closer to the end product.

00:05:29.440 | And at the same time, I started getting into open source.

00:05:34.080 | I'd kind of done some popular open source before.

00:05:37.640 | But this was now 2019.

00:05:41.000 | It was it was locked down. I had a little bit more time.

00:05:43.680 | I also had a daughter, so not that much more time.

00:05:45.920 | Or I'm sorry, it was 2020.

00:05:49.520 | And yeah, I guess that open source that I started getting into

00:05:54.400 | started swinging back toward some of my kind of artistic interests

00:05:58.000 | or studio interests and kind of visual interests.

00:06:00.200 | Those are the parts where I felt like

00:06:02.560 | the problem space was was really underserved and really

00:06:05.920 | it wasn't necessarily like technical problems that were really hard.

00:06:11.600 | It was more subjective problems where I think the thing that was lacking

00:06:16.280 | was the taste or the opinions

00:06:20.960 | or the like the feeling for what good solutions were.

00:06:25.400 | So the first kind of problem like this that I got into was arrows.

00:06:29.840 | Like I had, you know, two boxes or two points arbitrarily placed.

00:06:34.160 | I want a good looking arrow, like a quote, Mark, like good looking arrow between the two.

00:06:38.640 | You know, well, that could be anything.

00:06:40.840 | That's not a math problem.

00:06:42.880 | Maybe it involves some angles and linear geometry and vectors and all that.

00:06:46.680 | But it's like the good looking part really was the

00:06:49.600 | was just like my own taste and my own eye and like tons and tons of iterations.

00:06:55.520 | And arrows are super tricky.

00:06:57.520 | And there's a million ways for this, you know, edge cases

00:07:00.120 | when things are overlapping or things are too far away or too close and all this.

00:07:03.880 | But I was working on this and I was working on this in public on Twitter,

00:07:08.120 | recording GIFs of boxes and arrows kind of squishing together and all that.

00:07:11.640 | And I think people really like that.

00:07:15.320 | And they liked kind of following me on this somewhat obsessive journey,

00:07:19.880 | which was technical, but it wasn't like it wasn't like trying to crack an algorithm.

00:07:24.640 | It was like trying to trying to figure out and identify the the rules

00:07:29.680 | governing an aesthetic experience or an aesthetic, you know,

00:07:32.720 | thing, which was a good looking arrow

00:07:35.800 | that became perfect arrows.

00:07:37.960 | And that was pretty popular.

00:07:39.600 | But the next one really is what what kind of broke my popularity

00:07:43.800 | on Twitter or just in the space, and that was a project

00:07:47.680 | that ended up being called Perfect Freehand. This is

00:07:50.600 | a little hard to describe.

00:07:53.960 | If you've ever used like an iPad pencil or drew with like a stylus

00:07:57.960 | and Photoshop or something like the harder you push,

00:08:02.000 | the thicker the line gets and the lighter you push, the thinner the line gets.

00:08:05.800 | It kind of is like this ink experience.

00:08:08.120 | And that's it's not a easy problem.

00:08:13.920 | But if you're doing it in a kind of a Photoshop style, like raster environment,

00:08:18.680 | you know, the solution is pretty straightforward.

00:08:21.400 | You just draw like.

00:08:22.520 | You interpolate like tons and tons of tons of whatever shape you're drawing

00:08:27.600 | in between each point that you've actually moved your mouse to.

00:08:30.960 | And you just change the size of that little stamp that you're making.

00:08:34.320 | So it's like a little circle, slightly bigger circle,

00:08:36.960 | slightly bigger circle, slightly bigger circle.

00:08:39.080 | But they're all really tightly packed together.

00:08:41.120 | And so it looks like a kind of a

00:08:42.600 | a line that's changing its width as it moves.

00:08:46.040 | My. Angle on this.

00:08:50.360 | The reason why I spent so much time about it

00:08:52.600 | was that I wanted to do that using vectors.

00:08:54.320 | I wanted to get a bunch of points in and then like a polygon

00:08:58.120 | that sort of defined the outside of that shape coming out,

00:09:01.600 | because I like to work in SVG and.

00:09:03.200 | And it turned out that this was like an insanely hard problem

00:09:08.200 | that no one had solved.

00:09:09.160 | And if they have solved it, they certainly didn't open source it.

00:09:13.080 | But I couldn't find any good example of a variable width line

00:09:17.000 | that actually worked fast enough and

00:09:19.920 | consistent enough, et cetera, for for it to be digital ink.

00:09:24.560 | And so, again, I did this in public, did this on Twitter,

00:09:27.240 | a million GIFs of lines that look terrible,

00:09:31.760 | but, you know, like slowly attracting more like getting closer to the solution,

00:09:36.480 | attracting more people who had solved this problem or tried to do this

00:09:39.400 | or they wrote their Ph.D. on ink.

00:09:41.360 | And let me tell you about, you know, how arcs work in this environment

00:09:44.880 | and all this stuff. Wow.

00:09:46.520 | And I yeah, it was it was fantastic.

00:09:49.160 | Like I met so many good, you know,

00:09:52.160 | people who had like we're experts on this or or something like it.

00:09:56.400 | And slowly we made a

00:09:59.440 | like a really, really good, tight

00:10:02.760 | little library for for doing exactly what I wanted.

00:10:06.200 | Like here are a bunch of mouse points or just arbitrary points.

00:10:09.560 | Like, give me back a polygon that surrounds them.

00:10:11.600 | And let me essentially draw a line around the edge of that polygon,

00:10:15.680 | fill it in, and it'll look like ink.

00:10:17.920 | And so that was perfect freehand.

00:10:20.760 | And that's now used in like Canva uses it like Draw.io uses it.

00:10:25.360 | ExcalDraw uses it. We use it at Tealdraw all over the place.

00:10:27.920 | Yeah, it's it's just like a significantly better

00:10:33.480 | than the next best solution in that space.

00:10:35.880 | And there really wasn't even any known solution in that space.

00:10:39.280 | So someday I'm going to be checking out at a hotel and see my own ink

00:10:42.320 | and, you know, a little iPad or something like that.

00:10:44.240 | And so. I might as well

00:10:49.920 | just kind of finish the whole origin story or something.

00:10:53.400 | But that's kind of led right into Tealdraw is that I had integrated my ink

00:10:58.280 | into ExcalDraw and I spent time in that that code base.

00:11:02.600 | And I'd also like created several

00:11:04.760 | like infinite canvas like tools to help me

00:11:08.640 | build perfect freehand and visualize it and sort of, you know,

00:11:12.840 | do my ink and pan and zoom in and program against this thing.

00:11:18.280 | And so I had done including Globstuck design, which I won't necessarily talk about.

00:11:22.040 | But it's a yeah, it's kind of like a weird experimental design tool.

00:11:28.080 | But anyway, it was like as an infinite canvas.

00:11:30.400 | It was very much like, you know, Framer, Figma, et cetera.

00:11:33.160 | And after doing ExcalDraw and been working on these kind of projects

00:11:37.120 | that were in the same area, I was like, you know, maybe there's

00:11:40.560 | there's a market here for or not even a market.

00:11:44.440 | It was just like, I think the thing that I want to work on next is

00:11:48.200 | is like a general purpose, kind of like whiteboard like engine.

00:11:52.880 | Mostly for myself, I built globs, but the only thing that you could put

00:11:57.640 | on the canvas in globs was a glob.

00:11:59.880 | So I had all this like code and these solutions that I was like hanging around.

00:12:04.040 | It could kind of see how I would adapt it.

00:12:07.280 | And so that's what I started doing.

00:12:08.560 | And that was the next story that I was kind of telling on Twitter is that like,

00:12:12.360 | OK, here's how selection works in something like Figma or or something like Miro

00:12:16.720 | or Framer or Sketch.

00:12:19.000 | These these kind of like not heuristics, these sort of conventions

00:12:23.760 | that are part of this really complicated thing called like the infinite canvas,

00:12:27.720 | you know, going all the way back to whatever

00:12:31.120 | Flash and before then, you know, Adobe Illustrator and before then all the way back.

00:12:36.240 | And they're all pretty consistent between products.

00:12:40.040 | Like if you're making a canvas this way, you have to kind of do them all.

00:12:42.960 | Like your undo, redo should work in a specific way.

00:12:45.200 | Your selection should work in a specific way.

00:12:47.960 | Like, you know, the camera position and how the camera moves

00:12:50.360 | should work in a certain way.

00:12:52.360 | All the modifier like option drag to clone and then all of those

00:12:55.720 | became their little vignettes of of how I was building this thing.

00:12:59.840 | This was now like spring of twenty twenty one.

00:13:05.240 | And I had everyone

00:13:08.280 | from any infinite canvas related creative product

00:13:12.960 | kind of like in my inbox being like, hey, can you come work for us?

00:13:15.800 | You know, like, let's talk. Let's do this.

00:13:17.920 | And so I was either going to go work for Figma or Adobe.

00:13:20.720 | And I ended up going with Adobe in part because I think FigJam had just come out.

00:13:27.640 | And the team at Figma were like, well, this this is competitive with with FigJam.

00:13:33.480 | I'm like, this thing is like nothing.

00:13:35.880 | It's like a little open source.

00:13:37.400 | You know, it's like no one uses this.

00:13:39.560 | It's just me trying to get to 10,000 Twitter followers.

00:13:43.040 | But, you know, it's mine.

00:13:44.280 | So, no. So I went with Adobe and I told but I told them, I'm like,

00:13:48.320 | I don't want to start for six months.

00:13:51.160 | Like, this is actually a pretty fun project for me.

00:13:53.160 | I want to get it out of my system.

00:13:55.560 | You know, let me start in January and just work on this.

00:13:58.600 | And so they said yes.

00:14:00.680 | And I, you know, quit my working with play

00:14:04.680 | and and said, I'm going to go work on this little open source thing for six months.

00:14:09.000 | I have some, you know, contracting money in the bank.

00:14:12.120 | Let's drain the company account and and do this.

00:14:15.160 | That's not what happened.

00:14:16.480 | The like I went full time from a Wednesday on Thursday.

00:14:22.920 | I had a very large communications company say, hey, we're moving our

00:14:28.520 | whiteboard that we've designed for specific touchscreen devices.

00:14:32.960 | We're moving that into the browser.

00:14:35.680 | It turns out people want to use the whiteboard on their phones.

00:14:38.640 | And on their laptops and all that, like like they do with Miro.

00:14:41.440 | And so we need to make this thing that we wrote in C++ to be highly performant on these,

00:14:46.960 | you know, kind of tiny microcomputers that were part of these interactive touchscreen TVs.

00:14:52.160 | We have to make this work on the web and we don't think it's going to be good enough.

00:14:55.360 | We have to build from scratch.

00:14:56.920 | We don't have the team. Can we just build on what you're building?

00:14:59.480 | At the time, this thing was an open source.

00:15:01.040 | It was just sort of but it was getting there.

00:15:03.160 | And I'm like, yeah, sure.

00:15:04.760 | Like, give me like seventy five thousand dollars.

00:15:06.600 | I'll let you see the source code.

00:15:08.160 | Just, you know, don't I don't want to talk to you very often.

00:15:12.120 | You know, I'm not working for you.

00:15:14.080 | I never want to see your code, but you can look at mine.

00:15:16.080 | And they said yes to that.

00:15:19.280 | And so I was, you know, funded for those those first six months.

00:15:22.640 | And I got to work on this, you know, without having to feel bad about it.

00:15:27.520 | And I'd also opened up, eventually opened up TealDRAW to be

00:15:32.960 | like sponsorware that if you were sponsoring me on GitHub,

00:15:37.280 | you could you could access it, you know, in its kind of primitive state on TealDRAW.com.

00:15:42.000 | And it had like a couple hundred people join that way and sponsor me.

00:15:46.720 | So at one point, like my sponsorship was, you know, over five thousand dollars a month,

00:15:51.600 | which is not massive money, but it's like I wasn't doing anything different.

00:15:55.360 | So it was pretty good as a kind of a passive thing.

00:15:57.440 | Anyway, I shipped it at the end of November 2021, and it was very popular.

00:16:06.800 | I just open source everything.

00:16:08.000 | I was just like, you know, the TealDRAW.com app, the library, the canvas.

00:16:12.880 | And, you know, I was organized in a certain way.

00:16:14.960 | I just made it all public.

00:16:18.480 | Everything was MIT.

00:16:19.840 | You know, let's just let's just throw this out into the world and see where it goes.

00:16:23.040 | Well, it went pretty far.

00:16:25.200 | It was like number one on Hacker News for a while.

00:16:27.280 | It was like the top trending repo on GitHub.

00:16:29.280 | A lot of people like 40,000 people showed up at TealDRAW.com to use it on that launch date.

00:16:36.720 | Which was all good.

00:16:37.520 | Like so far, this was all within my same narrative of, OK, this is cool.

00:16:41.600 | I'll make this and then I'll go do something else afterwards.

00:16:43.680 | The thing that really surprised me was how many teams wanted to build on this.

00:16:49.840 | And they weren't like they weren't building whiteboards.

00:16:53.520 | They weren't like Miro competitors or Figma competitors.

00:16:56.720 | They were just like apps that you wouldn't expect to have infinite canvases inside of them.

00:17:04.240 | But and they wouldn't have built it except that I had suddenly made this very easy.

00:17:10.000 | And I had suddenly shrunk the development time of this like whiteboard like feature

00:17:15.360 | in their product from like three years and three people to three weeks and one person.

00:17:21.120 | And not even one person.

00:17:24.160 | Just like, you know, no new developers, no new team, no new graphics experts.

00:17:28.320 | No, no, no.

00:17:29.360 | Like computational geometry guys like, you know, we can do this.

00:17:31.920 | We the canvas itself is like React all the way down.

00:17:34.480 | So even if you wanted to customize it, you'd just be writing React components and then

00:17:37.760 | a little bit more code on top.

00:17:39.200 | It got like I was totally overwhelmed by inbound from companies who were like, I want to build

00:17:44.720 | this or I want to acquire you or I want to, you know, I want you to build something for

00:17:49.200 | me or, you know, I want this in my app.

00:17:51.120 | You know, how do you help me or how can I do this?

00:17:54.080 | And and people were shipping things also like within two weeks, three weeks, like production

00:18:00.000 | ready, like people had taken this and run with it.

00:18:02.400 | And so the the story that I started to get around Tealdraw was that like, OK, well, this

00:18:08.240 | is this is a cool little whiteboard, but it's also kind of like filling a gap that no one

00:18:15.120 | knew was there, which was that like in the same way that like Mapbox or Google Maps,

00:18:22.640 | you know, provide maps for apps that would definitely not build maps themselves.

00:18:26.400 | Like maps are insanely hard, like your little local food delivery app like wouldn't just

00:18:30.800 | wouldn't have a map in it, you know, like easy.

00:18:32.720 | But it is a value add.

00:18:35.040 | If they can have it in there, then absolutely it is a value add.

00:18:37.920 | It's just completely impractical to do themselves.

00:18:40.800 | And what I learned talking to folks was that like every PM had used Miro or used Figma

00:18:47.840 | or used, you know, one of these other collaborative tools.

00:18:51.200 | And every creative product person was like, well, this is fun.

00:18:55.760 | Collaboration is fun.

00:18:56.720 | This canvas thing is pretty cool.

00:18:59.040 | Like, why don't we do why don't we why can't we put our information on, you know, why can't

00:19:03.920 | we put our CRM on the canvas or why can't we do our sales stuff here?

00:19:07.520 | Like we're already kind of using Miro for this.

00:19:09.360 | Like, why don't we give why couldn't we give this to our customers as well?

00:19:12.640 | Like, why don't we build a product around this?

00:19:14.160 | And it was just a technical no until, you know, November 24th, 2021, when suddenly it

00:19:23.600 | was like a technical maybe and yeah, there was there was absolutely demand.

00:19:29.760 | So hence the, you know, I had to call Adobe and say, no, I'm not going to come in on Monday.

00:19:35.440 | Like it turned out that the best possible outcome of this happened and there's actually

00:19:40.960 | a company here.

00:19:42.080 | And then I went out and I raised a seed round from Lux in New York and Amplify in California

00:19:47.280 | and a whole bunch of really great angels.

00:19:50.640 | You know, on the story of, yeah, this is this is cool.

00:19:52.640 | It's good app feels good.

00:19:54.480 | Companies want it.

00:19:57.120 | And, you know, by then I had had almost $200,000 of sponsorship, you know, and people were

00:20:02.080 | just signing up and signing up because there was no way to even be a customer.

00:20:05.360 | Like sponsorship, sponsorship.

00:20:09.600 | And but also, yeah, there's a massive 200, you're not saying 200k a month.

00:20:14.560 | That's no, no, no.

00:20:16.240 | But like, I mean, I had had up to then, the amount of sponsorship that I had received

00:20:22.480 | was around $200,000.

00:20:24.000 | I think some of the recurring stuff was like, like 5,000 a month.

00:20:31.680 | But yeah, like it was.

00:20:33.200 | Which is in the top echelon.

00:20:34.960 | A lot.

00:20:35.360 | Yeah.

00:20:35.600 | Oh, yeah, like, yeah, yeah.

00:20:37.840 | Certainly the but yeah, and just the amount of like kind of validation that had come in

00:20:44.640 | around this was like more than more than usual.

00:20:48.080 | So race company or race around, put together a team here in London, and basically had just

00:20:56.000 | been building this, this whiteboard SDK.

00:20:58.400 | Since then, you know, we reconfigured the project around, okay, we're going to be building

00:21:02.480 | this not necessarily for end users, but for teams to use as kind of an infrastructure

00:21:07.200 | product, a developer product, something closer to Mapbox.

00:21:12.000 | And, you know, we were making demos to kind of like show different ways that it could

00:21:16.160 | be used.

00:21:16.640 | Certainly the collaboration thing is a big one, but the fact that you could put anything

00:21:21.040 | on the canvas that you can put on a website, just because it is a, it is a web, like all

00:21:26.240 | HTML, CSS all the way down.

00:21:28.000 | And that was going really well.

00:21:32.560 | It was already a good story.

00:21:33.840 | And then I just raised like a 2 million extension for the company.

00:21:40.320 | While I was on the final pitch for that, the Dev Day was happening at OpenAI.

00:21:47.520 | And in the morning I woke up and I was getting all this kind of action on Twitter because

00:21:53.360 | a developer at Figma had used teal draw to make this little demo where you could draw

00:21:58.480 | a website, click a button and get back a big pop up that had your website in there.

00:22:05.840 | It was like a prompt, like, you know, hey, you're a developer, you just got this wireframe

00:22:11.040 | from your designer.

00:22:12.000 | Can you give it back as a single page HTML file?

00:22:14.400 | And it would do it and it could do it.

00:22:18.400 | And then you could show that website to whoever is using the app.

00:22:22.480 | And we took that and we're like, wow, you could do so much more with this, like with

00:22:28.640 | teal draw, just like it's only scratching the surface of the type of integration that

00:22:32.560 | you could do.

00:22:33.520 | So again, we had just finished the race.

00:22:36.640 | Pressure was off a little bit.

00:22:38.720 | It was kind of getting towards the end of the year.

00:22:40.240 | I was like, all right, let's let's just take this and have some fun.

00:22:42.720 | Let's make some some viral shit.

00:22:44.000 | Maybe we'll get like 200 likes or something like that.

00:22:46.320 | And it exploded.

00:22:49.280 | It was like, I think we're at like for this month, last 30 days, like 22 million views

00:22:55.920 | or something like that.

00:22:56.640 | It's just like it was Kanye West numbers.

00:22:59.360 | It was really, really, really popular for a couple of days.

00:23:02.000 | If you're on Twitter and at all technical, you might have just seen a ton of teal draw

00:23:06.320 | stuff on your timeline or about two weeks ago, three weeks ago.

00:23:10.480 | Well, so, yeah, that kind of brings us up almost to today.

00:23:15.200 | You just released something two hours ago, which we'll talk about.

00:23:17.600 | But maybe this is a good time to bring up the screens, you know, for those who.

00:23:24.560 | Oh, yeah, yeah, let me share.

00:23:26.080 | Yeah, so we're recording a video as well.

00:23:30.080 | You can jump over to the YouTube to see stuff.

00:23:32.720 | But this is an inherently visual podcast, so we have to show stuff.

00:23:38.400 | OK, the incremental thing I got from your blog post.

00:23:41.680 | So you did do a write up, which thank you for that, because I actually didn't know that

00:23:45.200 | you did a write up.

00:23:45.840 | It was just drawn up.

00:23:47.280 | Oh, yeah.

00:23:47.760 | All the videos.

00:23:49.040 | This is the power of open source, right?

00:23:52.800 | That someone else had the idea.

00:23:54.960 | You weren't even focused on Dev Day.

00:23:57.200 | Someone else had the idea and just made it without your permission or talking with you.

00:24:03.120 | And then the idea could spread back to you and you could run with it.

00:24:06.160 | Yeah, exactly.

00:24:07.040 | And we had made a lot of the bits and pieces in place already based on, you know, I mean,

00:24:12.800 | it's well documented or it's documented.

00:24:15.360 | It's documented.

00:24:16.720 | It has tons of examples and all that.

00:24:20.400 | Yeah, and it's not, I mean, it's a big library as far as an open source library goes.

00:24:28.400 | But it is, yeah, you can work with it.

00:24:31.600 | And once this thing got popular, like the first thing we did was create like a starter

00:24:39.280 | kit so that someone could take it and like run with it.

00:24:41.600 | Yeah.

00:24:42.160 | So this is normal teal draw where, yeah, you can, you know, you can draw, you can whatever,

00:24:47.680 | move things around.

00:24:48.560 | It works if you've used Figma, if you've used Miro, it's kind of familiar to that.

00:24:54.320 | And you can put pretty much anything on this canvas that you want, like YouTube links,

00:25:01.680 | etc.

00:25:02.720 | Just because it is, I always have to remember to open YouTube in a incognito window so that

00:25:14.480 | people don't see my embarrassing interests here.

00:25:18.000 | So yeah, because this canvas is HTML and CSS and like divs and stuff all the way down,

00:25:23.920 | you can put things like YouTube videos on there.

00:25:26.160 | You can even make them play.

00:25:27.440 | Because again, like anything you can do in a website, you can do on teal draws canvas.

00:25:33.600 | What's fun is because it is a canvas all the way down, you can also like draw on top and

00:25:38.800 | like do the kind of canvas manipulation stuff that you might do with normal shapes, but

00:25:43.600 | also with this type of content.

00:25:46.400 | So that ended up becoming like a big part of why Make It Real got kind of popular.

00:25:54.640 | So anyway, I'll show you Make It Real now.

00:25:57.040 | This was a hastily built layer on top of the kind of teal draw engine SDK that we sent

00:26:06.400 | up.

00:26:06.560 | And the idea here is that you can make a wireframe and we're going to send it to

00:26:13.040 | GPT-4 with vision with like a prompt like much like the original one that that Sawyer

00:26:18.320 | Hood had come up with, which is you are a web developer.

00:26:21.520 | You work with designers, they give you wireframes and notes and screenshots and all sorts of

00:26:27.840 | stuff could be anything.

00:26:28.800 | Your job is to come back with a single page or a single HTML file that has all the styles

00:26:35.280 | of the JavaScript, all the markup necessary in order to make a real working prototype

00:26:42.080 | based on what you've been sent.

00:26:44.480 | It also has slight emotional manipulation like you love your designers and you want

00:26:50.240 | them to be happy and like the better your prototype is, the happier they are and all

00:26:55.200 | that.

00:26:55.280 | Oh, in the prompt?

00:26:56.000 | Yeah, yeah, yeah.

00:26:57.520 | Again, it's open source.

00:26:59.600 | You can read the prompt.

00:27:00.720 | It's kind of a funny one.

00:27:01.840 | But we do some other tricks that I'll like, for example, we'll have a second as I do this.

00:27:08.000 | So you select something like this, you click the Make It Real button and you get back a

00:27:11.600 | little waiting box.

00:27:14.240 | We've considered running ads or like in this waiting moment, but no, I don't know.

00:27:20.480 | You could.

00:27:21.040 | Maybe like a kind of Zelda style like tips, you know, tips and tricks like here's different

00:27:25.760 | ways you can do.

00:27:27.280 | But yeah, for example, like we also send it.

00:27:32.240 | I mean, this is part of the joy of like a multimodal prompt is we send it the photo,

00:27:35.760 | which kind of looks like the same as if you had done a copy and paste thing.

00:27:41.440 | So like an image.

00:27:42.240 | And you have all this functionality worked out, you know, like that's what I find so

00:27:50.320 | poetic about this that we're just ready.

00:27:52.160 | Yeah, it's like it really I mean, it feels like we had gone off in this like this very,

00:27:58.960 | you know, as collaboration and AI and stuff was going in one direction, we kind of just

00:28:02.960 | went off in our own weird like, hey, the world is really going to need a whiteboard at some

00:28:06.640 | point direction and then and then it just they kind of met us where we were at and then

00:28:12.720 | we've been able to just be like show up like show up on day one of this new world of possibility

00:28:19.840 | with like the thing that like if I hadn't spent the last two years building this, I

00:28:26.560 | would spend the next two years building this like it is the right product for this type

00:28:31.680 | of type of feature.

00:28:33.520 | So anyway, they give us back a HTML, we stick it into an iframe, put that onto the canvas,

00:28:38.240 | just like we did with that YouTube link, and I can, I can interact with it.

00:28:41.840 | So it should be going from orange to pink, orange to pink.

00:28:46.160 | Hey, it's given us a hex code, I can click the hex code.

00:28:50.960 | And it gives me, you know, YouTube links, etc.

00:28:55.760 | Just because it is, I always have to remember to open this open YouTube in a way that in

00:29:04.560 | something like v0 or some of these other kind of prompting environments, like the only way

00:29:10.640 | for you to then make this better.

00:29:12.880 | Maybe you can do this with chat GPT or something and you could you could write like, oh, actually,

00:29:18.320 | you know, instead of or like you missed the labels like it should say orange and pink,

00:29:23.520 | you know, on top of this thing.

00:29:25.360 | And, and it doesn't.

00:29:26.880 | So you could go back here and like, you know, make, make sure that this is whatever you

00:29:33.520 | can change the input.

00:29:34.320 | But because this is teal drawn, because you can draw on top of this stuff, you could also,

00:29:40.720 | you know, right on top, like you could kind of modify this, and maybe even give it so

00:29:47.600 | the same type of markup that you would give to a designer, you know, a designer or something

00:29:54.320 | like that, you know, draw some arrows, or maybe paste in a screenshot and say, hey,

00:29:59.040 | make it look more stylistically close to this other thing.

00:30:01.760 | And I'll say this as well.

00:30:05.760 | I'll say like full width for that, that button, and anything else that we, well, let's just

00:30:14.320 | see what it does.

00:30:14.880 | And then what you do is you select, you select the website that they gave you back the previous

00:30:23.120 | result along with all this markup.

00:30:25.120 | And you use that as the new input.

00:30:28.400 | And so that's going to give you something kind of like, yeah, it's going to give you

00:30:34.160 | an image that looks like this that you've now sent.

00:30:36.240 | But we've also kind of tweaked the prompt a little bit when you when you do include a

00:30:41.200 | previous result and say like, hey, this, you know, the, the, the wireframes coming back

00:30:46.240 | are annotations or markup based on, on what you sent before.

00:30:51.120 | And there, there you go.

00:30:51.920 | So now we have a new prompt that sure enough, the, the labels are there, you know, it still

00:30:58.080 | works just like before the button is full width and, and, you know, it still works just

00:31:02.400 | the same.

00:31:02.800 | So we send it back.

00:31:04.880 | Again, we send it the image, we send it the text, the prompt.

00:31:11.600 | We also send it all of the text items themselves separately, because ChetchiBT is not really

00:31:16.400 | great with, with recognizing text.

00:31:18.720 | So we say like, oh, by the way, your vision's not so good.

00:31:21.120 | So we've, we've ensured to have our copywriter, you know, list out all the, the copy that

00:31:26.000 | you can use.

00:31:26.480 | I think we even send it back the HTML that they used for the, the, the previous result.

00:31:32.400 | So we just dump like as much information as possible at this, the, the GPT-4 with vision.

00:31:39.200 | And that's how you're able to, to get these sort of iterative results.

00:31:44.400 | And the, it is, it is like legitimately good to, like, it feels like work.

00:31:55.600 | It feels like, like you're actually doing stuff when you're iterating through this way

00:31:59.360 | and, and slowly shaping and adding complexity and doing step-by-step, you know, as you're,

00:32:05.440 | you're, you're building something.

00:32:07.040 | And when you're done, you can, you know, copy a link to that and, and open that in a new

00:32:12.640 | tab, like we, we host it, it's there forever.

00:32:15.040 | You can, you can bookmark this if, if you really just needed a slider between orange

00:32:19.520 | and pink.

00:32:19.920 | Well, now you have one, you, you, you know, whether you could code it or not, or you maybe

00:32:24.720 | not worth building or using a no code tool to build, but, you know, we just made that

00:32:29.040 | in five minutes.

00:32:29.680 | If you are more on the coding side and you want to use this as a kind of a foundation

00:32:33.520 | of, of a real project, or maybe just to like, see how it like, well, how did, how did, how

00:32:39.760 | does that actually work?

00:32:41.360 | You can open it up in StackBlitz or CodeSandbox.

00:32:44.960 | I think tomorrow we'll have Replit.

00:32:46.240 | And yeah, see all the code, see what Chachapiti came up with and, and kind of use it or adapt

00:32:53.760 | it or, you know, keep it going or do whatever you want with it.

00:32:57.200 | But yeah, cause it is, it is like, it is real and yeah.

00:33:05.040 | You make real.

00:33:06.080 | Yeah.

00:33:07.360 | Yeah.

00:33:07.600 | It's, it's interesting that you can also, I've seen some of your other demos.

00:33:11.120 | It looks like you're, you're about to move us on to another.

00:33:13.040 | Yeah.

00:33:14.880 | I'm, I'm going to grab a couple of the, the ones that I have showed before.

00:33:18.560 | This one's a really interesting one because it is.

00:33:22.720 | Okay.

00:33:26.320 | So what's on the screen now just to, to kind of.

00:33:29.360 | Narrate.

00:33:31.280 | Describe it is, is I have a, I have a drawing of a stop, like a kitchen timer, you know,

00:33:38.640 | where you can add a minute, add a second, you know, start or stop the timer or, or reset

00:33:43.040 | the timer.

00:33:43.520 | And then next to it, I also have a state chart, like state machine describing the three states

00:33:49.840 | of the timer starts or stopped running or complete.

00:33:54.320 | And like what each one of those buttons could, should do in terms of transitions or changing

00:33:59.440 | the state.

00:34:00.640 | I think you can hand this to pretty much any designer or developer and get back a, you

00:34:05.440 | know, a working result.

00:34:06.560 | Like it's fully, fully spec'd sort of, sort of.

00:34:08.560 | But what, what you can do with this.

00:34:12.480 | I made a friend that did it, David Crofied might say.

00:34:14.400 | Yeah.

00:34:14.800 | Develop a state chart first and then, you know, plug it into it.

00:34:17.280 | Yeah, exactly.

00:34:17.920 | And what's cool this way is that you can, well, let's do a couple of things in parallel.

00:34:24.720 | First thing I'm going to do is I'm going to,

00:34:28.880 | I am just going to make a box over here and I'm going to say, kitchen timer.

00:34:34.720 | Right in the middle of the box.

00:34:37.200 | And, and this is going to be the only prompt that I'm going to, I'm going to give it.

00:34:41.760 | I'm just going to click make real and just the, the kitchen timer box.

00:34:44.480 | Sometimes you see with these multimodal prompting, like someone will draw a calculator, like

00:34:51.280 | in, in a lot of complexity and, and say, you know, make this, make this real.

00:34:55.280 | And sure enough, you get back like a really complex full calculator.

00:34:58.400 | But if you did the same thing and you just said empty box, but just the word calculator,

00:35:04.160 | it would give you back the same thing is that it knows what a calculator looks like

00:35:07.200 | and it knows how it works and all that.

00:35:09.680 | So next, let's also give it the, just the, the user interface, like without the state chart.

00:35:15.760 | Well, we'll leave the state chart out, but we'll do just the user interface.

00:35:21.680 | And then we'll do just the state chart, you know, and say, Hey, make this real.

00:35:28.720 | And then we'll do both the state chart and the UI.

00:35:33.920 | So we have four different prompts with four potential different results based on, you know,

00:35:44.400 | variations of the same, same input.

00:35:47.120 | So first off our kitchen timer, where all we did was we sent it a box with the word

00:35:52.960 | kitchen timer.

00:35:53.680 | It has, I don't know what this box is for, but we have a time.

00:35:58.560 | We have start, stop and reset.

00:36:00.560 | I can double click in.

00:36:02.560 | I can click start.

00:36:03.440 | It doesn't do anything.

00:36:05.520 | Oh, what is this?

00:36:06.560 | Oh, whoa.

00:36:07.600 | This is the, if this, okay, well, if the number is there, yeah, then it'll, it'll stop.

00:36:14.800 | If I stop it, it goes, it stops.

00:36:17.200 | I can start it.

00:36:17.920 | It'll keep going again.

00:36:19.040 | Okay.

00:36:19.440 | And I can reset it.

00:36:20.320 | And there we go.

00:36:21.280 | The only weird thing is that it's, yeah, it has a, a number input field for the number

00:36:27.280 | of seconds that I can, I can type out, but yeah, you know what, in a pinch, in a pinch,

00:36:32.320 | I'll take it if I really needed just to count 60 seconds or something.

00:36:35.760 | Next, we have the input where, or the result where the input was just my drawing of a kitchen

00:36:42.480 | timer.

00:36:43.280 | I didn't tell it it was a kitchen timer.

00:36:45.280 | I didn't send it the words kitchen timer, and I didn't tell it how it should work, but

00:36:49.280 | it did produce something that kind of looks the same.

00:36:52.560 | Let's see if it works.

00:36:54.560 | So I'm going to click minute, second, start, reset.

00:36:58.080 | No.

00:36:58.240 | So unfortunately, it did not make any working UI, although it did, you know, put the buttons

00:37:05.120 | in the right place or something like that.

00:37:06.480 | Maybe it over focuses on the UI because you, you told it, you just, that's all you gave

00:37:12.560 | it, you know?

00:37:13.520 | Yeah.

00:37:13.920 | Yeah.

00:37:14.240 | I mean, there is in the prompt kind of language around, like, you use what you know about

00:37:19.760 | the way that applications work in order to sort of fill in the blanks here in terms of

00:37:23.040 | the behavior and all that.

00:37:25.680 | But let's go to the next one.

00:37:27.120 | This one is where we only sent it the state chart.

00:37:29.680 | There's also something in the prompt that says like, if it's red, it's not part of the

00:37:33.760 | UI.

00:37:34.640 | Like if it's red, then like treat that as an annotation rather than like a, a thing

00:37:39.840 | that you should, should actually make.

00:37:42.080 | Uh, okay.

00:37:43.200 | So this time it actually looks a lot like the previous one, uh, but it does have these,

00:37:49.840 | um, minute, second buttons.

00:37:52.400 | Oh, weird.

00:37:54.080 | It has plus and minus for minute, minute, seconds.

00:37:56.720 | Uh, and it also has this like stop state written at the bottom.

00:38:01.760 | So there's four buttons, you know, minus minute, minus second, plus minute, plus second, and

00:38:06.080 | then there's starts, uh, start and reset.

00:38:08.240 | So does it work?

00:38:10.080 | I can add a minute.

00:38:11.360 | I can also subtract a minute.

00:38:12.640 | All right.

00:38:13.440 | I can add a second.

00:38:14.320 | I can also, yeah.

00:38:16.400 | And if I press start, we're now in the running state.

00:38:18.800 | Uh, unfortunately it's going up rather than down.

00:38:21.840 | And I, and I can reset it and, and okay.

00:38:27.040 | Uh, I'm just curious if I, if I do give it a, an additional prompt here and I say like,

00:38:32.240 | uh, uh, this should count down, not up.

00:38:39.200 | Uh, and just kind of do an arrow towards the start button here.

00:38:42.320 | Let me see if that, uh, that'll make a real one.

00:38:44.480 | But, and then finally we look at the other example, which is where we sent the state

00:38:48.400 | chart and the UI.

00:38:50.240 | We get something that looks much, much more like our user interface.

00:38:55.360 | The question is, does it work?

00:38:56.480 | Yes, it does.

00:38:58.080 | Uh, perfect.

00:38:59.840 | I can stop it and start it.

00:39:02.240 | Yep.

00:39:02.640 | That's a reset it.

00:39:03.680 | Wonderful.

00:39:05.440 | Uh, and in this case, yeah, my, my, my, my feedback was accepted.

00:39:14.080 | I went back to the one where I, I'd asked it to count down and not up and it all looks

00:39:18.960 | the same, but now it's counting down.

00:39:20.240 | So I think for folks, especially who have like, uh, who have worked in design and who

00:39:28.800 | have worked in sort of like user experience design in particular, like this should feel

00:39:33.520 | pretty familiar of, of kind of sketching out and trying to do your best to specify, like

00:39:38.160 | what it is you want and see what you get back from your designers.

00:39:41.840 | You can see what you get back from your developer.

00:39:43.520 | Um, but having like a, an environment in which to have that, like game loop, that like iteration

00:39:50.480 | cycle, uh, alone and in instantaneous and essentially free, uh, is really, really wild.

00:39:57.520 | And, uh, you end up spending a lot of time kind of like, uh, not only getting into the

00:40:03.280 | head of the, the AI and sort of being like, okay, well, why are they getting confused?

00:40:07.680 | You know, what am I sending that is confusing?

00:40:09.440 | How do I send more information in order to like produce a better result?

00:40:13.120 | Uh, but also it really forces you to clarify your own expectations of like somewhere up

00:40:19.680 | here, I have a, uh, uh, uh, drag and drop list, you know, where you can drag list items

00:40:25.920 | between, and like, I started working on this and started specking it out.

00:40:29.040 | I was like, man, this is like, actually like not only really hard to, um, like to produce

00:40:36.160 | a good result, but it's also like just really hard to describe is that like the failure

00:40:40.000 | was really on, on my end for just not knowing how to get the information in there because

00:40:44.320 | I didn't actually know how this thing should work.

00:40:46.480 | Um, but yeah, but you know, I could figure it out.

00:40:50.400 | I have an environment in which to figure that out.

00:40:52.400 | It's fun.

00:40:55.920 | That's amazing.

00:40:59.440 | Uh, I'm still processing.

00:41:01.440 | Oh, you have a picture of it.

00:41:02.640 | Oh yeah.

00:41:04.000 | All right.

00:41:05.360 | So this is, this is kind of like, uh, slightly obnoxious, but, uh, during this math, like,

00:41:11.120 | cause this thing went massively popular on, on Twitter.

00:41:14.080 | Uh, uh, yeah, like thousands of retweets type of level.

00:41:20.240 | Um, and there, there were some folks who like were subtweeting it about like, you know,

00:41:25.360 | get over it.

00:41:25.920 | It's just a wireframing or no code tool or something like that.

00:41:29.520 | Uh, one guy did say like, you know, I prefer to, to wireframe like the old fashioned way

00:41:34.480 | with pen and paper.

00:41:35.520 | And, uh, I was like, oh yeah, no, that, that works too.

00:41:39.040 | I like this works with screenshots.

00:41:40.800 | I can just take the screenshot here of, you know, that the, the dude posted of, of the,

00:41:45.520 | uh, the, the drawing that he had made, you know, it's not even like a good photo.

00:41:50.880 | There's a pen, you know, across one of the screens, et cetera.

00:41:53.760 | But if you just give that with no other information, uh, to like, as a prompt, uh, you get back

00:42:02.000 | a pretty good result, like it loading right now, but, uh, I've done this before and yeah,

00:42:08.960 | like you get back, um, you know, just from this like photo of a, of a piece of paper

00:42:15.200 | on the, the, the guy's desk, you know, you have a, um, like not, not completely arbitrary,

00:42:25.760 | uh, result like working website here.

00:42:28.640 | Um, that was, that was, that was inferred from just that picture with no other input,

00:42:33.200 | not, not even like titles or anything else.

00:42:34.800 | And of course it's like responsive and all this stuff.

00:42:37.440 | Uh, and so the idea of like, yeah, like our tools, yes, I've, I've worked really hard

00:42:43.760 | to make all of our shapes, you know, really good in our arrows, obsessively good and all

00:42:49.680 | this stuff.

00:42:50.080 | But like the fun of the infinite canvas and teal draw in particular is that you could

00:42:55.760 | just dump like whatever you want onto the canvas, screenshots, text, images, other websites,

00:43:04.480 | uh, sticky notes, all that stuff.

00:43:06.320 | And, uh, the, the, the model, even as something that was in preview, like the very, very first

00:43:12.480 | sort of multimodal, um, model, uh, can do a really good job at just taking all that

00:43:18.880 | stuff as the input.

00:43:20.160 | And, um, and yeah, like, so we accidentally made a really, really good visual multimodal

00:43:26.880 | prompting, uh, uh, application environments or, or user experience byron.

00:43:34.160 | I'm not even sure what we're going to call this thing.

00:43:35.840 | It's a teal draw.

00:43:38.880 | You also had a, you know, pre-show prep.

00:43:43.760 | You also talked about parallel prompting.

00:43:45.520 | Is that basically just, uh, prompting and then moving on to something else?

00:43:48.560 | Is that, is that what?

00:43:50.240 | Yeah.

00:43:50.480 | That's kind of what we did up here with the, uh, the stopwatches, the fact that we could

00:43:53.840 | get multiple prompts going at the same time and like arrange them spatially.

00:43:58.480 | Uh, and, um, people have done this also with, with imagery, uh, to say, okay, well, here

00:44:06.320 | are, we're going to use Dolly and we're going to kind of like make a tree of prompts as

00:44:13.200 | you go, you know, different iterations based on, um, which, whatever you make for iterations,

00:44:19.680 | you pick your favorite one, you keep going kind of like what you do in mid journey.

00:44:22.960 | But to have that spatial and to have that like, uh, arranged on a canvas so that it

00:44:27.920 | actually can make sense to you.

00:44:29.040 | And you can kind of look back and follow it, follow forward that like, I don't know, like

00:44:33.760 | whiteboards, infinite canvas stuff is just really good for a lot of things.

00:44:37.520 | Um, and, uh, so organizing like a whole bunch of different content that is, uh, irregular

00:44:44.720 | or ephemeral or, um, or, or has a kind of like ad hoc meaning configuration, like, you

00:44:54.640 | know, things that are next to each other or things that are in a grid or in this case,

00:44:57.920 | you know, uh, just even what we're, what we have here for what we did with the stopwatch,

00:45:03.360 | like there's an implicit meaning of like, okay, the source is on the left, the result

00:45:07.920 | is on the right and any further iterations are further on the right.

00:45:12.320 | Right.

00:45:12.640 | Like we didn't put that into a data model.

00:45:14.640 | We didn't structure that in any way.

00:45:16.640 | It doesn't actually, that meaning relationship doesn't really exist in any part of the product.

00:45:21.680 | It just exists to us because we, we can make sense of it.

00:45:25.520 | Um, and yeah, so for, for this type of thing, not only is it cool that now a model can make

00:45:33.920 | sense of it as well, but, uh, but yeah, for, for organizing, uh, complex iterations of

00:45:41.680 | imagery, complex iterations of, of, of outputs, et cetera.

00:45:45.600 | Like, yeah, the canvas is a place I really do believe that.

00:45:49.440 | Yeah.

00:45:52.720 | Um, I mean, that's, that's, that's really incredible.

00:45:55.360 | It's also, it's a little, you know, I think a few developers are kind of scared about,

00:46:02.400 | you know, how much of this, uh, their jobs, uh, this does, obviously, uh, there's a lot

00:46:06.320 | more that they can't do.

00:46:07.360 | Yeah.

00:46:07.920 | So the, the, will this take my job story is, is interesting.

00:46:12.960 | I mean, I'm not actually concerned, but I think this augments, like, uh, my, actually

00:46:20.720 | my concern as a developer is that this is good, but not good enough.

00:46:23.920 | You know, like it's good for throwaway UI.

00:46:26.560 | Um, but I, would I actually export the code and take that code?

00:46:31.120 | Um, I don't know.

00:46:32.240 | Uh, it, it looks like, uh, your first MVP was just HTML files, which, you know, if it's

00:46:38.720 | a single HTML file, it can, it can have some JS or some CSS.

00:46:41.760 | I saw some problems with layout, uh, in there, which I don't know how good it is at layout.

00:46:47.920 | Uh, it's, it looks like you could just prompt it for tailwind if you want tailwinds.

00:46:52.080 | Uh, I assume it can generate react.

00:46:53.920 | Uh, I don't know.

00:46:54.720 | What are the limitations of this thing?

00:46:56.400 | Well, there's, there's the limitations that are in that particular demo, which is that

00:47:01.120 | like, well, it couldn't do react because it needs to just be a single compiled thing.

00:47:05.360 | Uh, you can't do any multi-page stuff or, or anything like that, but that's more of

00:47:09.600 | like how we're structuring the, uh, the project rather than like a specific requirement of,

00:47:16.640 | of the project itself.

00:47:17.600 | The, I mean, we've talked about having, well, there's, there's two kinds of things.

00:47:23.520 | There's one is like, how big is the input window and how big is the output window or

00:47:26.960 | something?

00:47:27.280 | Uh, so in theory, you could have, uh, the input be, here's a entire full stack react

00:47:36.640 | application, uh, together with all my UI and all this, all my components, et cetera.

00:47:41.200 | And here is a screenshot that I took of the landing page where the menu, uh, is in the

00:47:50.000 | wrong spot, you know, and I'm going to annotate that with some arrows and some text in order

00:47:55.280 | to say like, here's what I, where I want it to be, or here's what I want, et cetera.

00:47:58.800 | And for the output to be, um, you know, a diff that I can apply to my code base, like

00:48:06.720 | produce the, the, the, the git, like basically like produce the commit, um, that would change

00:48:14.000 | this and have that commit be against multiple files and et cetera.

00:48:19.600 | Um, in order to, to have potentially like a solution that is just, uh, like ready to

00:48:26.640 | go applicable, like a patch or a PR that you can make, um, there really isn't any, any

00:48:36.320 | limit in that.

00:48:37.200 | And we've seen with copilot, et cetera, the challenge is more on the input side than the

00:48:42.560 | output side.

00:48:43.760 | Like, like absolutely you could figure out a way for this thing to spit out like a working

00:48:49.120 | iOS app or something like that.

00:48:50.960 | Uh, the question is like, how do you tell it what you want and, and how do you iterate

00:48:56.080 | when it gets it wrong?

00:48:57.360 | And, and just doing zero shot, zero shot, zero shot.

00:48:59.760 | It's like really a frustrating, um, process.

00:49:02.640 | Uh, but if you do have a way of, of iterating, if you do have a way of kind of like step

00:49:06.800 | by step moving towards the, the solution that you want and kind of like, you know, getting

00:49:11.600 | it into like, okay, well, this is good, but it's not great.

00:49:15.200 | Could be better, et cetera.

00:49:16.960 | Um, that's, uh, that's how you actually make that type of complex output more practical

00:49:23.120 | or more realistic.

00:49:24.320 | Um, is that you probably won't get ever get the prompt just right.

00:49:28.400 | Even if you have like a really, really, really good, you know, three generations from now

00:49:33.040 | agent, like you, you still have to put that information in.

00:49:36.000 | Um, but you're never going to put the, all of the information in the first time you need

00:49:40.800 | to be able to, to iterate on it.

00:49:42.320 | And so with visual stuff, uh, I feel like the canvas, like what we were looking at,

00:49:48.160 | um, that's part of what it unlocks is that like space of iteration at space of, uh, um,

00:49:54.080 | you have a way of marking up the result and using that as the new prompt.

00:49:58.720 | Um, and that's, that's kind of new.

00:50:00.560 | Yeah.

00:50:02.800 | Yeah.

00:50:03.520 | Um, multi-modal prompting is such a brilliant concept that, uh, you know, I think it's,

00:50:10.080 | uh, it's going to be a norm for, for, for some things.

00:50:12.720 | Um, in my mind, you demonstrated, you know, coming from Photoshop, there's this concept

00:50:19.600 | of layers.

00:50:20.160 | Um, you can, you can kind of simulate layers in, in TO draw.

00:50:24.000 | And I see like emergent property of layers in, in this kind of prompting, which is there's

00:50:30.560 | the UI layer.

00:50:31.600 | And then there's the state chart layer.

00:50:33.120 | Um, and those two things seem like pretty useful in, in, uh, spec specifying a prompt.

00:50:39.600 | I was just wondering if you've, if you thought a little bit about like other dimensions or

00:50:43.200 | other layers that would be useful in multi-modal prompt.

00:50:47.200 | Yeah.

00:50:47.840 | So, uh, one thing that we've done is to bring in screenshots of other apps, like here's

00:50:57.280 | stripe.com, like make it look like stripe, you know?

00:50:59.680 | So that's sort of like, like, or like here's linear.com.

00:51:04.720 | Like let's, let's do it this way, uh, you should just, you should just like give a design

00:51:14.160 | and ask you to make pop instead of make real.

00:51:16.000 | Yeah, exactly.

00:51:17.440 | Make it, make it more, make it more, make more pop.

00:51:20.880 | Uh, so there, there's the idea of like bringing in style as like a, as another, um, another

00:51:29.200 | in like part of the input, uh, flow charts are absolutely useful.

00:51:34.240 | Uh, I mean, this is, it really just boils down to like, what would you really give a

00:51:38.080 | developer who you are working completely asynchronous with?

00:51:41.200 | You know, if you had to, uh, spec out a project and put that printed out on paper and mail

00:51:51.200 | it to a developer and they were going to mail back a disc with an HTML file on it, like

00:51:57.120 | what would you send, uh, if you were sending this to the moon or something?

00:52:00.720 | Uh, uh, so yeah, definitely like descriptions of how the state should operate and specs

00:52:07.040 | on that.

00:52:07.440 | Uh, we've, um, what else have we done?

00:52:11.200 | We've done flow charts, we've done screenshots of other apps.

00:52:16.160 | We've, uh, I think we've, we've even just pasted in code, like, like here's a whole

00:52:24.320 | bunch of Jason that you can use, um, and have it just read that as the, as the input data.

00:52:31.280 | Uh, we can, you can point it at specific endpoints.

00:52:34.960 | You can say like, I want you to hit this endpoint and then display the results, you know, as,

00:52:40.560 | uh, as cards or as items or something like that.

00:52:43.680 | Uh, not, I mean, you don't even have to like wire this up.

00:52:47.120 | It's not like retool or anything where you, you have to register that or, you know, it's

00:52:51.360 | not built into the tool.

00:52:52.400 | You're just from an endpoint.

00:52:53.760 | Oh yeah, yeah, yeah, yeah.

00:52:55.200 | I'm trying to think of, uh, what a good demo endpoint would be.

00:52:59.040 | We could, maybe we could do one more, uh, more test.

00:53:01.600 | What is it like dog.co?

00:53:03.440 | Is that?

00:53:04.240 | Yeah, it's a good demo.

00:53:06.160 | Yeah.

00:53:06.560 | Yeah.

00:53:07.040 | I've used that one.

00:53:07.680 | Um, I mean, this, this might be, this might be kind of like the box with the word calculator.

00:53:13.280 | Like it might just know because it's probably been a bunch of tutorials.

00:53:16.800 | Oh, it's in the beginning set.

00:53:18.240 | Yeah.

00:53:18.740 | Yeah, but, but, uh, you know what, we'll do it anyway.

00:53:21.680 | We'll, uh, I'll, I'll share it and, uh, we'll, we'll try.

00:53:25.840 | So for those who don't know, dog.co is, is one of those like, uh, demo APIs that, uh,

00:53:31.600 | you just set up just because it's not offensive.

00:53:34.080 | And, uh, yeah, exactly.

00:53:37.520 | You can, you can get dog.co.

00:53:39.360 | Yeah.

00:53:40.400 | Um, I'll try and think if there's a, there's a more complicated endpoint that we could

00:53:44.960 | hit.

00:53:45.600 | Uh, maybe, maybe while I'm doing this, if you want, it gives me more ideas.

00:53:48.400 | Yeah.

00:53:49.920 | It gives me more ideas of what to promise and what, what can go in.

00:53:53.200 | Uh, I definitely didn't think about hitting endpoints just because I, it's just not in

00:53:57.840 | any of the demos I've seen.

00:53:58.800 | Yeah.

00:54:00.320 | Uh, but it works.

00:54:02.400 | Um, let me see.

00:54:04.160 | I'll, I'll have a big button down here.

00:54:06.000 | Uh, show me a dog.

00:54:09.680 | Okay.

00:54:12.160 | So that's going to be our, show me a dog button.

00:54:14.400 | Uh, this should be a picture of a dog.

00:54:18.480 | Oh, that's a great dog.

00:54:23.280 | No, thank you.

00:54:24.160 | Oh yeah, yeah.

00:54:26.160 | Yeah.

00:54:27.680 | Uh, and, um, and then we'll, we'll do some annotations here.

00:54:33.920 | We'll say like, uh, uh, okay.

00:54:37.520 | When, when this is clicked, get a new dog.

00:54:42.240 | Um, yeah, there's those perfect arrows coming in.

00:54:47.040 | Yeah, exactly.

00:54:49.440 | When clicked, get a new dog from dog from, I'll just paste in this, um, and put the result

00:54:58.080 | in the image.

00:54:58.880 | Okay.

00:55:01.680 | So it's more of a, more of an instruction than you would normally, you know, I don't

00:55:07.200 | know.

00:55:07.360 | Yeah.

00:55:08.720 | One thing that it's going to have to guess is, uh, you know, the, the response format,

00:55:12.560 | right?

00:55:12.800 | Cause it could be anything.

00:55:14.160 | This is true.

00:55:16.160 | Let's see if it works.

00:55:19.120 | Well, Hey, all right.

00:55:20.720 | Yeah.

00:55:22.240 | And let's see if it, if it actually hit it, hit the, uh, the right, um, the end point

00:55:29.760 | in the right way.

00:55:30.400 | So, um, dog button.

00:55:33.920 | Yeah.

00:55:34.960 | Okay.

00:55:35.280 | It hit the right red endpoint, Jason dog image, uh, and then put it in.

00:55:40.240 | So there you go.

00:55:41.600 | Uh, you have yourself a, uh, JavaScript tutorial in a box ready to go.

00:55:48.560 | And I think like, I mean, we probably wouldn't do this on camera, but like, you can say,

00:55:54.240 | you know, like, like use the auth token, you know, whatever.

00:55:58.480 | And like, you know, go like really get, you know, real data back from this thing.

00:56:04.800 | Um, yeah, there's no reason why it wouldn't be able to do that.

00:56:08.080 | Uh, so yeah, it's really the only, yeah, yeah.

00:56:14.080 | Well, not really, because again, what inside of the prompt for this, we do give it like

00:56:21.440 | an array of all the texts that you've put in.

00:56:24.400 | We say like, look, I know your vision isn't so good, or you have a hard time reading texts

00:56:30.480 | sometimes when it's small.

00:56:31.600 | Uh, cause what are like the input that, that you get is pretty wild.

00:56:35.920 | It's like, it takes this as a PNG.

00:56:39.120 | And then it like, um, I can't do this in teal draw, but it, it resizes it.

00:56:44.240 | It squishes it into a 512 by 512 image or something like this.

00:56:49.360 | And tiles it, yeah.

00:56:50.320 | And then, yeah, so it's like, um, it, the text especially can get kind of like chunked

00:56:57.440 | up, especially if it's small.

00:56:58.800 | So we, we send those strings separately, uh, so that it can kind of reassemble, um, anything

00:57:05.600 | that it can't read, uh, right off the bat.

00:57:08.560 | Yeah, it's, it's a weird future that we've, uh, we've found ourselves in pretty cool.

00:57:15.360 | It's pretty cool.

00:57:17.680 | Yeah.

00:57:17.840 | I mean, you know what, one layer I automatically think of is, is backend, right?

00:57:21.520 | Like, uh, as someone who has, uh, works at AWS, um, I see a lot of systems diagrams,

00:57:28.640 | like cloud diagrams, um, entity relationship diagrams for database schema.

00:57:32.320 | So I wonder if like anyone's tackled extended in this to backend.

00:57:36.800 | And then obviously the next level from that is, uh, full stack apps where you have backend

00:57:40.800 | in front of it.

00:57:41.360 | Yeah.

00:57:41.920 | Um, short answer is yes.

00:57:44.560 | There's someone on Twitter that was using this to generate, um, yeah, like doing like

00:57:50.960 | flow charts.

00:57:51.760 | I'm not a backend guy, so I don't actually know exactly what the output was.

00:57:55.760 | Um, but I, I believe it was like a, like a configuration script for AWS, um, that was

00:58:03.440 | built off of this.

00:58:04.400 | Like, I think he just copy and pasted, uh, a diagram that he had made in teal dry anyway,

00:58:10.560 | and said, okay, well, let's throw this at this thing and see what it comes up with tweaking

00:58:14.800 | the prompt to say like you, um, rather than building single page websites, you just return

00:58:21.520 | the Jason, you know, description of this configuration or something like that, or return a script

00:58:26.480 | that would set this up.

00:58:28.560 | Um, you could tweak it to say like, here are like, yeah, all the entity relationships

00:58:34.320 | between, um, uh, different tables or items and tables, uh, and, and give me the, give

00:58:40.880 | me back the SQL, you know, like, you know, initialization or something that said that

00:58:45.600 | would make all these tables and, um, and set up these relationships.

00:58:49.440 | Uh, yeah, it's just, again, like the hard part is getting that information in, but we

00:58:57.760 | I don't know, pictures are really good.

00:59:00.480 | And yeah, if you can tell it that way, that works.

00:59:06.720 | Yeah.

00:59:08.000 | Yeah.

00:59:08.500 | Awesome.

00:59:09.760 | Um, you are also part of, uh, so you were one of two, one on what I think about, uh,

00:59:16.000 | multimodal, um, viral hits in November.

00:59:19.360 | Uh, the other one also, you had to play a part to play in it, which is the local consistency

00:59:23.360 | models, uh, trends, uh, where I think you worked with Phal.

00:59:27.840 | Yeah.

00:59:29.360 | So we, uh, and actually I do have something to show here.

00:59:32.800 | We, we actually have a couple of things to show here.

00:59:35.120 | Uh, we connected with Phal because they used, um, TealDRAW to, to create a demo for, uh,

00:59:42.800 | for their, their, uh, LC, LCM, right?

00:59:48.400 | Yeah.

00:59:49.600 | Latency consistency.

00:59:50.480 | Yeah.

00:59:50.980 | Um, but we took that and we, we made a draw fast, uh, TealDRAW.com, which is, uh, basically

01:00:01.360 | you get a, these like shapes, these little draw fast shapes.

01:00:06.000 | Um, and it puts the result basically grabs that new image and puts it right next to it.

01:00:11.440 | And these are, um, these are extremely fast.

01:00:16.320 | So as I'm moving things, you should see the image, um, updating as well.

01:00:21.200 | Uh, and I think it, I think this was originally not a goat, you know, whatever, this is a,

01:00:28.080 | um, uh, uh, uh, wise princess.

01:00:35.040 | I don't know.

01:00:36.000 | I play this more with my, uh, my daughter than anything else.

01:00:39.760 | I like what this looks like, you know, and, um, oh my gosh, she, she does.

01:00:45.920 | And actually we had a lot of, uh, a lot of folks on, uh, Twitter, um, being like, this

01:00:54.000 | is, this is not good.

01:00:55.200 | Like, whatever.

01:00:55.840 | Like, cause I had a video of whatever my, my daughter drawing.

01:00:58.640 | And, you know, she made this awesome drawing of a mermaid and we turned it into like this

01:01:03.760 | really anonymous crappy version of like an illustration of a mermaid.

01:01:08.880 | And, uh, they're like, no, no, the children's drawing is much more interesting.

01:01:12.400 | And I'm like, yeah, yeah, yeah.

01:01:13.280 | Come on.

01:01:13.920 | Who cares?

01:01:14.640 | Of course it is.

01:01:15.600 | But I, this is, uh, you know, this is, this is fun.

01:01:18.960 | So yeah, I think it's pretty wild.

01:01:22.320 | Animations, like some kind of, like you could make, uh, some kind of, it's, it's, this is

01:01:28.160 | almost like stop motion film.

01:01:29.760 | Um, yeah, yeah.

01:01:31.440 | I mean, we need to do more work on consistency, but like, this is getting there.

01:01:37.920 | Yeah, uh, it is a car driving through a busy market marketplace.

01:01:45.920 | Uh, the fun is that like, you end up, uh, after playing this with this for a little

01:01:51.120 | while, you end up like, uh, getting really into the particularities of the input.

01:01:56.000 | Like the, uh, like what can you do with a design tool?

01:01:59.360 | Okay.

01:01:59.600 | You can move things around, right.

01:02:01.040 | I can grab some of these and move them around.

01:02:02.720 | Um, and like, oh yeah, there's a highlighter here too.

01:02:06.960 | So we could like, we could do some highlighting, you know, that'll, that'll do stuff.

01:02:10.640 | Uh, and then you kind of like, we, we couldn't help ourselves.

01:02:15.680 | We started making these like stories.

01:02:17.520 | So, uh, one thing you can do is kind of click this little button and that'll make the, the

01:02:22.720 | drawing and the result kind of on top of each other.

01:02:26.080 | All right, well then I'll move on to the other one that we, uh, that we released earlier

01:02:30.480 | today, which is called lens.taildraw.com.

01:02:33.360 | So that was drawfast.taildraw.com.

01:02:35.520 | Uh, and again, this is probably not making a good podcast audio, but it is, uh, the image

01:02:42.720 | updates as soon as possible based on what the input drawing is.

01:02:46.880 | Um, and it is pretty hypnotic.

01:02:49.600 | So the, uh, this one's a little riskier cause it's live.

01:02:53.520 | So this, this is, uh, we took a project called together, which is a vertically scrolling

01:02:59.680 | infinite drawing collaborative experience.

01:03:02.800 | Uh, a little bit like a chat room as you're drawing, everything's just sort of moving

01:03:06.400 | up and it just disappears off the top of the screen.

01:03:08.640 | Never to be seen again.

01:03:09.600 | So it's kind of just fun to play with.

01:03:12.720 | Um, by the way, one of the most magical chat experiences I ever had was with you.

01:03:17.440 | Uh, I think you were like with your daughter or something and I was, I was, whatever, showing

01:03:21.520 | off together and you started writing, I started writing and we had chat, uh, on together.

01:03:25.840 | Yeah, I was like, what is this?

01:03:29.280 | It's super cool.

01:03:31.920 | Like, you know, inevitably someone will write like, you know, where are you from?

01:03:35.520 | And, uh, and everyone's like chiming in and talking about, yeah, you are like, this is

01:03:40.720 | cool where, you know, yeah, you can chat.

01:03:42.880 | Um, so what I'll describe what's on the screen now, which is basically we've, we're taking

01:03:52.000 | like a screenshot of the center, uh, like a square out of the center of this chaotic

01:03:58.160 | vertically scrolling chat experience, and we're sending that to the, to the LCM and

01:04:03.280 | putting back the image based on like a prompt, uh, like, you know, desert scene or busy marketplace

01:04:09.680 | or, uh, futuristic cityscape or something like that.

01:04:13.200 | And so it is updating like, you know, 10 times a second as, uh, as we go, but yeah,

01:04:19.280 | I would say like it's updating surprisingly quickly, like 10 times, 10 frames per second.

01:04:23.600 | Yeah, here we go.

01:04:25.360 | Yeah, yeah, uh, no, I think it's now like to 32 milliseconds basically, uh, as you go.

01:04:32.240 | And, um, so if I draw like a big orange thing down here, it's going to kind of show up into

01:04:39.760 | the drawing.

01:04:40.320 | Um, maybe I'll do a big black one so you can see better.

01:04:44.080 | Uh, like it just sort of becomes part of the, the input to this prompt.

01:04:49.200 | Um, and it is extremely hypnotic.

01:04:54.000 | Uh, this is again, like lens.tealdraw.com.

01:04:57.600 | Um, this isn't live, so no one should come on here and say hello, Steve, or anything

01:05:02.400 | like that, but it is, uh, it is, yeah, it's like this like slow moving, um, collaborative

01:05:11.600 | kind of like hallucination experience and it just never ends.

01:05:15.920 | I mean, yeah, I'm probably going to be funding fell completely for the next, uh, you know,

01:05:22.320 | their series A or something like that.

01:05:24.960 | But the, uh, yeah.

01:05:27.440 | Are you on here, uh, Sean?

01:05:29.200 | I'm not, but maybe I should.

01:05:31.600 | Oh, yeah.

01:05:31.920 | Well, whatever.

01:05:32.400 | Yeah, I, I, I have like, I don't know.

01:05:38.160 | I have a healthy respect for like the amount of processing that must be going on behind

01:05:42.320 | these things.

01:05:43.360 | Um, I just, yeah.

01:05:47.840 | Well, what's funny is that, uh, oh, cool.

01:05:52.080 | Perfect.

01:05:52.560 | Someone's doing that.

01:05:53.280 | It looks like I can't really draw the way I normally draw on Teal Draw.

01:05:56.080 | You, you blobbed it somehow.

01:05:57.360 | Yeah, it's, everything's a little bit bigger.

01:06:00.480 | Um, when there's 36 people, uh, it's, it's kind of slow as well.

01:06:06.000 | So, uh, yeah, we, um, what's funny is that like, yeah.

01:06:13.280 | We're using the Cloudflare workers to do the, uh, the, the, the updates and the CRDTs to

01:06:19.200 | do the collaboration and, you know, all this, like whatever LCM models to, to, to, to populate

01:06:26.240 | this image or create this image.

01:06:27.520 | Yeah.

01:06:27.920 | But, um, there's also a laptop in my living room right now, uh, that is doing the actual

01:06:34.080 | screenshotting, uh, and sending that up.

01:06:36.640 | And so there's a big note that I had to write, you know, for my family to say, like, don't

01:06:41.440 | turn off this laptop, don't close this laptop because, uh, this needs to be on in order

01:06:45.600 | for this thing to work.

01:06:46.480 | Uh, and yeah, so, you know, we'll, no matter how good our tech stack gets, we'll always

01:06:52.800 | come back to, uh, some, some laptops stuck in the corner that can't possibly be turned

01:06:59.280 | off.

01:06:59.520 | Uh, it's pretty fun.

01:07:01.760 | Yeah.

01:07:02.000 | I've heard of major businesses being run that way.

01:07:03.920 | Yeah, exactly.

01:07:05.200 | There's a Raspberry Pi in the closet.

01:07:06.960 | Yeah, I, you know, it's weird because it's really funny because like, you know, you,

01:07:11.280 | you are inventing your own art form.

01:07:13.120 | Uh, this is a fine art, you know, going back to your degree.

01:07:16.800 | Um, it's just a definitely a, uh, you know, it's funny because like the output of this,

01:07:26.080 | like, while it is like a visual output, the output like doesn't actually matter.

01:07:29.920 | Like it's gone in 16 milliseconds and it's not coming back.

01:07:33.680 | Uh, and I think with all this AI stuff right now, just where we are with it and just how

01:07:39.040 | completely unknown it is in terms of like, where is this useful?

01:07:42.960 | Uh, like the best thing that you can get out of this is like the experience.

01:07:48.800 | Uh, and so I think of this much more as like, you know, the thing that people will walk

01:07:53.200 | away from, from playing with like lens.tiltro.com, uh, should be more of like that experience

01:08:01.600 | of, of having interacted with this thing or interacted with it, you know, among, with

01:08:05.840 | others, uh, rather than like, oh, there's, it made my favorite image or something like

01:08:11.600 | that.

01:08:12.320 | So I think the, uh, yeah, I don't know, as a, as a former image maker, like the idea

01:08:20.080 | of having, having like an aesthetic experience where the image is, uh, a major part, but

01:08:27.600 | it's, it's not necessarily like the, the important part, uh, or any one of these images

01:08:32.800 | isn't the important part.

01:08:33.600 | I don't know.

01:08:34.160 | There's, there's some, like, there's some, uh, there's something, something new feeling

01:08:41.440 | about this kind of fun.

01:08:42.960 | Certainly I wish I was, uh, could do a big critique with all the new media artists, people,

01:08:51.440 | um, about this and about like, what, you know, where does this, where does this fit into

01:08:56.640 | the, sort of the, uh, yeah.

01:08:59.280 | Yeah.

01:08:59.840 | Like other people.

01:09:00.480 | Well, that's, that's for them to write and, you know, for you to build.

01:09:03.760 | Uh, and I think that's a, that's a, yeah, you know, I would encourage you to keep building

01:09:07.840 | there because, uh, you're definitely not done exploring with your explorations.

01:09:11.680 | Um, thank you.

01:09:13.440 | Cool.

01:09:15.200 | Yeah.

01:09:15.680 | Well, so I can kind of round it out by sort of looking towards the future.

01:09:19.520 | Uh, you hinted a little bit, uh, you're working towards TL draw 2.0.

01:09:24.240 | Um, so first of all, actually, so, uh, it seems like you're very focused on the core

01:09:30.560 | mission of canvas.

01:09:31.520 | Um, and the AI stuff is, is a side project for now.

01:09:36.400 | Um, why, why not pursue it as like a full, why not pivot and like be an AI company?

01:09:42.080 | Right.

01:09:42.160 | Like that's, yeah, I guess a lot of those questions.

01:09:44.800 | Yeah.

01:09:46.240 | I mean, when you, when you get something as viral as, as Tiltraw got, like, I think I've

01:09:50.960 | talked to everyone, um, certainly every, every investor, uh, and, um, okay.

01:09:59.600 | So I guess the way that, the way that, yes, we, we probably could on for something like

01:10:04.800 | together or that draw fast thing, uh, make a tiny little SaaS app, you know, give me

01:10:11.360 | $10 a month, play with this thing.

01:10:12.800 | And, uh, you know, it could make it, make it good.

01:10:16.960 | We could go in that direction.

01:10:18.640 | Um, there's not much of a moat around any of this stuff.

01:10:25.760 | And we're seeing that just in, you know, I don't know, Gemini is going to come out in

01:10:30.400 | a couple of days, weeks or whatever.

01:10:32.960 | Um, and if it's better than people are just going to use that until the next better thing

01:10:38.400 | comes along.

01:10:38.880 | Like, it's not, there's not a lot of like unique, uh, it's not, there's nothing really

01:10:45.280 | defensible about like, Hey, it's an, it's a drawing app plus an LCM like model, uh,

01:10:49.760 | because there's going to be a lot of those models and there's going to be a lot of drawing

01:10:52.560 | apps.

01:10:53.360 | The, the thing that I think is really unique for teal draw, the thing that we have added

01:10:59.600 | that is not easily created is the canvas itself.

01:11:04.080 | Is that like this, uh, you know, web-based, uh, hackable, extendable, uh, super refined,

01:11:13.120 | um, interactions and, and all that stuff.

01:11:16.080 | Like, you know, all the thousand table stakes features that drive people nuts when building

01:11:21.520 | something like this, like they're all there, they're all good.

01:11:24.400 | Day one, you could build a really great experience, uh, whether it's AI driven or not, like using

01:11:29.360 | teal draw, um, in a way that it's just not practical to do if you're building it yourself.

01:11:34.240 | And especially if you're not doing like graphic stuff, there's really not that much else out

01:11:38.000 | there, uh, oriented towards, towards this type of thing.

01:11:41.280 | Um, and I think in a world where like these types of experiences are going to, or not

01:11:47.920 | these experiences, these types of AI driven capabilities are just going to keep coming

01:11:52.400 | out faster and faster.

01:11:53.600 | And like, you know, I don't know, next, next year is going to be wild.

01:11:58.080 | Like every month there's going to be some new, uh, you know, capability or something.

01:12:02.800 | The, the thing that I would want to see both just me as a, as a person and as me as having

01:12:12.160 | built the business that I've built, uh, is for teal draw to sort of become the, the place

01:12:20.400 | where some of this prompting, some of these ideas are explored.

01:12:23.280 | Um, even if we decided to, okay, we're just going to close everything up.

01:12:28.080 | We're going to build a product, but based on this, and maybe it's a great product, but

01:12:31.920 | it's, it would only be one kind of one direction, one ray kind of into this infinite space of,

01:12:39.280 | of, of possibility.

01:12:40.400 | Uh, and that, that could be successful, good, but like the idea of building the, um, I mean,

01:12:49.200 | we've built the, the sort of the direct manipulation manipulation core, but there are so many,

01:12:53.440 | even like AI specific APIs that we could build around teal draw for having, you know, like

01:12:59.040 | a virtual collaborator, uh, or, or, or working with images in a, in a more, more rich way.

01:13:04.560 | Like there's just so much that we could build in order to make this the best possible place

01:13:09.680 | to explore, not just one direction, but like, you know, many, many, many directions.

01:13:14.480 | Um, and I think that, that narrative gets me out of bed in the morning in a much more,

01:13:20.160 | uh, that, that gets me much more excited.

01:13:23.360 | Um, and I think we're also just like the team that we have and the, the, the tech that we

01:13:27.920 | have and the skills that we have, we're more of the team to build that rather than like,

01:13:32.880 | to, um, become like a SAS product, uh, company.

01:13:37.120 | I'm not saying we'll never do like a, you know, a past 10 bucks a month and we can,

01:13:42.080 | you can play with our magic toy.

01:13:43.920 | Um, but, uh, primarily my goal is to make, uh, to draw the, either the place to explore

01:13:53.200 | these different models, or you might, you might think of it as like, um, the battleground

01:13:58.480 | on which the winners will be kind of identified.

01:14:01.920 | Um, like right now we're using open AI for the make real thing.

01:14:05.440 | Um, maybe next week we'll be using a Gemini and now it's, now it's a question of, okay,

01:14:11.840 | well now we have an environment in which to compare these two models, um, with the same

01:14:16.320 | input and a very advanced form of input.

01:14:19.280 | Uh, but yeah, like, let's see which one does better now.

01:14:23.040 | Nothing would make me happier than to be at sort of like the, the, the battlefield, um,

01:14:27.280 | for multimodal prompting and multimodal, uh, uh, AI experience.

01:14:30.640 | I should also shout out Baklava as I want the open source, um, vision and, uh, multimodal

01:14:38.080 | model.

01:14:38.960 | Um, yeah, I mean, like, uh, so, so I fully understand you want to, you want to own the

01:14:45.040 | light cone of multimodal prompting.

01:14:47.200 | Uh, I think that that'll probably be the title of the episode.

01:14:49.840 | Um, what's, uh, what's coming up for TealDRAW 2.0.

01:14:53.440 | So really the TealDRAW that you are using now and that I'm using are, is basically 2.0.

01:15:01.040 | It's been in pre-release for a long time.

01:15:03.200 | Uh, really the only change that's going to happen once we launch it is, uh, we're going

01:15:06.880 | to start selling commercial licenses for it.

01:15:09.120 | Uh, so if you are using TealDRAW in a commercial product or if you want to, then, um, you know,

01:15:14.640 | if it's, if you're funded or if you have revenue, then, you know, you'll buy a license and I'll,

01:15:20.640 | um, add you to our, our special list of customers.

01:15:24.720 | Um, so yeah, it's mostly just go to market and the necessary changes around there.

01:15:29.280 | There will be some kind of fun changes, secret saucy changes that, that launch, but nothing

01:15:34.160 | substantial, nothing breaking.

01:15:36.160 | Uh, we've put a lot of effort in the last, like, it's crazy that we've only had an open

01:15:41.840 | source since May of this year.

01:15:43.680 | Uh, this, this new version, right.

01:15:46.080 | And yeah, it's, we've been very busy since then, but it is, uh, it's stable.

01:15:51.040 | It's robust.

01:15:51.600 | We put it through, you know, a lot of usage and caught a lot of the issues.

01:15:58.400 | So it's absolutely ready to go.

01:16:00.160 | Um, but I have, uh, one or two conversations with my lawyer before we, uh, we turn, turn

01:16:06.400 | over the license and start, start moving it that way.

01:16:08.400 | Gotcha.

01:16:10.400 | Um, and then, uh, maybe, uh, I think if I could get your commentary, uh, before we close

01:16:14.960 | on, on, um, just the, the competition out there, like, um, uh, so you are not, you are

01:16:20.080 | not the only sort of canvas tool.

01:16:22.000 | I, I think I get now that I was going to ask about like Figma, FigJam, and they have some

01:16:26.720 | AI thing that they're also doing.

01:16:28.640 | Adobe is also working on similar things.

01:16:30.960 | Canvas also working on similar things, but they're all individual point solutions.

01:16:34.000 | Whereas you're more of the open source, uh, uh, canvas to power all of them.

01:16:39.920 | Um, yeah, I feel like it's just Excalidraw.

01:16:42.560 | That's like the other alternative, uh, that remains.

01:16:46.480 | I think Excalidraw, and I like Excalidraw a lot.

01:16:50.160 | I, I contributed there and, you know, we, we retweet each other and tease each other

01:16:54.000 | on Twitter and stuff.

01:16:54.880 | And, uh, early on I was copying features from them.

01:16:59.760 | Now they're copying features from me.

01:17:01.680 | So I, uh, but no, it's, uh, it's like the collaboration space is so, has so many dominant

01:17:10.800 | players like, um, that I, uh, I think me and Excalidraw are tiny within that.

01:17:18.240 | Um, yeah, well, I, I, there's two things.

01:17:22.720 | One is that we made this very strange bet on using a kind of a web canvas that our canvas

01:17:31.360 | is not like an HTML element or HTML canvas element.

01:17:34.800 | It's like normal React components all the way down.

01:17:37.520 | So if you wanted to add something interactive and have that participate in the sort of space

01:17:42.480 | of the canvas, uh, the way that we were doing our, our, um, iframes and kind of like being

01:17:48.880 | able to write on top of an iframe, you can't do that in Excalidraw.

01:17:52.880 | You can't do that anywhere.

01:17:54.560 | Um, that is like a very strange tech choice that we made around, uh, Tealdraw that is,

01:18:01.040 | you know, finding its home in, uh, in a few different ways.

01:18:04.080 | Most of the people who pick Tealdraw and approach me like the inbound that I get are folks for

01:18:10.080 | whom that's like the killer feature, um, to be able to, to put interactive widgets on

01:18:14.480 | the canvas using just React.

01:18:17.120 | Uh, the, yeah, I, I was kind of repeat the same point that you, you, you kind of suggested,

01:18:25.200 | which has said like, no matter how good Figma's, uh, like AI solution is, and I hope it's great

01:18:32.960 | because I love Figma and I use it.

01:18:34.720 | Um, it's not going to solve every possible problem in this space.

01:18:38.960 | It's not even going to like touch, you know, like, like you can't like none of these things.

01:18:43.360 | And I mean, I already had identified like, okay, uh, there was a point where like any

01:18:49.680 | Kanban board was like, uh, uh, was Trello, right?

01:18:53.440 | When you, when you talked about Kanban boards, you were talking about Trello.

01:18:56.480 | Kanban boards are in every productivity app now.

01:18:59.360 | Uh, I think the same thing is going to happen with collaborative whiteboards.

01:19:02.400 | It's like people like them.

01:19:03.360 | Uh, I'm making it easy.

01:19:05.440 | Uh, people are already doing it even without Tealdraw when it's hard.

01:19:10.480 | Like, uh, like, yeah, that's going to become a kind of a commodity user experience and

01:19:14.960 | a lot of different products.

01:19:15.920 | Um, probably, you know, give me a diagram from a text prompt.

01:19:22.480 | Like, yeah, that is probably going to be a commodity too.

01:19:25.600 | Give me an image from a text prompt.

01:19:26.960 | Like, yeah, that's just going to be everywhere.

01:19:28.320 | We're just going to assume that that's, you know, it's like adding a gift to a, to a chat

01:19:32.240 | or something that there's no moat there.

01:19:34.400 | Um, I do hope that Figma has an amazing AI integration, but I think the thing that it

01:19:40.880 | will help you do is use Figma.

01:19:42.800 | Um, like, like generating an image won't be super useful, but like generating it, uh,

01:19:51.280 | extended and autocomplete this, uh, this, uh, design absolutely would be.

01:19:56.560 | And, and I hope that they launch something amazing there.

01:19:59.520 | Uh, but yeah, there's, uh, I, like I said, there's just a million different directions

01:20:05.920 | that this stuff could go in.

01:20:06.960 | Uh, the canvas is just like a input device.

01:20:11.840 | It's, it allows a certain type of user experience.

01:20:15.040 | Um, and that's certainly not limited to design.

01:20:17.680 | That's not limited to whiteboarding.

01:20:19.040 | It's not limited to collaboration or anything like that.

01:20:21.280 | So, um, yeah, my hope is that there are those like kind of 10,000 products that, that could

01:20:27.520 | be made with what we're making.

01:20:29.440 | Uh, yeah.

01:20:31.680 | Yeah.

01:20:31.840 | That's, that's a really, uh, great mission.

01:20:34.400 | And, uh, I, I see why you're so passionate about it.

01:20:36.400 | You're the right team for it.

01:20:37.600 | Um, okay.

01:20:38.560 | Uh, uh, you know, a couple of lightning round questions.

01:20:41.520 | Um, uh, one, which is like, if you had some AI capability that you would wish for that

01:20:48.800 | you don't have yet, what would it be?

01:20:50.400 | Oh, that's a really good question.

01:20:52.880 | Uh, helps people to do some research, I think probably related to, it's not quite a CRM,

01:21:04.240 | but like a human, uh, just normal relationship management.

01:21:08.240 | Um, this is something that I've never had a problem with until, uh, until I had a startup

01:21:14.160 | actually, where there's just a lot more people involved in my life.

01:21:17.680 | And it's, uh, um, it's hard to keep up with them all.

01:21:21.600 | And I think this is probably something that like an EA like kind of does of saying like,

01:21:26.000 | Hey, there's a birthday coming up or something like that.

01:21:28.400 | Um, but also just, you know, identifying opportunities to, to work together, uh, to,

01:21:33.440 | to connect or, you know, who's an expert on this thing that I'm working on.

01:21:38.160 | Like that doesn't always occur to me.

01:21:40.080 | Um, and I think there could be, I don't know.

01:21:45.200 | I think the, the, the value of your network, um, that even if you're good at that, you're

01:21:51.760 | probably only scratching the surface of like, you know, how, how you could be helping the

01:21:56.960 | people around you and how they could be helping you based on like the specific context of

01:22:01.120 | like what you're working on and the problems on your table today.

01:22:04.320 | Yeah.

01:22:05.920 | I've also wanted to build a CRM on top of Twitter, uh, because you have all the info

01:22:10.480 | there, but what people are working on your past conversations with each other, um, and

01:22:14.880 | your shared interests, you know, like it should be able to search, you should be like a bare

01:22:18.480 | minimum to search it, but to proactively suggest is, uh, the next layer.

01:22:22.880 | Um, and I guess AI chief of staff, AI executive assistant, you know, something like that.

01:22:30.160 | Yeah.

01:22:30.720 | Yeah.

01:22:30.960 | Yeah.

01:22:31.440 | Um, I think like some people are working on that, but the problem is so big that they're,

01:22:36.560 | they're working on like the automation piece.

01:22:39.200 | So like Lindy, I had at my conference, um, where they're, they're like, it's, it's, it's

01:22:44.080 | a virtual assistant that you can trigger on your desktop or via email.

01:22:49.040 | Um, and it's, it mostly deals with scheduling, um, but also it helps you do a little bit

01:22:53.120 | of research.

01:22:54.160 | Um, so that, yeah, I think the agents field will progress there.

01:22:57.760 | We might take 10 years to do it.

01:22:58.960 | Yeah, I can wait.

01:23:01.680 | It's all good.

01:23:02.160 | And then, and then finally, um, uh, invite your founders, like what, what has helped

01:23:07.040 | you the most, uh, as, as a founder, you know, you're two years into your journey.

01:23:10.720 | Yeah.

01:23:12.560 | I think the, uh, so this, this kind of comes a little bit out of, uh, like what you learn

01:23:21.040 | in art school, uh, type of thing.

01:23:23.040 | Um, but yeah, uh, but one thing is that, that basically like when you're a studio artist

01:23:30.800 | or you're in a studio or whatever, there's no external constraints.

01:23:33.520 | You just kind of are running on, well, what do I feel like working on?

01:23:36.880 | Uh, and the further you get away from like, what do I feel like working on kind of like

01:23:41.360 | the worse your work becomes.

01:23:43.040 | So having like a really good, uh, feeling for that sort of desire and being able to

01:23:48.960 | respect and follow that, that desire as like, uh, um, because it's not arbitrary.

01:23:55.840 | Is that like, if, if you really, really feel like working on, um, uh, on, on a thing, like

01:24:03.360 | it, that might be the sort of the, the tip of a very complex, uh, iceberg of analysis

01:24:11.280 | of like the field or like what people are talking about or, or something, uh, that you,

01:24:16.400 | directions and market or something like that.

01:24:18.720 | Like, I don't know, I think with, with Tealdraw and with, as, as a founder on this, um, the

01:24:25.040 | thing that I've tried to do and I've tried to preserve is like the, uh, being able to

01:24:31.200 | prioritize based on like, what is most interesting right now.

01:24:33.920 | Uh, and that is, that is true for, uh, what code we write and like what features we work

01:24:39.600 | on.

01:24:40.240 | That's true for like which partners we, you know, we spend time with in terms of who's

01:24:44.240 | using Tealdraw, uh, the types of problems that we want to solve, like using your own

01:24:50.240 | sort of sense of what's interesting as a filter, uh, and what you want to work on, like what

01:24:55.840 | sounds like a fun thing to work on right now as a filter is not, it's not naive.

01:25:00.560 | Um, and it can be kind of part of your, your secret sauce.

01:25:04.480 | And I think it's, uh, I think a lot of early founders are encouraged against that and to,

01:25:10.000 | to be working backwards from a certain outcome and all that.

01:25:13.120 | Um, and yeah, you, you do have to have to do that.

01:25:15.440 | You have to put that into the, um, into the mix as well, but, um, you know, be sure that

01:25:21.680 | you're, you're picking the best parts, um, out of that mix.

01:25:24.960 | I don't know the parts that you want to work on.

01:25:26.320 | Yeah.

01:25:28.080 | Well, I mean, what's the point of doing this if you don't have, uh, have some fun, indulge

01:25:32.000 | in curiosity.

01:25:32.720 | The worst case you'll, you'll build something that you love.

01:25:35.920 | Yeah, exactly.

01:25:38.240 | Good things can come out.

01:25:39.200 | Good things can absolutely come out of like, uh, you had an 8,000% increase in your followers

01:25:45.040 | or something.

01:25:45.440 | Should I put this on camera?

01:25:48.720 | I'll, I'll share my screen.

01:25:49.760 | We'll look at my, uh, my Twitter analytics.

01:25:51.440 | It's on your sub stack.

01:25:52.320 | Yeah.

01:25:52.820 | Oh yeah, it is.

01:25:54.480 | It is.

01:25:55.200 | I need to update that image.

01:25:56.240 | I've done it once already.

01:25:57.760 | Uh, yeah, if you, if you're a, if you're a sub stack reader, the teal draw sub stack 72

01:26:02.880 | hours into this big make real virality explosion, I, I sat down and wrote a blog post and I,

01:26:09.680 | uh, I wanted to at least capture that, that vibe, um, of what it felt like in the middle

01:26:14.640 | of that, that hurricane.

01:26:16.320 | Uh, yeah, so it's, it's a pretty fun one.

01:26:18.240 | It's good to read back.

01:26:19.920 | Uh, well, I'm sure it's not the last time we'll see you do something crazy viral.

01:26:25.360 | Uh, I'm sure that a lot of people will be exploring to draw.

01:26:27.840 | I hope a lot of people, honestly, um, one thing I'm thinking about is like embedding

01:26:32.640 | tail draw into my, my input box.

01:26:35.120 | Like why can't, I can't see our job be like, you know, part of the, the input.

01:26:39.680 | Hey, I'm, I'm talking to, uh, the, the good folks over at open AI tomorrow.

01:26:44.480 | Fingers crossed.

01:26:45.200 | Maybe we, uh, maybe we get it in, inside of a chat GPT or something.

01:26:49.600 | Cause yeah.

01:26:50.000 | Like, I don't know.

01:26:50.800 | I want to, I need to take a drawing or take a photo and then annotate it or like, you

01:27:01.840 | know, just sketch something out.

01:27:03.200 | You should be able to do this.

01:27:04.880 | Yeah, exactly.

01:27:05.920 | Yeah.

01:27:06.160 | It's just a good, it's just a good thing.

01:27:08.800 | Uh, yeah.

01:27:10.080 | The, the people cry out for it.

01:27:12.240 | I, uh, failed it fast enough.

01:27:14.800 | Yeah.

01:27:15.840 | Um, well, thank you for inspiring the rest of us.

01:27:19.040 | Uh, thank you for everything.

01:27:20.160 | And I'm sure we'll, we'll hear from more from you over the, over the next few years.

01:27:23.520 | Uh, so, uh, thanks.

01:27:25.120 | Thanks for your time.

01:27:26.240 | Awesome.

01:27:26.720 | Thank you for your time.

01:27:40.080 | Bye.

01:27:47.120 | [BLANK_AUDIO]

The Accidental AI Canvas - with Steve Ruiz of tldraw

Chapters