AI Engineering Without Borders — swyx
00:00:00.000 | I've been thinking a lot about borders recently for no particular reason and borders are a very
00:00:18.420 | innately human thing if I don't have the right piece of paper I cannot cross this line in the
00:00:23.320 | sand like it's a very very real problem that I face and many many people face every single day
00:00:27.640 | and borders are just one kind of constraint that humans just make up and I think that's very
00:00:33.360 | interesting that we respect borders so much but AI does not AI is a border disrespector it is very
00:00:38.040 | very easily multilingual so if you trained an LLM on mostly mostly English text corpus it's going to
00:00:44.360 | learn other languages just as a side effect it's going to be very natively multimodal because it
00:00:50.680 | can you can turn llama into a vision language model with just like 100 bucks of just post training
00:00:55.320 | there's it's very disrespecting of ground truth borders because it can just doesn't know the
00:01:01.360 | difference between hallucination and memorizing from a world model and it also doesn't respect
00:01:06.560 | copyright which is a whole other topic that we won't get into today but it's also super fascinating
00:01:10.340 | and how does that relate to do with AI engineering right like I think a lot of you here are here
00:01:15.400 | because you are interested in that concept at least maybe you identify as an AI engineer maybe you're
00:01:19.200 | trying to hire an AI engineer so there are a lot of definitions floating around and I confess that
00:01:25.220 | you know I've contributed to that is engineering an API line right that's the that's the line a lot of
00:01:30.600 | people have and that's come under some debate recently and yeah that's one form of AI engineering and I think
00:01:35.660 | that is useful to some people for understanding like where the responsibilities in a team might stop
00:01:40.280 | in and start with the other people in the team or maybe it's there's different subtypes right like
00:01:46.220 | last AI engineer summit I talked about the three types of AI engineer that I was seeing emerge the AI
00:01:51.760 | enhanced engineer the AI products engineer and the non-human agentic AI engineer or it could be a job
00:01:59.280 | description that you try to sort of list out and this is something that on the latent space podcast we
00:02:03.860 | recently went through with illicit talking about the different roles that they see within their teams
00:02:08.860 | as well so okay if I broadly have any of these three things do I have I nailed down a good definition of
00:02:15.360 | AI engineer that is workable yes right but is that something that we're happy with this is something
00:02:20.200 | that we can is there nothing left to explore I think the answer is no I think there's more to explore
00:02:24.580 | I think the very easy cop-out as well for people discussing this is that you have your opinion I have my
00:02:31.160 | opinion you come from your point of view I come from my point of view we agree to disagree or we agree
00:02:35.760 | that you know the different different strokes different folks and we move on I don't really
00:02:40.500 | like that just because there's no shared agreement on the things that is ground truth to everybody
00:02:45.960 | so I want to raise that challenge a little bit more I was in a podcast with Fraza Habib who's one of
00:02:52.340 | the speakers today talking about what this conference is and why this conference is does what it does
00:02:57.700 | and I always say that AI engineer conferences are effectively my highest stakes expression of what
00:03:03.220 | I think the state of AI engineering is so this time last year 2023 AI years are two times of human years
00:03:10.580 | we had a few tracks and we had a few topics that were up for debate we had RAG code gen and then agents
00:03:17.440 | and multi-modality all those tracks are repeated here today I have some speakers illustrated here just for
00:03:23.000 | illustrative purposes these obviously are not every not everyone involved but I think just like
00:03:28.520 | the inside-out metaphor that I've been thinking a lot about as the engineer matures so there's the
00:03:34.040 | number of concerns that you have to juggle in your head so this year you know after you're a competent
00:03:38.900 | AI engineer this year you're now faced with like okay I have to migrate to open models I have to build up
00:03:44.120 | my evals maybe I should have done that first that's a whole topic of discussion maybe I should scale up my
00:03:48.920 | inference or maybe I should deploy it to the fortune 500 and maybe on the management side of things I
00:03:54.120 | should be hiring teams of AI engineers and managing AI strategy for my company I think the last track
00:04:00.360 | you know like I talked about the nine tracks in in the engineers it's always about the network the
00:04:03.720 | community the network that we're building that's probably the single most important part of this
00:04:08.360 | conference and that's the part that we cannot sell we cannot I cannot put on the website hey we have
00:04:11.480 | good community because no one will believe us you have to come and see for yourself but please for those of you who've been uploading to the google photos album
00:04:17.320 | we've been tweeting out your photos and sharing them on LinkedIn please keep doing that that's a
00:04:22.520 | way for myself and everyone else who is not at the conference to try to join in on the fun
00:04:27.720 | the reason I'm not comfortable with any of these tracks because I is because I know how they were made
00:04:33.640 | because I made them up and I know that because I was looking at the the original document for the rise of the
00:04:40.840 | AI engineer it's we're celebrating the one year anniversary today and just down in the document
00:04:47.160 | somewhere I just listed out the you know disciplines that I thought the engineer would have and those
00:04:50.760 | eventually became mostly mapping to the tracks that I have been exploring in these conferences and the
00:04:55.640 | meetups that I do and it's arbitrary like why is there a separate agents track from code gen why is
00:05:01.880 | there a separate rag track from open models like these are all related what you know they're all of a kind
00:05:07.720 | and obviously as a competent engineer you should be familiar with all these things and that brings us
00:05:13.480 | back to this mindset of having boundaries and borders right these are all made up by someone I made them up
00:05:19.640 | for this one but you know you're going to live in a world where your boss made it up at your company
00:05:23.640 | and these are not reflective of how reality actually has to operate right if you start with all the rules
00:05:31.160 | and a different group of people came in would they agree on the same rules probably not just because
00:05:35.640 | they're made up but the laws of nature are hard to make up because the reality actually works that way
00:05:40.680 | if you had an alien civilization come down to earth they would discover that gravity works the same
00:05:44.360 | the sun the energy from the sun works the same the magnetic field works the same so what are the
00:05:49.640 | laws of AI engineering we'll come we'll come to that via definitions of software engineering and real
00:05:54.920 | engineering so I went and looked up definitions of software engineering and IEEE talks about the application
00:06:00.200 | of engineering to software with the design implementation testing and documentation of
00:06:03.960 | software Google developers talks about mostly the same thing they also mentioned software life cycle
00:06:08.760 | management okay like really reasonable definitions of kind until you look at real engineers real
00:06:15.800 | engineers talk about applying natural science sciences utilizing them for benefit of humanity
00:06:21.560 | both the IEEE and the national association of engineers agree on that and it's curiously missing from
00:06:27.160 | software engineering like the benefiting humanity part the understanding natural laws part completely
00:06:31.480 | missing from software engineering so my proposal to you is that AI engineering is somewhere in between
00:06:35.400 | right the the the the software engineering but then encountering a lot of the more of the real world
00:06:40.280 | constraints than you would in a typical software engineer career so if we know the laws of earth and
00:06:45.880 | they are independently derived they cannot make no matter what point of view you're looking at they all
00:06:49.960 | are the same laws then what are the equivalent laws of AI engineering I have a few you can come up with more but I'm just going to propose some to start off the base
00:06:57.080 | there's constants so for example if you're designing for humans you should respect the fact that
00:07:03.000 | humans only speak at 80 words per minute but they read at 200 words per minute right so there's an inherent
00:07:08.360 | disparity there there's also constant contingent facts that things that are true for now instead of true
00:07:13.560 | forever and true for now facts are for example like the apple intelligence when they ship a local model on
00:07:19.480 | every phone then that inference speed of 30 tokens per second that they're advertising becomes the baseline speed limit
00:07:25.400 | a speed barrier of what intelligence that is too cheap to meter should look like
00:07:29.960 | and these things they're not set in stone they're not actual physical laws so they also trend over time
00:07:36.040 | due to forces and momentum and I want to establish a little bit like I think it's under it's very
00:07:41.400 | beneficial for AI engineers to understand what the Moore's laws of AI is so that you can plan for them so
00:07:46.120 | that you don't have to make the bad bets that are not going to last just obviously just because of overwhelming evidence
00:07:51.960 | the first bet is the improving of context right a year ago I was interviewing mosaic and talking about
00:07:59.080 | mpc7b with their whopping 60 000 70 000 token context with a lot of loss today sitting in the audience we
00:08:05.640 | have people who have trained million token context windows and we've also uh have from anthropic which
00:08:12.840 | just released cloud three five years last week um the the fact that you know that we have complete
00:08:17.160 | utilization it's not just about the length of context it's also about the utilization of context and I've been
00:08:21.000 | Greg who's sitting in the audience as well would be very happy with how Claude is improving on their
00:08:25.240 | utilization of their very very long context windows there's also the cost of intelligence the commodification
00:08:31.480 | of intelligence so in the past two years we've seen a 99.55 percent decline in the cost of GPT-3 level
00:08:37.880 | intelligence the cost of GPT-4 level intelligence has probably come down maybe 90 percent maybe maybe 80 percent
00:08:43.240 | 80 percent um from from GPT-4 to Lama 3 and now all the the newer models that are finally I think it's worth
00:08:49.240 | commenting a little bit on where AI engineering stands in contrast to other AI philosophies there's EA versus
00:08:54.680 | EAC and maybe we're in the middle like we we care about safety but we also want to accelerate right it's
00:08:59.640 | kind of like a weird combination of the two things my proposal is that that one dimension isn't enough to
00:09:04.680 | express how AI engineer differs from the other philosophies and actually need to add a second dimension to talk about
00:09:09.720 | utility we are utility maxis above all else we see what's out there and we want to use it to benefit
00:09:14.360 | humanity so my message to everyone at the world's fair is to try to disrespect borders a little bit
00:09:20.280 | try to avoid your own dogmatic beliefs lazy consensus of other people or passive reactions
00:09:24.680 | and in other words try to disagree disagree more disagree with your own conclusions disagree with
00:09:29.400 | each other productively and disagree with the status quo and I think there you'll find that this
00:09:34.680 | conference becomes more of a useful landmark in your careers rather than just the party which it can very
00:09:40.360 | well be but my final analogy which I really like is that AI engineers are the kind of person that looks
00:09:44.840 | at shoggoth and sees instead of a monster that cannot be tamed they want to turn them into
00:09:49.480 | mass rapid transit and the kind of person that looks at that looks at you know a force of nature and wants to
00:09:55.880 | turn it into tools that are useful for people is the kind of engineer that I would love to speak to and welcome
00:10:00.680 | at conferences like this one so that's my view of what borders and engineering without borders should look like
00:10:07.400 | I very much encourage you to jump between tracks to jump between friend groups to jump between disciplines
00:10:11.560 | and modalities because here's the one place that you can do that outside of your work and to mingle
00:10:16.440 | with everyone else that we've gathered so I hope you enjoy doing that I wish I was there in person but
00:10:20.840 | just share them online and I hope to see you in person at the next one bye