Structuring a modern AI team — Denys Linkov, Wisedocs
00:00:00.000 | -
00:00:02.000 | - All right, thanks everybody for joining today.
00:00:16.400 | My name is Dennis Linkov.
00:00:17.480 | I lead the machine learning team at WiseDocs
00:00:19.240 | and I'll be talking about hiring a modern AI team.
00:00:21.660 | So who's heard this message before?
00:00:25.440 | We are now an AI first company.
00:00:27.900 | We've seen companies like Shopify, Duolingo, Zapier,
00:00:30.820 | all make these announcements
00:00:31.940 | saying that they're AI first companies.
00:00:33.900 | And they're saying that there are new expectations
00:00:36.680 | that before you hire a person,
00:00:38.460 | you need to make the claim
00:00:40.300 | that you can't hire an AI agent or use AI.
00:00:42.860 | We're now seeing big tech companies
00:00:46.060 | and many companies in general
00:00:47.300 | sharing how much code is being written by AI systems
00:00:50.020 | and how this is going to lead
00:00:51.320 | to the extinction of the software engineer.
00:00:53.420 | So now, if you're in the position to hire people,
00:00:56.800 | you'll ask the question, now what?
00:00:59.000 | So we're going to talk about three main themes today.
00:01:02.480 | The first one is the anatomy of an AI team.
00:01:04.780 | The second is the evolution of a generalist.
00:01:06.720 | And the third is the question of hiring.
00:01:08.580 | So let's start off with the anatomy of a team.
00:01:12.200 | So there is a spectrum of different companies
00:01:15.460 | and this is where we should start.
00:01:16.780 | We have technology companies
00:01:18.020 | where technology is the core value proposition
00:01:20.360 | that is being offered.
00:01:21.580 | We know big tech companies, we know many startups.
00:01:24.160 | There's also verticalized solutions
00:01:25.700 | or services companies such as Palantir
00:01:27.940 | or the company I work at, WiseDocs.
00:01:29.560 | And there's also tech enabled companies
00:01:31.220 | where the core product is not technology,
00:01:32.920 | but it is something that benefits dramatically
00:01:35.220 | from good tech.
00:01:36.340 | You can think about banks, retailers,
00:01:38.340 | small and medium businesses.
00:01:40.700 | So who here is in each group?
00:01:41.900 | Who here is at a tech company?
00:01:43.380 | Who here is at a verticalized or services company?
00:01:47.260 | Who here is in tech enabled?
00:01:49.540 | Okay, pretty good mix.
00:01:51.260 | Now each of these different companies
00:01:53.820 | have different challenges.
00:01:55.400 | Typically in a technology company
00:01:56.840 | where we've seen blunders is the lack of domain knowledge
00:01:59.060 | when launching a product,
00:02:00.400 | there's usually some kind of business
00:02:01.700 | misalignment with technology.
00:02:03.620 | In the middle, either everything goes right
00:02:05.440 | or everything goes awfully poorly.
00:02:07.860 | And on the tech enabled side,
00:02:09.140 | there's usually some kind of tech challenge, right?
00:02:10.880 | Because tech is not your core value proposition.
00:02:13.320 | So you make different decisions based on this, right?
00:02:16.760 | You typically buy data or buy expertise
00:02:18.820 | if you're a tech company, right?
00:02:20.300 | You go to a vendor and say, give me label data.
00:02:23.280 | In the middle, you either have everything
00:02:25.480 | or you have nothing.
00:02:26.960 | And from a tech enabled company,
00:02:28.260 | you usually buy technology,
00:02:29.480 | either through a service provider
00:02:30.880 | or a true intent solution.
00:02:33.520 | Now I bring this up because every company,
00:02:38.100 | every organization and every person
00:02:39.920 | has a different perspective on the role of technology
00:02:41.960 | solving our world's problems.
00:02:43.460 | This is my stance.
00:02:45.220 | I think we have 90% of the technology
00:02:46.820 | to solve the problems of humanity.
00:02:49.780 | Now this might be a controversial perspective,
00:02:52.000 | but I'll show you why.
00:02:53.120 | The fax market still exists.
00:02:56.520 | Billions of dollars are spent on faxes.
00:03:00.740 | And the market is growing.
00:03:01.760 | In 2017, only 3% of payments in the US were contactless.
00:03:10.700 | That number is now higher,
00:03:12.680 | but we're still paying in archaic ways.
00:03:14.620 | Checks are still a massive part of the market.
00:03:16.660 | And it took 40 years after the introduction
00:03:20.540 | of personal computing of the internet
00:03:22.420 | for medical systems and electronic medical records
00:03:25.940 | to become digital.
00:03:27.360 | This number is much higher now,
00:03:28.580 | but it takes time for technology to be adopted.
00:03:31.640 | And many of you might have seen this in your industry,
00:03:33.620 | in your job as well,
00:03:34.460 | that technology is not the thing
00:03:35.600 | that's stopping you from achieving success.
00:03:37.540 | So this is the core question.
00:03:40.140 | Is technology the limitation of our success?
00:03:45.820 | And it's not about technology is how we use technology.
00:03:47.800 | And the way you build your team should reflect this
00:03:50.040 | by understanding the problems that you have.
00:03:51.940 | So going to this question of,
00:03:55.200 | do you need to hire an AI researcher?
00:03:57.440 | A lot of times when ChatGPT was coming out,
00:04:00.060 | every team is like, I need an AI engineer,
00:04:01.920 | I need an AI researcher,
00:04:03.300 | and it's not always smart to do so, right?
00:04:05.280 | Up until you hit a certain scale
00:04:06.660 | or a certain need of specialty,
00:04:08.460 | it does not make sense to hire an AI researcher
00:04:10.460 | to work on models.
00:04:11.860 | Free training models,
00:04:13.060 | even fine tuning models of a certain capacity
00:04:15.200 | is not necessarily the first thing that you need
00:04:16.880 | to achieve the value that you need to get.
00:04:19.460 | There's a lot of transformation work
00:04:20.520 | that goes in before that.
00:04:21.900 | Now in certain domains, the best tech is essential, right?
00:04:26.220 | So if you're working at OpenAI or a model provider,
00:04:28.920 | Anthropic, Google, or some of the startups,
00:04:30.840 | you want the best team who's working on that,
00:04:32.420 | 'cause that is a product as we covered.
00:04:34.280 | So here I'll propose a wager for you.
00:04:38.240 | Are people here familiar with Pascal's wager philosophy?
00:04:41.960 | I'll give you the successor of that Ampere's wager
00:04:44.040 | if you're familiar with graphics card architectures.
00:04:47.140 | Here's your trade.
00:04:49.860 | You trade your team for five researchers from the top labs.
00:04:53.580 | And maybe you need to throw in some cash
00:04:54.940 | and first round picks for that as well.
00:04:57.040 | But do you make this trade?
00:04:58.300 | Do you trade your team that has domain knowledge,
00:05:01.800 | has worked in the area for five AI researchers?
00:05:04.600 | I want you to think about that.
00:05:06.080 | So we go back to the question of what does an AI team need to do?
00:05:10.520 | There's a lot of stuff, right?
00:05:15.360 | Start off with defining use cases.
00:05:16.840 | We want to go through and integrate with products, right?
00:05:19.380 | We're not doing greenfield everywhere.
00:05:21.060 | We want to measure ROI, find the right data.
00:05:23.700 | We want to test and refine workflows,
00:05:25.740 | build the interfaces we need for success,
00:05:27.740 | sell this product, and make our customers care.
00:05:30.020 | And it's not one person who does this job.
00:05:34.500 | You can't just say, AI researchers,
00:05:36.460 | go make me $10 million from this product unless a very specific niche.
00:05:43.460 | So this means your success is not one job, unless you're a founder, but we'll skip that.
00:05:49.580 | So the goal here is that you need to have a comprehensive AI team,
00:05:52.180 | and you need to figure out how you're going to structure that.
00:05:54.180 | And the thing that we need to remember is that companies aren't just one team.
00:05:59.620 | It's not just my AI team owns this small segment, this deployment, or whatever.
00:06:04.260 | Otherwise, you ship your org chart, and you get some weird product behaviors.
00:06:08.980 | So identify to yourself, what is your bottleneck?
00:06:12.820 | What is stopping you from achieving success?
00:06:14.740 | Is it shipping features?
00:06:18.100 | Is it acquiring users?
00:06:20.140 | Is it retaining users?
00:06:21.260 | Are you monetizing correctly?
00:06:22.900 | Are there scalability issues?
00:06:24.900 | Are there reliability and observability issues?
00:06:28.460 | All of us have probably run into these things as we were deploying AI products,
00:06:32.260 | so we need to make sure we can prioritize all these things and hire accordingly.
00:06:35.860 | And these are all questions that you need to answer when building an AI team.
00:06:39.900 | The key takeaway here is, what kind of team do you need?
00:06:44.740 | And only you know that answer.
00:06:46.340 | Let's talk about generalists and why I think they're important.
00:06:50.060 | So in 2021, I was building a first machine learning team,
00:06:55.020 | and I adopted an approach where we hired generalists,
00:06:57.300 | and we supported them by automation across the board.
00:07:00.820 | So at the time, I was hired to a conversational AI company working on a platform.
00:07:07.140 | Sorry, let me rephrase that.
00:07:08.420 | AI agent building platform.
00:07:09.700 | I just wanted to make sure you guys understood what that meant.
00:07:12.980 | And I was hired with the mandate of we want ML.
00:07:16.020 | That was my job description.
00:07:17.700 | Change that.
00:07:19.420 | We want AI.
00:07:21.380 | So after working with the business teams and leadership team, this was the final set of goals we set.
00:07:26.660 | We want to serve hundreds of thousands of concurrent models.
00:07:29.140 | It needs to be multi-domain.
00:07:30.260 | It has to be low cost.
00:07:31.220 | And we want to support real-time training and serving.
00:07:33.140 | Those are some tough goals.
00:07:36.580 | So this is what we did.
00:07:37.460 | We wrote a custom ML Ops platform for deployments to match our requirements.
00:07:42.580 | We mainly fine-tuned encoder models.
00:07:44.660 | We built RAG as a service.
00:07:46.020 | And as a team, we own six microservices on 10.
00:07:50.980 | So the three areas I focused on building the team was model training, model serving, and business acumen.
00:07:56.900 | Now, you might say, I want top grades in all these things, but that's a lot of money, right?
00:08:03.540 | And as a team leader, as somebody who manages the budget, you don't have infinite money.
00:08:08.180 | So we have to pick along this axis, where do we want each of these skills to lie?
00:08:13.460 | For model training, we don't want somebody at the very bottom, but we don't need somebody who can train GPT-3.
00:08:19.860 | And basically, we went across and said, okay, what are the key requirements?
00:08:22.980 | For model training, we said somebody in the upper half who knows general architectures of models,
00:08:28.420 | can do encoder fine-tuning, does some data engineering, using hugging face is okay.
00:08:33.220 | That was the bar we set.
00:08:34.180 | On the model serving perspective, on the first round, I was the first engineer at the company.
00:08:39.300 | I spent a lot of time on building the ML platform.
00:08:41.220 | But that was something I was comfortable with coming from a cloud engineering background.
00:08:44.420 | Now, after that, there was enough abstraction built in that we didn't need somebody who knew
00:08:48.740 | the intricacies of how Kubernetes works and how we did serving or training, but the capability to use
00:08:55.380 | these abstractions and understand the trade-offs that were being made.
00:08:58.820 | And what I did focus on is the ability of our engineers to get on calls with customers, right?
00:09:04.660 | We didn't need a business development rep who would just cold call people for fun.
00:09:09.220 | But we need engineers who didn't say my job is coding in a basement.
00:09:12.740 | So we went through and understood these trade-offs that needed to happen.
00:09:17.300 | In 2024, I was building another team, the new organization that I joined, and similar approach,
00:09:25.620 | but open source had advanced. When I was building the original ML platform, we didn't have things like
00:09:30.260 | shadow deployments or A/B testing on a lot of the platforms that existed, and we had a specific use case.
00:09:34.820 | Now, since then, what's important to recognize is that all these skills that you're prioritizing
00:09:41.140 | don't necessarily need to be one person. They can be multiple people. You just have to find a way to
00:09:44.900 | make the team work. So once again, we set similar structures. And in this case, because open source
00:09:53.460 | had advanced in a number of different ways and commercial malls had advanced, some of the things
00:09:58.580 | shifted around. On the training side, using commercial APIs and prompt tuning and model fine tuning
00:10:06.100 | commercial models became important, but we also expanded our scope. We're now using decoder and
00:10:10.740 | encoder models, which each have their nuances. On the serving side, because we're using an open source
00:10:17.140 | offering, we didn't need to write our own platform, which is nice. And on the domain side, again, because
00:10:22.420 | of the nature of our business of doing medical record processing, there's a whole nuance of what
00:10:27.060 | that domain knowledge was. So that bar increased in a different way. So now that we know what kind
00:10:33.940 | of skills we need for our team, we can identify this threshold and balance the budget, right? We can't
00:10:38.660 | just ask for infinite money unless you're a specific subset of companies. You might have this question,
00:10:44.660 | what if I already have a team? I have 40 people, 100 people, what do I do? How do I reskill, upskill?
00:10:50.420 | How do I manage this team? So we need to figure out what the goal of the team is, as we were referring
00:10:55.700 | to. And I typically like to think about it through inner and outer loops. So inner loops are the daily
00:11:01.700 | activities that the team needs to accomplish together every day to be successful. And the outer loop is the
00:11:07.620 | broader set of activities that will set you apart. And you might not need constant interaction with that,
00:11:12.100 | but they're really important. So in my current team, this is how we typically structure it.
00:11:17.460 | So we have model training, prompting, product requirements, model serving, some domain experts,
00:11:23.380 | and the capability to build business cases as the core nucleus of our team.
00:11:26.820 | And again, as you're building your team and your function within your domain, these will be different,
00:11:32.340 | but this is a framework to understand what are my priorities. And we need to have the expertise
00:11:37.380 | in our outer loop as well to further differentiate our company and our team.
00:11:41.140 | And if you have a weak technical loop on the inside, you're going to struggle with the technical
00:11:46.340 | execution. If you have a weak domain loop, you're not going to find product market fit.
00:11:50.420 | So you need to make sure that you really understand those feedback loops and the collaboration loops
00:11:54.980 | that exist within your company.
00:11:58.980 | Now, depending where you are at the stage of your AI strategy, all of us fall on a different spectrum.
00:12:04.420 | You win with different types of people. You win with generalists at the beginning when you're
00:12:08.900 | trying to find that fit, trying to make that basic progress until you get to the point where you exhaust
00:12:13.780 | the knowledge and you need to move into a more specialist model.
00:12:17.060 | So once again, on the general side, most companies as they're going through transformation fall in
00:12:22.260 | that category. Once you get to a really good stage for your model training, serving and so forth,
00:12:27.220 | you need specialists to push the extra 5% of performance there.
00:12:31.140 | So generally, my perspective is generalists are good because they're adaptable. And in most cases,
00:12:38.500 | you're good enough with a generalist who can do many different things beyond just writing code.
00:12:44.260 | Let's talk about upskilling, reskilling, and hiring.
00:12:48.340 | So I think there are three main things as we continue to go through this AI wave that you need to do.
00:12:54.980 | People need to learn to build, you need to become a domain expert, and you need to be human facing.
00:12:59.540 | So we've talked about vibe coding and prototyping. We should go from static product requirements to
00:13:05.220 | functional prototypes that take those details and elicit them. We never want to have those
00:13:10.340 | conversations again, those dreaded conversations with PMs and engineers being like that wasn't in the
00:13:15.460 | requirements or that wasn't an edge case. We want to shorten that feedback loop. We want to make sure
00:13:20.900 | that people are writing evaluations, that domain experts aren't just providing input and feedback,
00:13:24.900 | that they're the ones writing the use cases, defining them, and having the literacy to work with
00:13:30.180 | elements directly. We need to make sure that engineers are on customer calls so we shorten those
00:13:34.980 | feedback loops. If your engineers say sorry I can't talk to a customer, that's a learning opportunity.
00:13:41.540 | Finally, you need somebody to sell your product. Now the way my team works is that we have weekly
00:13:48.740 | cadences to learn. Every week we have a new topic either with myself or other members of the team that
00:13:53.460 | is brought to the table for 30 minutes and we learn the underlying key priorities of our team and our
00:13:58.340 | company. And we make sure that every week we're upskilling ourselves. If this sounds intense, the consequences of not doing this are much
00:14:04.740 | much higher. Let's close out on hiring. When do you need to hire? I believe that people need to be hired for
00:14:12.660 | two main reasons. One is to hold context and the other is to act on context. So it's important that if you have too few
00:14:19.300 | people on your team, things are getting dropped and you can't execute on your priorities. You might ask the question,
00:14:24.900 | can't AI agents with a massive context window do this? Maybe to some extent, but you need expertise
00:14:31.060 | to be able to verify that this context and this execution is correct. And to have expertise, you need
00:14:36.900 | to have context. And finally, humans should be accountable for the systems that we build, as we have in the old IBM
00:14:43.060 | quote, right, we can't hold a machine accountable. So who do you who do you need to hire? So we're hiring on a budget.
00:14:50.980 | And going back to everything that we've talked about today, you need to know your team composition and
00:14:56.420 | the needs that you have to set up this budget, right? If you're trying to hire the top researcher, it's going
00:15:01.060 | to be very expensive. If you're going to hire a generalist AI engineer, we'll be quite a bit cheaper.
00:15:07.060 | Now, it's also important when you're hiring is that you're not just following trends.
00:15:10.500 | Who here has heard the trend that junior engineers shouldn't be hired or just using AI agents?
00:15:14.900 | Okay, some people are asleep.
00:15:17.940 | Now, the counterpoint here is why is YC running a school, an AI school for students and young people
00:15:29.540 | on AI? 2,000 people coming to San Francisco in two weeks. Why are they doing that? Certainly entry level
00:15:35.300 | positions. If they were useless, they wouldn't be bringing in all these young people. So make sure
00:15:40.420 | that you verify the trends that you're seeing and think from first principles. What do I need?
00:15:45.220 | What is the team composition? Is it new grads? Is it people with 30 years of experience? What are the
00:15:50.980 | retraining opportunities, right? There's lots of ways to build a great team.
00:15:54.820 | Now, just repeating this because I've seen so many companies do this, ask relevant questions to the job,
00:16:02.740 | stop putting people through lead codes that have nothing to do with the job.
00:16:06.100 | And now that LLMs can solve it, it's not a great way to evaluate either.
00:16:09.940 | So we go back to Ampere's wager. You have the question of, am I going to have five researchers
00:16:18.820 | from the top labs? Or am I going to build my team in a domain specific way? So for example, in my company,
00:16:25.380 | I'd rather have the team on the left with the domain expertise, the ability to sell, work, have empathy
00:16:30.420 | with customers, rather than just having five researchers. That's the way that our domain and
00:16:35.060 | company are structured. Now, you can also answer Blackwell's wager, which is do you want GPUs or a
00:16:41.060 | team? So that's a story for another day. So overall, we have three main lessons from today. The first one is,
00:16:50.260 | it's important to start off from the beginning and say, what team do you need to win? Once you know that,
00:16:55.860 | you'll start noticing that cross-functional teams will continue to be effective, but they'll be built in
00:17:00.260 | different ways. The overlap will be greater, but all of us will have the opportunity to work with AI
00:17:04.740 | systems and contribute to our product. And finally, we need to continue learning. This is a must, right?
00:17:11.300 | The world moves too quickly. We have Pelican evaluations now for the past six months rather
00:17:17.140 | than the past year, right? Hopefully, that's an illustration of how fast the world works as well.
00:17:21.860 | So keep up to date, keep moving, make it part of your culture to keep learning. So thanks, everybody,
00:17:27.300 | for joining. These are my handles if you want to connect afterwards, and I'll be here
00:17:32.100 | later on if you have any more questions. Thank you.