How to Hire AI Engineers when EVERYONE is cheating with AI — Beth Glenfield, DevDay
00:00:00.000 | *Music*
00:00:15.000 | Hi everyone. So I'm Beth Glenfield. I actually flew in from Ireland.
00:00:20.000 | So I will try and slow down my accent for everybody.
00:00:23.500 | I have heard that I do get a bit American sometimes.
00:00:26.500 | So yeah, it may not be the Irish accent you're expecting.
00:00:30.000 | So I'm going to talk to you today about how I believe AI is breaking how we hire technically.
00:00:37.000 | So I know everyone's very busy, but just a few questions to think about.
00:00:43.000 | Is that okay?
00:00:44.000 | Perfect.
00:00:45.000 | Is, are you using AI today in your recruitment process, both as an interviewee and an interviewer?
00:00:53.000 | Are you competing against Google or Meta employees, both for positions, but also for candidates?
00:01:00.500 | And are you recognizing that lead code puzzles just aren't really up to scratch, like they
00:01:06.500 | used to be for knowing if people will build great AI products?
00:01:10.500 | So welcome to 2025, where the technical recruitment process is very broken.
00:01:17.000 | And small companies are getting crushed in the talent war.
00:01:21.000 | So just to refresh our memories, let's look back at what happened over the last 18 months.
00:01:27.000 | So we had Cluey come to the table and they have raised a 5.3 million round and it is a AI cheating
00:01:34.500 | service from a ex-Columbia student.
00:01:37.500 | And they're currently heading towards 1 million in ARR.
00:01:40.500 | And we're also seeing 93% LeeCo Wizard success rates for Google and Meta interviews.
00:01:49.000 | And we're seeing like one in three interviews now are having AI assistants.
00:01:54.000 | So really, whenever you're interviewing candidates, you're just interviewing for who has the best
00:02:00.000 | AI coding assistant.
00:02:01.000 | So Sam Altman said this very plainly.
00:02:05.500 | The technical thing to do is to learn the best AI tools.
00:02:09.500 | We also saw that Marc Benioff announced that Salesforce will no longer be hiring software engineers
00:02:15.500 | this year because, according to them, they have seen a 30% boost in their productivity
00:02:21.500 | since they've replaced people with AI.
00:02:23.500 | I would like to see the data on that.
00:02:25.500 | But really, jobs have fundamentally changed.
00:02:29.500 | And you're not just competing on compensation anymore.
00:02:34.000 | You're competing on brand recognition, career prestige, and perceived stability.
00:02:39.500 | There's a lot of people being laid off and people are looking for security.
00:02:43.000 | So when a candidate has to choose between your Series A startup and Google's AI division,
00:02:49.000 | we kind of know what really happens there.
00:02:51.000 | So the candidates you actually want for AI development, the creative problem solvers,
00:02:59.000 | the collaborative leaders, and the ones who can work with AI instead of being replaced by AI,
00:03:05.500 | they are not being optimized for lead code performance.
00:03:09.000 | Instead, they're building AI tools and using AI libraries.
00:03:13.500 | They're contributing to open source.
00:03:15.500 | They are understanding how the business impact will be impacted.
00:03:19.500 | And that is just not what lead code is indexing for.
00:03:22.500 | So what if instead of asking candidates to solve puzzles that they will never actually use in a job,
00:03:33.000 | you could observe how they collaborate with AI teammates on actual business scenarios?
00:03:40.000 | What if instead of measuring their ability to memorize algorithms, you can see how they delegate tasks,
00:03:46.500 | how they handle ambiguity, and how they deal whenever requirements change mid-sprint?
00:03:52.500 | So you're not competing on brand recognition.
00:03:55.000 | You can demonstrate to a candidate at the beginning of a process
00:03:59.000 | what your engineering culture is and see how they would perform in your environment.
00:04:04.000 | So, at Dev Day, we are completely reimagining the technical hiring process in the AI era.
00:04:14.500 | Here are some examples of what we're seeing today, like I mentioned earlier.
00:04:18.500 | And then we'll get into what we're considering is the new kind of process you should be looking for
00:04:24.000 | in a hiring process.
00:04:27.000 | What we do instead of getting candidates to code interviews is we create real-life workplace
00:04:32.000 | simulations.
00:04:34.000 | Candidates work alongside AI agents with very different personalities.
00:04:38.000 | You can have the perfectionist, the pragmatist, the security expert, and even the junior developer
00:04:43.200 | that needs extensive mentoring, so that now they have to make trade-offs on day-in, day-out situations.
00:04:49.700 | They're not solving contrived problems.
00:04:51.700 | They are building features specifically for your business domain.
00:04:55.700 | So, instead of measuring how they can code, we're actually measuring them on skills that matter.
00:05:02.700 | How do they collaborate with AI?
00:05:05.700 | How do they handle ambiguity?
00:05:07.700 | How do they communicate these technical decisions in pull requests, in comments
00:05:12.700 | on tickets, and how do they mentor others and adapt when everything changes on a daily basis?
00:05:19.200 | Now, if you do work at Google or Meta, you can just brute force your hiring process.
00:05:25.200 | You interview 100 candidates, you hire five, and you give them great paychecks.
00:05:31.200 | But for the majority of companies, we don't have the luxury.
00:05:35.200 | You cannot afford to make one bad hire, and you can lose up to 20 to 60K in that bad hiring
00:05:41.700 | process.
00:05:42.700 | And you definitely can't afford to hire someone who doesn't know how to ship AI products.
00:05:47.700 | So, look, the future is coming for us whether we like it or not.
00:05:54.200 | Mark Zuckerberg says that AI will handle mid-level engineering work by later this year.
00:06:00.200 | TechCrunch is also reporting that we're wiping out entry-level engineering jobs.
00:06:06.200 | That doesn't mean fewer engineering jobs.
00:06:08.700 | They're just very different engineering jobs.
00:06:11.700 | Jobs that require creativity, collaboration, and the ability to work according to business
00:06:17.700 | judgment, and not just code.
00:06:20.700 | Jobs that require working with AI instead of being replaced by it.
00:06:26.700 | So, we're currently working with select design partners.
00:06:29.200 | If you're interested in thinking about this process or learning more or just want to chat
00:06:33.700 | about how you're thinking about hiring, let me know.
00:06:36.200 | My details are on the screen or just come find me at the event.
00:06:39.200 | Thank you.
00:06:39.200 | Thank you.
00:06:39.700 | Thank you.
00:06:40.700 | We'll see you next time.