Back to Index

How to Hire AI Engineers when EVERYONE is cheating with AI — Beth Glenfield, DevDay


Transcript

*Music* Hi everyone. So I'm Beth Glenfield. I actually flew in from Ireland. So I will try and slow down my accent for everybody. I have heard that I do get a bit American sometimes. So yeah, it may not be the Irish accent you're expecting. So I'm going to talk to you today about how I believe AI is breaking how we hire technically.

So I know everyone's very busy, but just a few questions to think about. Is that okay? Perfect. Is, are you using AI today in your recruitment process, both as an interviewee and an interviewer? Are you competing against Google or Meta employees, both for positions, but also for candidates? And are you recognizing that lead code puzzles just aren't really up to scratch, like they used to be for knowing if people will build great AI products?

So welcome to 2025, where the technical recruitment process is very broken. And small companies are getting crushed in the talent war. So just to refresh our memories, let's look back at what happened over the last 18 months. So we had Cluey come to the table and they have raised a 5.3 million round and it is a AI cheating service from a ex-Columbia student.

And they're currently heading towards 1 million in ARR. And we're also seeing 93% LeeCo Wizard success rates for Google and Meta interviews. And we're seeing like one in three interviews now are having AI assistants. So really, whenever you're interviewing candidates, you're just interviewing for who has the best AI coding assistant.

So Sam Altman said this very plainly. The technical thing to do is to learn the best AI tools. We also saw that Marc Benioff announced that Salesforce will no longer be hiring software engineers this year because, according to them, they have seen a 30% boost in their productivity since they've replaced people with AI.

I would like to see the data on that. But really, jobs have fundamentally changed. And you're not just competing on compensation anymore. You're competing on brand recognition, career prestige, and perceived stability. There's a lot of people being laid off and people are looking for security. So when a candidate has to choose between your Series A startup and Google's AI division, we kind of know what really happens there.

So the candidates you actually want for AI development, the creative problem solvers, the collaborative leaders, and the ones who can work with AI instead of being replaced by AI, they are not being optimized for lead code performance. Instead, they're building AI tools and using AI libraries. They're contributing to open source.

They are understanding how the business impact will be impacted. And that is just not what lead code is indexing for. So what if instead of asking candidates to solve puzzles that they will never actually use in a job, you could observe how they collaborate with AI teammates on actual business scenarios?

What if instead of measuring their ability to memorize algorithms, you can see how they delegate tasks, how they handle ambiguity, and how they deal whenever requirements change mid-sprint? So you're not competing on brand recognition. You can demonstrate to a candidate at the beginning of a process what your engineering culture is and see how they would perform in your environment.

So, at Dev Day, we are completely reimagining the technical hiring process in the AI era. Here are some examples of what we're seeing today, like I mentioned earlier. And then we'll get into what we're considering is the new kind of process you should be looking for in a hiring process.

What we do instead of getting candidates to code interviews is we create real-life workplace simulations. Candidates work alongside AI agents with very different personalities. You can have the perfectionist, the pragmatist, the security expert, and even the junior developer that needs extensive mentoring, so that now they have to make trade-offs on day-in, day-out situations.

They're not solving contrived problems. They are building features specifically for your business domain. So, instead of measuring how they can code, we're actually measuring them on skills that matter. How do they collaborate with AI? How do they handle ambiguity? How do they communicate these technical decisions in pull requests, in comments on tickets, and how do they mentor others and adapt when everything changes on a daily basis?

Now, if you do work at Google or Meta, you can just brute force your hiring process. You interview 100 candidates, you hire five, and you give them great paychecks. But for the majority of companies, we don't have the luxury. You cannot afford to make one bad hire, and you can lose up to 20 to 60K in that bad hiring process.

And you definitely can't afford to hire someone who doesn't know how to ship AI products. So, look, the future is coming for us whether we like it or not. Mark Zuckerberg says that AI will handle mid-level engineering work by later this year. TechCrunch is also reporting that we're wiping out entry-level engineering jobs.

That doesn't mean fewer engineering jobs. They're just very different engineering jobs. Jobs that require creativity, collaboration, and the ability to work according to business judgment, and not just code. Jobs that require working with AI instead of being replaced by it. So, we're currently working with select design partners. If you're interested in thinking about this process or learning more or just want to chat about how you're thinking about hiring, let me know.

My details are on the screen or just come find me at the event. Thank you. Thank you. Thank you. We'll see you next time.