Built In A Weekend? How 3 Leaders Answer Rising AI Expectations

Josh Anderson sits down with three engineering leaders: Kyle Crum (VP of Engineering at Reverb, about 50 engineers), Karl Jackson (leads AI accelerated software engineering at CDW, roughly 200 engineers), and Dom Scandinaro (CTO at Cameo, 20 people across engineering and data). They start with hiring. All three let candidates use AI in interviews, and Kyle compares it to allowing Stack Overflow.

From there they get into junior engineers and mentorship. Dom says Cameo stopped hiring juniors after layoffs and the rise of AI, then started hiring more junior people again for automation engineering. The three also question whether the pull request is still where apprenticeship happens. Kyle has removed PR approval gates, and he says nobody has taken him up on it yet.

Dom explains how Cameo's automation engineering team went from 1 hire to 3 after the first engineer built three or four always-on agents for the celebrity-recruiting team. He also describes rewriting the 10-year-old website with Claude in about 2 months, with 2 people on the first half and 6 to 8 on the second. The goal was to clear out tech and product debt so Claude could work faster in the codebase afterward.

The panel keeps coming back to expectation inflation. Boards ask why the process isn't faster when a prototype took a weekend, and the leaders say the gap is planning, testing, go-to-market readiness and engineering rigor. They end on the slop cannon, with Dom's reply to 12-page unedited LLM output: AIDR, AI didn't read.

Dom Scandinaro 00:00

There's just so much chatter in the in the ecosystem today from clients that I meet and boards and leadership teams who are like, "Why is the process not going faster? Don't you have AI?" The biggest trouble with this expectation inflation that we're talking about where it's like, well, I, you know, created this little website in Claude Code over the weekend. So, like if I could do that in an hour, why can't we do that, you know, next week 100 times with in 100 hours?

You know, there's all these trade-offs to be made whenever you start like making new features in the product that I think need human decision, need human eyes. I I don't see that going away. So, this idea that we need less people to do that, I don't see that happening. The distance between vibe coding and production grade digital product is still engineering rigor. We put somebody in that position for the first time about 4 or 5 months ago. He's built three or four agents that are running kind of 24/7. With that success, we actually just hired two more automation engineers.

So, went from a team of one to a team of three. I've removed approval gates on PR. If you were working with someone and two people agree this is the right solution, go for it. I don't think people have taken me up on that quite yet. We've pushed a lot of the human collaboration, the tutoring, and the apprenticeship into the pull request process cuz it was a really effective place to do it. I wonder if that's the place it's supposed to happen anymore.

Josh Anderson 01:28

Welcome to Tech Teams. Today, joining me in studio today is Kyle Crum, who runs engineering at Reverb, Karl Jackson leads AI accelerated software engineering at CDW, and Dom Scandinaro is the CTO at Cameo. Onto the episode. First off, we want to learn about your team's general size, if there's any special technology or anything that you do. So Kyle, we'll start with you and then we'll go around the horn before we get off to regular questions.

Kyle Crum 01:54

Yeah, I'm Kyle. I'm the VP of engineering at reverb. com. so our engineering team is around 50. total company's around 200 people. I don't like to brag about special technology or anything. but yeah, we we're the largest marketplace for musical instruments and yeah, serve millions of users.

Josh Anderson 02:17

Nice. All right, Carl.

Karl Jackson 02:19

My name is Carl. I'm at CDW. our team, my team is around 200 people. I'm on the CDW professional services side. and so I run our software engineering team there. CDW is a big place. and so you might know them for much larger teams in many different areas. but that's part of the business side of it.

Josh Anderson 02:36

Nice. All right. Dom,

Dom Scandinaro 02:38

I'm Dom, CTO at Cameo. Our whole company's about 65 people. 20 or so of them are between engineering and data that report to me.

Josh Anderson 02:47

Nice. Okay. All right. So, we're going to jump in. Hiring AI. It's a whole thing both in the process and the resume creation, everything that goes into that. There are a lot of people in our roles wrestling with should I allow AI in the hiring process or should I exclude it? Kyle, I know that you've taken a stance with that. So, can you tell everybody the approach that you've taken and why? Yeah, we've taken the

Kyle Crum 03:13

Approach like we have in all hiring, which is we're trying to create the kind of way that you would work, you know, with a company anyways. And usually when you work with us, I don't I don't say turn off the internet, you can't use Google, you can't use whatever. So we we allow people to use AI. we give them instructions ahead of time that says this will be there. You can use it. we're not evaluating you on your usage or lack of usage as it may be. We're just evaluating you based on your solutions and how you think through the solutions.

So, please be transparent and talk with us along the way. but you know, we want people to use the tools available and see see how they use them.

Josh Anderson 03:53

So, it's like an open book test, right? Like you still have to get the right answers. We're just allowing you to use some extra the technology you would use on a daily basis, right?

Kyle Crum 04:02

Yeah. I mean, for me, I don't think it's any different than allowing people to use Stack Overflow back in the day, right? I mean, you can go to Stack Overflow and copy and paste an answer and put it in there. This is just Stack Overflow, but really quick. give you answers much faster. And the same thing like I I just want to see how people think through the problems, how they use the resources, and how they trust or don't trust the solution, the resources are giving you to this the problem that we've given them.

Josh Anderson 04:31

Where do you guys stand on this? kind of the same. I mean, I I think the general consensus 6 months ago or so

Dom Scandinaro 04:37

Was like, don't let people use AI. And pretty early, I was kind of confused by that thought process, right? Like, we were trying to get our team to adopt it faster. And the most obvious way to me to do that is to bring in more people that have used it dayto-day at the same level or better than we have to kind of level all of us up based on their experience. So we use CoderPad for interviews for a while.

they have a little like agentic AI built in that we started to try and use but that was it felt pretty foreign to people based on their day-to-day. So we switched our interview process now to be you know bring your own laptop your own ID your own coding assistance so that we weren't having people like you know accustomed to codeex and then confused by Claude Code or what have you in the interview process. So we look for a lot of the same things as Kyle dope but that's that's kind of our thought process [snorts] on it. yeah, generally

Karl Jackson 05:28

Agree. I think the interview style and structure can determine a lot about where AI fits in it. So I think intentionality matters. and you know when you're hiring I think most of us are hiring for problem solving. I'd say that in the services org we have a a very large client base that is very varied in terms of technology usage and AI usage for that matter. And so I think we're we're we we have less technology to test on because people are expected to learn everything really fast. And so, sometimes our interviews are really focused on, hey, how fast can a person learn?

And so, much like, you know, Kyle said, we start with real world problems like things that teams that they may join are already facing. and and so using AI in that problem solving construct is is definitely part of the process because like Dom said, we're expecting them to use that and get better at using it on the job. And so it doesn't make a ton of sense to me either to to exclude it from the process. Unless your interview process is incredibly academic, in which case then maybe maybe a different different path

Josh Anderson 06:42

Forward. It makes sense. Google is well according to the internet Google is bringing people back in to the traditional whiteboard seeing people in person to be able to validate because I've interviewed some talent acquisition leaders that have seen people where they're pretty sure they're they're they're cheating. So, have you seen anybody in a remote interview process where you're pretty sure they're not giving you that like the answer like they're cheating or has it always been like that was never an issue? I haven't seen it. I mean, I'm I'm I'm curious what the cheating would be.

I mean, for me, I guess the cheating would be like if someone else was literally there feeding you the answers, but that was always the case any remote interview. But if someone is [snorts] using AI, you know, whatever transcribing what you're saying and then spitting out the results, I mean, one, there's a little bit of the the interviewer's chance like push them on the answers and what's a different, you know what's a different response you have or what's a, you know, what's

Kyle Crum 07:48

An alternate path? Why would you think this is good or bad? two, I think people are pretty adept now at understanding like AI speak. You know, maybe a year ago you weren't. You're like, "Wow, this looks like amazing writing." Now it's like, you know, this sounds a lot like Claude cuz I've written I've I've written and read so many things by Claude and I was like, "That sounds like Claude." And I think that people are, you know, you can kind of filter that language out and know if people are being real to you or not. So, I haven't haven't heard of that.

I mean, I've heard of some bigger companies like actually hiring people that were, you know, not actually living in the same country and they had those problems, but, you know, we're not at the scale that we have to worry about that. Like, we can deal with people individually. Yeah. Yeah. No, I've seen it all. I've seen like transcript transcribed replies, pauses. I've seen different people

Karl Jackson 08:43

Show up that interviewed on day one than than the person that actually showed up with like two different people. there's all sorts of stuff and I think there's a yes to all of it. and so I think there is depending on the rate of hiring some some some vigilance and and practicality that goes to it. But I do find that if you're running an interview in the way that with an experienced interviewer that is really looking at a real world problem you sus it out pretty fast.

Josh Anderson 09:18

Is there any part of the process where you would exclude like AI isn't even to be a thing? Like we're just going to talk, just me and you. Like there's things you're trying to sus out that you just want to have a conversation with

Dom Scandinaro 09:30

Somebody, right? Yeah, of course. I mean, we always bring people in. We're hiring in person in Chicago. So, at some point of our interview process, they're going to come to the office. We're going to go walk and grab a coffee or grab a beer and just chat about, you know, their interest in tech and why they're interested in the job, what they've been doing previously. So, obviously at some point we're going to have those conversations for sure.

Kyle Crum 09:51

Yeah.

Yeah, I mean I as when I screen for managers a lot of times it's you know look at the resume and I want to start off with like kind of an open-ended thing like you know pick something recently that you're super proud of tell me about it why you're proud of it what problem you solved like what your what your role in that thing was like that is a very open-ended thing that I think if AI is transcribing that transcribing that probably wouldn't have the context to respond and you know I allow people then I use that to start with and then kind of go from there where yeah it is a conversation.

It's less of a a canned process and it is more of a you know let's sit down get to know each other and kind of go from there.

Josh Anderson 10:32

Yeah. I mean we're hiring people right so you got to talk to them. So moving through the hiring process let's talk about the kinds of people that we hire. And so Dom I know some of the best members of your team started out as an intern with you and then have grown into that role. So, how do you see where juniors fit? Cuz that's such a hot topic

Dom Scandinaro 10:54

In our space of does the does the need even exist anymore? What does it look like? And nobody has the answer, but I think there's a ton of value in everybody hearing what you all think about that. So, Dom, we'll start with you and kind of the journey that you're on. Yeah, I mean, I I'm just kind of a little bit lost on the journey to be to be totally transparent.

like you said, a lot of our best engineers at Cameo today and over the time that I've been there started as either interns or very junior people that we, you know, build up in house through, you know, training and just investment in them day-to-day. And both through downsizing you know, after some layoffs at the company over the last few years, as well as with the rise of AI, we just haven't been looking for junior people in in kind of either of those situations.

we've been skewing much more senior and obviously it's like the topic of conversation everywhere that you know do junior people have a place on smaller teams like the ones that some of us work on here and I think more recently for us the the first place we've got back into hiring folks that are a little bit more junior is on the automation engineering side so we're building that discipline out at Cameo for the first time and I think that at least from my perspective somebody that's a little bit more junior might be just as success essful and more happy in the position that we're asking them to do that type of work at Cameo than a more senior person anyway.

So that's where we're fitting it in. It's very early days for us. So to be determined if that works out well, but that's that's the first area where I'm excited to bring on a junior person and think they'll excel.

Karl Jackson 12:25

Yeah, I'm similarly I feel like the journey's been windy over the last couple of years. you know Dom and I said last night we talked about talent, taste and training, right? And you can get all those things if you hire more senior and definitely for the last couple of years as AI's sort of made become a more used tool in in the the software engineering community. We've seen junior talent hiring drop off and we have found that senior people will scale the their their ability to you know produce and manage teams faster.

but I would say just even in the last couple months this summer I've started to change my opinion on that and say hey you know we are eventually going to need more senior people and we can't get more senior people if we don't hire more junior people. And so particularly as we look at talent planning and horizons we want to make sure that we're absorbing talent that's that's early and quite frankly in some ways that are experimenting with the tool sets in a way that's sometimes more advanced than our senior people. So I think there's a good blend to be had.

I think there's good apprenticeship to be intentionally created as well as we bring as we bring younger talent on.

Kyle Crum 13:48

Yeah, I just I haven't thought about it too much. We have we have a we have really good retention and so you know we've got just a ton of senior talent and so I'm trying to like build down to that. So if if there is if if senior engineer that does leave, I try to replace them with the mid-level. That way we're building at that one. And then once we have like some mid-levels, then you know, we will hire some more junior talent, assuming the mid-levels don't just go to senior and then I get I'm always like filling this mid-level thing.

But but yeah, we've had pretty good retention. So I'm I kind of have a lot of senior folks at the moment.

Josh Anderson 14:25

Yeah, I think there's a lot of potential on the junior side because of that hunger and willingness to just kind of do whatever to get the job done. even if it's stumbly and I get it wrong sometimes, but sometimes you stumble on something new and better, you don't even know it. And so there's a ton of value there. But the apprenticeship is one of those things I want to ask you all about if your seniors are looking around and are they missing that mentorship that they used to give? Is that something that has come up at all?

Like, you know, I really kind of wish I was able to give back more or is that even not a topic? it's there are my impression I mean I haven't asked people directly but some people do mention that and they do say I want to have juniors and I want to bring up the next folks you know I would say it is [snorts] it's not like a bunch of seniors saying that there I think there's you know there's some folks that are really adamant about it and really want to do it and so yeah that is

Kyle Crum 15:27

That is out there for sure.

Karl Jackson 15:29

Yes. I feel like there's like this curve that people go through where particularly as like in terms of senior talent where they're like I don't ever want to work with anyone except someone who's just like me. and then but then there's there does seem to be some inflection point that they turn around and be like all right what's what's next and and how do I continue to grow my own skills maybe even outside the technical space and and you know teaching and mentoring and growing a team all become part of of that growth for that person as well.

So yeah, I see I see a fair amount of desire for that particularly as you know sort of consulting draws a certain kind of personality that's that's used to a lot of change or desires a lot of change. another piece there is just the shape of the work. It was always easy to look at the items that are on the backlog and say, "Okay, that's good for a junior or this is a good first PR or things like that for a new hire."

The work doesn't feel like it filters out like that anymore because it's just kind of like everything can quickly get gobbled up or you already have an agent that has already scoured the backlog for all the stuff that it can do without needing to ask. So all that stuff is kind of like blown away. Do your backlogs have anything that looks like something [snorts] that you could handle or hand to somebody in that less experienced

Dom Scandinaro 16:54

State? I I don't think so. I mean, we're definitely pretty intentionally focused on trying to have an agent tackle the stuff that is is easier. You know, the the small copy change, the easy bug fix. we're not definitely don't have that all figured out. They're not opening pull requests that are perfectly ready to merge 100% of the time, but we're definitely trying to to tackle that stuff with agents for sure. So, I think it's a little less obvious what what work is the right thing for a junior to do. I think well, you said the shape of the work is changing.

I mean, I think that there's technical level of work that is appropriate for a junior. I think the decisioning level like we might have to increase that a little bit as in to say like you know there are a number of things that come by it's like listen there are three kind of I don't know easy solutions technically but they all have like some trade-offs where like you

Kyle Crum 17:48

Know some stakeholder might not like this one another stakeholder might like this one etc etc you might have to talk to product and design and get the answer like ultimately like the implementation isn't isn't that material but like getting everyone together to say like this is the thing we want to do is actually the work. so I would see that as like a, you know, kind of training a junior to be like, you know, a lot of the job is is going to be, you know, not just doing the solution.

It is saying, yes, there's a variety of solutions and they're all within your capability to implement, but like which one of these should you should you do? go here's the people we should talk to and then like you know kind of work with them to like understand how to how to get to a decision and communicate those things and etc etc. Yeah, I think

Karl Jackson 18:37

That's the that's been the human element of software engineering for a long time, right? The actual number of letters you bang out on the keyboard that gets compiled is really small part of the process sometimes. And I I haven't thought that's a really good way to think about it that there's a lot of learning to be done. you know, as we as we talk about training that taste you know, with people that have a lot of talent, it's exposing them to those parts of the business that we know or parts of the SDLC, let's say, that we know is frictionful.

This idea that says, hey, a stakeholder might have an idea of what they want, but how does that get implemented? How does that get measured? and that kind of motion, that kind of muscle is a really

Kyle Crum 19:18

Good muscle to build. Yeah. Yeah.

just like just like even simple like there's a bug you know it's like whatever there's some null poin error like some no reference and it's like you know the simple thing would be like oh just check for null and then like move on but it's like well okay should it be null in the first place and if it shouldn't be like why was it null was there some something upstream that didn't happen or does you know is there some human element to it where somebody didn't put something right in a form or whatever and that that's where the solution should be I think it's like that stuff where you know I almost think of like a almost like a an analyst role versus like a junior engineer role like you you kind of have to do some other stuff to figure out you know yes it is just a null error but like what what should we

Karl Jackson 20:07

Actually really be doing what's the problem we should be solving and I see agents still living in that spot where it addresses it fixes the bug but the root cause still lives somewhere and And and I catch a lot of that like okay Claude I understand that yes you fixed it that's not going to happen again but like why to your point why did this happen and then so getting it to look deeper or just knowing where that likely is going to be and point it hey we should probably look here that's that evolution of that problem solving that you want that junior to evolve into so that's the that that's the I still see that with agents and that's

Kyle Crum 20:47

Something that a human has to go through as Oh, I know. I haven't solved it. Maybe maybe the cameo agents have solved it. I don't know. Have they?

Dom Scandinaro 20:56

No. No, definitely not. It's it's it certainly would just be checking for null and then not throwing the air in that example. Well, I mean, we have we we had like a we had a brief outage today and and so part of like we have AI that went through and here's what happened and here's like just go just go revert this commit. and it was wrong. It just wasn't wrong. like there were there were two things that happened at once and it picked one of them said well this is the reason why it failed and I it wasn't it wasn't that

Kyle Crum 21:23

So yeah like I think part of like the the automating stuff and the agents is like it does it does do some things and reduces some toil and helps like bring some context to it but there's still some human decision on like what what am I fixing and and where am I fixing it and and that's that today that's an agentic problem. But I've made that mistake myself, right? Like picked the wrong thing. I was like, "Ah, this is this looks like it'll fix it." And early on in my career, I've made that mistake.

And so I think there's something to be said about what are the controls and

Karl Jackson 21:58

Structures we're putting in place to as the speed of the actual coding part speeds up, you know, what controls we're putting in place to manage that. And I think we pushed a lot to the idea of the pull request. you know, the pull request didn't have the revered status that it has today like 20 years ago, you know, if it didn't really exist back then. And and so we've pushed a lot of the human collaboration, the the the tutoring and the apprenticeship into the pull request process cuz it was a really effective place to do it.

I wonder if that's the place it's supposed to happen anymore. And and now you've got agentic systems that are opening pull requests. I don't know if the pull request is a source of truth. Maybe it's the agentic system itself. it's too late for me. I would like to catch all of that before it gets to the PR. So I do a lot of work trying you know traditional [clears throat] shift

Kyle Crum 22:49

Left right trying to shift that discovery left but the PR has been that's the one central repository where everybody can see the branch and see the changes and see what's going on. So, I haven't figured out how to get that same dialogue, which is different cuz if Dom and I were working on something before I put it to PR, I'd be like, "Hey, Dom, can you come over and look at this?" And he would roll over and we'd look at it be like, "Oh, you dummy. You it it should be null, right? Why are you letting it not be null?" You know?

So things like that where I'm trying to figure out how to get that to shift left because it does it feels so late and it feels like a very expensive and there's a lot of churn there of like back and forth and there's all these coding agents that are doing all the reviews and they all have very strong opinions, right? You know, which humans always had as well, but like that's I've been [snorts] trying to figure out how to address that problem. Have you guys made any headway in that space?

No, I I I was just going to I I don't want to use strong words like I hate PRs, but I don't like I don't like the I don't like the affordance that it creates. because yeah, what I really you know what I really like and I've like sent this to people, but I can't I can't enforce it or it's really hard for me to tell people to do this way. Like what I'd really like is that like I I

Josh Anderson 24:02

Like pair programming. like that is how I learned almost every thing that I know how to do was like just sitting next to someone else and like cool what's that what's that keyboard shortcut what is that you do like what's this what's this Vim thing that is amazing and and then like I I would I went and did Vim you know like I didn't I didn't learn that through a PR was like sitting next to someone like

Dom Scandinaro 24:22

Watching them do it and and like and and getting into their thought process so like you know I've told I've told my teams like especially in the agent world where like you can write code and there's it's quick like I would like people to pair more that way you do a thing like it comes out and you can decide is this the shape of the code we want is this what we want to do you can make those decisions faster versus like waiting for a PR and then like you put it up and then however many hours later someone looks at it and then you look at the comments and then you do a thing and then you know it's just this cycle you know I've I've removed you know I've I've removed approval gates on PRs and like you don't need anyone's approval like merge it in because if you're working with someone and two people agree this is the right solution, go for it.

I you don't you don't need an approval from anyone else. Just go just go just go do it. you know I don't think people have taken me up on that quite yet. So I'm still thinking about

Josh Anderson 25:18

How how how I can design some affordances so that they can do that a little bit more. But that would be my my dream on how people would work. Yeah.

I used to call it Jira tennis bugs would get logged and then they would reply like comments like just like go talk to the person as opposed to you know I'm going to reply in a comment it no this is broken I fixed it okay I tested it it's broken again it's just like back and forth you could watch it and and it to your point it would drive me crazy just like just go talk to the person how are you guys dealing with that how

Dom Scandinaro 25:55

Is the collaboration has it has the agentic nature put everybody like back in their cubes and in their dark corners. Like I feel like that's a thing that's happening in a lot of places as well. Well, I think the way we've been thinking about, we're just talking about this in a meeting this week, is that the process of coding as we see it today at Cameo is kind of three steps. The first step is writing a plan. You know, thinking through the architecture. The middle step is writing the code, which you know, yesterday we wrote by hand today and we use Claude Code to write.

And the third part is testing it,

Karl Jackson 26:30

Finding the edge cases, and then maybe like going back in the process and and doing more coding. and obviously the middle part has got sped up tremendously. But I think the first part is probably more important now than than it ever was to avoid to the point of this conversation getting to the pull request and it being completely wrong and needing to start to start over again.

And I think bringing that all the way back to the junior engineer thing, that's the part that like makes me struggle the most with understanding where juniors fit in because it's, you know, a couple years ago, we would have a more senior person, you know, think through the architecture work with the manager to break out some tickets and then the, you know, the mid-levels and junior people would pick up those tickets and build it in pieces that somebody else had already thought through.

And now I think we're trying to or more expect everyone on the team to to be writing that architecture plan, giving it to Claude and then responsible for opening the pull request and and shipping the feature. So we we haven't figured it out for sure. We're parallelizing a lot more and you know have less opportunities for pairing which I agree is like a you know a big a big miss and a big loss compared to how we all learned so much from one another previously. That's a good point.

I I wonder if you know when we we think about the progression of of careers, we think about engineer, senior engineer, and at some point architect like I wonder if we're we're all just skipping to architecture as a skill pillar. that doesn't have to necessarily come after you write a lot of code and maybe it's you know you get you start your job

Kyle Crum 28:05

And you start architect apprenticing because those other two parts and I agree like I think there's just so much chatter in the in the ecosystem today from clients that I meet boards and leadership teams who are like why is the

Karl Jackson 28:19

Process not going faster don't you have AI and you've got to explain to them that the constraint doesn't necessarily It's not in the coding piece of it. It's in the how do we get the requirements? How do we test? How do we validate? What are our edge cases? so there's it's an old quote right from from lean startup thing that talked about any any any increase in capacity. That's not the bottleneck is an illusion. So maybe lean manufacturing thing. Eric Reese would be so bra or it's like all right, I think the the computer one is doll's law.

Like as soon as you if you speed up one part of it,

Kyle Crum 28:54

Then it just points out the flaws in some other part of it. So like you know and there's no there's no end [snorts] to like which bottleneck you're going to fix. Like cool I fixed this one and then like it just exposes like three other bottlenecks. Okay, then go fix that one. I was like okay there's these other bottlenecks like there's always there's a next level of problem and that's okay for that to be the work.

Josh Anderson 29:12

Yeah. Yeah. Right. Like that is the technical work that happens is hey let's go chase every constraint and technically or or in the process.

Karl Jackson 29:20

Yeah. So, we're talking a lot about learning and learning has become such an important piece of what you have to do because the pace at which everything's changing is changing weekly. There's a new model and everybody's saying it's the best model ever and it can do all these things and you know it's a it's an endless new cycle and hype cycle but also a learning cycle and with an organization of any size more than just like the four of us.

As soon as you get bigger than that, how do you how do you transform that group into a living learning organization that can absorb all of that and not have it become too disruptive like you want some disruption of something new, a new discovery that's going to empower something's going to teach me something just just like the PR, right? Like, oh, I learned a thing and that's going to accelerate me. So, you want some of that, but you can't have it be every week there's a new thing and how do and we all have to switch, right?

Because otherwise you're you're always trying to get your feet up and under you. So, in your organizations, how are you going about encouraging and empowering but also guiding learning across the board?

Kyle Crum 30:34

Carl, we're going to start with you. I've told this story before, but I I have little kids. I have a a 9-year-old and a 5-year-old. And there's a playground down the street from us, right? And I would take my daughter there when she was, you know, till today. And it's it's funny, every time she's there, she learns some new physical move. She can jump a little further. She finds a different way to twist off the monkey bars. and it really struck me that like the playground never changes, but her experience at it changes her.

And so I think really I use that metaphor when I think about learning. I say what are the playgrounds we're constructing for our teams so that they can experiment so that they can exercise curiosity. and and moreover, you know, put whatever they learn back into the system. I think I'm a really strong believer in in in a in a in an engineering culture has to be insatiably curious and relentlessly collaborative, which means you got to go find out whatever that new thing is.

and I think lots of people that end up in this field, they're they're geeks about something, usually technology, and or maybe it's music instruments or lockpicking or whatever it is. and but the other half of that is like they find a thing, they learn a thing, they're so excited about it, they turn around and they're like, I'm doing a lunch and learn and I'm going to learn I'm I'm going to teach 50 people how to pick a basic lock, you know, or or or tune a piano or some, you

Karl Jackson 32:04

Know, whatever it is. and so I'm always thinking, hey, what are the what are the playground structures that we have? lunch and learns certification funding experimental hackathons, like you know, celebrating people who do stuff like that. It is interesting that we're I think we're leaning into to celebrating and encouraging it, right? Because, you know, go back to a couple years ago, we all have people on our team that it's like, oh, there's this new JavaScript framework out, there's this new front-end framework, there's this new backend language. like we should rewrite every we should move from rest to GraphQL because right and you had to

Dom Scandinaro 32:43

Balance that every day and you needed some people in your org to be pushing those sorts of things but you know most of the org to to be getting the tickets done and shipping the features and and building value for the company and I think now at least at Cameo we've skewed a lot more the other way we definitely encourage people to be you know posting whenever they see about a new model or you know a new use case of a model and trying out the latest version of of Opus when it comes out and telling us the rest of us, oh, this one's, you know, way more verbose, like to stick with Opus 47 until lately with 55 that we actually are on the other side of that and everyone's excited about a new model again.

so I think we we definitely haven't figured it out, but right at this point, we're we're heavily encouraging it. Yeah, I mean, we we try to encourage people to share both successes and failures with stuff, especially with AI. for some reason people don't want to share the failures. I don't know. Like I I I try to push people a little bit. because yeah, I want to have the place where we can just be honest about what's working, what's not.

And that's that's the learning is, you know, you can learn from people who have a success in something, but you can also learn by someone who just went the wrong direction and said, "Oh, like this thing complete failure. you know, don't don't do this thing. Cool." but I do, you know, I do like the, you know,

Kyle Crum 34:03

The framing of just being insatiably curious. So I think it it comes down to like learning learning what so like there are sure there there are times to learn about new technologies and and all that. There's also like you could learn about the business.

So you can go talk to, you know, one of the stakeholders and like sit down like what tell me about your day and what your what your job is and how how do you use the thing that I make and that's and that's learning you know I've learned so much about the finances of our company and how our CFO thinks that that to me is like super interesting as well as well as learning about AI as well as learning about you know new security techniques with AI and also new exploits that AI is coming up with like all that stuff is available in the day-to-day like if you are curious like it is there for people to go and grasp and and learn

Dom Scandinaro 34:58

That day-to-day. Yeah. And there's too much for one person to learn, right? Like the the the pace at which all this stuff is coming at us is so fast that the only way to effectively do it is like what Dom said like everyone's got to go find a different thing u and learn that and then tell everybody what they think about it. like the group has to learn at scale. And Dom, it sounds like you've formalized that a little bit. You've created channels where for for people to talk about that. Is that right? Yes.

I mean, we have a Slack channel called AI Avengers, very creatively named, where it's it's certainly heavily encouraged to like I think one of the most recent conversations is we have an open- source version of Jev Kev, creatively named running on a a Mac studio in the office and some people trying it out in the office for different use cases compared to traditional LLMs or ML models and comparing contrasting use cases versus like jumping in to try and build that tool into something we're trying to productionize right away.

but you know, even with a small team in that channel for anyone with the distributed, you know, learning and contributions, it's still pretty hard to keep up with everything. Like just today, I think somebody was like, "What's Kev in another channel?" And it's like, "Well, here's the link to where we talked

Kyle Crum 36:16

About." It's just like there's so much flying by every day that it's like if you mark a few too many messages as red on Slack, you you know, you missed a bunch. For sure. Yeah. I mean we have as a companywide we have like a especially for for AI we have like a I'm going to mess up the name but basically like a time to like kind of share what you are are kind of doing internally with it and it's just a way of you know seeing how people are using the tool. Is it like a demo day or something like

Karl Jackson 36:46

That kind? Yeah, it's it's it's it's kind of like a demo day, right? And so, you know, it's interesting for me to see how people outside of engineering you know, are using this. so, and I I make myself available when people say like, "Hey, I'm doing this thing." like I go sit with them like, "Cool, what are you doing? Like, do you need any help with this? Can I answer any questions about how some of this stuff works?" so yeah, part of that is me just being super curious and I like to hear about what other people are doing.

But yeah, we're trying to just again kind of have that culture of just sharing what what people are doing and if what someone is doing is interesting to you and it's relevant to your job, cool. If it's not, then we've learned something and that that's great too.

Yeah, I think you know I I came from small teams also and I as as as teams are start to scale I find myself in this point in this in this kind of perspective where I'm like maybe innovation isn't planned and exercised maybe it's harvested as long as the culture of the group you know subscribes to that kind of curiosity and collaboration. Yeah.

Josh Anderson 37:55

And that's one of those things that you really have to model. Like two of you have said, we try really hard to celebrate it because that's a new skill you're trying to build across the organization of, hey, I'm going to try a new thing. I'm going to be bold enough to talk about it. I'm going to be bold enough to try it. I'm going to be bold enough to put it in front of everybody.

Dom Scandinaro 38:13

And it takes leaders like you to encourage people when it works and when it doesn't because otherwise that habit doesn't get formed because they've never been asked to operate like that. So, like that's one of those things I see is really important is everything that you guys have said is we have channels, we talk about it, we encourage it. that's the only way I think you can make it become part of the culture, right, as you're talking about.

Karl Jackson 38:38

Yeah. I talked to someone years ago, I can't remember who it was in on their teams, they would they would throw failure parties. I was saying like like they had an outage or something. They would like literally buy pizza and a bunch of beer, hand out gift cards, and everybody else was supposed to come and listen to like what that team learned from the experience. I think that's a way you can celebrate the learning, but also really make a public statement about what is our culture's risk tolerance, right?

Like do you know is it is it is the cost of failure very high or is the cost of failure low because we prioritize

Josh Anderson 39:17

Learning from those failures. Okay, so let's talk about AI across the organization. Many people in your roles [snorts] were the technical leaders. So you had the technical teams. Yes, you would interface with finance or product or sales and marketing just as part of the job. It called it called for that.

Dom Scandinaro 39:35

But so many folks now are asked to own AI for the entire organization. So how are we going to roll out AI for accounting and HR and what does it look like? what's the tooling? How do we manage it? How do we keep it secure? So, we already had a hard job, but now it just got harder. And Dom, something that

Josh Anderson 39:56

You and I have talked about is the automation approach that you're taking and how that's working. And to me, I see it as the next evolution of DevOps, just like DevOps across the entire organization. And it sounds like it's

Dom Scandinaro 40:08

Working really well. So, can you talk through that? Yeah, it it is working well. It's still early for sure. I'm super thankful for AI in this perspective because a lot of Cameo is very manual, right? Like we have teams of people that are Instagram DMing celebrities all day, emailing their managers, you know, do you want to join Cameo? How about now? How about now? How about now?

And for years, I felt like if we want to grow the company faster, outside of, you know, trying new product idea after product idea after product idea and having some of them work and some of them fail, we really needed to to further scale the go to market teams, the people onboarding celebrities onto the platform, the people reaching out to marketers to buy our B2B product. And there's always been engagement around those conversations about like, hey, how about we build internal tools for these teams, but they've never got prioritized over the maybe sexier product initiatives that were also part of the planning process.

And that changed abruptly I think with AI because people started to see agents in non-technical organizations at other companies and hearing about that you know at different conferences and people having a legal bot or a you know a finance bot or what have you and that's put us in the place where we've gotten to the point of actually prioritizing that within Cameo. So we call it

Josh Anderson 41:28

Automation engineering. I think that's a somewhat used industry term but also kind of an antiquated industry term. people have been automation engineers for 10 years. but we put somebody in that position for the first time about four or five months ago. Very senior engineer on our team started working

Dom Scandinaro 41:46

With our the the team that recruits celebrities to join the platform to just sit with them, understand the manual tasks they do all day every day and work to prioritize things and then build tools to help them. And some of that is, you know, an actual agent. I think he's built three or four agents that are running kind of 24/7 doing different things in the background to help them streamline that process. But some of it's just, you know, you could be working with Claude. We have this connector to your Gmail. they send campaigns through Mixmax.

Like we can we can integrate that all together into your process and just teach you how to use the tools you already need access to more efficiently. So there's kind of a scale I think of the work that we do there from just teaching people how to

Josh Anderson 42:28

To use the tools they have better to building an agent and we don't want to jump to building an agent for everything. But with that success, we actually just hired two more automation engineers. So went from a team of one to a team of three. both of those people started four weeks ago, I think now. so we're starting to scale that out to every other department at the company.

So, we're just going through an exercise with each of the department leads to have them map out all the things they do from the things that take the least time to the most time on one axis and the other axis is things they believe that could be automated, things that they believe could be semi-automated with a human in the loop and areas where they believe we would always want a human doing, you know, the majority of the work.

And then we're sitting down with each of those department leads and talking through all those things right now to collect basically just a a massive backlog of stuff that we can prioritize for that group to to start branching out into all these other departments at the company as well. Do you also own AI for the entire org as

Dom Scandinaro 43:28

Well? maybe unofficially. yeah, I mean I I you know we we have we have guidance. and so I mean there's a there's a couple of things like, you know, there's like official guidance like here's what you should do. I mean people I think they read it but they don't really you know ingest that in their day-to-day. so I generally tell people a couple things like one even for engineers like I want I don't want AI making art. I want AI doing the dishes.

I think there was a famous like tweet or someone that did that, you know, like I'm just like, you know, you should be like automation is the actually the word that I use like it's just for me it's a little old school like we're just automating some tasks, right?

And if you sit with some people it's like yeah and I take this number from this spreadsheet and I put it over here and it comes from this system and then I copy and paste this number and put over here and it's like you know I I know that there are bigger problems that you could be that you could be working on and so there's a lot of that.

but we you know I give some you know we have some guidance for folks of like here are the tools you can use like we have claude like yeah you can make your skills here's how you can share your skills like we have default MCPs and connectors and like you can use these and generally here's how you use these just know that like you are responsible

Karl Jackson 44:49

For the things that you make so you know if you make a little app to help your day that is that's fine then that runs on your computer good for you and Now, if you want that to be hosted by us, like here's, you know, that's a whole separate thing. Like, here's a list of how we would do that. And usually most people say like, "Okay, I get that. I don't want to do that. I'm fine with having my own little tool to help my my day." And I'm like, "Great. Go have go have fun. Go crazy.

I don't want to I don't want to stop you from doing that." So and yeah, we we try to provide a nice safe environment. So we have a a better VPN which now we can make sure again that that there's not some rogue MCP or some prompt injection or something else in there. We can we can kind of give people this nice safe environment to be in to say like yes in here I think you're safe. Do do your thing. and hopefully they can they can use those tools. Right. Gotcha. Carl, what's the shape in your or that's a big question.

I I don't I don't own AI but I do sit on there is a team of people in charge of you know that is guiding that process at CDW. I have a seat at that table. but the the size of the problem is is quite immense. there are 15,000 employees at CDW. As of last week are our our running agent account was at 9,800. And so we're a big organization that's leaned heavily into this idea of how does AI accelerate workflows across the entire enterprise.

And so there's been some really fascinating stuff that I've seen that have come out of that u both you know in the in in the in the finance in the legal and in the sales process. I'll say that the the the shape of what it looks like right now and I' I'd say this is probably the same inside the technical organizations as well is that people are wrapping a they're molding AI into their own image, right?

They take their own workflows and the things that they have to do and everyone does that in a fairly there's some sort of customization and so they've built some kind of agentic experience in the myriad of tools that they have access to. that replicates that particular process. but there's very little sharing of that and and when it and it's not that

Kyle Crum 47:14

For want of that people are protective, it's that when you publish something like that, it's so hyper customized to the way you run your workflow, whether that is responding to a client quote or writing an S. W or reviewing some legal language that the next person takes a look at that and it's enough different that they're that it's it's it's feels unusable for them. So I think we're in the in the w you know in the in the diver diverging part of the experimentation cycle here. and as things start to become a little bit more consistent we'll pull that together.

but we're we're really you know focused on security. we our clients put a lot of trust in us and so we're first party you know that that security governance and reliability is and the safety of our client base is really top of mind for us. so it's a it's a really fascinating thing to watch. and and I'm you know I'm obsessed with the software engineering part of it like everyone at this table. but it is fascinating to see everybody else also use all this tooling.

I do think well it it reminds me like a lot of the stuff reminds me of again sometimes these old school like build versus buy decisions and I see people making this you know now now they're exposed to like what engineers do a lot of the time and so like it's yes it's really exciting and fun to build something and I'm just I'm waiting for a year from now we're like I'm getting really tired of supporting it's like yes really it really is kind of annoying to support it so maybe just use like a vendored solution right cuz you know some of the builders versus buy is like, yeah, any vendor solution that doesn't do 100% of what you want, doesn't do it exactly how you want to do it, but you know, it takes care of all these other

Dom Scandinaro 48:55

Problems and also everyone else is using it in the same way. So, you're all on the same page about just how this system works and that's good enough for whatever you need to do. Like, yeah, you can always make your own personal solution, and that's fine.

The trade-off, which I'm sure people will figure out down down the line, is like now you have to maintain that and keep it up and the world around you changes and then like how much time do you want to spend, you know, messing with your own little personal system versus just saying like, okay, I'll just use a vendored solution on my own. So, it' be interesting to see a year from now how some of these like kind of personal things shake out in the future. Yeah. I mean, I think internally we're trying to to be the vendor, you know, with this automation team at Cameo.

So, we we rushed to get the, you know, the enterprise agreement signed with Claude so we get Claude desktop to to our whole company earlier this year. Previous to that, we had kind of our own homegrown chat GPT looking interface on top of AWS Bedrock to get people private and secure access to these models. And at some point it didn't make sense to try and keep up with you know the frontier software anymore and we just signed an enterprise agreement got everyone cloud desktop.

By the time we signed that agreement and got it to everyone you know when we started that process I was like we got to get this to everyone get them learning how to create skills you know having lunch and learns teaching them all how to do all these things.

And by the time we got it to them, which is only a couple weeks later, I really started to question, I think a lot of the stuff that you're expressing now, is like, how many people do we want building their own skills, building their own agents, managing these things in isolation where they're going to potentially be like 80% repeated by the person next to them and often wrong, right? Like just like people trying to, you know, write a SQL query without the data analyst [clears throat] reviewing it. like they might get something that

Josh Anderson 50:48

Looks right or build something that looks great but it's like you know creating a pitch deck to send to the customer with the wrong data in it kind of thing right so I think we really

Dom Scandinaro 50:56

Quickly decided we are still interested in that we do still want to enable people but we also want to embed these people throughout the the other departments at the organization to see the similarities and problems within the department and try and build kind of an internal vendor tool to give to that department so they can all be using the same thing we can support it we can scale it for them over time. but I think that's the challenge is like what's the right answer to this? We're still in the the early days of it.

So, how did you So, you've hired three, you took a stab at one as a trial run, make sure it kind of worked out and it worked. So, you tripled down. What are the skills? Does it look a lot like a developer or is it developeresque? Like, what's the profile there? I mean, I think they have to love technology because I think it's a super creative person, right? Like somebody that is interested in working with a a bunch of different APIs. They're interested in, you know, writing scripts that run on, you know, a Mac to try and send an iMessage type thing.

Like, it's got to be people that are, you know, interested in, you know, building things. But I think like you know somebody that works with a you know if they were saying in the interview process they're working on like this project with their Raspberry Pi on the

Josh Anderson 52:12

Weekend to do X Y or Z home automation thing. I think that would be [snorts] kind of the perfect candidate cuz I think that describes the day-to-day like uncertainty of like taking these seven tools. There's no documentation. Claude

Karl Jackson 52:24

Doesn't know exactly how to combine them together. You just have to kind of understand each of their interfaces and figure out the right way to put them together to build something that's actually useful to the person that like in the example earlier is copying this thing from here to there and here to there and then you know turning the spreadsheet into a PDF and emailing to this person like you got to be somebody that's willing to to kind of think outside the box I think to come up with the best solution to be helpful there.

So a classic tinkerer, someone that, you know, took apart old tech took apart the VCR, you know, and just kind of figured out how that work. So that same mindset [snorts] that we've seen in DevOps or various things like that where there isn't a very well- definfined path. It's there's this new thing, we need it to work, but there nobody's done the work. You can't say give me somebody with 5 years of experience doing that because they don't exist. So you've got to chart your own path there. Maybe you just figured out the interview process, though.

We'll just take apart the VCR and put it back together. There you go. Problem solved. Problem [snorts] solved. Okay. So, the job gets harder. Expectations get higher. I say expectation inflation is one of the biggest things that we wrestle with as leaders. Sometimes we are our own worst enemy there because we like to learn and there's so much to learn and we have a responsibility to the organization to be the one that has all the answers but there's sometimes where it feels almost

Dom Scandinaro 53:52

Impossible to keep up. Are you guys getting more internal or external pressure for AI to solve all of the things or is it about equal? Where does it sit? Start with you. For me, I don't I don't get a lot of external pressure at the company to use AI for stuff. And I think that's great.

Josh Anderson 54:16

You know, for me,

Kyle Crum 54:18

You know, the way that I think about solving problems is like, you know, first like defining what problem we're solving is and then like what's the actual cost and outcome of a solution. Like there were times where like you know I want to be careful of using AI to solve a problem and then it's like well you know okay we saved $100 a year or something like that's not that's not a good use of my time.

And there are times where like I thought like wow I would be great at doing this and that and I go look and like okay even if I approve the efficiency of this you know process by 50% like it's it's marginal not worth the support time and engineering time and all that stuff to do it. So it's like you know not not worth it. so I haven't gotten a lot of [snorts] external pressure.

you know, I have my own internal pressure like this is like this powerful tool and I've been, you know, doing tech professionally for over 20 years and, you know, tinkered with computers for probably over 30 years at this point. And so like this, you know, for me this is like this [snorts] is amazing. like I can I can put all of this knowledge I have like to use to do like all of this stuff. and there's there's a neverending like list of stuff. it's just too easy for me to be like yes I'm going to do that. I'm going to do that.

I can run that. I can do that. I can do that. And I can I can do all that stuff. But it's you know I am I'm getting back to the the kind of leadership principle of like deciding like the signal versus noise like what should I really be spending my time on. Yes, I can do a lot more but like should I be or should I really be focusing on these few things? So for me yeah a lot a lot of my pressure is from me. I want to do stuff. I want to be I want to be on top of it.

and I don't know how true that is, but that's just that's just my my internal compass. yeah, I don't know if I don't know if it's not maybe direct pressure is that gentle pressure continuously applied. It's it's this idea that we're we have a we have as a client base with our engineering team. and so a lot of our client base is coming to me looking for they look you know you're going to consulting company you're going for one of two things right expertise or capacity.

And so on the expertise front I get a lot of like hey how should I be thinking about AI in my ecosystem? and so the the pressure there is to be relevant to that conversation when it shows up. and there's definitely a expectation inflation in the fact that everything's moving so fast that if that you're expected to stay up on on that speed. So there's definitely there's definitely pressure and I think it becomes how do we intentionally learn together is the thing I always come back to.

and also saying sometimes like hey we don't know like this is what we do know and this is what we don't know and this is what as a community of software professionals we do and don't know and how we might experiment to find the right answer. Yeah. Dom you and I talked about your rewrite of your website and something that you've had technical debt there for a long time and AI enabled you to tackle something that had been haunting you for a while. Did that change the conversation of like, oh, well, we could do that now. What else can we do?

Yeah, in good and bad ways, I think, as you kind of implied in the the tone of your question there. but yeah, we it it's hard to say. It was a big project how long it would have taken. I'd estimate it would have taken 6 to 12 months for us to do with several people working on it. But we took our 10-year-old website, a lot of product debt, a lot of tech debt.

the main motivation for rewriting it was actually to get faster with AI right we're looking at all these other examples sometimes on Twitter sometimes in real life of colleagues that you know at other companies where they're building in something more green field that is a more AI native product and they're

Josh Anderson 58:30

Having a lot more success with cloud than we were having at Cameo at the time and we were first investing earlier this year in well let's update you know all of our dependencies react navigation all

Dom Scandinaro 58:43

That sort of stuff to the latest and greatest because that's the documentation that the version of Opus we're using is based on and when it tries to to build on our version it gets confused and everything starts tripping over itself as well as the fact that we've got three different engineering patterns to solve the same thing in the codebase. We have all this product debt for features that don't really get highly used or supported anymore that confuse the agent when we ask it to to do something.

so the idea was after trying to to upgrade all of that unsuccessfully, you know, in the background over a couple months to just pitch this idea that we're going to take a couple months to slow down to speed up and on the other side of it be more of an AI native company was kind of the the tagline of the project. And that was a surprisingly easy conversation I think at the the leadership level versus what I thought it would be to get that kind of signed off on. But yeah, we spent a couple weeks to create the foundation of the new website.

with everything being very agent driven, all the documentation, the cloud MD, agent, MD, all that sort of stuff being being very agent first, describing all the business domains and you know, language and all that sort of stuff in a really friendly way. And then being really thoughtful as we built each of the features we needed on top of it with Claude, saw where it went wrong, tried to understand what was missing in the context to to make it less than desirable, change it, and kind of rinse and repeat until we were done. Took us about 2 months.

Two people did the first half of the project, like six to eight people finished the project for the the second month of it. but on the other side, we had a completely new website. It's a lot more performant just cuz it's built on the latest and greatest, but we can ship when it is a feature that's built in the front end of our website, we can ship a lot faster now as a result.

I think that's been really good and really confusing to to a lot of people at the company at the same time because, you know, we we have a product roadmap for the second half of the year. Some things are going faster, other things aren't. And, you know, it's it's frequently like, well, we did this website rewrite. like doesn't that mean this thing can go faster? And you know it's it's obviously a complicated answer

Karl Jackson 1:00:58

In many of those cases.

Kyle Crum 1:01:00

That is such a great description. I've been walking around telling everyone that the distance between vibe coding and production grade digital product is still engineering rigor. Yeah. And that's what you described right there. Right. You've got to look at the process. You got to break it out.

you're going to find out how you and those some of the things that you said I think is a lot of the new software development abstraction layer we're operating in which is what are my instruction sets what are my scaffolds and harnesses and guardrails and all these new terms that are showing up that are basically just hey how do I effectively write software that I know I can depend on in production and so for every person who's know gone into claude and said make me a new cameo website you got to listen to like all those pieces that that that it actually takes and how difficult that can be.

Yeah, I mean I I do think it's worth, you know, I we've been using AI in engineering for a lot of similar things of just like migrating code or you know upgrading dependencies or we need to upgrade you know a language version like normally you can do that stuff by hand and it's really just tedious and it's boring and and AI is great at that stuff because it's like here's here's the output you need to have you can give it you know we can constrain it can test itself and it can it can do it and that's that's great.

I I think where I I haven't seen people figure it out and I don't know if people will like but actually like building new things into the product is a completely different you know mode of mode of working right cuz it's not it it really like people think that like you can provide a perfect spec you know like this is what it needs to do and then you feed it to you know AI and like you get you get out what you describe. Right? Like we've thought this for years even with humans like no we just have to write like the best PRD ever.

and then we give it to engineers and they're going to implement it and like that's all we need to do and like oh it didn't end up the way we wanted to but we got to write a better PRD. Like that's not that's not how it works. Like you know it's really about the process of doing it. Like yes you have you know direction. You say this is what we want and then like the actual

Dom Scandinaro 1:03:16

Making process is like it's a lot messier than people want it to be. It's like you get into the code and you realize oh like by the way like we changed this thing. We found this 5-year-old feature in there and I looked through the logs and there's like nobody uses it except for this one person who used it like five times a day. you know what should I do about that one? Like I you know we have to make that decision you know like should I shut that off or should I like you know should we deprecate this feature etc etc.

you know, there's all these trade-offs to be made whenever you start like making new features in the product that I think are I, you know, need human decision, need human eyes. I I I don't see that going away. so this idea that we need, I don't know, less less people to do that. I I I I just kind of I don't see that happening. Yeah. Yeah, I mean I think one thing I' I'm struggling with a ton is that it's this tech has made it really easy to start more things, right?

Whether you're, you know, at our level, whether you're an engineer on our team, whether you're somebody completely non-technical at the company that has Claude Code and, you know, has a fun idea to type into it, it's really easy to start a bunch of things. But the more things you start, the more trouble I have finishing any of them.

And I think that that for me is the biggest trouble with this inflation expectation inflation that we're talking about where it's like well I you know created this little website in Claude Code over the weekend so like if I could do that an hour why can't we do that you know next week 100 times with in 100 hours and it's like well we can start a bunch of stuff that runs on your machine but then like actually fitting it into the rest of the product thoughtfully is is a much different process. Yeah. I mean I Yeah. I think Claude

Kyle Crum 1:05:01

Itself is just like a example of just like yes you can do stuff really fast but I think they've gotten to the point where that product like you know I guess I'm an admin of the cloud instance so it's just like you know that here's a new product like do you want to enable it in beta and it's like sure and there's actually one on there I said yeah I want to enable it and then I was like but what does it what does it do so I was you know I couldn't find any documentation on what it did and so I actually asked cla I'm like what does this feature do and they're like I don't know I finding documentation like where did you find it like in the settings of your thing you know like they're going so fast that like even the users of the like even cloud itself can't keep up what's what's going on you know and I think about that like if you if you have a product that's salesdriven you can't go back to like you know the potential buyer you know every or client every time every every month and be like I got a new feature check this out I got a new feature check this out like they're going to be dude, like I I just adopted the feature you had like last month.

Like like chill out. Like we need to we need to figure out how this works out with us. like why don't you think about how to make all these other things better instead of just like you know putting more stuff on my plate. So I I do think there's like a you know a little bit of a speed limit there with users and and you know then I think it's really become a fidelity thing like who can actually make things better.

There's tons of, you know, there's tons of Jira like apps, Jira linear, all this other like, but it's going to be like who actually makes the thing that's better like more performance, more high fidelity, not like who's adding all these features quicker. Yeah. I' I've I've never been in a company that was good at getting features into the go to market organization effectively and well and fully understood at the pace that we could go at. historically. Now our pace is even faster.

So we already were bad at getting sales and marketing to understand what's in it to getting the support team to actually know how to support it. So when they when Claud asks or when you ask Claude, they can respond. That's the huge hurdle that I see right now. Is that tripping up your organizations at all? Yeah, there's there's a there's the product community is talking a lot about, you know, is a feature done when it's in production or is a feature done when it's adopted, right?

And I think the the rate of digital production far before AI has allowed us to release feature sets at such a velocity that closing the loop to say what's valuable and what's not is just not a thing that we've been historically good at. however, I do think the fact that those loops are getting faster to run means that the user feedback is getting more and more important. So there's definitely an increased level of intensity in saying, okay, what are we releasing or why are we releasing it?

Tom, I know that you're driving a lot of bottoms up planning within your organization and I'm a huge fan of that. That also has some challenges, right? So, I'd like to talk through with this crew how you're attacking the problem

Josh Anderson 1:08:22

Because Dom, just like you said, the capacity to build useless stuff is greater than ever. And so, I go back to historical lean and waste. Your capability to create waste is at an all-time high. How do we corral that? How do we educate our engineers so they understand what waste is, what waste

Dom Scandinaro 1:08:43

Isn't? you start to dip into some of the the judgment and taste challenges that that are really hard to figure out. So, I just want to understand from each of you the way you're trying to figure that out. Again, nobody has the answer here. This is just like, hey, this is where we're at with it. Yeah. So, I mean, we're trying to move to to smaller squads where it's just a couple engineers building together.

We're also trying to move from I mean I we're early in this but like trying to move from software engineer to builder where builder is maybe a more broad term that you know a product manager who wants to get more involved can be a builder. Our designer actually is is opening up more pull requests at this point I think than most of the engineers on our team. So really changing what it looks like to to build at Cameo is a big piece of it.

trying to give the teams more ownership of individual areas, build them up into subject matter experts versus having to depend on the product manager or the tech lead for every single thing to decide how to get it done and instead just say like you own these surfaces. We're trying to get you to move these metrics generally here's the road map go. so that's what we're trying to do. I think it's causing a bunch of communication burden for us internally though on just the teams are you know they're all they're going they're building they're opening pull requests they're merging they're shipping things to production.

some areas where it's been really successful is just rapid experimentation in the the consumer funnel right trying to get more people from one phase of the process to the next. We're trying to get more people to buy cameos. We're trying to get more talent to fulfill the videos. So those things are really easy to just kind of ship in isolation, run experimentations on and and you know, ship the treatment if it wins, roll back to the control if it doesn't without needing to communicate, you know, a ton of things to everyone because they're just minor changes that that aren't like changing people's day-to-day significantly.

but I think it's really hard for us to actually communicate reliably right now all the things these individual teams are doing up to the stakeholders to the point where they feel appropriately aware of what's being built, why we're being why we're building it, and how it ladders up to the company's goals. even though we were aligned on it kind of going into the second half of the year when we were doing it.

But now that things are just happening so quickly, I think we're just kind of stuck in this this process right now where we're like overcommunicating around who's doing what, why they're doing it, if we're if that's still the right thing to do because there's so much kind of interest in doing things faster and trying to ship bigger swings and and find the thing that'll help the company grow even faster than we're

Josh Anderson 1:11:19

Already growing. So, it's a good problem, right? Like everybody's trying to do the right thing, but it's still messy, right? Going from that very structured, you know, I am deciding what you're going to work on and telling you what to work on versus, hey, everybody now has the capability to work on the next most important thing. That's a that's a huge cultural shift, a huge cultural change. And that's even with like build with intent from the start, that's still kind of messy as you sort it out and figure it out. Everybody gets their feet up and under them. What about within your organizations?

How's how's product driving or is it along for the ride?

Kyle Crum 1:11:58

No, I mean yeah I mean a couple of things I you know I have a really good relationship with my product partner and with the design lead. so I don't you know for me personally I I I consider myself half in engineering and half in in product. So I I you know yeah it's hard for me to think like I'm a pure engineer and all I care about is implementation. Like I I you know I care about the user problems and how we're solving those. What's the strategy there and what's my part in that.

So I for me like I haven't seen it change that that much. and I I don't know if I expect it to or if it will. Like one thing that I've noticed is is people's tendency possibly to use AI for other people's job functions. And you know for me like I I I want people to use AI respectfully. part of respectfully is like understanding other people have skills in other areas that are you know they've built an expertise in that for a very long time and they know more than you're going to know even though the LM will tell you one thing or another.

and so you know I'm always really careful with it. especially you know with with engineers to say like yes just because you're good at engineering doesn't mean you are good at design or that you are good at product like you can understand

Josh Anderson 1:13:34

Product and you can dip your toes like I read a lot of stuff on product but at the end of the day like my product

Karl Jackson 1:13:38

Partner thinks about product 100% of the time so she's going to be better at that sort of stuff than I am you know we're on some things so I know I can I do some stuff and I pass it by her and she's like yes like that that aligns what we're doing but you know these other job functions like if I'm working with people on finance like my first thing is to be curious about them and their work and like tell me what you do and like I can learn from them but just because I learn from them doesn't mean I can go do their job from working with even customer service like I would be a terrible customer service agent like I you know like I could not respond so gracefully you know in the way that a lot of them do like they're they're just very very good at their job.

And so like part of like the AI thing that I've seen is like you know and that that I worry about is people being like oh like yeah you don't I don't know you don't need product people you don't need design because there's cloud design or whatever like you know I'll spit out a strategy doc from claude and I just you know I

Josh Anderson 1:14:38

Want to be careful of of that kind of thinking. so I haven't seen the roles change that much. And in terms of like velocity, we've always been pretty good at at at shipping quickly. and it's always been a it's always been a contention that like we ship stuff and someone doesn't know about it somewhere, I I you know, I tend to like, you know, do the best we can, but but realize that it's it's going to happen.

you know, road maps don't work how people think they are, which is like, you know, I say I'm going to ship things on this date and it's always going to happen. So, you know, I could give you a road map or schedule, but it's going to change. so try to keep people

Karl Jackson 1:15:21

Informed. We're not perfect about it, but also talk to people about just why it's going to be imperfect and we're going to miss a communication sometimes. Apologies, please work with us. And I would assume the same from you. You might change something, your process is going to affect me, I will be graceful about it. So, let's just be graceful with each other about, you know, how we do these things. And Carl, you live in a slightly different world. I do. we're constructing and deconstructing teams fairly consistently.

And so part of my job is to understand what a client is attempting to do, which is a person that I potentially have never met before and a company I know very little about. our industries change pretty dramatically. I can go from, you know, from retail to oil and gas in kind of a heartbeat.

And but at at the root we're all we we're all doing the same thing right and an engineering team in many ways is just like an agile engine right you try and put requirements and features and stuff on one end and you get working product out the other and we're all working hard to expose the controls of that agile engine that we're building to the our stakeholders we whether that be our product teams it you know our customers in some cases that that that you know influence our products.

And so I think when when I'm looking at at clients today versus maybe two or three years ago they expect a lot more controls in the system and then the engine like Dom said is a lot smaller.

Josh Anderson 1:16:52

It's producing the same amount of horsepower per se with with fewer people and a lot more blurring of roles. So I don't think of them as overlap as much as potential extensions. and so I think yeah being respectful about how people think about other people's work is really important which is another way maybe of saying respecting the expertise that comes from someone's career of talent training and taste right and

Karl Jackson 1:17:23

So we're seeing as those roles blend together we have to be much more explicit in how we communicate that to our clients that are looking to get some kind of business objective done. Yeah. One of the things that respect thing I want to talk through that this will be our last big big topic is [clears throat] the slop cannon and how that's my favorite term and how you've had to educate your organization cuz I've talked to a lot of leaders where they're just like I'm so tired of people clearly sending me something that Claude said.

So, they got what I asked them or they read an email from me or they saw something and their response is, "I'm going to go ask Claude. I'm going to hand it back to you." And it's 50 pages long. And I've talked to a lot of people. It's like, I hire people for their brains because I want that to be engaged. So, how are you guys retraining your team members on I need

Josh Anderson 1:18:24

Your brain to be engaged. I don't need you to just like turn it off and ask Claude. and whatever Claude says you do because sometimes Claude's wrong and sometimes Claude gets a little overzealous. So talk to me about how you guys are tackling that cuz that's an everpresent problem today. Hopefully we're going to kill it, but right now it's everywhere. Yeah. And maybe there's two kinds there. There's the slop cannon is for sure, right? There's a lot of just stuff that you've got to go out and rebuild or doesn't make sense or isn't deeply thought about enough. I'd say there's a content cannon too.

Not even outside code like the amount of white papers I get and decks about stuff or things that even I'm generating.

I I even just this week too have had to step back and say like all right there's I can now say a lot of things right what is valuable to say and I think that's actually two parts I think that's that's obviously some you know selfediting but there's also some empathy right I think what we're missing here is the sense that because we can say a lot of things we just say a lot of things as opposed to saying like hey I'm going to talk to Dom today.

What does Dom dom what is most important for Dom to hear about all the things that I said? Like does it make sense for him to be like, "Hey Dom, we're going to we have a 30-minute meeting. The first 15 minutes you're going to read six pages of claw generated content." That I think is actually really good even if it's not

Karl Jackson 1:19:58

Slop like is that the best use of his time as opposed to being like, "Hey Dom, I've got an idea. You grab a whiteboard pen and I grab a whiteboard pen and let's go work through something together." So I think there's a there's an overall content explosion that or generation that all this generative AI is creating for us. but then there's definitely inside that a lot of quality quality problems and and that I think comes back to say they have empathy which is who am I building for who am I writing for and how is this going to affect them? Yeah.

I mean, my new favorite acronym is AIDR, AI didn't read, and I'll just respond that to people when they send me, you know, their 12page product spec that's filled with M dashes that like doesn't really make any sense when you start reading through it. But I think it all comes back to for me

Dom Scandinaro 1:20:47

Anyway, like I'm pretty sure there's a famous quote that's like, "If I had more time, it would have been shorter."

And I think that that's like a big piece of this is like sure, you know, if if you don't like writing this stuff, use Claude to create the first draft, but like spend the time to go through that, understand it, edit it, try and make it really concise to your point to like, you know, if we're going to take time to talk about something or you want me to take the time to read it, like try to get to the point where it's really easy to understand, it conveys your opinion well and concisely versus like you expect somebody to take the time to read a 12page page document while everyone else is sending them 12-page documents and people are posting about the new model that came out today and everything else, right?

Like it's like if you want people to actually pay, you know, to review your pull request, make it as short as possible. If you want people to read your document, try and make it as short as possible. so I think there's just a lot of effort to go into that. And we're trying to be pretty, I think, respectful towards one another at the company and just not send anything that's fully cloudgenerated. if it is cloud generated, we're asking people to, you know, say that it's cloud generated up front versus like representatives as their own work.

But I also read something today about I think somebody wrote a blog post that was like, stop sending me like, hey, I asked Claude and this was Claude's answer. It's like if I wanted that, I would have just asked Claude myself, right? Like yeah. So I think there's you know two sides of the story there is like you do want people to not pass something off as their own work if it is just output from claude but like at the same time we all have cloud access.

So like don't just send me the thing that Claude told you and say I asked Claude and here's the answer. It's like I if I wanted that I would have just done it myself too

Kyle Crum 1:22:25

Right. Yeah. This is this is a kind of scary thing happening where you know you talk about like claude has a tone you read this stuff and like fact this is copilot or claude generated you can kind of get that tone that people are starting to write that way too I've got some friends in the education space and obviously they're wrestling with AI in the classroom like that's a big topic for them and and they talk about how writing styles are modifying as kids are reading so much more claw generated content that they're starting to write like that as well.

and I don't know what that means for the future of content and cannons of whatever. Yeah. I mean, yeah, I think I mean even even before Claude like I I mean one of the things that becoming like especially like a manager of managers is becoming comfortable with asking people to do things in the format that works for you. and so, you know, like I learned a while ago that like decks just don't don't work for me. I don't I don't I don't like reading decks. I don't because it feels like a sales pitch and I just I don't I don't like it.

If you're going to send something, please like I'd rather have a doc and then I can comment and like I just ingest information that better. So the same.

So like you know I tell people like I don't I don't mind if you know you have something that is clogged just just tell me and like I do it sometimes too like you know investigating something and then I get pulled away to something else and I'll just say like just whatever your current state is like put it over here and then I'll just tell someone like hey like you know here's where I left off it's totally clotted. let me know if doesn't make sense. but can you help me figure this out?

but yeah, I and for me like I I do well with you know there's there's a lot of stuff that comes by like just what's the TLDDR? Give me the give me the bullet points and then if I want to like look at the details I can. But if I look at the bullet points and I agree with it like great I will remember that and we can we're we're good. But yeah, don't make me read 12 pages to figure out what what what's the what's the point? you know because really the top level bullet points are probably going to be

Josh Anderson 1:24:44

What I what I remember. AID is my new favorite acronym. [snorts] Yes, I I was just going to close with that of I am now on a mission to make that a industry standard term that just gets applied everywhere and just instantly responds and is every there's a Slack emoji on every Slack server like a like a very specific aid that just like you send it everybody knows like oh yep sorry Dom I won't do that again. So Dom thank you for that. Okay, everybody, thank you so much for being on the show. Been fantastic guests. Appreciate all the knowledge and experience that you've shared.

So, Kyle, Carl, and Dom, thank you so much for being on the show. Thanks for having us. Thanks for having us.