IT Horror Stories

The scariest stories in IT.

About This Episode

When a huge cloud platform migration stalls out mid-flight with no hope of changing course, panic lunges to take the wheel. In this episode, Mahesh Guruswamy, CPTO at Kickstarter and author of How to Deliver Bad News and Get Away with It, recounts the night he took down production and had no other choice but to push through. Tune in for the blow-by-blow — from anxiously watching the clock as queries time-out to the tightrope walk of breaking bad news without breaking trust. You’ll leave with the brutally simple playbook Mahesh now swears by to make sure your next migration doesn’t become an IT horror sequel.

Host

Jonathan Crowe

Jonathan Crowe

Director of Community, NinjaOne

Guest

Mahesh Guruswamy

Mahesh Guruswamy

Author and CPTO at Kickstarter

About Mahesh Guruswamy

Mahesh Guruswamy is a seasoned product development executive who has been in the software development space for over twenty years and has managed teams of varying sizes for over a decade. He is currently the Chief Product and Technology Officer at Kickstarter. Before that, he ran product development teams at Mosaic, Kajabi, and Smartsheet. Mahesh caught the writing bug from his favorite author, Stephen King. He started out writing short stories and eventually discovered that long-form writing was a great medium to share information with product development teams, resulting in his book How to Deliver Bad News and Get Away with It: A Manager’s Guide Greenleaf Book Group (January 14, 2025).

Audio Transcript

[00:00:00] Mahesh: I was there for about six months now. And this, the project that I was leading was supposed to be one of those big things you do in the first 90 days. So people are like, okay, this guy knows what he’s doing. It was supposed to be that initiative where I’m gonna make a big change, propel the company forward in a positive direction, and that’s when this… this issue happened.

[00:00:24] Jonathan Crowe: Hello, and welcome. Please come in. Join me. I’m Jonathan Crowe, Director of Community at NinjaOne and this, this is IT Horror Stories. Brought to you by NinjaOne, the leader in automated endpoint management.

Introduction

[00:00:40] Jonathan: Hello and welcome to another IT Horror Stories. I’m your host, Jonathan Crowe, and today’s guest is Mahesh Guruswamy. He is the chief product and technology officer at Kickstarter. And in addition to that, he’s also the author of a new book, which is very appropriate for our subject, it’s “How to Deliver Bad News and Get Away With It.” He’s also got a Substack, The Sensible Manager. Mahesh, thanks so much for joining us today.

[00:01:05] Mahesh: Oh, thanks for having me.

[00:01:06] Jonathan: So before we get into your horror story, and also you can share how you deliver the news of that horror story to others, tell us a little bit about your background and talk about this book, actually.

[00:01:18] Mahesh: Yeah. Yeah, sure. Sure. So yes, I’m Mahesh Guruswamy. I’ve been in the tech industry for about 20 some years. The first 10 years I was a software developer, kinda rose through the ranks, got to staff engineer, principal engineer. The last 10 have been in management roles. I’ve done management stints at Amazon, Houghton Mifflin, a bunch of smaller companies, large companies.

[00:01:44] And the last five I’ve been in executive type roles. I was an early VP at Smartsheet. I was a CTO at Kajabi, and I’m now the chief product and technology officer at Kickstarter. And as an engineering leader, I deal with messiness all the time. So this, so I’m looking forward to the conversation here.

[00:02:04] And, I mean, as a people leader, you know, messiness is part of my day to day. And one of the things I struggle with is how do I deliver these messy bits of news to employees, stakeholders, even bosses? And I realize there’s no playbook for it. So about 10 years ago, I started writing these essays for myself, refined it over the years.

[00:02:25] So I would deliver a piece of bad news, do it poorly, go back home and figure out how to do it better the next time around. And then those essays, you know, refined over time, ended up becoming this book because I felt that there was a gap in the market right now, which talks about the bad stuff. So I was like, okay, maybe I’ll be the guy to talk about the bad stuff and here I am.

Getting into the bad stuff

[00:02:51] Jonathan: Here to talk about more bad stuff. I appreciate you and your eagerness to do that and to help people learn from your mistakes so they can hopefully fast forward that process a bit because one thing’s for certain, if you’re working in tech, if you’re working in it, there’s going to be messiness as you said, and there are absolutely gonna be horrific days.

[00:03:15] Mahesh: That’s right.

[00:03:15] Jonathan: You’ve been in several different engineering roles. You’ve been in leadership and executive roles. For our horror story now, that you’re gonna be talking about, where are you, what stage are you at in your career?

[00:03:26] Mahesh: So I was a CTO.

[00:03:28] Jonathan: Okay. And were you an early CTO? Is this one of the first roles or is this at this point, have you been doing that kind of executive role for some time?

[00:03:38] Mahesh: No, this is my very first time at CTO.

[00:03:41] Jonathan: And was this new, when this incident happened, had you been at the company very long?

[00:03:48] Mahesh: No, not long. I think I was there for about six months now. And this, the project that I was leading was supposed to be my, I guess like one of those like big things you do in the first 90 days. So people are like, okay, this guy knows what he’s doing. It was supposed to be that initiative where I’m gonna make a big change, propel the company forward in a positive direction, and that’s when this issue happened that we’re gonna talk about now.

[00:04:09] Jonathan: So let’s talk about that mindset because it’s clear, you know, you’re, this is your first CTO role. You want to show everybody, Hey, you’re here, you’re legit, you’re the real deal. You’re a leader and you wanna make a splash. That 30, 60, 90 days, you actually wanna make an impact and do things the right way. Can you tell us a little bit more about the pressures you were feeling or what you were excited about there, how you were feeling in that role?

[00:04:33] Mahesh: So this was, I was CTO. This is my very first time as CTO at a mid-size company. And the company, it’s like a large eCommerce company. And we would process about a billion dollars of transactions every year.

[00:04:46] And this is money our customers are making through the platform. So a lot was at stake. We’re a global company, so there were no, what I would call off times, you know, every day, we would make money for our customers. The big issue back then was the stack was running on Heroku. You know, the company was one of those early Ruby on Rails shops that adopted Heroku as their backend or as like, platform for development.

[00:05:15] And when I joined, it was clear that they had outgrown it significantly. The database was like tipping over every day and it was causing outages for the company and causing lost revenue for our customers. So I came in and went, okay, well you, we have to like move this off of Heroku and go to something big and scalable and maintainable.

[00:05:38] So we all decided that we’re gonna go to AWS. The original plan was to migrate the web pieces first and then do the database next, because the infrastructure team was fairly new to AWS. They were like, we wanna do the web pieces first because it’s easy or easier to do than the database.

[00:05:57] And, and I was like, well, let’s do the hardest one first, because that’s the one causing the most pain. And I don’t think the team was a big fan of me saying that, but it was something that I strongly felt was necessary to prevent additional revenue loss for our customers. So that was, that was the stage.

Don’t migrate without a plan B

[00:06:14] Two other bits of information. One is that the team proposed a plan B. You know, if things go south on the day this cutover was supposed to happen, you know, if we had more time, we can develop a plan B so you can walk back and go back to the state you were before. But that plan, like developing plan B required additional investment of like a quarter, and I was like, well, this is like way too much time.

[00:06:43] We can’t keep losing the revenue we’re losing, so it’s okay to not have a plan B. We’re gonna go through this one way door, and the only way out is through this door. So if we don’t make it, we have to like find this, find a path out to the other side. That was one thing.

[00:06:58] And the second thing I realized as CTO, there’s nobody above you to help you, per se. So if you are like a, even if you’re a Director or a VP, you can go to the CTO and say, Hey, I need help with this thing. Can you help me out? Here, the person above me was the CEO, and the CEO was trusting me to get this job done.

[00:07:16] It’s not like I can go to them and go, Hey, you know, let’s brainstorm on this solution. What do you think about this? They had like zero time for it. They’re like, yeah, get this thing done. You think it’s important to do so you have the freedom to go do it, but get it done.

[00:07:29] Jonathan: Were you feeling pressure in terms of, you know, you mentioned the stakes here that, the system’s tipping over and you know, it is costing customers. There is real pain here. Were you feeling that pressure from the other executives to really make this happen quickly? Or is that something that you stepped in and you’re like, okay, this is a clear problem. I wanna make a statement here and fix this problem as soon as I can.

[00:07:53] Mahesh: For sure. It was, the pressure was on from multiple fronts. One is, the CEO was like, we’re losing revenue to go fix this. That was one pressure. The second pressure was the cutover to AWS required us to take a small downtime. Well, we’ll find out soon. It was not small, but the small agreed upon downtime.

[00:08:16] Back then, when we designed the plan it was five minutes, so it was like, we’re gonna take a five minute downtime to cut over. And the CEO had concerns about the five minute downtime as well. Because they were like, we’re gonna lose revenue here. Customers are gonna lose revenue, so this better be just five minutes.

[00:08:35] It can’t be more than that because, the more revenue we lose, the more our customers will like hate us. So pressure was on both sides. It’s like you have to fix this issue you have right now and you cannot have additional downtime when you do the cutover. So it was on multiple fronts. And customer support, the leader there was like do whatever you can to prevent, like, loss of revenue for customers. So it was on from multiple fronts, from multiple executives.

[00:09:06] Jonathan: Yeah, it seems like you’re in between a rock and a hard place at this point because you’re facing the pressure, yes. We need to have a change happen. At the same time, that change needs to have zero impact if possible.

[00:09:19] And you’ve been presented with, okay, here the team presents you with some, here’s some plans we could do, here’s ways we could go about this. And you’ve decided, okay, we’re going to go with the route of, we need this to happen sooner. Burn the boats. We’re not going to come back. We’re just gonna go full steam ahead.

[00:09:39] Mahesh: That’s right.

Burn the boats

[00:09:40] Jonathan: Yeah, well, let’s take us to the day. So, you mentioned businesses worldwide or customers are worldwide, so there’s not that convenient time that you can actually just kinda sneak this through.

[00:09:50] Mahesh: No, there, there wasn’t. The closest we could get to was a window from 3:00 AM to 5:00 AM on Sunday morning. Just a little bit more detail about the database itself. We were on Postgres on Heroku. We went to Postgres on Aurora, which is AWS’s managed database.

[00:10:12] So it was me, four other engineering infrastructure engineers, 3:00 AM on a Sunday morning, ready to push the button to make this happen. The team had done all the legwork to get us to this point, and now we were like logged in. And then we pushed the button. And we’re looking at the console inside AWS because that’s when we ran the script. The new database should come up. It should spin up the nodes and then start populating it, and we’re looking at the console and the database is not coming up. And so it would come up, it would get pegged to a hundred percent CPU. And then AWS will be like, oh, this cluster is unhealthy. I’m gonna bring it down. I’m gonna bring another one back up as a backup. And that one would get pegged to a hundred percent too. So that’s when I realized we’re screwed now. Something is up.

[00:11:05] Jonathan: And so here you are at 3:00 AM, you’ve done it, like you’ve pushed the button, you’re kind of holding your breath, and you see these errors start to come in. You mentioned it’s you and your team. You’ve got a few others there. What’s the mood in that room there?

[00:11:17] Mahesh: I mean, we really thought that this would’ve been a slam dunk ’cause we had been working on it for so long. When we saw the hundred percent CPU, everybody immediately realized that something was terribly wrong. And the other thing that was acting against us to a certain extent was the team was fairly new to AWS because they’ve been in Heroku-land for so long, and now they were learning AWS, so somebody was like, okay, well this is broken. We have to do something.

[00:11:51] The lead developer who developed the plan started looking at the scripts that he was running to get this thing set up and up and running. I think it was Terraform Scripts that brought the database up, and he realized that you can set the number of virtual CPUs inside Aurora. You can say anywhere from, I think it’s like two to, I don’t know, 200 or something you can set when the database comes up. So our plan was to bring up 24 virtual CPUs, basically like we’re getting, giving like 24 course to like run, run our load. And there was a typo in the script.

[00:12:29] So instead of 24, he gave it 12. It was an honest mistake. Anybody would’ve done it. I could have done it. Nobody’s fault. That’s when we, that’s when we realized that, this is not gonna work. Because the minute we bring this, the database up it’s just getting flooded with requests that it cannot handle because it has a lesser number of CPUs than originally intended.

[00:12:48] Then we had an outage because of that.

An outage ensues…

[00:12:52] Jonathan: Because you also can’t say, well, we’re gonna go back now. Let’s just revert back to where we were. That’s not possible.

[00:12:59] Mahesh: That’s right. It’s not possible. Because as you said, we burned the boats. Right. And it was, I take it as, it’s learning for me. It was my mistake. I should have made the additional investment to come up with a plan B, but I was fairly confident in the migration because I thought it was going from Postgres to Postgres. This cannot be complicated. It should be straightforward. And in reality it was, it’s just like we made a typo. It’s like a small typo that caused this outage. The other thing was the engineer didn’t realize this immediately. So we were in the console messing around the server, trying to figure out like what’s going on.

[00:13:38] And at the 30 minute mark I was like, we have to tell customer support that this is down, the app is down, the site is down. We can’t do anything. So I reached out to customer support and let them know that this is down.

[00:13:55] And then I texted the CEO saying, this is down. They were not happy, but they realized that, you know, the only way out is through. So we just gotta let the team work through this. And then I think at the 30 minute mark is when we realized that typo in the script, that it was 12, not 24. And those, it was, it was wild. The team was hesitating to push the button to fix the number of CPUs.

[00:14:24] Jonathan: Sure.

[00:14:25] Mahesh: Because they were like, the app is somewhat working because some requests were coming through. If we push the button their worry was, we don’t know how long this is gonna take. This could be five minutes, could be 20 minutes, could be like an hour.

[00:14:43] It’s down anyway. We can’t be more down, so might as well just push the button to see what happens. And we pushed the button and it came back up in about 10 minutes.

[00:14:52] So it was not, it was not the worst case situation. It was 10 minutes. But by the time all of this had happened, it was about an hour. So we had an hours, you know, 60 minutes of outage. And then the fun began with explaining to customers why this happened.

[00:15:08] Jonathan: And hence the title of your book.

[00:15:11] Mahesh: That’s right.

[00:15:13] Jonathan: I’m just imagining, you know, having to make that text or that call or take that call when your CEO, you know, gets back to you in the middle of the night on a Sunday morning. You know, were there any things that were kind of prepared?

[00:15:27] Were people already potentially ready to be on alert or was everyone just kind of trusting that, okay, this is gonna work, we’re gonna be okay. Did you have to scramble anyone? Was anyone kind of at the ready?

[00:15:39] Mahesh: Oh, so customer support was definitely ready.

[00:15:43] Jonathan: Okay, yeah.

[00:15:44] Mahesh: Because they’ve seen situations like this in the past, so customer support was like logged in. They were like, yeah, we’re ready to go. The engineering team wasn’t fully ready because we had just rolled out on-call procedures. So people just like call each other when they needed help in the middle of the night or whatever. So it wasn’t formalized. But when I joined, the team was about 80 and it eventually grew to like a hundred fifty, a hundred and twenty, a hundred fifty people. And without on-call it’s hard. You can’t remember everyone’s cell phone number. You can’t like tell everybody to be prepared all the time.

[00:16:17] Like you just can’t do it. So I rolled out on-call procedures and we didn’t do any fire drills or we didn’t do any extensive fire drills of getting paged and responding to pages, et cetera. I had an actual beeper for when I got paged. But these days, these things are apps like PagerDuty, it goes on your phone. And you have to explicitly tell the phone when you, or like tell PagerDuty that it can still make an audible even when your phone is on silent.

[00:16:47] And a lot of people didn’t check that setting or deliberately put it to silent. So we actually paged one of the dev engineers, like the lead dev engineers when the outage was going on because we thought there was maybe a bug in the code.

[00:17:01] And we couldn’t get a hold of him because he put the phone to silence at like 3:00 AM or like 4:00 AM on a Sunday morning. [Thinking] like nobody’s gonna be awake, and then I eventually got ahold of him at 6:00 AM when he was making coffee.

[00:17:16] Then he joined the call and I asked him like, Hey man, like why didn’t you like, respond to the page? He was like, my phone was on silent, so that’s why you’re like, oh, crap. Like if he put it on silent, chances are most of the engineering team also put it on silent.

[00:17:29] Jonathan: This is the point where in the horror movie you see that, you know, the person makes that little move and you know, that’s gonna come into play later.

[00:17:37] Mahesh: That’s right.

[00:17:39] Jonathan: Grand scheme of things I mean, an hour. Of course, I’m sure it felt like an eternity. Grand scheme of things, you’re able to move quickly and figure out a solution and resolve that fairly quickly. I mean, did that period, was there relief there when you had it done, were you guys like high fiving? Did it bring you a little closer together as a group?

Relationships are build in the trenches

[00:17:58] Mahesh: I say this to everybody who gets introduced to on-call for the first time, that relationships are built in the trenches. And in engineering that is especially true. I remember, you know, not just at that company or in prior companies. Even at Amazon, when engineers are in the trenches fixing production incidents up at night trying to get stuff to work, that’s when they build friendships, that’s when they build camaraderie.

[00:18:27] And we were definitely much closer after that. Because now we had gotten through this and then we always reminisce about it in future incidents. It can’t be as bad as the time on the app went down and we are trying to do this migration. Right?

[00:18:45] Jonathan: Forged in fire. Well, let’s talk a little bit about some of the changes that rolled out. Obviously now we need to have this in place or that, let’s talk about that.

[00:18:54] And then I want to also veer over into how you deliver that bad news and talk a little bit about that. But, yeah, first things first, with those, what are some of the big changes that you made as the CTO having gone through this?

[00:19:08] Mahesh: Well, a few things we did. One is, we mandated dry runs for on-call every quarter. So every team would go through their own fire drill every three months so that people are used to responding to pages. Because as an engineer on a specific team, you could go months without needing to respond to a page.

[00:19:28] So when it actually happens, you’re like, oh, shit. Like, what do I do now? So, so that we did mandate that. The other thing I realized was the team wasn’t super familiar with AWS, so we ended up paying extra for the enterprise support for AWS, which is expensive and I hated doing it. But it, because I always feel like, well, I have engineers, like why do I need somebody else to explain this thing to me?

[00:19:51] ’cause we should already know this. But for the first year we paid extra for the enterprise support where, as part of the package, like a solution architect from AWS came over, looked at our data schema, looked at our setup, and gave us feedback on what we can do better. And it also gave the team an opportunity to ask this person questions like learn from them, et cetera.

[00:20:13] That was super helpful. And the third thing we did was we made a joint decision about Plan Bs. So not unilateral decisions anymore. ’cause I realized that I should have trusted my team’s council. So every time I was about to make a one-way door decision. Okay, it’s only one door. No coming back.

[00:20:37] I’ll get my team’s point of view on it. Oh, okay. What do you think about this? And, that has been super helpful and I still continue to do that even in my current job when I’m about to make a decision that’s gonna change the trajectory of the team or, or the company. I wanna do it somewhat together.

[00:20:53] Jonathan: Those are great. And now let’s talk about the communication part of it.

[00:20:56] Mahesh: Oh, the, yeah. Right.

[00:20:58] Jonathan: Yeah, absolutely. I mean whenever you’re, we’re hearing these horror stories, there’s absolutely the technical component of it. Trying to troubleshoot an issue, trying to fix the problem. And really when those situations happen, you’re kind of in the moment you’re, it’s just go time.

[00:21:12] You’re just kind of heads down and, and you’re working together to, to just fix the issue. I found that hearing people’s stories, what tends to last a little bit longer is, or be another completely separate challenge and a scary element in itself is the communication part of it. It’s one thing to, ‘okay, something’s broke. I’m going to fix it,’ than to have to talk to people who are upset, who are scared, who don’t want to, you know, want some updates now. Who may be angry in some cases. Especially because that’s what your book’s about, would love to get your take on how that process went back then and how that shaped how you approach these situations now.

How to deliver bad news better

[00:21:52] Mahesh: Up until that point, I was used to just updating a status page. With the, I guess the status of the incident and expecting customers to be okay with it. And this is the first time when the incident happened and we were doing the standard updating the status page saying we are in the middle of an incident.

[00:22:12] We’re investigating, next update will be at this time. Talking to the CEO really helped me ’cause the CEO’s advice was, imagine what your customers are going through right now. Put yourself in their shoes. What do you think customers would wanna know? If you were a customer of this product, what would you wanna know?

[00:22:34] And that really changed the way I looked at things. I originally thought that who needs the details? You know, most customers are not technical. It’s like, who needs these technical details? But then, that’s when I realized they do appreciate the details. So I ended up going to the community page and writing up a blog post, not a blog post, like an update of what actually happened.

[00:22:58] And because without it, the customers were just like spinning up their own stories.

[00:23:04] Jonathan: Right.

[00:23:05] Mahesh: They must have gotten hacked or, you know, maybe their logins are down or whatever may be the case. So I went there and I explained to them what actually happened. I explained to them what we did to fix the issue. I explained to them what are we gonna do next, like in the future to prevent incidents like this from happening. I also gave them my direct email and said, Hey, if you want more information or you wanna talk to me or you wanna talk about refunds or whatever may be the case, you know, just reach out to me.

[00:23:38] We did end up giving a bunch of refunds to people who asked. We also, in certain cases gave back the money they lost to some big customers because it was like a significant loss for them. Even if it’s like an hour loss, an hour of outage.

Lessons learned

[00:23:55] Basically my takeaway from that exercise was that customers want transparency. That’s their number one thing. They wanna know what is going on. So that’s number one. Two, they wanna know the details of how are we gonna fix it. Not just, we’re gonna fix it with code. So they want more than that.

[00:24:17] Like, what are you gonna do to like, make this better next time around? And the third piece is, what are you gonna do for me? Because we have lost revenue, because we have lost time, or we lost productivity, whatever might be the case, what are you gonna do to help me with my situation? And the last takeaway for me was, if you wanna be the CTO or an executive for any meaningfully sized company, you have to develop a thick skin. People will yell at you. People would question your credibility, your credentials. Not a majority, but a small group of people, customers, were like, well, you need a better team. Or maybe like, you need to go like, learn this thing, or variations of, you’re not good at your job, basically.

[00:25:01] Right. And you cannot take that to heart. I think it’s just like customers lashing out because they’re upset that their livelihood is being affected. But yeah, you do have to like to develop a thick skin.

[00:25:12] Jonathan: It must be incredibly tough, especially when you’re having a stressful moment yourself. You’re the main character in your own horror story, but this is having a ripple effect. And the other members of your team, customers, they’re now having a horror story too, and they’re the main characters.

[00:25:28] And being able to kind of acknowledge that, and treating them with really the kind of consideration and respect that you would wanna get treated with, right, with transparency and all that. But it must be incredibly tough too.

[00:25:40] Mahesh: That’s right. That’s right. It’s funny ’cause when the incident was done, I reached out to the CEO and I said, if you wish to fire me because of this, I’ll be like, not upset. And they were like, nah, you’re not getting fired. You have work to do.

[00:25:56] Jonathan: Amazing. Is there anything, looking back on this now, obviously, you learned from it. You mended those relationships, you adjusted your approaches, any other lessons learned, I guess that you want to let the folks listening know about?

[00:26:16] Mahesh: It always helps to have plan A, plan B and plan C vetted for any big change, engineering or otherwise. Even if you’re making a product change, you’re making a business decision, you wanna change the name of the company. It always helps to think through things completely. Especially if it is a one way door.

[00:26:38] If it’s gonna be extremely expensive for you to walk out of that door, then you really have to think through what could go wrong, what would you do if X happens? And, just be prepared for it. ’cause you never know what’s gonna happen. As Mike Tyson said, everybody has a plan until they get punched in the face. And that’s especially true for tech decisions.

[00:26:57] Jonathan: There’s that getting feedback, there’s considering the different options, making sure you have options available. But in the end, you know, you mentioned it yourself, especially when you’re CTO, when you’re an executive, there’s no one above you to make the decision for you. You have to be decisive in the end. And so you did decide to push the button a second time. How do you balance that with, you know, knowing that okay, you’ve been humbled a bit. There’s, you need to take another input, you need to make sure that there’s options, but at the end of the day, you still need to be able to act confidently and decisively. How do you manage that?

[00:27:27] Mahesh: That’s something that I learned, even as a middle manager when I was just sort of director, senior director. There is a cost associated with delaying your decisions, and I will always optimize for just making the decision and see what happens. The thing that I try to do quickly in my head is if it’s, is it a one-way door or is it a two-way door?

[00:27:50] If it’s a one-way door decision, like getting too AWS is a one way door decision, but pushing the button is a two-way door. It cannot be more dead than what it is right now. So if it’s a two-way door decision, I don’t tend to, you know, think about it too much. So I say, okay, it’s a two-way door. You know, we can quickly get out of this situation if we, if we need to.

[00:28:10] I think that’s something that I teach the people reporting to me as well, is how do you differentiate between a one-way door and a two-way door? And how much time do you spend on these decisions? ’cause if it’s a two-way door decision, just make the decision and chances are like you’ll recover from whatever happens.

[00:28:25] Jonathan: Mahesh, is there anything that keeps you up at night? This is the IT Horror Stories podcast, after all. You had the horror story that you’ve been able to overcome. You’ve come out stronger on the other side, but what keeps you up at night these days?

[00:28:42] Mahesh: If I think about where we are as the larger tech industry, we have seen unparalleled growth starting from 2009, 2010. And we’re at this point where it feels like the tech industry is undergoing a major restructuring. Companies are realizing they’re too expensive. Investors, board members are applying pressure on companies to trim their costs.

[00:29:07] The growth of AI is really questioning, do you really need, you know, really these expensive engineers and infrastructure engineers, developers to go do this work? So I think we’re in the middle of a giant restructuring of the entire tech industry, unlike we have ever seen in our lifetimes. I mean, the tech industry is only like 20, 25 years old, so we haven’t seen a restructuring like this ever, and I’m both excited and worried about the future.

[00:29:39] Excited because I’m waiting to see what it looks like in the next year. Worried because in the short term there will be pain that tech companies have to undergo. If you take that and also apply the average age of a manager in tech, high tech is about 35, which means they haven’t seen a restructuring like this in their careers ever. They haven’t experienced a .com bust. They might have experienced a financial crisis, they might have experienced a Covid crisis, but they’re, I feel they’re not equipped to handle whatever’s coming their way now. So, that keeps me up at night.

Closing

[00:30:27] Jonathan: I’m sure you feel that, especially as a leader and having a lot of people looking to you and you’re responsible for a lot of folks. Anything that you fall back on, then, when you find yourself kind of worrying about these things? I mean, if it’s even just focusing on something, the more immediate, you know, and not so amorphous.

[00:30:48] Mahesh: Yeah, I think the, well, a few things, like one of the reasons why I wrote this book is to equip managers with these skills when they have to navigate tough times and tough decisions that they would have to make. I think the other side of it is, it’s interesting how a population, a subset of tech population, has become Luddites to a certain extent.

[00:31:08] Like, oh, this generative AI stuff, that’s a fad. We’re not gonna use it. Our code is better than what it can generate. We have to babysit it. All those might be true to a certain extent, or maybe to a large extent depending on your use case. But you can’t ignore it, right? You can’t ignore it. You have to embrace it.

[00:31:27] And, and see what you can make out of it. Because even if you don’t, if you consciously ignore it, the market will force you to do it at some point. And when they, when it forces you to do it, it’s gonna be hard. It’s gonna be a hard choice and it won’t be your choice.

[00:31:41] Jonathan: You wanna be able to have those options. I’ve heard it said that, you know, for a lot of folks, it’s not that you’ll be replaced by AI, it’s be, you’ll be replaced by someone who does use it.

[00:31:50] Mahesh: That’s right. That’s a hundred percent right. I was talking about this with my wife. I don’t think the size of technology organizations will shrink dramatically. Right, I think it might drop by maybe 10%. I think the size of engineering organizations is gonna shrink in the next like 1, 5, 10 years.

[00:32:12] I think the size of product managers, analysts, program managers, that’s gonna increase because they’ll be using these tools to extend the application themselves. You know, there are tools right now where you can write a requirements document, feed it to ChatGPT, and it’ll spit out. Whatever code it needs to write to satisfy that requirement.

[00:32:28] So I suspect people who can build or have a view of how to build products and how to build features well, they will become better.

[00:32:47] Jonathan: I hesitate to go too far down the AI rabbit hole anyway, but, I mean a lot of, I think you’re right, navigating change, it seems to be a core skill that will never become unuseful.

[00:33:00] Mahesh: That’s right, that’s right. Or I could be completely wrong, right? I don’t know. But it’s, this feels like something that you can’t, you can’t ignore.

[00:33:10] Jonathan: Absolutely. Mahesh, thank you so much for joining us. Chief Product and Technology Officer at Kickstarter. Definitely check out his book, it’s a new book, it’s called How to Deliver Bad News. Mahesh, appreciate you taking the time, sharing your horror story. Really loved the conversation.

[00:33:26] Mahesh: Thanks for having me.
×

See NinjaOne in action!

By submitting this form, I accept NinjaOne's privacy policy.