Decisive Moments Podcast (by Plato)

Introductions

MUSTAFA: 0:00 – 0:17
Take this moment, Gautham, why do you, why don’t you give us a quick background. I know you’ve worked at PagerDuty now, you’ve worked at Zandesk. Let’s just hear a little bit more about, uh, about your background.

GAUTAM PRABHU : [00:18]  Sure. So I am currently the VP of engineering at PagerDuty. So a SaaS platform for monitoring and alerting. When, you know, our statement is we tell you when, we tell you when the computers are broken. So just in terms of how I got here, I’ve been biased towards startups, my entire career. Generally on the smaller size I’ve worked at, I’ve worked at bigger companies, but that’s almost always been because I was working at a small company that got swallowed up by a bigger one, which is how I ended up at Sun Microsystems which had about 25,000 employees for awhile.

GAUTAM PRABHU: [00:57]
Or because they started small and got fairly big. So like Zendesk, which is now a few thousand people, but was a small little startup when I got there, but I’ve generally, always felt most comfortable joining very small places. So PagerDuty is actually somewhat of an anomaly. The biggest company I’ve ever joined before PagerDuty was probably 60 people that might’ve been Zendesk. ANd PagerDuty was about 500 when I joined. So it felt like a big company to me, even though in the grand scheme of things, it’s pretty small. Uh, my, my first management role was abou seven, eight years into my career. Uh, it was at a startup where I was you know, one of the co founders. Um, and I got that management role, you know, I’ve said this actually at a, at a Plato event for the two reasons that a lot of people get their management roles because, uh, there was no one else to do it.

GAUTAM PRABHU: [01:51]
Um, and they made me so, um, that’s, that’s how I got started management. So about, I don’t know, 12 years ago or so I, uh, I became, I went straight from, you know, writing code to being the VP of engineering at a small company, which meant I was in charge of one or two people. Um, and I’ve kind of stuck with it from there. I am sort of an engineer at heart and I think the way I’ve made my, my peace with kind of this pivot from no active development where it was a hundred percent of my time to now where it’s, you know, maybe 1% of my time, if that that’d be generous Mmm. Is by, by finding the analogs between sort of the software engineering world and the management and organizational engineering world. So that’s me.

DAVID MURRAY: [02:40]
Excellent. And can you hear me okay. Do you hear an echo?

GAUTAM PRABHU: [02:43]
That’s great.

DAVID MURRAY: [02:43]
Okay, great. I was listening in the background as I was grabbing stuff. So it sounds like you’re, you know, you’ve come from actually a place that Other folks have certainly shared a similar story in terms of kind of filling a role that needed to be filled. You kind of knew it was there or not. Right. Some people kind of have this dream of where they want to be from the beginning and others kind of find it because of the environment that they’re in and then they land in something that just works really well for them. So it’s great to know. And of course, we’re very happy to have you here. So with that, I’m going to start diving the questions in the Q and A. So again, as folks have questions, feel free to post them in the Q and A at the bottom of the screen. Just click that button and you can also upvote other people’s questions. Right now we have about six questions and not a ton of up front. So I would love to get folks engaged in there. So feel free to go in there, dive in. We’ve got 117 people on the call right now. So with that, I’ll start with the first question here. How do you find a balance between technical and product impact and make sure that engineers understand the tradeoffs and are inspired to make the right decisions?

The Role of the “Voice of the Customer” in Product Development

GAUTAM PRABHU: [03:52]
Yeah, that’s a good question. So I’m going to start with sort of a philosophy on product development that I have. So there’s a couple of roles within a team that I think have to be there. And one of them is : Who is the voice of the customer? And so by that, I don’t mean just someone who’s, you know, basically asking customers what they want and bringing it back and saying this is what the customer is asking for. You know, that’s a very primitive role. What I’m talking about is someone who is in contact with, first of all, let’s start with the idea that every engineering team should understand. We are building this software for somebody. And if you don’t know who that somebody is, that leads to a lot of misalignments and, you know, incentives and confusion and frankly sometimes wasted effort

GAUTAM PRABHU: [04:47]
So every team should understand, we have customers. And that doesn’t necessarily mean it’s. someone out in the public, you could be a team that’s done internal customers, but someone on your team has to be the voice of those customers, meaning I’ve understood their asks and their needs. And I’ve kind of synthesized that into things that I think we should be building. And there has to be someone else on this team who is the voice of the engineering side of the house or the internal asks. And the way I always put that is there should be someone who is the person who is the keeper of: “These are things our customers are not asking for but if they were given the choice, they would choose them.” Like if you offered the customer like, “Hey, listen, we have two choices for you.” There’s a pretty, you know, there’s a feature we could build that looks interesting but doing that will come at the cost of stability and maybe, maybe we’ll have a, you know, a major incident or downtime and the product won’t be as reliable as you want the customer. If they were given those two choices would often say like, well, I’d rather have something stable first. But we never put that question to them. So someone’s got to be that voice, the internally focused voice on what are the things we need to do to make sure that we can deliver a good quality of service. And ideally those are two different voices. We have someone. so that first voice of the customer is often filled by someone called a product manager. On internal technical, on internal infrastructure teams, it’s probably not a product manager. There’s someone else who’s sometimes, you know, in that role either explicitly or implicitly.

A Balance of Voices: Customer and Engineering Team

GAUTAM PRABHU: [06:20]

So you had these two different voices and what is now required is a collaboration between those two voices to say, what are we actually going to do? And it kind of goes to another question I’m kind of seeing in here about technical debt. That’s often one of the things that sort of that internal voice is speaking to is like, what are the things that we need to do to make this product more scalable or more reliable? Um, and then you have to kind of merge that together with a list of “what are the things that our customers are actually asking us for?” So now the question becomes like, “how do you actually merge those two things?” And the answer is that is art, That is not science. I don’t think it’s healthy to say, well, the voice of the customer always wins or, or the voice of, the internal, you know, engineering needs always wins. It has to be a collaboration. And this is really the secret sauce of, I think most successful engineering teams is that there’s a healthy collaboration between those two voices to understand sometimes you interleave it. And it’s like, okay, we have a technique. You know, we have a big marketing launch coming up. We are going to defer a bunch of things that are internal technical debt type projects, because we need a line around this big marketing launch. And then there’ll be times, and it often happens after bad incidents and they get sufficiently bad and everyone’s like, Whoa, we need to slow down. We need to focus on the reliability now, ideally you’re not basically, uh, you know, getting to that point and you’re interweaving it, but it’s really a collaboration between the voice of the internal engineers and the voice of the customer to figure out what is the right balance. And you’ll hit a different balance at different times. Sometimes it’s interleaved. Sometimes it’s much more chunky than that.

DAVID MURRAY: [08:30]
Yeah, it’s interesting because the takeaways I get are sort of clear roles, obviously clear communication and for lack of better words, teamwork, but what’s interesting is I think as an engineer, especially kind of going into management for the first time, or even if you’re just earlier on in your career and you haven’t collaborated in much with product management, maybe cause you’re in a highly sort of silo technical role. It’s interesting because we come to the table with what we see, what we think is the right thing to do for a particular product or experience. I think the most sort of obvious one is: the site should be up and it should be running and, and not go down right. But there are some circumstances and some situations where the, when you’re choosing what technical debt to take on it actually is more prudent the business to take on the technical debt of stability, right? Like you may be earlier on, in your life cycle as a product or as a company where that’s less of an issue or you have a product that people actually don’t physically log into very much, but there’s like a service that’s kind of ongoing and that’s happening in the background. For which that 99.99% of the time, reliability isn’t as critical. So I think it’s astute to point out that it’s not, it’s not just simply about kind of having a clear blanket rule of. this person always gets to make the decision, but that you have to collaborate and you have to understand the business.

GAUTAM PRABHU: [09:32]
Yeah. And actually one of the things that I think makes that collaboration even more effective is having people who are that internal voice of technical debt, or internal technical concerns, to think about what they want to work on the same way a product a person would, right? Like there’s usually a stack rank list, there’s usually some sort of: What is the business value we get out of paying down this technical debt? What are the ways we’re going to measure, whether it was successful or not? And then you have an apples-to-apples comparison versus black box and, kind of like a, an open card.

Preferred Way to Deal with Technical Debt

DAVID MURRAY: [10:07]
Totally. So actually transitioning to actually the top uploaded question right now, and very similar is: What is your preferred way to deal with technical debt as a project in general, or continually as part of every sprint? How do you navigate that?

GAUTAM PRABHU: [10:25]
Like I said, I think this one is, you have to take it on a case by case basis. You know, obviously at the beginning of a life cycle of a product you don’t want to prematurely optimized things. You know to basically build, build something in a way where you feel like it’s a good building block for the future, but not necessarily dot all the I’s and cross the T’s for the future, because you want to see if someone’s going to use this first. And then you get to the more mature products and, you know, I’ll use PagerDuty as an example. Like our, our core offering has, has hard publicly defined SLAs which guide us a lot. And frankly, if we’re trending for any reason towards being in breach of those SLAs, um, that is a very good signal that we are going to focus entirely on simplification, technical debt, reliability work. I think you really have to look at where you are in sort of the life cycle of what you’re working on, um, to make that call.

DAVID MURRAY: [11:33]
It makes sense. It’s, there’s another kind of technical debt question there. And I think what, what you’re alluding to, and kind of what we’re discussing maybe means that answering this is you can’t, but I’m curious to hear your thoughts, how do you execute and deliver with speed without taking on technical debt? And is that a thing?

GAUTAM PRABHU: [11:52]
One of the things that’s really interesting about technical debt is, you have to be very careful about declaring it. So I’ve had an observation, which is that anytime a service or a software system is handed from one team to another or inherited by a team, the first declaration is, man, this thing has to be rewritten. And that’s been kind of more universal for me. So I think we have to acknowledge, first of all, that it’s much easier to write code than to read code. That’s a lesson that I’m paraphrasing from Joel Spolsky from 20 years ago. So taking on technical debt, first of all, whether you’re taking it on requires some judgment about what is technical debt and one person’s technical debt might not be another person’s technical debt. So there is a judgment call that has to be made there about whether something is actually technical debt or whether it’s just difficult to get into the mind space of a person or a team that built something in the past and get to a place where you can feel comfortable iterating on it. So I would say that one way to minimize technical debt is to minimize the handoff of things between teams, because that is a very clear sort of source of the declaration of technical debt existing. And it’s, I’m not saying it’s a lie it’s or the fact is it’s someone built something, I don’t understand all the principles, Some of them might not even be here, So it’s harder for me to move as fast as they used to move. So if you’re able to minimize handoffs, and I think that like, I’m kind of really wildly diverging there, but this kind of gets to my philosophy on like a project-based mentality or a product based mentality. And in a project-based mentality, you have people as resources and you have priorities and you’re constantly matchmaking people to the highest priority. Okay, this thing’s really important. Let’s pull these people or pull this team and have them go work on this thing. So first of all, it leads to a bunch of short term building without long-term ownership. So maybe the choices you make are not as long term. And the second thing is you’re moving complicated systems around between a lot of people and that in and of itself is the cause of technical debt often.

DAVID MURRAY: [14:13]
Yeah. It’s so funny because I mean, what you described of a handoff between teams, and then they declare the technical debt from what they see. I’ve seen that so many times, so it resonates quite a bit. The other ironic thing is oftentimes when that handoff occurs, there’s a creation of new technical debt as that new person or team is trying to understand what’s going on and thinks it works one way and implement something else. And meanwhile, they’re introducing a bunch of new technical debt. But what’s interesting in this question that was asked is there’s kind of this implied assumption. And I think it’s a cultural assumption, right? Because we experience, particularly as engineers, we experience the downsides technical debt. And so we immediately kind of conclude it’s this bad thing that we need to get rid of like a plague, right. And yet if we think about it like cash debt, I mean, nobody wants it, but it, like anything, is a tool. It’s a tool that people use to be able to take on something that they wouldn’t otherwise be able to take on right. In a faster time period than they otherwise might be able to. And the way that you described how to navigate it is a healthy one in that you’re describing taking it on consciously and being very clear about what is that debt, as opposed to just kind of labeling everything as debt.

Invest or Disinvest, Don’t Maintain

GAUTAM PRABHU: [15:33]
Yes. We have a head of product at PagerDuty and one of the things she says is that we should be in this mentality of “invest or disinvest” in something. And that “maintain” is a difficult place to be. And so that also is a good lens for deciding how and when to pay down technical debt. Like if things are in maintain mode, it’s on that fence and fuzzy boundary. If it’s invest, you know that’s the right thing to do. If it’s disinvest, you’re like, we’re winding this thing down. Let’s wind it down without, let’s minimize the effort that it takes to wind it down. So that’s much easier said than done. You know, I do find a lot of things kind of fall into the maintain bucket, but I think that’s another tool that we have to bring to bear, which is more than just prioritization and figuring out the right code structures and systems to pay down technical debt is maybe to look at the source of technical debt and whether we want to maintain it at all and whether it’s worth the effort or whether that, the fact that it is, accumulating this cruft is a reason that we should be looking at deprecation in some cases.

Most Effective Metrics to Use

DAVID MURRAY: [16:42]
Totally. It certainly makes sense. I want to shift gears a little bit to talk about metrics, which seems to be a high one up here. And then after that, we’ll kind of transition to more like scaling. So blanket kind of question, which obviously is a complicated one is, What metrics do you use to measure impact and related to that, How do you measure internal team impact versus external teams? And do you have similar metrics? It’s actually kind of curious internal versus external teams and what that is. Feel free to share your thoughts as it relates to that.

GAUTAM PRABHU: [17:04]
Yeah, sure. So if we’re going to focus on the team, one of the things that I think is important is to have some sort of rough definition of what a, we call it a “well-formed team” looks like. Is this team basically given the basic building blocks of being able to be successful in delivering software? So for us that means, a sufficient size to maintain the healthy on call rotation,  a mix of sort of like senior technical leadership, having those roles sort of well-defined somebody who can serve as the voice of the internal systems and one who could serve as the voice of the customer. So we do have this idea of, is this team at a place where we can truly measure its impact. And if the answer is no, our first order of business is not to go and measure how they’re doing, it’s to actually build that into a team that is ready to be measured. So I think that’s step one for us. Now step two is ultimately every team should be measured in terms of: “What is the impact they’re having on the business?” Now that’s an easy thing to say. It’s a hard thing to do. You know, the business generally will have things like revenue targets, and it’s very hard to basically directly map a revenue target for the business onto your SRE team. So what becomes important is sort of these layers of understanding well, here are the high level business targets. We now need to have engineering wide targets that say: “What are the things that we are contributing to this revenue?” Or whatever the business targets are. There’s other ones in addition, revenue, number of customers, geographic distribution of customers. Um, so, and at a department level, you need to have some sense of like, well, what are the metrics we’re trying to move to sort of contribute to these things. And you’ll often find the same ones at lots of places and it’s things like, okay, reliability, obviously, no one’s going to use, or it’s going to be hard to get people to use an unreliable product. You get like, SLA targets. You often get a growth and engagement user growth, how many people are actually using it. You know, if we’re selling to a lot of customers, but they only have one person using it, is that success? Or do we want to see that we’re able to sell to customers and then expand to that so you have lots of people using us within a single customer.

GAUTAM PRABHU: [19:30]
And then engagement metrics of features. We’re building something. How do we know if it’s actually getting used? So it’s not going to be a one size fits all. And often your internal infrastructure teams will be trying to move metrics on quality of service often on cost, cost is another metric. Like how much are we spending? Are our costs actually keeping in line with our growth? You know, I was just talking to someone today, like if you’re spending a million dollars a year to basically, you know, uh, support a $10 million year business, and then the next year you find yourself spending $5 million a year to support a $12 million year business. That’s not inherently a disaster, but you have to look at like, are we controlling costs? So you often find that infrastructure teams are supporting, there’s a, there’s like a grab bag of metrics that an engineering team can use to say, are we supporting the higher level business? And you often will find teams, have to gravitate towards a subset of that grab bag. So infrastructure teams will obviously often be on like quality, reliability, costs, sometimes speed of development, you know, there’s teams that are sort of like internal enabling teams, they don’t build pipelines and things like that. Product team, product facing teams or externally facing teams will often be looking to move metrics of user engagement numbers.

Measuring Teams’ Performances

DAVID MURRAY: [20:51]
It makes sense. The first thing that comes to my mind is as you’re kind of sharing like measuring impact and, you know, kind of being product centric in doing that is the unfortunate engineer to probably many people on this call, kind of know this experience, right, where you’re put on a project that probably by definition is a low impact project, maybe because it’s a necessity to maintain, or maybe because of actually poor decision making from a product person, you know, that causes you to kind of no matter what you do have low impact. So I guess maybe the thing to call out is like measuring impact versus measuring performance. And that maybe they’re not the same thing. I don’t know if you have any reflections on that.

GAUTAM PRABHU: [21:34]
That’s fair. Right? Like when you’re measuring the performance of a team as a whole, you’re often looking at okay, “Did they deliver what they committed to deliver?” Was it reliable? And then there is a higher-level thing, which is okay, is this team focused on the right on the right work? You know, and, and I often say that one of the things I’m trying to do is to build teams that are able to, not be purpose-built to do a single thing. Because they might be, if you build a team that basically knows how to work well together on a variety of products, different sort of tasks or projects or initiatives, and you find like, okay, they delivered something really well, but no one’s using it. It’s nice not to say like, okay, we’ve, we’ve done all of the careful work that we as managers have to do of assembling a team with a distribution of skills that distribution of personalities, ideally a diverse representation and they’re working well together, but they built something that no one wants now we’ve got to break it up. So I would say in that case, if they’ve built something that is what they committed to and it’s working well, and just, no one wants to use it, I would say, well, we’ve got a good team here. We’ve got a high performing team. This is not working in a high impact initiative. And so  instead of breaking apart the team, we should be giving them something else to work on. So you’re right. I think team performance is heavily correlated with impact, but it’s not a hundred percent Venn diagram overlap.

DAVID MURRAY: [23:09]
Makes sense. So kind of in a similar vein, when we’re thinking about measuring impact and performance and scaling, scaling being a large theme of the conversation today. I imagine that there were probably a lot of startups and other organizations of various sizes that may find that they actually aren’t sure if they have what you just described, that team, that naturally is cohesive and has worked together and built history and rapport together and they need that, right?  It seems like there’s only in my mind, a couple of ways that you can do at one is that you pull in people that you’ve worked in the past, or that you have like history and rapport with, I think the other one is to work with a third party that has like sets of teams that work really well and cohesively together. I’m curious, for yourself, as well as for PagerDuty, have you all used any, any services that do that sort of thing and what’s your experience been like?

GAUTAM PRABHU: [24:05]
Sure. I’ll talk about scaling teams in general and then talk about my experience. One of the things that’s super important is as the business grows, can you scale the team appropriately to meet, to meet business needs? And ideally be ahead of business need because having demand and demand signals and being unable to deliver on it, you’re wasting time and opportunity. And you’re giving other people a chance to sort of get into, and take an opportunity that’s right in front of you. So scaling of a team often requires building out new locations and, being able to ramp up and take advantage of talent in other locations and wherever your core home base is. Now, I know that as we go more and more towards completely distributed teams, that paradigm is becoming a little bit foggier, but most software companies that I’ve encountered are still in this model of like, we have a headquarters. And then we have these, these sort of like, um, satellite offices are these growth spots that we’re building, you know, in physical points of presence elsewhere. And so at my last company Zendesk we had many of these. And one of the things we realized is that for a team to have positive impact, it’s necessary for them to be, to get to what I call that well-formed team very quickly. You know, it’s a pretty bad experience to have one person sitting by themselves and Copenhagen or Dublin waiting for the second person to show up who shows up a couple of months later. And so whenever we open new offices, sometimes it’s by acquisition and so you have a fully formed team there. Or if we built it organically, there was an expectation that very quickly, like on week one or week two, like when we opened that there was already a functional team there that we can give something to and get them excited about it.

Building High-Performing Teams Quickly

GAUTAM PRABHU: [25:53]
So that’s not easy because it carries this requirement in a tight, tight… there’s a tight hiring environment, not just in the Bay area, it’s all around the world. So it’s a big requirement to say that like on Day One or shortly after Day One, when you build out a new point of presence, you have a fully functional team. So that’s something that PagerDuty he was able to do, starting in San Francisco, but expanding out to Toronto. But it’s difficult to do. And so we do use, we do use a third party, we use Lohika. So there are, one of the advantages there is that we can basically get to that point of I’m building out a team very fast. And one of the failure modes for projects I’ve seen is, “Okay, we have a project, here’s a budget, go build out the team and then we’ll start the project.” And that process of building out the team itself can take months. So the lead time when you got that money, might not always be understood to people where they think, okay, we gave you the dollars, you can magically translate that into a team of like fully functional, software engineers who have great working relationships with each other. And I think all of us know that that takes time, it takes a long time. And if you need to move fast on something, you often don’t have that sort of a multimodal timeline. So PagerDuty does work with third party, like Lohika to be able to ramp up teams quickly and on Day One, start moving on big initiatives.

DAVID MURRAY: [27:26]
There you go. And there’s like a, a side question that somebody had asked relevant to this. Can you tell me how working with a specialist type of engineering from Lohika works differently, better or worse in this new remote era? Does that actually change things? Does it make things easier or harder working with any third party firm in this kind of capacity as you scale?

Benefits of Working with Remote Teams

GAUTAM PRABHU: [27:47]
You know, with PagerDuty, I had really embraced the remote nature of work ahead of time. So in addition to, like I said, having points of presence outside of, outside of headquarters, they made a decision that predates me, which is intentionally striping teams between locations. So I’m a much more common configuration that I’ve seen is, okay, you’ve opened a new location, there’s a team that basically works on something together. So everyone in Dublin works together on this thing. And everyone in Singapore works together on this thing. And maybe every once in awhile teams need to work together but your day to day is: “I am in a location and I work with other people who are in that location.” And so PagerDuty decided to stripe the teams. So we have multiple offices and we intentionally construct our teams so that there’s people on a single team and multiple offices, even though we could consolidate and say “No, there’s teams that are in Toronto or teams that are in Atlanta and teams that are in the Bay Area.” And so that’s done a few things for us. First of all, it enables us to integrate people who are truly remote, and not working out of any of our offices much more easily. And it also makes it a much easier for us to work with a team like Lohika, because working remotely is in our DNA. At this point, like working in a distributed fashion is basically something we’ve been working on for quite a while. So, I think that this shift to remote work has been something that was, that trend was coming. First of all, I think this is accelerated it by five or 10 years in terms of, where the trend was already going. And so I think that being able work with a third party like Lohika in a remote location is, is something that everyone is going to have to get used to. The idea of like remote and distributed teams.

DAVID MURRAY: [29:50]
Yeah. No, I think it’s really astute. You’re the, the decisions to “stripe” your teams as you call it. I hadn’t used it before this, it’s a good one. Um, it’s really an astute point that, and this was mentioned in one or two AMAs ago kind of like working fully remote or not remote at all are great, but like the hardest is when you’re kind of half and half, right? That’s because you get the, the echo chambers, right?  Of folks that are kind of in their own bubble and they create and build a whole reality of something that’s happening. And maybe you don’t want to surface it to another office. And then all sorts of dramas get kind of created stories, get created that maybe are completely false. And so the idea of striping of having people from different locations allows you to eliminate in the same way that you would with a fully remote organization, this possibility of these echo chambers, because everybody’s kind of, you know, always elaborating virtually kind of on, on equal footing. So I guess you’re inspiring in my mind that for those who are in a hybrid world, things get back to normal, that striping their teams is a really great way to navigate that.

GAUTAM PRABHU: [31:07]
You know there’s another advantage, which ,keeping it on this theme of scale, which is that at that point, it doesn’t matter where in the world you find… I mean, the number one problem for most software organizations now I’m excluding like the five companies that everyone knows who have these infinitely sized funnels of people who come in every year. But for a lot of us, the number one challenge is finding, you know, the talented software engineers that we need to build out the things that we know are going to be successful. And it’s kind of baffling to me as someone who participated in this system that I used to say “Man, I wish that we found this person in Melbourne Australia, instead of we found them in Dublin.” So we can’t, it’s like they would have been such a better fit for that team. And so I guess we’re going to have to let them go. And it’s crazy to me that 10 years ago, I would do that. Today you know, especially given the model we’ve set up, it’s like you find talent. You find someone who’s talented and is able to contribute. It doesn’t matter where they are. You’re not constrained by this, these boundaries. So those, those points of presence actually become an asset to you, a bigger asset. I there’s people who want to sort of feel like they are connected to an office, but don’t necessarily, want to be tied to whatever people in that office are working on currently. If you have data science in a different location, you find a great data science in a place where, you weren’t expecting to. Having this ability to match-make people around the world is incredibly powerful.

Engineering in the Time of Coronoavirus

DAVID MURRAY: [32:39]
Totally. So I’m going to shift gears just a little bit. We got about 15, 20 minutes left. How are the priorities for scaling for you changing in this time of the Coronavirus?

GAUTAM PRABHU: [32:47]
That’s a good question. I mean, so I assume everyone knows what PagerDuty does but if not, we are an alerting platform and incident response platform. So you know what my boss always says is we need to be at our best when everyone else is at their worst. And just given what’s happened, there are certainly companies out there who are experiencing drops in demand as a result of coronavirus. There’re obviously companies out there, we’re on one of those platforms right now, which has experienced massive increases and usage. So we really do take our responsibility seriously because we support a lot of those companies. And so we’re seeing increases in traffic. And we also, so even outside of a business sense, we do feel a heightened sense of responsibility, that there are certain companies that are not just experiencing hyper business growth, they’re actually helping keep society running in some ways. So internally we have had a much, we’ve always have a focus on reliability given what we do. But we have a higher focus on reliability and scale and being able to rapidly respond to spikes in traffic and demand as a result of Coronavirus. And it’s purely because of the nature of the customers that we support. They’re seeing it and therefore we have to make sure we’re prepared for their business needs. So, the short answer is yes, it’s become an even bigger focus than what was already a big focus for us before. And we have a bunch of initiatives going to make sure that we keep up with that. That’s sort of like uptick in traffic and demand.

Communicating Engineering Initiatives to Rest of Company

DAVID MURRAY: [34:34]
Makes sense. Speaking of initiatives, and this is very relevant. I imagine you’d have a lot of wisdom on this particular because PagerDuty is a very technically driven and focused product. How do you communicate engineering initiatives towards the rest of the company? This person have sales and marketing stuff, but no tech during company all hands, how do you make it worth listening to for non-tech people?

GAUTAM PRABHU: [34:59]
So I think one of the things is you just have to force that cadence. There’s got to be, sometimes, it’s like building a habit and it’s one of those things where sometimes what doing what feels like jamming a square peg into a round old, but you know, you keep doing it. I’m going to make a bad analogy, but you keep doing enough. Maybe that square peg gets shaved off a little bit. And then all of a sudden it fits in the round hole or the round hole breaks a little bit and you fit in the square peg. But the fact is that all hands is an all company audience. And so this is not an uncommon phenomenon. Yeah. There’s tons of stuff about sales and marketing there. The fact is that’s important, but for an engineering team, they don’t always know what all those terms mean or what the sort of like underlying takeaway is meant to be from those things. I think you have to view an all hands as an opportunity to educate people about something they didn’t know about before. Otherwise it’s an echo chamber, like you said. Sales will have its own all-hands where they’d get in talking about this stuff to people who understand it very deeply. Engineering can or should have its own all hands where we talk about this stuff very deeply to an audience who understands it. The company-wide all hands is the opportunity to explain to people what other departments are up to and give them that sort empathy. And often they’re very curious about this kind of stuff. To be honest, one of the things that I find is that engineering often operates from this very privileged position of, I call it the “black box.”  I don’t know if other people feel this way, but there’s something about a sales deck or a marketing deck. It just ties to concepts that we understand. Everyone knows about how businesses work and that there’s money that comes in and money you spend and you have to advertise to get business. And so the concepts there are things that we just kind of pick up and we can sort of latch onto. The concepts in engineering are often just viewed as a black box to people. They don’t know the difference between someone who’s doing something in react and doing an optimization of a database query. And they’re interested, they don’t like that it’s a black box. It’s this whole part of the org where everyone has to just trust us, it’s really kind of a privilege and sort of a responsibility we have. There’s a lot of trust put in engineering teams where it’s like, “Okay, it’s a black box. I don’t fully understand what’s going on in there. We just trust that you’re doing the right stuff.” And I think the more that we can actually kind of open up that box and show them what we’re doing. I think it builds trust. And I think they’re really interested in it, given that it’s often like a huge part of an organization.

DAVID MURRAY: [37:43]
Totally. The takeaway I get from that is, “Make me care, give a context, give background” Allow people who are nontechnical to actually understand what’s going on. And like, why does it matter to the organization? Right. One of the things that also reminds me of is being on the Facebook campus before I’ve seen signs kind of saying, “We don’t use like project code names here” we avoid that because it kind of creates a like “you’re in the circle or you’re not in the circle kind of vibe.” Like certainly even if you have a project code name when you’re in that global company audience, if you want to make somebody care, the worst thing that you can do is say “We just completed project Blackbird and it was very successful.” Right? You’ve got to explain what is project Blackbird actually doing for the business and why does it matter?

GAUTAM PRABHU: [38:33]

Absolutely. So I think “make me care” is a good way of putting it. I would say, “Make me care by explaining it to me in a language that I can understand.”. And I think that’s a good skill for everyone to build in engineering or otherwise is to explain what you’re doing in a language that the audience will understand. And then also don’t underestimate the appetite of the rest of the audience to hear what you’re saying. I think that’s a common misconception that people at these companies want to hear only about the sales and the marketing efforts. I think they actually would like to hear from engineering, especially if it was in a language they could understand, cause it’s an area that’s harder to follow sometimes.

DAVID MURRAY: [39:11]
Totally. And as technical as we are, we also live and survive by having meaning and purpose in our lives. And if we, as engineering leaders have meaning and purpose in our day to day, simply sharing how we get that meaning and purpose from the projects that we did is the most organic way to do it. What you just described, right. This is great.

Handling Technology Changes Over Multiple Teams

DAVID MURRAY: [39:31]
Well, we have about 10 minutes. I’m going to do a little bit of, not rapid fire, but quickish answers and I’ll go through more questions. So how do you handle technology changes that span multiple teams, for example, the re-architecture of a platform.

GAUTAM PRABHU: [39:46]
Good question. You need to have a culture of team collaboration. And I would recommend a group of senior technical representatives of teams that know how to both advocate for team needs and understand them deeply, but also understand one level of abstraction up.  Who collaborate with each other to figure out if we need to evolve this architecture across 2020 teams. Have sort of representatives from each, not necessarily all those 20 teams, but –I’m just using rough math –  five people who can speak to, “I understand what these teams are doing, what their needs are. “ And we’re going to bring that to a collaborative conversation about what are the things that we need to do at an infrastructure level, that are both achievable for these teams and also sort of impactful for these teams.

DAVID MURRAY: [40:44]
Got it. So it’s like, if you have something that spans multiple teams or systems, bring the experts in from those various systems, have them come together in some way and then possibly have the decision maker in the room. And if you don’t have an expert for one of those, there’s probably an underlying problem you should address first. Right?

GAUTAM PRABHU: [41:00]
Absolutely. So that kind of gets to my well-formed team thing. If there is no one who can serve in that role, that’s a, that’s a smell in and of itself.

DAVID MURRAY: [41:07]
There you go. Where have you seen management structure get in the way of scaling or what management structure have you seen that support scaling?

GAUTAM PRABHU: [41:16]
I’ve been part of an Instructure that got in the way of scaling. I think a really healthy trend that’s been happening for the last 10 or 15 years is the separation of the management track from the technical track. I think the management structures that get in the way the most are where management is viewed as a promotion out of technical contributions. And my old boss put this in place at Facebook and was the one who first told me, “No we do separate but equal tracks. And frankly, our goal is to make the technical track greater than or equal to in terms of attractiveness..”  And that means from a compensation standpoint, it also means from an investing, decision-making authority to them. So to come back to my previous thing about this collection of experts, that collection of experts that’s making the decision needs to have some decision making teeth. It should not be okay, we’re waiting for management to make decisions. I think empowering your technical track and sort of saying like, okay, these are separate but equal. And there are equivalencies between someone who’s moving up the technical track and a management track in terms of the authority they have to make certain decisions is really important.

DAVID MURRAY: [42:26]
Excellent. If you are at an organization, those who are here, that does not embrace that idea, that you can have separate tracks that are equal from a pay and the seniority and respect perspective, point them to Google, point them to Facebook point them to every other company that’s out there points them to PagerDuty. Because that’s a big one and culturally 100%, you have a manager that feels like they have to do it because that’s how they make more money. You’re not going to get very enthusiastic managers.

GAUTAM PRABHU: [42:53]
You’re not going to get enthusiastic managers and you’re setting them up for failure in terms of being able to hire from the outside. Because you’re going to get someone who comes in has no context and is being asked to preside over things, which they don’t really have any business doing. And maybe they don’t even want to do it.

DAVID MURRAY: [43:07]
Totally. Middle managers, some of the hardest people to hire out, right? Much easier to promote internally, for that exact reason. Totally makes sense.

DAVID MURRAY: [43:14]

How do you measure the impact of setting up patterns and paying down or not technical debt? I know it’s another technical debt question. Make it quick!

GAUTAM PRABHU: [43:26]
So how do you measure the impact of setting up pounders or paying down technical debt? you have to have a pretty firm goal on, uh, what is the, what is the metric or the outcome that we’re trying to hit as a result of the technical debt and you have to get the team to sign up to it. If it’s speed of development. Okay. What are the measures we’re going to put in place on velocity to say, once we’ve done this thing that we can sort of move faster on other stuff. If it’s quality of service, there’s many things like can we increase our SLAs or internal SLOs? Can we see a decrease in support? Tickets is often one that we, that we find. Are we able to actually deflect a bunch of questions that are coming in higher up to the support team?
These are all things that you should be looking to do if you’re paying down technical debt. But the fact is just like a product manager when they build something. If it doesn’t sort of generate the results they want, there’s got to be some feedback loop for like, “Okay, what did I get wrong? What am I going to do next time?”  You’ve got to have those sorts of internal measures of technical debt. And I’ve given some examples of, “Can we build something that’s more reliable, something that’s more scalable? Can we move with higher velocity?” Those are examples. Sometimes it’s going to be less quantitative and more qualitative and that’s okay too.

DAVID MURRAY: [44:48]
Having metrics of success as table stakes when you’re doing a project. It seems like that’s a great theme. And even if you don’t know the exact number that you’re going to get at least whether it’s picking ballparks or having hypotheses so that you can at least have a set of expectations. It allows us to not fall into that trap of like, “I told you it was going to happen anyways.” You know, hold holding yourself accountable to what those metrics are. That makes sense.