S4 E14 - Rob Park - Navigating Software Evolution through Deming's Principles

In this episode of the Profound Podcast, I have a conversation with Rob Park. Rob shares his journey from early involvement in extreme programming (XP) to his current fascination with W. Edwards Deming's principles and their application in modern software development.

Rob's career path offers a rich tapestry of experiences, from using case tools and adopting scrum practices before they were widely recognized, to working with Kanban and continuous integration/continuous delivery (CI/CD) processes. He reminisces about his introduction to Deming's work through the influence of notable figures in the agile community.

A significant portion of the discussion delves into the integration of Deming's theories into software practices. Rob talks about the impact of statistical process control (SPC) and control charts on his work, emphasizing the importance of understanding variation and employing data-driven decision-making. He highlights the utility of Dr. Donald Wheeler's teachings on process behavior charts and the critical difference between enumerative and analytical statistics.

The episode wraps up with Rob reflecting on the broader implications of Deming's work, advocating for its relevance in addressing contemporary challenges in software development. He shares his experiences of applying Deming's theories in real-world scenarios, illustrating how these timeless concepts continue to drive quality and efficiency in the digital age.

You can find Rob Park's LinkedIn below:
https://www.linkedin.com/in/robpark-4ls/

Transcript

John Willis: [00:00:00] Hey, this is John Willis again. We got another profound podcast and with a fun person I've gotten to meet and gotten to know pretty well. And it's been kind of cool about, you know, even my Deming journey that just continues. Like, again, I always say, you know, beware of becoming interested in Deming because it actually, it may change the contour of your travels Rob, you want to introduce yourself.

Rob Park: Sure. So, Rob Park, I've been a software developer engineer for over 30 years now, I guess. Yeah, I think even way back was involving case tools. I actually did some scrum before there was a scrum book. I, I realized like when I was looking that I actually was introduced to extreme programming XP in 99, but I'm curious because my book has a copyright of 2000.

So I'm not sure how that all worked out because we didn't even have Amazon pre [00:01:00] release back then. But so did that. I actually didn't know very much about Dr. Deming back then. But I did a lot of work with, like, I did read the goal way back then, but long before the Phoenix project ever came out.

And let's see, I paid a lot of attention to Marion Popendiek's work and. David Anderson's work with Kanban and moving from say scrum boards or, or even just regular XP planning and doing a lot of Kanban with software teams. I've also been very much in the, I guess what we refer to mostly like CICD type process these days, but back then, so, you know, using cruise control, if people are familiar with that I actually worked with one of the I guess a couple of the core committers, Paul Julius, PJ he was actually one of the original people on the cruise control team, but and [00:02:00] now actually runs KITCON with Jeffrey Frederick, who I think, I don't know if you've had him on here or not.

Oh, Jeffrey, yeah, yeah. All in the IT revolution family there yep. And they'll be in in Croatia this year actually with Yvonne, right? Yvonne I forget his name, but he was in the book club at the beginning when we did, when you started the profound book club. So I also have listened to podcasts for.

Ages now decades. And I, I probably most notably remember you from the DevOps cafe and from doing that. And so that's kind of the essence of the background there and how I, how I came across this, 

John Willis: it's funny, you know, I guess there's a, you know, being, you know, the, the guy who sort of does the book clubs gives the presentations.

There's an intellectual laziness, I guess, that just inherit to that process because I knew [00:03:00] you, but like, you just listed a bunch of really cool things that I didn't know about that. You know, I guess when we're in that setting, we don't have the time to do what we're doing right now. But I thought that was, you know, you get to hear my background, but I like that.

I've got a ton of stuff that I said now, I love because they're, you know, like, like, all that stuff you talked about. In fact yeah, yeah. We you know, I was just at DevOps Days Seattle and, and Ken Mugrage. . ,. He was there, he was one of the original DevOps Days.

 You know, he, he actually would do all the sponsors. For ThoughtWorks, and then he did a history of ThoughtWorks, and he had a bunch of stuff to date. I didn't know that they had created Selenium. So that ThoughtWorks, anyway, so ThoughtWorks is an interesting industry, but I guess the, the, the more interesting thing is like, what how did you wind up reading the goal?

Because that's, I think there's an interesting, my connection in my [00:04:00] research, just not being a developer, was You know, David J. Anderson story of him reading the goal and thinking that that might be a way. And so, you know, if there's a version where Anderson creates can, you know, software, I think in 1 of his books, he talks about how we read the goal.

And he saw that as the sort of. put it all together. And what was your experience there? 

Rob Park: Yeah, I don't know how I've managed to stay slightly ahead of all of these things. But like, even with David Anderson's stuff, I knew about his stuff before that, that book ever came out, his Kanban book ever came out.

And so I, I actually would, I think that I got to Goldratt through a guy named J. B. Randberger, who's he was very big in the XP community back in the early 2000s and in the Agile community and then Yeah. So, so kind of that crowd and there's more of the ThoughtWorks family [00:05:00] that overlaps with, with that part of, you know, back in time as to how I got to the goal, but didn't otherwise being interested in, in lean and and a lot of lean thinking are in trying to apply that to software development and software delivery in particular.

So there's a 

John Willis: little, you know uncovering of cool history. So there was, I mean, again, I, you know, you know, I've written about how I got introduced to the Gauls with Jean, right. And I had no idea who Goldratt was until, you know, I met Jean and Jean, you know, gave, and most people who've listened to this podcast have heard this story a hundred times, but Gene gave me this gift where he wouldn't let me, well, he wouldn't not let me, but he suggested before he gave me an early copy of, you know, what became the Phoenix Project that I go read the goal, right?

And I read the goal. But and, and the only other thing I found through my travels, cause I did actually wind up doing probably Damon Edwards and I did a DevOps Cafe podcast with David J. Anderson and Dominic DeGrandis way, way back when. And [00:06:00] I did some research on him. And actually he was at like the first or the second.

European DevOps days. He gave this session and so that was kind of interesting. That was in Germany somewhere, right? And he, and that's how I started digging into him. And that's when I read one of the forwards of maybe the Kanban book where he talks about how he came up with that. And he mentioned. The goal, but it didn't really stick until I had gone back later and read the goal.

And then I realized I went back and connected that. I guess the long winded question here was, it seemed like the lean committee in my research. I guess I didn't really know. There was a, a little bit of a pocket of, of, theory could change or the goal influence in the early sort of XP agile and potentially lean community.

Rob Park: Yeah, and I think it was more really the. XP Agile community, you know, and this is, I guess how I usually describe it is kind of [00:07:00] pre jumping the cat. You know, we're, we're in a different evolution of Agile these days where, you know, half of us complain and half of us are like it. And the other half think it's a thing that's in a box somewhere.

So, and so so back then it was Yeah. You know, we were all very much experimenting and exploring and, and trying to find those things. And so, you know, so Mary Papandeek again, I saw her speak a couple of times. Early on I think also back then. So I, I have a strong passion for test automation and for upfront test automation, so test driving everything that I do and, and being, and that's very much, I don't know, it seems like it's ingrained in the, from the XP background, but there were a lot of people doing that at the same time and using things like fit and Cucumber things like that.

John Willis: Okay. That's cool. Yeah, Mary. I, you know, I, I, you know, through sort of early days, DevOps days, I get [00:08:00] invited to, I remember one time they had this this group out of Minnesota that literally we're running this conference and he invited me out and it was all great. It was, I hadn't done a lot, you know, I've done a lot of DevOps days, but I hadn't done a lot of other conferences with sort of DevOps and, and I got set up all excited.

Jez Humble's there. And Mary Poppindick is there, and there's all these like really cool and they, they have my session is at the same time as Mary and Tom's like, oh, man, how you do this to me? I got, I got a little bit of a crab, but man, I got swamped man, like, more so I just wanted to see her present and then I.

And then I got to meet her at I'm actually going back to Budapest craft conference site and had, you know, pre pandemic, every periodically they invite me there. It's this really cool conference in Budapest where they invite like Dan North and they invite like all these people like Mary and Tom and like these amazing people.

Right. Yeah. And [00:09:00] the first time I went like the speakers, then I'm like, Oh my God, that's Mary Bob. And they go, Oh my God, you know, like, you know, like fanboying, like all over the place, you know? And 

Rob Park: yeah, and I was actually at so part of, you know, in this, during this time, I was at Agile 2006, which was actually in Minneapolis.

And and there was actually an invite to go to Mary and Tom's house, like for like, Oh wow. Oh wow. Party thing. And I didn't go like I don't but now it's like damn it. Why didn't I go that would have been 

John Willis: cool Actually if you want to 

Rob Park: fast forward on to more modern day here, and kind of how I found my way to you and your book again Yeah, yeah, because you mentioned budapest.

I actually so I did some consulting work where I was working through jabe and kevin's Praxis Flow was 

John Willis: the organization. Oh, you were working with Jabe. Full of mysteries 

Rob Park: here. I started doing some consulting with them and I actually was [00:10:00] got to hang out with Jabe. a bit in Budapest. Oh wow. And another guy who actually Bobby Norton was with us too.

And he ended up on the cruise team out of ThoughtWorks at one point earlier. So yeah, so it's all, all very nested and intertwined that way. That's 

John Willis: crazy. Yeah, yeah, yeah. No, I Yeah, you know, everybody knows I think Jabe was a just a special person, but a special sort of, you know, intellect. He just he's just a wealth of knowledge.

Right? So 

But yeah, 

Rob Park: I'm sure we'll get to, like, process control and that sort of bit too. But there's a little more to relate to when working with the Praxis guys from that as well. But yeah, so I think, you know it was after that and I was looking to do so. I've been both full time employee as well as a consultant, and I've done both and I kind of go back and forth. 

But [00:11:00] I was, I knew that you had been working with, you know, Jabe. And Kevin and Andrew and then and you know, and I had discovered that, that Jabe was starting up. I think it's ergonomic with Andrew and Sasha. Right. And so so anyway, so then I found you. And so I went looking for your podcast and that was how I even.

Came across this in the first place was I found the profound podcast and I just started falling in love with the, especially the back episodes, things like Doris Quinn meaning I had found it after those episodes were already out. I found the episode with Jabe from back then. Notable to, to me, like, I feel like a bit of a moron, but it's like You're like, I don't think I ever heard the word epistemology until that episode with with Jabe.

And so but not at all surprised that it would come out of out of that episode for sure. Yeah, and, and then, of course, the, the podcast episode with [00:12:00] Dennis Dennis surgeon. So, 

John Willis: yeah, yeah, no you know, I think, you know, it might have heard the word epistemology, but I don't think I could have explained it the way I can think about it, explain about it and written about it without.

Jabe influence. Well, that's pretty cool. So then so you, you didn't actually start listening to some of my work until actually we were at red hat or post red hat then I guess, or is that. 

Rob Park: Well, I, I knew about you and knew about the DevOps. Oh, that's right. Okay. 

John Willis: And I 

Rob Park: already had like yeah, I guess I had beyond the Phoenix project as well as as well as DevOps handbook.

I mean, I've, I've read all of that. 

John Willis: I got it. Yeah, no, and it's cool. And so that with the book club, that 1st book club was, was like, that was like, really awesome. Right? Because. It was, you know, it wasn't we did, you know, I've done 2 now, right? It's 1 where I just wanted to have 1 before the physical book got published.

Right? And then and that was sort of on my own. And [00:13:00] you know, and and the 2nd, 1, we started I guess earlier this year. That was an it revolution sanctioned, not sanctioned, but like run 1. But that 1st, 1 was, I think we all agreed that, you know, myself included, that it was like Friday mornings, depending on where you were based.

But it was like, I think. A lot of us would say, you know, like, I think this is the. Most interesting part of my week and it's not wasn't so much because of, I mean, it was, the book was the linchpin to get us, but we just had such great conversations, you know, because we could go exploring with like really passionate people.

It was, you know, we, yeah. 

Rob Park: Yeah. And I mean, I think all of us, best I can recall are, Our ver, you know, we have some DevOps background, like we are in sort of this, this space you know as well. So we have that in common. And I guess that implies that we're also all a bit on the same nerdy level. So so we just kind of [00:14:00] clicked and so, right.

So that's the book club, the profound book club that I've Yeah. I've contained, continued and kept that thing running because yeah, 'cause we're, we're just like, we're a group that enjoy. Investigating these ideas and digging and seeing like, who's using what, where, and where, where can we make different applications?

I, I am right. East coast U S based, but we do still have Mike Harris has been joining us on a very regular basis and he's in, he's an earlier episode in this podcast. So And he's a UK based. So 

John Willis: yeah, yeah, very, very influenced by test and development. I, you know Elizabeth Hendrickson, right? She was, you know, she's, it's interesting to test people team to People who are involved with tests, which makes sense.

You can kind of get the whole, why, why that, I mean, so I guess that question is, you know, like what, I mean, what is it about Deming that, that, you know that has connected you so [00:15:00] deeply into it where like you've taken over my original book club. And by the way, I have a, I have a really good book to recommend that I'll actually get back involved in it.

It's so good. It's Domenica Lepore's wife sent me an early copy of books. She's writing. And it's in a Goldratt style, but it has a ton of, a ton of Deming in it as well. So Interesting. Yeah, I, I'll, I, I'm gonna ask her if it's okay, we can maybe do a book club on it before she gets it fully published, so, but anyway, I just, yeah, and I 

Rob Park: actually, I love like that, that episode too that you had with him was super interesting.

Yeah, yeah. Connecting Goldratt and, and Deming, which is his. His book on that. Yeah. So, so sorry. The question was why Deming? So honestly, I think, you know, so really my big introduction into is your book. So, so once I, I got really hooked on the stories that were coming like out of people [00:16:00] like Dennis in the podcast.

 But then once we started getting into the book I got very fascinated by like the early parts where we're talking about Dr. Shewhart you know, now we're talking about math and and actually making data meaningful, like metric data meaningful, especially with, with consideration of like what I would call now continual software process improvement.

 Or I'm actually still taking a course with Dennis right now through Deming next, which is. continual quality improvement which has been, been pretty cool. But then there was actually more in there too. Things started to come into my mind of realizing. So when I, in the seventies, my father was actually, President of a division of a manufacturing company of sewing machines here in Massachusetts.[00:17:00] 

He actually traveled to Japan in 71, which is now like realizing like, oh, this is all exactly the same time frame. My father is now passed away, so I can't. Unfortunately, go back and ask him. 

John Willis: Yeah. 

Rob Park: So like, you know, about any of this or to, to know if he knew, like, he had to know that this stuff was going on, but I was just good.

So I didn't really know a lot of this was going on, but that was interesting. Then to find out about Hawthorne works, which is a factory town, which I had never heard of such a thing. So your book kind of led me to that. So the company that he was president of was called Boynton machine works.

And it was here. In a town called Whitensville which probably should be, I believe something more like Southbridge, but they've actually just renamed the town after the company because it was a factory town in the 18 hundreds. You know, like rules, like, if you need firewood to heat your house, you can cut down any tree on the [00:18:00] company property and use that to heat your family's house.

John Willis: Wow. 

Rob Park: Yeah. Oh, man. So it was just like ties like that. That really kind of hooked me in the early part. Yeah. Early parts of the book, and so yeah somewhere along the line in there, too, was where probably in the maybe even your podcast episode with Katie Anderson that, like, hearing about the Japan, like, or trips to Japan and.

 This whole concept of chain of learning and and connecting all of these things together. That, yeah like you said at the top, right? This is a, this is a rabbit hole that falls very deep. Yeah, no doubt. No doubt. And so once you sort of get your toe in there and get get hooked. I have more books than I can keep up with right now.

John Willis: Yeah, no, it's and, and, and you know, I guess the thing is, is it just makes sense. That's the thing about like, sort of Yeah. You know, it starts with [00:19:00] Deming, it starts with Goldratt. I mean, you know, it's sort somewhere, you know, my, my, my story I told over and over, right? I was all in on, on Goldratt and I felt, you know, I read a couple of books after I read the goal and, and then we're at this Open Spaces and Ben Rockwood, you know.

Another great influence of mine and he, he's sort of like John, John, it all goes back to Deming. I'm like, who's this Deming guy, you know, and then like, you know, like, like, and I've interviewed people over the years where I like, like, like, you know, I, there was a, I wanted a guy that I couldn't get on a podcast.

So he worked for a company that wouldn't know, it's just really like a jay black guy. And he couldn't get on a podcast because the company politics or whatever, and, and or PR stuff. And But he said that, you know, like he, he was a late bloomer to Deming as well. Like he had a whole career in computer science and, and then he said he just started tracking what lean was and, you know, he got into lean through DevOps and then went back to lean and figure out where lean come [00:20:00] from and he wound up at Deming, you know so, and, and like I said, it all, it just like all this stuff, you know, it all makes sense, you know?

Rob Park: Yeah. And thanks for saying that because on the, that point of where I think that's another thing with. Deming's work, that it does all make sense. There's been a lot of these concepts that have just been intuitive to me. Like, like inspecting at the end and doing that as the main way to deliver software, like it's terrible.

Like, it's just like, it's so like super slow and inefficient and and it, you know, like quality, like you can't assure it at the end. Like it's already, it's already. You know, you can say, like, it doesn't work, like, throw it back, but so then you start to understand that Deming's saying all of these same things, and like, Like, yeah, that's what it feels like.

So now there's like a framework a whole system, right. A whole system to put around it. Other things too, like [00:21:00] you know, I don't know if there's anyone who really enjoyed performance reviews, but like, so that's another intuitive thing. I know I've heard you talk about like Deming wouldn't be a big fan of OKRs.

I can't say I am either. So. You know, things like that, there's just like the targets having targets and targets tied to merit, like such a bad idea, but heart, you know, I don't feel like I'm going to justify it on my own, but, but it's been very interesting and just nice, I guess 

John Willis: Yeah, just affirming.

It's amazing that all the things we've known, you know, I mean, DevOps was this like revolt. You know, finally, we've had enough almost right of, like, things we knew were wrong and, like, nobody stood up and did anything about it, like the wall of confusion. Right? And and then, but even even that's just 1, you know, that's, like, really, the strength of everything that DevOps has given us, which is quite a [00:22:00] bit.

Just one variable , because there's other, the other variables are things like, you know, you know, OKRs and MBOs and, and performance reviews and, and we just, like, we know they, you know, we, it's like we, we've all been sort of the, the, what is it, the, the sort of tyranny. You know, these things the other thing I thought, you know, so to me wanted to, like, it's funny, like, there's a lot of things that, like, I ensure this is, you know, I've written a lot of my books have been coauthored.

This book was really mine from the get go. You know, obviously, Derek Lewis has been an incredible partner of mine, and we're working on more books together and, but the, but this is my book, right? And, and so I, you know, I think for me, what's interesting and I've heard other authors say this is like, you know, so you go through this thing and then like efforts over to so many more things and like, wow, you know, I wish I would have known that.

And then you know, like and, but for [00:23:00] me, a couple of things mid stream, I started picking up on like, you know, fortunately, a little early, I started really understanding the layering of not just. Control charts, not just statistical process control, but what Deming was really talking about when he talked about the difference between enumerated statistics and analytical statistics, right?

And, and the whole analytical statistics thing. So I'm glad I caught that wave enough to really sort of use it in the book. And then there's operational definitions. I think I caught that a little later that when I but, but but so once, you know, for me, that was, I think it was really important for me to get over that hump because I, like, I understood, you know, I think intellectually.

What, you know, like, when you look at like, just cross control, and you talk about common cause and special cause and all that, right? And you sort of okay, I can see it. You can visualize [00:24:00] it, but I couldn't put it and actually it was jabe again here that really sort of helped me. I started reading what Deming had talked about analytical statistics is an old podcast in the Deming Institute with a guy named Rod Moen and talks about how he met Deming and how he, you know, he, he, he.

He was, like, get, you know, pissing off all these enumerated or academia statistics guys because he was calling something analytical statistics. But then Jabe was, like, really showing me the difference between a histogram and a control chart. And like, oh, my God, I get it. Right. And and I tried to. Play around a little bit, you know, with, like, some data that I had, but then, you know, we had Dennis surgeon and he was so, so, helpful in that podcast.

I think a bunch of us said, hey, Dennis, why don't you just the next week? You just give us a tutorial on on this stuff. Because I was at my limit of like, okay, let me try to 6 time explain the way I think the difference between common and special causes. Right? So And I think [00:25:00] after that, a couple of you guys really sort of went off to the races, right?

And I think you in particular found, you know, so some real value in your day to day work with this, right? 

Rob Park: Yeah, for sure. Yeah, because actually before that call that. Dennis did that, that intro. I had actually, one of the great things about this community as well. Like, I think we talk about that sometimes in the DevOps community, like very open inviting community, but, but this, the Deming community is kind of been off the charts, inviting and kind and generous with their time.

I mean, I've had time multiple times besides the course now with Dennis. So I had actually been on a call with Dennis privately before. That particular day. Yeah. And starting to build control tarts. I think the next layer is. Like, intuitively, a lot of these things can make sense. The math is pretty easy if you read Dr.

Wheeler's books to figure out, like, how to put these things together. And that's [00:26:00] important. Can 

John Willis: you spend a couple minutes on that, why Wheeler's books are so important here? 

Rob Park: Yeah, so, so for me being very nerdy that I am like there's, it's kind of a very interesting topic, at least to me about number of points and number of points that you need to make this actually relevant. 

I don't know. I'm glad you went over the analytical versus Enumerated, enumerated. Enumerated with, with John Des on the, in the previous episode. So reference that. 'cause I'm the, I have listened to Mark Dr. Toi like his episode on the Deming podcast is great about that too. But but when it comes to yeah, what was I talking about?

 So with the number of points and why Wheeler was important. Oh, right, Dr. Wheeler. So. 

Rob Park: There's this whole idea of should you use standard deviation to to come up with the limits, you know, the upper and lower control limits, right? Or, or is [00:27:00] it not standard deviation? And Dr.

Wheeler is the best I can find that Really talks about using these other dispersion type statistics to calculate those sigma, the three sigma limits. Okay. And you can find multiple places and that refer to just talking about like the, those it says three sigma, which is kind of like, commonly known as standard deviation, but it's really not standard deviation, unless you go like to the Deming website and you look about control charts and, and Shewhart is still a bit of a mystery to me.

So I can't really tell if he's saying that or not, but Dr. Wheeler is the one that's really moved us towards Process behavior charts and and like John Dues is consistently talking about process behavior charts. And the other difference there is when you see a change in the pattern, you know, so the ones that we really, the ones that I've [00:28:00] learned to sort of look for, it's like a pattern 1 and pattern 4, like you talk about pattern 1, which is just like your one special cause.

Right. Which, you know, and, and what was that? And can we, can we investigate that? But when you have like a true shift in the process, well, now, what do you do with the chart? And one of the really fascinating thing to me that, because you see this in, in his understanding variation book, which is a very approachable book.

It's a small little book. And it's like, I, I know certain things by page number in there. So yeah, and 

And so but when there's a shift in the process, there's a shift in the upper and lower limits, and And the, the key is that you actually can, you know, you can now see that this is what you know, we made a change to the process or something happened to the process.

And that was the cause of the shift. And what's interesting about actually starting to track the data [00:29:00] for yourself, you have to figure out whether it be Excel or I'm usually in a Google sheet, but but it's the. Well, so a getting it to map out and how you actually get it to, to appear. But when you now see the trend coming you have to have patience if you don't have back data, because like you're collecting, you know, one point a week and you need 20 something points.

But but you only need. Actually, you only need like four points to actually get the chart really started. And that's, that's the other thing. And, and so it's been explained to me. So one of my friends from when I lived in Colorado in the Kanban community was Frank Vega. And Frank Vega actually used to work in that same consultancy with Jabe and Kevin as well.

So I had been reintroduced to him from there. It turns out like. He knows all about this stuff too, so I actually have reconnected with him on a regular basis to talk about, it's the difference with the limits is a [00:30:00] dispersion metric. And so you can use standard deviation, but you're more likely to get I don't know if the right term is like false positives, but you're more, likely, like the range is potentially going to be wider or smaller than it should be.

And, and that's why apparently, and I have seen and read about, but it's hard for me to explain because I'm, I'm a bit. Well, what 

John Willis: is the alternate to standard deviation? Because this is where I've gotten this, you know, you see, you've got a little beyond, I know there's other sort of, you know it's like mode.

Like, it's different sort of forms that you can use. Yeah. Come up to me. It's not just you find some that have 

Rob Park: the median in the middle and not. Yeah. Yeah. And from looking at a bunch of stuff with, like, Doug Williams is a another guy actually in Don Berwick in the at the, healthcare Improvement Institute of Healthcare Improvement.

 They, they actually explain this pretty well in one of their courses of when you only have a [00:31:00] few points, you, you can start with a run chart and a run where a run chart is really just the trend in it, and then they'll put a median in the middle of that particular chart. But when you turn it into a process behavior chart, it's the mean in the middle.

And so that definitely had thrown me, but the, the limits Dr Wheeler in understanding variation. gives you the formula. There's actually a really good little pdf too that okay that dr wheeler wrote so he's created 

John Willis: his own statistical formula basically 

Rob Park: yeah i don't yeah i don't know where or how he developed those particular okay formulas but they're they're again they're simpler you don't even need to calculate the the standard deviation it's it's all based off of the, the mean, and then the part of the chart, it's an XMR chart so that you have the moving range and he's tracking the, the range that it, that you get the variance in between, you know, point A to [00:32:00] point B to point C.

John Willis: Right. No, I, I have actually read some of that and then I'm like, all right, unless I have real data and then like, then like, it gets, it's really 

Rob Park: interesting. It was interesting at first to start figuring out, well, what data do I put on here and then collect it. And then you go and look at some back, like collect some data that you have from, that I had from a couple of years or from a year previous, it was actually some support call data.

And then you see a special cause that actually happened. Yeah. In the system back in, you know, back in June or whatever when I investigated that a little further, it turned out, yeah, there was actually a problem that we were having with the vendor with one of our vendors at that particular time. The problem got fixed.

It took like a month and a half or so before the problem got rectified, but once it did, the process came back to normal. 

John Willis: And 

Rob Park: it's interesting to see that in the past, but when you see it, Like, now you have a point and it moves, like, you can see [00:33:00] the, you can see the special cause coming, but you have to wait.

Yeah. Because I'm collecting data now. 

John Willis: You know, and that's the, you know, going back to the analytic works and numerated, and I think there's a bunch of cool things to sort of, The couple there, right? One is, I love this sort of like Deming you know, sort of like, and I know it was mangled as quotes. I shouldn't even call it a quote, but, you know, like, it's the, it's not the statistician's job to find the problem.

It's to see, show the pattern. So the subject matter expert can go investigate. And that's really to me, like, one of the core values of analytical statistics. There are these physics You know, there's there's sort of a physics in the way patterns will move because they're based on sort of human conditions, you know, 

Rob Park: Yeah 

John Willis: Widgets being made you know, software being delivered, support calls, asking about problems.

And so so when you can sort of totally decouple [00:34:00] what's going on to just pure data, and this was the beauty of Shewart. And then you add this visualization, which is the sort of run to chart, if you will. You get to see that, this, that, so I, I can't imagine either at this point you're in three conditions or you're listening to one, you're like, you shut the podcast off and you stop listening.

or two, you've got a whole, you're gonna go back and write down all this stuff and you're gonna go read about it. And, and we'll get to a point where I think we'll entice you to that because I'm gonna ask you in, in a, in about a minute or two, what are some examples, but, you know, or I guess there's a third that is like, well, maybe I need to give you a little bit of an explanation, but I can't imagine you've gotten this far in this podcast series and and have not been.

Yeah. But, but the idea is that in very simplistic terms, right? The idea of statistical process controls, you take data over time and you sort of, you know, you sort of, you use some algorithms to basically create, you know, the, the, the, the mean. And then the [00:35:00] points then become the sort of the, this running time series of standard deviation.

And you're looking for a three sigma, which is by default three standard deviations above the mean and three below. And when Rob talked about the pattern one is any sort of point or points. That are beyond that 3 sigma or below the 3 sigma and then there's a way to think about that. And then you've read my book, we've covered that for Debbie, but the more interesting ones are like, what Rob said is these sort of ones that are inside the 6 sigma, if you will, that, you know, or the 3 sigma above low and I, and the pattern ones are interesting. I mean, I was looking at the, what the 8, you know, the sort of the bell labs, there's 8 patterns, you know, like, like 8, 7 or 8 increasing or decreasing or, but you're saying the ones that I heard John Due's talk about this to the ones that like, are like, you're going along.

I [00:36:00] got to say 1 more thing, right? Like, you know, according to damning the original physicist is. There should always be randomness like in in the process. Right? So what you really should be seeing in these random points below and below below, you know, nothing, no specific pattern. So you see a pattern.

That's an anomaly. 

Rob Park: Yeah. And really kind of backing up to, you know, some of like, well, what's the point? Like, yeah, 1 of the thing, you know, from Dr Deming is, is that. Management is prone to react to variation. That is just common cause variation. Right. And yeah, and I think the other thing that, that intuitively before this, like, just has bugged me in the past is that we, you hear, I guess, particularly management being you know, trying to be very cheerleader type effect about you know, how great this is when it's maybe two points, like we went from, from Like, you know, a bug [00:37:00] rate of like collecting, I don't know, a hundred defects a week.

And now we're only collecting 95 defects a week. And so we're making progress in the right direction where it's, it's not necessarily the case. Right. But the other thing that's been cool, that's also very tied to the whole system of profound knowledge is when you start getting into the PDSA loops and, and trying to figure that out.

That is a whole nother interesting thing to me of when, you know, you can read about it, which is very straightforward and actually sounds pretty, pretty easy to do. And then you try and do it and and finding the actual words that you need to put down and to , you know, talk with the team about, so what is the aim?

Like, what, what are we actually trying to accomplish here? And then what are we looking to improve? But then you come back to the control chart. To that is where you will actually see if you've made an improvement or not 

John Willis: Yeah, no, I think that you know, that's I mean [00:38:00] then, you know So what we're really talking about is the system of profound knowledge, right?

which is at least two elements of it, but they all four apply right which is You know and I and I was I I was you know I've said this and i've been some people i've been criticized a couple times. I think there is an order I you know, I know I know Theoretically, there's no order because all four have to work together, but I think you start with the theory of knowledge, you know, how do we think, you know, do we really know what we think we know, right?

We have this sort of theory of what we know, but it is a theory because we don't really know and it may even change even if we did get it right. And then we use the, you know, the understanding variation to apply. A way to sort of, to your point, you can do the PDSA, the Plan, Do, Study, Act, which really is a theory of knowledge, which is sort of epistemology, which is theory of not, you know, which is a scientific method, if you will, but, but like, what do you do?

[00:39:00] How do you, you know, like, okay, you know, and, and they do in a very many cases is the sort of the process control chart, the statistical process control to be able to see, you know, like, what is the effect? What is the The study part of it, right? And 

Rob Park: yeah, I mean, you can also see the negative, right? You can, but you can make a change.

And if you're paying attention from the perspective of the cycle, then you can see that the change actively did not work like it 

John Willis: was worse. Right? Or that's the whole experiment part. Like, you know, there's what I love about or epistemology is, you know, the, the There is no wrong or right. It's the like we tried something and now we know what we need to do next, right?

And maybe that's what we tried actually, you know, the other thing you made me think about, which is, you know, I guess I might have connected this, but the idea that management get all excited about less defects, go back to Rother's story [00:40:00] about the Toyota plant. You know, where they were pulling like, you know, a thousand and our cords a day and it went down to 800.

Right. And, you know, the sort of the the the Western culture would be gray, you know, 800, we went from a thousand, 800 and then it closed and the plant manager, no, we got a problem. We're learning 200 times less a month or a day or whatever it was. Right. So, 

Rob Park: yeah, I sometimes feel like, like the grumpy old man sometimes too, because I, I don't care as much about the successes.

I wanna hear about like what's yeah. Where the problems are. And that comes outta that same kind of culture. Yeah, I think Deming, 

John Willis: Deming would be having a blast right now with like, I, you know, I think Allspaw and those guys love John and all that, but I, I, I don't think I've, I would, I, if I couldn't get Richard Cook to, I did get Decker to, to literally acknowledge Deming and, and tell me he thought it was relevant.

But I don't think I'll ever get John and, and, and dear old Dr. Richard Cook, who's a brilliant man who [00:41:00] passed away a year or so ago. And you know, but I, I would have loved to just gotten sitting Deming down with Dr. Woods. And, and, and, and, you know, Richard Cook and John and like, like, oh, I get it.

Yeah. You know, because he would have loved those guys. Right. I mean but like, so let's get 

Rob Park: into the, I've been trying to map those charts on things like just. Regular things like whip counts, you know, how much whip we have in the current system defect counts things like flow metrics. So through cycle times but then also like Dora type related things.

So like lead time for changes and, and put those on a. Process behavior chart, as well as, you know, how many deployments do we have in a, in a certain time period. 

John Willis: See, that's where it gets interesting. You know, you know, I'm, you know, I, I think Dora, we've had this before every time you, you know, I think you, you're even like, don't mention Dora for a John, and this is going to get ugly, [00:42:00] right?

Like in some of the book, but But like, I, like, let's be clear, it was incredible for our industry. Like we had nothing, we were doing no data, you know, Nicole says I'm sciencing, we're, we're sciencing the shit out of DevOps, you know, or DevOps, you know, like that. And she's right. We were doing, you know, we were just.

Putting our finger in the air. Right. And so Dora was, I think where I sort of fell off the cliff was, you know, when it met, you know, non operational definitions and, you know, then what is the lead time really mean if you're not crystal clear about what starts when it ends and what is at some point the operational 

Rob Park: definitions are important.

John Willis: Big time, right? And I know we've talked a lot about that but, but here again, like, I think Dora data matched with, you know, control charts and SPC, then now you're, you know, you're circling back with now you know, I can have an opinion about deploys a day and I could tell you, I think they're not as useful once you get to a [00:43:00] certain achievement level in DevOps, but then I think I'll let you sort of tell me some examples, but like, I think, you know, If I'm tracking that and I can look for these patterns, then it can be incredibly interesting.

Yeah, yeah, 

Rob Park: I mean, I think you're, you're spot on in the trouble and, you know, we've seen this before, right? Like, agile jumped the chasm, we have problems, DevOps is now getting a little grayer in the tooth. And so you know, longer in the tooth, whatever the phrase is, and, you know, and so. Now it's about, you know, we get a lot of people that are focused.

You hear DevOps, or you have a DevOps team or a DevOps engineer. And some of us cringe at that kind of concept, like because it's, it's not really what I believe it's about, but so we're, we're likely to have the, it seems like the same thing. Like the tool vendors are all over the Dora stuff now, but they're all over it with pull requests and pull request links and things like that, which is.

[00:44:00] Not not necessarily my cup of tea. So the so the things that that I think you're spot on with is, is needing your, like, you try to actually measure these things yourself. And I'm usually recommending, like, do it with a spreadsheet, like, go and look at deployments and get and and then put your stuff on a, on a spreadsheet, but write down your operational definition of what I'm actually what it really is.

John Willis: Yeah, yeah. That's right. 

Rob Park: And the other one that always gets me is actually back to Dr. Schuhart's rule of the data should be available. Also the other thing that I don't know if you've seen this, but if you put a control chart and like the next challenge is teaching management that especially in software doesn't see this stuff.

Like they don't know about these things. You know, Deming is kind of foreign to a lot of software. And so they'll see like the special cause Item above the line and they'll want to chop it off because it's an anomaly. So so we're going to, you know, [00:45:00] chop it out. Cause I don't want it in my data set.

I don't want to cluttering up my quote unquote average that I'm going to base all of my estimates on. And this is just very problematic and this is where it's. Exciting that from Shewart to Deming to Dr. Wheeler that they're explaining like how to really, how to really do this, 

John Willis: how to understand all this is a whole science 

Rob Park: to it.

John Willis: My favorite quote in my book is the misunderstanding variation is the root of all evil. Right? And I heard that somewhere along the way, and I could never, you know, I used to just say, you know, unknown. And I said, you know, I've been saying I know so many years. I'm just going to say dash box loop. Right? So, right.

But, but that is the whole point, right? Like the whole idea of like, going from 1000 to 800 or like that. I mean, that is the enumerated versus analytic in the most simplest sense, right? Not be a decision, right? Is that enumerated tends to like, trend us towards, you know you know, more is better or less is better, you know, and, you [00:46:00] know, or like an anomaly, you know this anomaly is bad.

Like there's no scenario where anomaly is good, where analytical approach is. It's just data and it's points. So let's go investigate what that means. It's not, you know, a point on a chart, you know, or, you know, a line on a chart. It's just data, you know, and analytical forces you to sort of step away from the chart to say, that's bad.

That's good. That's bad. It's like, no, no, that's something. Let me go figure out what that is. 

Rob Park: Right. Yeah. And as far as I understand the levee, like this dispersion metric has a lot to do as well. Like, you don't have this population of. Thousands and millions of points like you're not doing a full statistical study.

You have like a couple dozen points and that that's what we're working with. And, and, and also, by the way, it's not normal. Like, it's not a normal distribution. So so trying to counteract those effects and still be able to make. [00:47:00] Meaningful judgments out of it. So, so just 

John Willis: to sort of close it up, then, like, you got any good sort of gems and things that like, like, as you've been tracking some of this stuff that, like, would just call it like DevOps data, which is really cool.

Some like interesting findings or, you know, like, 

Rob Park: Let's see, I don't know, probably anything that I say, it gets me in trouble. Okay. No, no, 

John Willis: no, it's fine. It's a, it's a, it's a company in Ohio that does, you know, makes green widgets that, you know, 

Rob Park: but, but tracking some cycle times of. You know, basically stories that we're pushing through or we're pulling, sorry.

Okay. Okay. Pulling through our system in a, on a software team where most of us were doing pairing or some amount of software teaming. But there was one individual not necessarily comfortable with that style and was, you know, Was doing this, the other very common style of put it on a branch, put it get a pull request and get some [00:48:00] reviews doing things very solo. 

And then you see on the control chart over time that there's blip blip blip, like every couple of weeks, there's 1 special cause item that's popping above the line. And sure enough, that's what it was. They were, they were tickets for that. The pull request tickets were the ones that got dragged out and, you know, this is a relatively easy story to tie back, but it's, again, it's affirming when you see it in the data and different teams will have different contexts and their situations will vary.

This was very particular to our situation. And 

John Willis: I think it's a good way to close it out because that is the, if I can, if we can teach people to be theory based. Then you get to like, then there's no wrong or right answer. There's no, I'm a senior person. Listen to me. I've only been here a week. I have a theory.

Let's run your theory and let's look at the data and let's see what happens, right? Like, you know, and everybody it's it's an equalizer, right? And it and it gets you to the [00:49:00] right, you know, at least a consensive and collaborative right answer or better answers. Yeah, 

Rob Park: yeah. And of course, the more the more we can talk about it, I'm, I'm excited to try and push or to get more P.

D. S. A. Thinking into. Like your more common retrospective instead of. 

John Willis: No, I think this is a really exciting. I mean, I'd encourage you to think about writing a book. Although I'm sure you're like, John, I got way too much other stuff going, but I think over time, you're going to compile a really, if 

Rob Park: I can actually start writing a blog, it'd be pretty handy.

John Willis: There you go. Yeah. 

Rob Park: So how 

John Willis: do people get ahold of you? 

Rob Park: I probably linked in is really the main social media that that I can be found on. I don't really pay attention to the others much anymore. I, I have started a very small blog, but, you know, you can email me at Robert park. Okay, we're like, software, so it's the number 4.

I can get, we can put in the show nights if you want, we'll get all 

John Willis: that in 

Rob Park: the show notes. So, well, that's fun, man. 

John Willis: Yeah, [00:50:00] 

Rob Park: it's like, I've been. A great learning journey and. Just wanted to say thank you. Like I, no, I, I knew you, you impact on my life, so, yeah. No, that was good. I 

John Willis: knew you were doing, you had taken sort of, I don't even call it the torch because I never really did anything significant other than try to figure out how to explain it to you.

It was just policy control and I knew you and there was a couple of people from the book club who were really trying to like put it into practical work, and we all know it makes sense. So I was excited to get you on a call and just talk about your journey. All the way through and then, you know, some of the stuff you're actually doing right now, which is cool.

And I, I hope I encourage you. I think it would be awesome to see whatever blogs you can write about it to show us more about, like, what our practical successes can be with using this, this, you know, just, you know, just great. Wealth of opportunity knowledge, which is Deming, Shewart, Wheeler, you know, and all the, you know, getting the, the.

The village of people who have, like, help. 

Rob Park: Yeah. And and then also, like, the existing [00:51:00] village of people to again, that's what I'm saying. Yeah, the village Dennis and you know, and Bill bellows and like, all these people are very accessible. 

John Willis: Very easy to just reach out and. Yeah, no, it's a great, great stuff.

And they're all excited about you know, and then it worked. The plan is, you know, I've said this, I'll start talking more about this, but, you know, I was invited to a bill ran for years into in thinking. Network of a lot of Deming industrial engineers, rocket engineers, bowie Boeing engine and like all these people and for like 10 years, they had not had a meeting.

And then it was Dick steel that, hey, you're going this thing. I'm like, where, when? And I was the only sort of non industrial engineer, you know, like, what the heck is DevOps? And I, you know, I, Bill was fascinated. He's gotten to meet a lot of you guys now. And. And so in June, I, you know, we're going to try to get the dates.

I, you know, the route now is to have all those people, double the size of the room, [00:52:00] all those people come back and we bring the other half of the room of a bunch of DevOps people and just watch this this glorious fusion. Of, like, what we're doing and what they've done, and, you know, people have worked with Goldratt people, you know, so I really just have this just I think it'll be a phenomenal event.

So. 

Rob Park: Yeah, well, that's also like, that's what Dick Steele's Deming study group is like as well. But yeah, yes, I have that the into in. I think it's already on my calendar for next. Okay, good. All 

John Willis: right. Good. Good. All right. Well, well, thank you. It was great. Thank you. 

Rob Park: All right.