How Software Groups Rot: Legacy of the Expert Beginner

Expert Beginner Recap

In my last post I introduced the term “Expert Beginner” to describe someone who has capped out in their learning at some sort of local maximum, convinced that the local is global. Expert Beginners are developers who do not understand enough of the big picture to understand that they aren’t actually experts. What I mean by this is that they have a narrow enough perspective to think that whatever they have been exposed to is the best and only way to do things; examples would include a C# developer who pooh-poohs Java without ever having tried it, or a MySQL DBA who dismisses the NoSQL movement as a passing fad. This isn’t to say that not liking a technology or not having worked with one makes someone an Expert Beginner, but rather the vaguely solipsistic mindset that “if it’s not in my toolchest or something I’ve had experience with, it’s not worth doing.”

Another characteristic of the Expert Beginner, however, is that they have some position of relative authority or influence within a software group. In my last post, I proposed the term Expert Beginner without explaining the rationale, planning to discuss that here. The Advanced Beginner is someone who is in the advanced stage of being a beginner, whereas the Expert Beginner descriptor is both literal and intentionally ironic; it’s literal in that someone with that much practice at being a beginner could be said to be an expert and ironic in that the “expert” title is generally self-applied in earnest or else applied by managers or peers who don’t know any better.

The iconic example might be the “tech guy” at a small, non-technical firm. He “knows computers” so as the company grew and evolved to have some mild IT needs, he became a programming dilettante out of necessity. In going from being a power user to being a developer, his skills exceeded his expectations, so he became confident in his limited, untrained abilities. Absent any other peers in the field, the only people there to evaluate his skills are himself and non-technical users who offer such lofty praises as “it seems to work, kind of, I think.” He is the one-eyed man in the valley of the blind and, in a very real and unfortunate sense, a local expert. This is the iconic example because it has the fewest barriers to Expert Beginnerism–success is simple, standards are low, actual experts are absent, competition is non-existent, and external interaction is not a given.

A Single Point of Rot…

So far I’ve talked a lot about the Expert Beginner–his emergence, his makeup, his mindset, and a relatively sympathetic explanation of the pseudo-rationality, or at least understandability, of his outlook. But how does this translate into support of my original thesis that Expert Beginners cause professional toxicity and degeneration of software groups? To explain that, I’m going to return to my bowling analogy, and please bear with me if the metaphor is a touch strained. For those of you who didn’t read the first post, you might want to give the second section of it a read because this is a lot to rehash here.

Let’s say that bowling alleys make their money by how well their bowlers bowl and that I live in a small town with a little, startup bowling alley. Not being able to find any work as a software developer, I try my hand bowling at the local alley. I don’t really know what I’m doing, and neither do they, but we both see me improving rapidly as I start bowling there, in spite of my goofy style. My average goes up, the bowling alley makes money, and life is good–there’s no arguing with profit and success!

Around about the time my score was topping 150 and the sky seemed the limit, the bowling alley decided that it was time to expand and to hire a couple of entry-level bowlers to work under my tutelage. On the day they arrived, I showed them how to hold the ball just like I hold it and how to walk just like I do. When they ask what the thumb and finger holes are for, I respond by saying, “don’t worry about those–we don’t use them here.” Eager to please, they listen to me and see their averages increase the way mine had, even as I’m starting to top out at around a 160 average.

As time goes by, most of them are content to do things my way. But a couple are ambitious and start to practice during their spare time. They read books and watch shows on bowling technique. One day, these ambitious bowlers come in and say, “the guys on TV put their fingers in the ball, and they get some really high averages–over 200!” They expect me to be as interested as they are in the prospect of improvement and are crestfallen when I respond with, “No, that’s just not how we do things here. I’ve been bowling for longer than you’ve been alive and I know what I’m doing… besides, you can’t believe everything you see on TV.”

And thus, quickly and decisively, I squash innovation for the group by reminding them that I’m in charge by virtue of having been at the bowling alley for longer than they have. This is a broadly accepted yet totally inadequate non sequitur that stops discussion without satisfying. At this point, half of the ambitious developers abandon their “fingers in the ball” approach while the other half meet after bowling at another alley and practice it together in semi-secret. After a while, their averages reach and surpass mine, and they assume that this development–this objective demonstration of the superiority of their approach–will result in a change in the way things are done. When it instead results in anger and lectures and claims that the scores were a fluke and I, too, once bowled a 205 that one time, they evaporate and leave the residue behind. They leave my dead-end, backward bowling alley for a place where people don’t favor demonstrably inferior approaches out of stubbornness.

The bowling alley loses its highest average bowlers not to another alley, but to an Expert Beginner.

…That Poisons the Whole

The bowlers who don’t leave learn two interesting lessons from this. The first lesson they learn is that if they wait their turn, they can wield unquestioned authority regardless of merit. The second lesson they learn is that it’s okay and even preferred to be mediocre at this alley. So when new bowlers are hired, in the interests of toeing the company line and waiting their turn, they participate in the inculcation of bad practices to the newbies the same way as was done to them. The Expect Beginner has, through his actions and example, created additional Expert Beginners and, in fact, created a culture of Expert Beginnerism.

The other interesting development that results comes in the acquisition process. As the Expert-Beginner-in-Chief, I’ve learned a pointed lesson. Since I don’t like being shown up by ambitious young upstarts, I begin to alter my recruitment process to look for mediocre “team players” that won’t threaten my position with their pie-in-the-sky “fingers in the ball” ideas. Now, I know what you’re thinking–doesn’t this level of awareness belie the premise of the Expert Beginner being unaware of the big picture? The answer is no. This hiring decision is more subconscious and rationalized than overt. It isn’t, “I won’t hire people that are better than me,” but, “those people just aren’t a good fit here with my ‘outside the box’ and ‘expert’ way of doing things.” And it may even be that I’m so ensconced in Expert Beginnerism that I confuse Competent/Expert level work with incompetent work because I don’t know any better. (The bowling analogy breaks down a bit here, but it might be on par with a “bowling interview” where I just watched the form of the interviewee’s throw and not the results, and concluded that the form of a 220 bowler was bad because it was different than my own.) And, in doing all this, I’m reinforcing the culture for everyone including my new Expert Beginner lieutenants.

And now the bowling alley is losing all of its potentially high average bowlers to a cabal of Expert Beginners. Also notice that Bruce Webster’s “Dead Sea Effect” is fully mature and realized at this point.

Back in the Real World

That’s all well and good for bowling and bowling alleys, but how is this comparable to real software development practices? Well, it’s relatively simple. Perhaps it’s a lack of automated testing. Giant methods/classes. Lots of copy and paste coding. Use of outdated or poor tooling. Process. It can be any number of things, but the common thread is that you have a person or people in positions of authority that have the culturally lethal combination of not knowing much; not knowing what they don’t know; and assuming that, due to their own expertise, anything they don’t know isn’t worth knowing. This is a toxic professional culture in that it will force talented or ambitious people either to leave or to conform to mediocrity.

You may think that this is largely a function of individual personalities, that departments become this way by having arrogant or pushy incompetents in charge, but I think it’s more subtle than that. These Expert Beginners may not have such personality defects at all. I think it’s a natural conclusion of insular environments, low expectations, and ongoing rewards for mediocre and/or unquantifiable performances. And think about the nature of our industry. How many outfits have you worked at where there is some sort of release party, even (or especially) when the release is over budget, buggy and behind schedule? How many outfits have you worked at that gave up on maintaining some unruly beast of an application in favor of a complete rewrite, only to repeat that cycle later? And the people involved in this receive accolades and promotions, which would be like promoting rocket makers for making rockets that looked functional but simply stopped and fell back to Earth after a few hundred feet. “Well, that didn’t work, Jones, but you learned a lot from it, so we’re promoting you to Principal Rocket Builder and having you lead version two, you rock star, you!” Is it any wonder that Jones starts to think of himself as King Midas?

As an industry, we get away with this because people have a substantially lower expectation of software than they do of rockets. I’m not saying this to complain or to suggest sweeping change but rather to explain how it’s easy for us to think that we’re further along in our skills acquisition than we actually are, based on both our own perception and external feedback.

Create a Culture of Acquisition instead of Stagnation

Having identified a group-(de)forming attitude that could most effectively be described as a form of hubris, I would like to propose some relatively simple steps to limit or prevent this sort of blight.

First of all, to prevent yourself from falling into the Expect Beginner trap, the most important thing to do is not to believe your own hype. Take pride in your own accomplishments as appropriate, but never consider your education complete or your opinion above questioning regardless of your title, your years of experience, your awards and accomplishments, or anything else that isn’t rational argumentation or evidence. Retaining a healthy degree of humility, constantly striving for improvement, and valuing objective metrics above subjective considerations will go a long way to preventing yourself from becoming an Expert Beginner.

In terms of preventing this phenomenon from corrupting a software group, here is a list of things that can help:

  1. Give team members as much creative freedom as possible to let them showcase their approaches (and remember that you learn more from failures than successes).
  2. Provide incentives or rewards for learning a new language, approach, framework, pattern, style, etc.
  3. Avoid ever using number of years in the field or with the company as a justification for favoring or accepting anyone’s argument as superior.
  4. Put policies in place that force external perspectives into the company (lunch-and-learns, monthly training, independent audits, etc).
  5. Whenever possible, resolve disputes/disagreements with objective measures rather than subjective considerations like seniority or democratic vote.
  6. Create a “culture of proof”–opinions don’t matter unless they’re supported with independent accounts, statistics, facts, etc.
  7. Do a periodic poll of employees, junior and senior, and ask them to list a few of their strengths and an equal number of things they know nothing about or would like to know more about. This is to deflate ahead of time an air of “know-it-all-ism” around anyone–especially tenured team members.

This list is more aimed at managers and leaders of teams, but it’s also possible to affect these changes as a simple team member. The only difference is that you may have to solicit help from management or persuade rather than enforce. Lead by example if possible. If none of that is possible, and it seems like a lost cause, I’d say head off for greener pastures. In general, it’s important to create or to have a culture in which “I don’t know” is an acceptable answer, even for the most senior, longest-tenured leader in the group, if you want to avoid Expert Beginner fueled group rot. After all, an earnest “I don’t know” is something Expert Beginners never say, and it is the fundamental difference between a person who is acquiring skill and a person who has decided that they already know enough. If your group isn’t improving, it’s rotting.

Series is continued here: “How Stagnation is Justified: Language of the Expert Beginner”

Edit: The E-Book is now available. Here is the publisher website which contains links to the different media for which the book is available.

  • Josh

    Great writeup. This has been making the rounds at the office for a little while and came up again today when I ran into a couple of people who hadn’t yet heard the term “Expert Beginner”.

  • http://www.daedtech.com/blog Erik Dietrich

    Thanks — I’m glad you liked and that you and your office mates have gotten some value out of the posts!

  • http://twitter.com/aamartynenko Anton Martynenko

    Superior post… That’s about almost every project I was involved in =)

  • http://www.daedtech.com/blog Erik Dietrich

    Thanks — glad you liked the post! Stay tuned, too. I’m going to be adding to this series.

  • Thomas Winget

    Don’t let this praise go to your head. Yes, it’s a solid post, but don’t think you can’t get better. Maybe your writing style is bad? Who knows??
    :D thanks for a good read. here from /r/programming, for what that’s worth.

  • Pingback: How Stagnation is Justified: Language of the Expert Beginner | DaedTech

  • Ben C.

    I violated the “Avoid ever using number of years in the field or with the company as a
    justification for favoring or accepting anyone’s argument as superior” rule when talking to a junior developer a couple weeks ago. I felt bad when I said it (made me feel old too) and I knew it was wrong at the time I said it. It shut him up and I know it wasn’t good for his morale. I’ve been too busy to apologize to him, but I think I’m going to do that now. Thanks for this post!

  • Plain Beginner

    First of all, thanks for a great read. I’m a fresh developer myself, trying to avoid becoming an expert beginner and my situation is in some ways similar to the iconic example described above (unfortunately.) With the little experience I have I know for certain that I’m no expert but sometimes I get a feeling that some are expert beginners out of convenience and that they in some subconscious corner of their minds know they aren’t that fantastic, or t least could do much better. Their problem is that they have reached a certain stature that requires some skill and if they don’t have it they need to fake it. It’s also easier not to make that extra effort to improve, even though they know its likely to bite them in the future. Maybe because they don’t exactly know how, the suggestions of things to learn and research that others have given them don’t quite work for them, they currently lack some fundamental piece of knowledge or understanding that would allow them to put the pieces together or something else.

    There are people at my workplace who are tricky to define. Boss believes in testing so we’re (finally) beginning to do that, but testing everything takes too much time. There’s no time put in to re-factor just fix the bug and move on to the next one. We know that we haven’t introduced a new bug with the fix (which is often the case) if we don’t hear anything from our users for a while (work with in-house tools so feedback is often instant.) Additionally, the tool is used in several countries so if someone fixes something in one country it often breaks something in another. So we end up endlessly fixing the same things, but boss doesn’t see the pattern and seems to think that every bug is unique and thinks they should be treated as such. So suggestions to improve our practices in a more general sense is not likely to be heard as he thinks we don’t have problems.

    Now to finally get to the point, I don’t think that boss is an expert beginner because he would probably be the first to admit that he’s no expert. He just displays many of the characteristics of one, maybe because he understands how much work it would require to do things right and we simply don’t have that amount of time and funds. Of course they need to be spent sooner or later. I don’t know if all this was fortunate in my case as I was looking for my first job and found it there, and at first I learned a lot, but after a while I began realizing that I would learn much more by doing things the right way. Not sure if people who are better programmers would have hired me though because to good jobs there are likely to be good applicants and even though I don’t consider myself bad I’m basically starting from the ground and building my way up.

    Or am I getting this wrong? Is he an expert beginner or one in coming? Is there likely to be a point in time where people who are destined to become expert beginners feel they have learned enough from their mistakes and experiences that they truly become expert beginners and just incorporate some of good practices in their work but misuse or misunderstand them? They get along well enough to fool some but not all.

  • Plain Beginner

    First of all, thanks for a great read. I’m a fresh developer myself, trying to avoid becoming an expert beginner and my situation is in some ways similar to the iconic example described above (unfortunately.) With the little experience I have I know for certain that I’m no expert but sometimes I get a feeling that some are expert beginners out of convenience and that they in some subconscious corner of their minds know they aren’t that fantastic, or t least could do much better. Their problem is that they have reached a certain stature that requires some skill and if they don’t have it they need to fake it. It’s also easier not to make that extra effort to improve, even though they know its likely to bite them in the future. Maybe because they don’t exactly know how, the suggestions of things to learn and research that others have given them don’t quite work for them, they currently lack some fundamental piece of knowledge or understanding that would allow them to put the pieces together or something else.

    There are people at my workplace who are tricky to define. Boss believes in testing so we’re (finally) beginning to do that, but testing everything takes too much time. There’s no time put in to re-factor just fix the bug and move on to the next one. We know that we haven’t introduced a new bug with the fix (which is often the case) if we don’t hear anything from our users for a while (work with in-house tools so feedback is often instant.) Additionally, the tool is used in several countries so if someone fixes something in one country it often breaks something in another. So we end up endlessly fixing the same things, but boss doesn’t see the pattern and seems to think that every bug is unique and thinks they should be treated as such. So suggestions to improve our practices in a more general sense is not likely to be heard as he thinks we don’t have problems.

    Now to finally get to the point, I don’t think that boss is an expert beginner because he would probably be the first to admit that he’s no expert. He just displays many of the characteristics of one, maybe because he understands how much work it would require to do things right and we simply don’t have that amount of time and funds. Of course they need to be spent sooner or later. I don’t know if all this was fortunate in my case as I was looking for my first job and found it there, and at first I learned a lot, but after a while I began realizing that I would learn much more by doing things the right way. Not sure if people who are better programmers would have hired me though because to good jobs there are likely to be good applicants and even though I don’t consider myself bad I’m basically starting from the ground and building my way up.

    Or am I getting this wrong? Is he an expert beginner or one in coming? Is there likely to be a point in time where people who are destined to become expert beginners feel they have learned enough from their mistakes and experiences that they truly become expert beginners and just incorporate some of good practices in their work but misuse or misunderstand them? They get along well enough to fool some but not all.

  • http://www.daedtech.com/blog Erik Dietrich

    Glad if it helped, and I wouldn’t sweat it. I think all of us have probably done the same at some point or another, and I think that having that conversation after the fact to say “this isn’t how I want things to work” will probably earn you a lot of respect in that dev’s eyes.

  • http://www.daedtech.com/blog Erik Dietrich

    Hard for me to say if your boss is someone I would consider to be an Expert Beginner or not — this is more of a composite of behaviors I’ve observed during my tenure in the industry than a measuring stick for an individual. It sounds as though, perhaps, your boss might simply be focused on the bottom line, which is often a self-aware, pragmatic position (or sometimes just an excuse to take shortcuts). In other words, as you point out, he might not fancy himself an expert at all, and thus he won’t use his ‘expertise’ as the justification for dictating software practices.

    To put it more concretely, the Expert Beginner would probably say something like “I’ve been writing high quality code for 15 years without unit tests, so if anyone knows how pointless they are, it’s me.” A bottom-line pragmatist would say something more like, “yeah, unit tests seem like a good idea, but we really don’t know how to do them and we don’t have time to learn right now.”

    I’m hardly in a position to be a career coach, but in your position, my litmus test for whether I’m in a good place or not would be to gauge the group’s willingness to adopt new approaches *if you can demonstrate their superiority*. So, maybe you spend a few nights and weekends prototyping a testing strategy, try it out in a sandbox, and then call a meeting to demonstrate the concrete value of your approach. If the reaction to that is interest and curiosity, you’re probably in a good situation. If it’s skepticism/hostility/apathy/non-comprehension, you might give deeper consideration to whether this is a good long term fit. Again, though, I’m a little more restless than average, and this is just what I, personally, would do.

    Hopefully your situation works out, and you become known as the go-getter that’s introducing good practice to the group :)

  • http://www.facebook.com/paul.vahur Paul Vahur

    Great analysis and it goes well beyond coding, this can, has and is happening in many other fields. The main reason for it’s emergence is the lack of market test because this person is in monopoly position – the tech guy in your example – his supervisors are too lazy or ignorant to look for alternatives in the market and thus remain ignorant of the price/quality ratio of particular skillset and the person remains in monopoly unaware of his inadequacy.

  • Pierre Rasmussen

    I just find you blog and I am a big fan of you “Expert Beginner” series.

    Keep sharing you thoughts, they are inspiring!

  • Pingback: 103 RR Ruby Gems

  • Pingback: Comfortably Numb « Thinking in Christ

  • Pingback: Most interesting links of April ’13 « The Holy Java

  • Pingback: How Developers Stop Learning: Rise of the Expert Beginner | DaedTech

  • Pingback: No Excuses for the Expert Beginner | ello!

  • ElleDmytryszyn

    But what happens when “I don’t know” becomes a crutch for the team member who doesn’t want to do more than the bare minimum?

  • http://www.daedtech.com/blog Erik Dietrich

    I was actually thinking about a similar theme last week and contemplating a post about it. As a team/technical lead (which happens to be where most Expert Beginners find themselves), “I don’t know” is a response with a shelf life. It’s perfectly acceptable and even good to say off the cuff, but once there’s time to research, learn, and investigate, it becomes unacceptable. Fine, you didn’t know off the top of your head, but it’s a week later and you’re still no closer?

    So maybe the real answer that’s acceptable is “I don’t know, but here’s what I can tell you” or “I don’t know, but I’ll find out.”

  • Kevin

    Very interesting read. Though regarding this point: “Provide incentives or rewards for learning a new language, approach, framework, pattern, style, etc.”. I’m not sure these are the things developers should be learning. They need to learn them in order to “keep up to date”, but becoming an expert involves a lot more than learning new languages. In fact, as you become closer to being an expert, the languages / frameworks should matter to you less and less.

  • http://www.daedtech.com/blog Erik Dietrich

    I’m glad if you liked the post(s), and thanks for reading. I’m not really referring to creating an incentive to make the developers marketable or “up to date” as much as I am suggesting the same thing that you are in a way. The more languages and frameworks they know, the more perspective they’ll have on solving technical problems (e.g. a developer that understands functional, OO, and structured paradigms is likely to have better perspective on solving problems related to, say, multi-threading, than one who knows only a single language). Encouraging and incentivizing developers to keep bringing fresh solutions to problems into the culture can help eliminate stagnation, I believe.

    In other words, I agree with you that no particular language or framework is important. The incentive to learn new things is just a means to an end.