To Succeed, He Threw Out the Management Books

Throwing handA book flyingIn 1980 when Ricardo Semler took over SEMCO, a manufacturing plant with 100 workers and $4 million in annual earnings, it was nearly bankrupt. By 1989 he had introduced many radical changes, and his outline of those and the results were published in Harvard Business Review (HBR). When Semler first took over, he tried to run the company as already organized. After working 12-18 hours a day, seven days a week, still feeling like it was not enough time, at age 25 he passed out from stress. He had physical symptoms of illness and nearly had a nervous breakdown. Besides realizing the job was impossible as structured, Semler wrote, “’SEMCO appeared highly organized and well disciplined, and we could still not get our people to perform as wanted.’”[1]

After bringing in consultants and trying a variety of standard improvement processes like TQM, Semler in effect threw out the management how-to books. By 1989, he said in HBR, he had wiped out most of the management levels; eliminated set working hours (even in his factories); abolished “norms, manuals, rules, and regulations”; “done away with security searches, storeroom padlocks, and audits of the petty-cash accounts of veteran employees”; stopped tracking employee expenses; and begun sharing all company financials with all employees. When the company needed a new plant building, top management (called “counselors”) let employees lead the hunt, vote on which to buy (over the counselors’ objections), design the layout, and hire a famous artist to paint it!

SEMCO had 800 employees by 1989 and was “one of Brazil’s fastest-growing companies, with a profit margin in 1988 of 10% on sales of $37 million.” He said it was regularly named as a best company to work for in Brazil, and they didn’t bother advertising jobs because word-of-mouth brought an average of 300 applicants per position.

Many success stories stop at this snapshot in time, with no evidence on how the successes held up. In this case we have a rare before-and-after comparison with outside corroboration, thanks to a TED Talk Semler delivered a quarter century later, and two academic studies I cite below.

Semler reported in 2015 that the company had not only retained most of those innovations but added more. Two seats at board of director meetings were left open for the first two employees to show up—janitorial staff appeared at a meeting and had equal voting rights to the people in suits. SEMCO’s hiring processes allowed any employee to interview new prospects, and those prospects were told to come back and “spend a day, talk to anybody you want. Make sure we are the bride you thought we were and not all the bull… we put into our own ads.” They allowed employees to see everybody else’s salaries—and set their own! Once sales or production objectives were met, people were told to “go to the beach.” Otherwise they would mess up other groups who had made plans based on those numbers.

The result? By 2015 the company had 5,000 workers, low turnover, was making hundreds of millions of dollars annually, and had bought other firms.

As echoed by a couple of management professors, SEMCO’s success was built on an unusual philosophy of management. “Another way of saying this is that a philosophy is a foundation that gives perspective, and through which the uncertainties and contingencies of everyday organization can be faced through a set of principles,” wrote Steven Segal and Kyle Bruce of Macquarie University in Australia. “It is the paradigm through which practices, routines and behaviours make sense.”

However this was not a prewritten philosophy, they add: “It was not through armchair speculation that he developed it, but in the context of the lived experience of action that he developed a democratic philosophy through which organizing took place at SEMCO.”

I admire Semler’s decision-making primarily because he gave most of it away to his employees—and that worked. The company became successful, workers were happy, and he is mentioned often as an exemplar of progressive management. His decision-making can be boiled down to asking three “why’s,” he said in his TED Talk: “Because the first why you always have a good answer for. The second why, it starts getting difficult. By the third why, you don’t really know why you’re doing what you’re doing.” No doubt he borrowed this from the “Five Why’s” of Six Sigma, but his results indicate three are enough to change an organizational culture.

This approach, and the courage to question everything he had been taught, led to a highly successful management philosophy so radical most leaders remain afraid to try it 30 years later. “But the fact is that it takes a leap of faith about losing control,” Semler said after his talk. “And almost nobody who is in control is ready to take leaps of faith.”

I hope you can gather the needed courage to take a flying leap—in that good way!

Subscribe

Please share this post at the bottom of the page.


[1] Quoted in Segal & Kyle, 2017.

Sources:

  • Jiro Takaki, Toshiyo Taniguchi, & Yasuhito Fujii. (2016). Confirmation of Maslow’s hypothesis of synergy: Developing an Acceptance of Selfishness at the Workplace Scale. International Journal of Environmental Research and Public Health, 13(5), 1. https://doi.org/10.3390/ijerph13050462
  • Segal, S., & Kyle, B. (2017). Breaking the chains of ignorance: Manager-philosophers in recent management history. Journal of Management History; Bradford, 23(2), 118–132. http://dx.doi.org.csuglobal.idm.oclc.org/10.1108/JMH-02-2017-0006
  • Semler, R. (1989). Managing without managers. Harvard Business Review, (September–October 1989). Retrieved from https://hbr.org/1989/09/managing-without-managers
  • Semler, R. (2015). How to run a company with (almost) no rules. Retrieved from https://www.ted.com/talks/ricardo_semler_how_to_run_a_company_with_almost_no_rules

 

Bad Teams Talk Too Much… and Too Little

Team members talking
A team talking just enough?

Presenters often extol the value of more communication. At the same time we hear complaints about “information overload” and “too many meetings.” I am among those who have advised managers to err on the side of overcommunicating, and in The Truth about Teambuilding I cover evidence that talking more improves team decision-making. However, I added the qualifier up to a point, without being sure where that point is. So I was excited to come across a couple of studies that provide evidence teams that talk either too little or too much are not strong performers, and some hints about where to draw the line.

Researchers surveyed “60 cross-functional teams from 25 corporate and government organizations” representing a variety of industries and projects.[1] Team members rated how often they communicated by various means like e-mail or face-to-face. Meanwhile, department heads rated team performance not only on schedule and cost, but on “goal achievement.” This meant “the degree to which the project was expected to be able to overcome all technical hurdles, meet its technical objectives, meet its business goals and provide the expected commercial value to the company,” the study said. These are closer to the kinds of goals customers really care about, so I will focus on that metric.

In line with the Agile Manifesto, face-to-face (F2F) communication correlated most strongly with performance. But the highest-performing teams did not speak to each other “often.” (That was the label for a 5 on the related questions; a 1 was “never.”) Specifically the average was a 3.81. This is significantly higher than the midpoint of 3, so my earlier comments about erring on the side of more talking held true. The main point, though, is that the line graph of the relationship between communication and performance was a hump: teams averaging around a 1, 2, or 5 on F2F communication were weaker performers. E-mail communication had lower correlations, and goal achievement peaked lower for that, around the mid-point. Phone use was not related to goal achievement.

“The downturn in team performance when more frequent face-to-face communications took place may indicate confusion or conflict within the team about project goals,” the study says. “This confusion or conflict could provoke more face-to-face discussions and could hinder project performance.” On the other hand, “low-communication frequencies may not supply enough information to team members and may not facilitate the innovative combination of information and expertise required for high performance.” Remember that correlations don’t prove causation, so we don’t know whether the poor communication patterns cause or merely reflect poor performance. In practical terms I’m not sure that matters. Either way, you should try to find the right balance.

Another team used the same dataset for a series of computer simulations that modified the complexity of projects along two dimensions[2]:

  • Multiplicity, meaning “multiple approaches to complete the task are feasible,” (and) “multiple end states exist that the task must satisfy.”
  • Ambiguity, meaning “conflicts among approaches and uses that require making tradeoffs must be addressed,” and “decisions regarding the approaches to be employed and the end states to be satisfied must be handled.”

The communication needs appeared to change based on the combination of these factors. The curvilinear nature still held for goal achievement, but the ideal point for each form of communication shifted as multiplicity and ambiguity did.

Most fascinating to me was that two of the resulting curves were U-shaped, the opposite of a hump: for projects that had low ambiguity, middling levels of phone conversations were related to lower performance than either low or high levels of calls! The scientists speculated that where ambiguity is low, more successful teams don’t need much information, but if they do, moderate amounts merely introduce more confusion without enough clarification. Hence in that case the need for higher communication. Though computer models have often proven accurate, I need to point out that these apparently haven’t been tested with real teams yet.

Too bad the questionnaire didn’t ask for quantified instead of subjective measurements. You, dear reader, might consider two hours of face time each week a lot, while I would argue that is too low for most teams. Actual numbers would have provided us better guidance. Then again, we humans are bad at estimating figures accurately.

“Too many meetings” is one of the complaints lodged against Agile. But there is good evidence people don’t mind meetings when they are run efficiently and accomplish results. A posting in an online-class discussion gave a perfect example. A student described a manager in his company so determined to be loved that she would not push people to change their behaviors. Thus her team meetings repeated the same issues week after week. (I recommended he suggest the use of action items.)

Someone coming from that low-performing environment might gripe about the additional face-time in Agile, but in my experience they come around when they see things actually changing. Indeed, often when this complaint is voiced in a group, someone else will point out the number of ad hoc and informal meetings the pre-Agile organization has to have to address problems. The lower number of formal meetings in a poorly run org can create the impression that new Scrum ceremonies are introducing more F2F time when they really aren’t.

On a side note, being physically collocated did not relate to performance in the original study, but being in a different city or country did, to the negative. This was consistent with earlier research I’ve seen suggesting time-zone similarity is more critical to teamwork than sitting in the same room. It’s also more evidence against “global teams,” which I put in quotation marks because I don’t believe global groups can ever be true teams. The 2003 dataset for these studies obviously can’t tell us to what degree video-conference and chat tools can capture the F2F experience. But I’ve blogged in the past about the considerable evidence they can’t make up for the time gap. Remember, the ideal F2F amount for goal achievement was nearly a 4 on a 5-point scale. Even with today’s video apps, I’ve never seen any virtual team come close to that level of interaction, much less a global one.

Regardless of the medium, the easiest answer to striking the balance between too much and too little F2F time is to ensure your team discussions have a goal and then achieve that goal as efficiently as possible. When I was doing teamwork consulting, I offered training on meeting facilitation, active listening, and persuasion skills. Together these helped ensure every perspective was heard and valued, which is critical for innovation and the best decision-making, while eliminating time-wasting behaviors. I believe a system like Scrum aids in this by prescribing specific goals for each meeting plus limits on what is discussed:

  • Grooming—Focused exclusively on clarifying and prioritizing user stories.
  • Planning—Deciding what stories to do in the next sprint and how to do them.
  • Standup—Answering three questions (and only those questions: if you do a “16th minute” discussion, you are missing the point to the ceremony).
  • Demonstration or Sprint Review—Showing what work was accomplished and getting customer feedback.
  • Retrospective—Answer what went right, what went wrong, and what you’ll do differently.

In terms of saving time, the most important duty of the Scrum Master is to keep meetings on-task through good facilitation. If you’re an SM and haven’t been trained on facilitation, pick up a book or attend a seminar. The underlying principles have been well understood roughly 150 years, since the first version of Robert’s Rules of Order was published, so there’s no valid excuse for an inefficiently run Scrum ceremony.

As in many aspects of work and life, a balance between extremes proves the most effective route to effectiveness. Listen to the complaints of your teammates and the team’s stakeholders. Whether you hear lines like, “no one tells anybody anything,” or like, “we yak about this stuff too much,” pay attention. Implement a system for work planning and apply formal meeting facilitation to make the system as efficient as possible. Either the complaints should go away, or more likely, you should hear a balance of both of these complaints, at which point the majority of folks probably feel you’ve hit the right mark.

Subscribe

Please share this post at the bottom of the page.


[1] R.R. Patrashkova-Volzdoska et al., “Examining a Curvilinear Relationship between Communication Frequency and Team Performance in Cross-Functional Project Teams,” IEEE Transactions on Engineering Management 50, no. 3 (August 2003): 262–69, https://doi.org/10.1109/TEM.2003.817298.

[2] Deanna M. Kennedy, Sara A. McComb, and Ralitza R. Vozdolska, “An Investigation of Project Complexity’s Influence on Team Communication Using Monte Carlo Simulation,” Journal of Engineering and Technology Management 28, no. 3 (July 1, 2011): 109–27, https://doi.org/10.1016/j.jengtecman.2011.03.001.

Should Startups Take Time to Get Organized? Here’s the Evidence

Example of a flowchartWhen I was working as a group manager at a mid-sized startup near Seattle years ago, a fellow manager said it was the fifth startup he had worked in. Of the other four, only one survived. That one had stopped work for two months to get its processes in order.

Our current startup was a disorganized mess, with no departmental budgets or project management. Eventually it went through three rounds of layoffs. (But I had created a budget and used project management, and my group survived the first two intact.) The company was finally acquired for a fraction of the original investment.

Now that I am marketing Agile coaching for startups, I wanted to find scientific evidence that taking time to get organized helps a startup succeed, or at least move faster within the existing burn rate—a desire expressed by every entrepreneur I’ve known! For any of you involved in startups, especially ones debating whether it’s time to get organized, I figured I would share what I learned. Some of it will get repetitive, but you may have to persuade teammates on the answer, so extra evidence could help.

My research into high-performance teams made clear that a degree of process formalization improves team-level performance.[1] Example studies covered in The Truth about Teambuilding found that creating a structure and plans improved team performance, and formal policies reduced nonprofit board conflict. Without wading into the debate about how much formalization helps rather than hurts, some definitely helps. In The SuddenTeams™ Program I recommend the “80/80 Rule”: “do procedures covering 80% of the team’s labor time, and only make them detailed enough to cover 80% of the times the process is used (allowing people to flex when circumstances warrant it).” Per the Pareto Rule, this would only be around 20% of the potential procedures you could write.

In few moments’ searching I found nine blog posts telling startups why and/or how to put business processes in place, not one of which offered any evidence for the “why” (see “Sources” below). Unfortunately, I couldn’t find scientific proof either. Nor could I find any that getting organized doesn’t help. Simply put, scientists don’t seem to have asked the question. Several study articles stated that little research has been done into operations and teamwork in startups.[2] A 2007 article, “Success Factors of New Ventures,” analyzed 31 studies.[3] Organizational factors were not found at all, negative or positive; that is, none of those studies covered internal processes or organizational structure, other than the process of creating a startup.

Furthermore, a 2010 Handbook of Entrepreneur Research complained the “number of research studies that have compared entrepreneurs who have successfully created new firms with entrepreneurs who have failed at this process, is very small.”[4] You should bear this in mind when you read a magazine article about successful startups: unless they have been compared to equivalent failures, the author has no idea what made the difference.

Still, there are hints about process worth considering. Many studies indicate, as you might expect, that the founder’s previous experience as a manager, in prior startups, and in the startup’s industry are linked to the likelihood of a startup surviving and growing. (How to define startup “success” is debated, but two common themes are survival past three years and growth in the number of employees.) Getting help from outside professionals, not just kith and kin[5], was helpful as well. A meta-analysis of 40 studies on startups in lower-income countries found that technical assistance had an impact on success, though small relative to matching grants.[6] More specifically, it concluded that training programs “should contribute… to firm productivity (for example, through the adoption of more efficient management practices).”

In a small study based on interviews with two dozen design and technology entrepreneurs, process was not directly addressed. Two items seemed relevant, though: “Entrepreneur’s competence” came in at #5 out of 20, and “Self-development” was #10.[7] Drawing from a longitudinal study (the kind that better indicates causality) of 2,000 startups in the Netherlands, researchers concluded three factors involving organizational expertise were critical to surviving three years[8]:

  • “The importance of work experience. Especially the young, inexperienced potential starters ought to be advised to obtain some work experience before they take the step of setting up their own firms, preferably in a paid job.”
  • “The importance of a business partner… preferably an experienced entrepreneur who actively guides the new entrepreneur and may also help to solve the (growth) problems in the first years.”
  • “The importance of a thorough preparation,” defined not only as having a well-developed business plan, but also “market orientation” and taking and following the guidance of courses on entrepreneurship.

After reviewing studies available by 1987, two management professors said, “Effective action also requires a detailed knowledge both of the startup process and of the key success factors of the industry to be entered…”[9] A survey of 79 Serbian startups found among the top six success factors three touching on organization and process[10]:

  • “Ability to manage personnel”
  • “Good management skills”
  • “Maintenance of accurate records”

More examples come from a systematic literature review of 74 studies published after 2003 in highly regarded journals. Of 21 success factors the authors found, none specifically call out organizational development. Six seem indirectly related, around the education and experience of the leader or team, including “business capabilities,” and, “Experience in management of the entrepreneur.”[11] The researcher comments, “The lack of experience in management is often the main reason for the failure of new ventures. The entrepreneurs of startups rely on their previous experience and… don’t want nor try to expand their knowledge in order to achieve a bigger business range.”

A 1994 dissertation[12] found that “positive organizational behaviors affected the firm’s overall performance, employee productivity, and employee commitment,” based on survey responses from top leaders of 328 successful startups, with sales growth an indicator of performance. Almost all of the 11 skills most of these leaders called “very important” related to internal processes: “developing a vision for the future, improving quality, team building, strategic planning, leadership development… managing innovation (and) implementing business plans, employee selection, effective delegation, managing change, and problem solving.”

In case you’re wondering, it seems writing a detailed business plan does improve the odds of success.[13] Solid evidence comes from a long-term study that sampled the business plans of 585 German companies and compared their content to survival rates.[14] The scientists concluded, “Initial planning is an important requirement of success, but cannot lift it, until certain minimum constraints are met.” Those constraints include issues like sales and funding.

Time to hit the books. North Carolina State University has respected engineering and business schools. Since most startups involve tech these days, I went to their library to look through the HB615 section, entrepreneurship books. I pulled each “how-to” book, as opposed to theory or specific-issue texts. Nine of the 12 (75%) discussed structure and process decisions, many of them extensively. For example, 35 of the 50 Steps To Business Success[15] relate to strategic planning, leadership, and operations. Two are “Identify Business Processes to Be Improved,” and “Appoint a Leader of Business Improvement.” Section titles in Mastering Entrepreneurship[16] include “Building and maintaining the entrepreneurial team—a critical competence for venture growth,” and, “Distilling a strong team spirit.”

Entrepreneurship[17] doesn’t get to “Developing Internal Processes” until Chapter 8, but it points out there is a “Malcom Baldridge Award for small firms that show exceptional skills at operational performance.” The authors make a compelling argument for focusing on operations just in the way they define it: “Operations are the internal processes that create the value… (which) distinguishes the firm from competitors, and it is the kind of value customers are prepared to pay for.”

Some data from these books suggest when to focus on operations. A textbook reported on a 1993 study in which the “dominant problems at start-up were sales/marketing (38 percent), obtaining external financing (17 percent), and internal financial management problems (16 percent).” That last is a process issue, and in the “growth” stage, its importance rose to 21 percent. It was joined then by the process issues “human resource management problems (17 percent) and general management problems (14 percent).” Problems with “organizational structure/design” entered there at 6 percent.[18] So in the growth stage, 58 percent of startup problems were process problems.

I feel compelled to push out process and structure improvement from where I thought they were needed. The research consensus is that creating a business plan, sales growth, and funding at all stages are more critical. Organizational development still seems important earlier than most startups get to it, though. Based on the current evidence, my recommendations to entrepreneurs for maximizing their chances of survival and growth are to:

  1. Do some market research before you invest more time or money.
  2. Especially if you don’t have much business and startup experience, attend some free Small Business Development Center (SBDC) classes, if not more formal business training.
  3. Earlier than you want to, create a detailed business plan even if you don’t intend to seek funding, and run it past SBDC volunteers.
  4. Emphasize developing the product, sales/marketing, and funding initially (the classic startup experience).
  5. Once you have four people, create a team charter and implement some work planning and tracking, even if just a spreadsheet of action items with names and report-back dates.
  6. Once you are too large for a single team, between 12 and 15 people, that is the time to implement formal work processes like kaizen or Full Stack Scrum, but with a lot of input from workers on specifics.

Subscribe

Please share this post at the bottom of the page.


[1] See for example Robbins and Finley 1995

[2] Lopez Hernandez, Fernandez-Mesa, and Edwards-Schachter 2018

[3] Song et al. 2007

[4] Gartner, Carter, and Reynolds 2010

[5] Mas-Tur et al. 2015

[6] Piza 2016

[7] Kim, Kim, and Jeon 2018

[8] Schutjens and Wever 2000

[9] Hofer and Sandberg 1987

[10] Stefanovic, Prokic, and Rankovic 2010

[11] Santisteban and Mauricio 2017

[12] Carlock 1994

[13] Schutjens and Wever 2000; Tarres, Melendez, and Obra 2006

[14] Schulte 2007

[15] Cleveland 2009

[16] Birley and Muzyka 2000

[17] Carsrud and Brännback 2007

[18] Kuratko and Hodgetts 1998


Sources (books appear only if they were cited above):

Aldrich, Howard, and Martha Martinez. “Entrepreneurship as Social Construction:A Multilevel Evolutionary Approach.” In Handbook of Entrepreneurship Research: An Interdisciplinary Survey and Introduction, edited by Zoltán J. Ács and Audretsch, 2nd ed. International Handbook Series on Entrepreneurship 5. New York: Springer, 2010.

Anderson, Mary Ann, Edward Anderson, and Geoffrey Parker. “How to Manage a Start-up Operation.” Dummies (blog). Accessed May 6, 2019. https://www.dummies.com/business/operations-management/how-to-manage-a-start-up-operation/.

Audet, Josée, and Paul Couteret. “Coaching the Entrepreneur: Features and Success Factors.” Edited by Harry Matlay. Journal of Small Business and Enterprise Development 19, no. 3 (August 3, 2012): 515–31. https://doi.org/10.1108/14626001211250207.

Birley, Sue, and Daniel F. Muzyka. Mastering Entrepreneurship. Harlow: Financial Times Prentice Hall, 2000.

Bozward, David. “Developing a Business Process Diagram for Your Startup.” Dr David Bozward (blog), January 16, 2017. http://david.bozward.com/2017/01/developing-business-process-diagram-startup/.

Brandall, Benjamin. “How to Build a Minimum Viable Process Pack for Your Startup.” Business 2 Community. Accessed May 6, 2019. https://www.business2community.com/startups/build-minimum-viable-process-pack-startup-01779333.

Bussgang, Jeff. “Why ‘Ops’ Is Taking Over Startup Land.” SEEING BOTH SIDES (blog), January 29, 2017. https://seeingbothsides.com/2017/01/29/why-ops-is-taking-over-startup-land/.

Butlion, Justin. “The Business Operations Playbook: How to Implement Ops in Your Startup.” Project BI – The Business Intelligence (BI) Blog (blog), October 19, 2018. https://www.projectbi.net/business-operations-playbook-how-implement-ops-startup/.

Carlock, Randel S. The Need for Organization Development in Successful Entrepreneurial Firms. New York: Garland, 1994.

Carsrud, Alan L., and Malin E. Brännback. Entrepreneurship. Greenwood Guides to Business and Economics. Westport, Conn.: Greenwood Press, 2007.

Carter, Nancy. “The Social Psychology of Entrepreneurial Behavior.” In Handbook of Entrepreneurship Research: An Interdisciplinary Survey and Introduction, edited by Zoltán J. Ács and Audretsch, 2nd ed. International Handbook Series on Entrepreneurship 5. New York: Springer, 2010.

Cleveland, Peter M. 50 Steps to Business Success Entrepreneurial Leadership in Manageable Bites. Toronto: ECW Press, 2009. http://site.ebrary.com/id/10173250.

Fulham, Liz. “10 Key Reasons Why a Business Process Is Critical for a Startup.” SalesOptimize (blog), August 25, 2016. https://www.salesoptimize.com/business-processes/.

Gartner, William, Nancy Carter, and Paul Reynolds. “Entrepreneurial Behavior: Firm Organizing Processes.” In Handbook of Entrepreneurship Research: An Interdisciplinary Survey and Introduction, edited by Zoltán J. Ács and Audretsch, 2nd ed. International Handbook Series on Entrepreneurship 5. New York: Springer, 2010.

Hofer, Charles W., and William R. Sandberg. “Improving New Venture Performance: Some Guidelines for Success.” American Journal of Small Business 12, no. 1 (July 1987): 11–26. https://doi.org/10.1177/104225878701200101.

Joshi, Neeraj. “Starting up a Business? Focus on the Business Process.” Medium (blog), March 6, 2017. https://medium.com/startupsco/6-steps-to-designing-a-business-process-a3e3830484b9.

Kim, Boyoung, Hyojin Kim, and Youngok Jeon. “Critical Success Factors of a Design Startup Business.” Sustainability 10, no. 9 (August 21, 2018): 2981. https://doi.org/10.3390/su10092981.

Kuratko, Donald F, and Richard M Hodgetts. Entrepreneurship: A Contemporary Approach. Fort Worth: Dryden Press, 1998.

Liao, Jianwen (Jon), and Harold Welsch. “Patterns of Venture Gestation Process: Exploring the Differences between Tech and Non-Tech Nascent Entrepreneurs.” The Journal of High Technology Management Research 19, no. 2 (January 2008): 103–13. https://doi.org/10.1016/j.hitech.2008.10.003.

Lopez Hernandez, Anna K., Anabel Fernandez-Mesa, and Monica Edwards-Schachter. “Team Collaboration Capabilities as a Factor in Startup Success.” Journal of Technology Management & Innovation 13, no. 4 (December 2018): 13–23. https://doi.org/10.4067/S0718-27242018000400013.

Mas-Tur, Alicia, Pablo Pinazo, Ana María Tur-Porcar, and Manuel Sánchez-Masferrer. “What to Avoid to Succeed as an Entrepreneur.” Journal of Business Research 68, no. 11 (November 2015): 2279–84. https://doi.org/10.1016/j.jbusres.2015.06.011.

Piza, C. “The Impacts of Business Support Services for Small and Medium Enterprises on Firm Performance in Low-and Middle-Income Countries: A Systematic Review.” The Campbell Collaboration, January 4, 2016. https://campbellcollaboration.org/library/business-support-services-for-sme-low-and-middle-income-countries.

Quora. “How to Create a Business Process for Your Startup Business? Is There a Book – Quora.” Accessed May 6, 2019. https://www.quora.com/How-do-you-create-a-business-process-for-your-startup-business-Is-there-a-book.

Robbins, H., and M. Finley (1995), Why Teams Don’t Work. Peterson’s/Pacesetter Books: Princeton, NJ.

Santisteban, José, and David Mauricio. “Systematic Literature Review of Critical Success Factors of Information Technology Startups” 23, no. 2 (2017): 24.

Schulte, R. “Pre-Startup Planning Sophistication and Its Impact on New Venture Performance in Germany,” 21. 295. International Conference for Small Business, 2007.

Schutjens, Veronique, and Egbert Wever. “Determinants of New Firm Success.” Papers in Regional Science 79 (2000): 135.

Sommer, Svenja C., Christoph H. Loch, and Jing Dong. “Managing Complexity and Unforeseeable Uncertainty in Startup Companies: An Empirical Study.” Organization Science 20, no. 1 (February 2009): 118–33. https://doi.org/10.1287/orsc.1080.0369.

Song, Michael, Ksenia Podoynitsyna, Hans Van Der Bij, and Johannes I. M. Halman. “Success Factors in New Ventures: A Meta-Analysis.” Journal of Product Innovation Management 25, no. 1 (December 7, 2007): 7–27. https://doi.org/10.1111/j.1540-5885.2007.00280.x.

Stefanovic, Ivan, Sloboda Prokic, and Ljubodrag Rankovic. “Motivational and Success Factors of Entrepreneurs: The Evidence from a Developing Country.” Zb. Rad. Ekon. Fak. Rij. 28 (2010): 20.

Tarres, Christian Serarols, Antonio Padilla Melendez, and Ana Rosa Del Aguila Obra. “The Influence of Entrepreneur Characteristics on the Success of Pure Dot.Com Firms.” International Journal of Technology Management 33, no. 4 (2006): 373. https://doi.org/10.1504/IJTM.2006.009250.

Zwilling, Martin. “8 Basic Business Processes Your Startup Can’t Survive Without.” Business Insider. Accessed May 6, 2019. https://www.businessinsider.com/8-business-processes-your-startup-needs-to-have-2010-8.

Agile Truths from an “Agile” Project before Agile

File CabinetI trained four administrative teams to meet the principles of the Agile Manifesto seven years before it was written. Their story busts several myths about Agile that continue to hinder its adoption 25 years later.

As mentioned in a recent post, Los Alamos National Laboratory faced a major risk in 1994. Their equipment management system had failed government audits. Roughly a billion-dollars-worth of equipment could not be unaccounted for, dating back to LANL’s birth as the design center for the Manhattan Project. The U.S. Department of Energy (DOE), which is responsible for the federal lab, was threatening sanctions if changes weren’t made. For example, they were going to require DOE approval for all purchases above some ridiculously small amount, like $250, across the 42-square-mile facility.

The means of documenting the new system was rewriting the old equipment management manual, a 500-page beast of dense type. I was hired as a technical writer to do that. But it quickly became clear rewriting the manual really meant redesigning the system and training workers on its use, since the manual had to reflect how things were actually done to pass later audits. That was the driver behind my learning how to train self-directed work teams (SDWTs) as described in the earlier post.

These were administrative or clerical teams, having nothing to do with software or new product development. I may not recall the titles and roles exactly, but I think they were:

  • “property managers” (PMs) who handled daily equipment management needs at the line level—not real estate, despite the title!
  • “property administrators” (PAs) who processed forms from PMs and others and recorded transactions
  • “property specialists” (PSs) who oversaw and supported these operations across the Lab, and
  • “fleet managers” who did PM-like work for the Lab’s vehicles.

The Agile Manifesto would not be written until 2001, and I was unaware of existing methods like Crystal and Kanban now considered “Agile.” To make my argument that these were Agile teams nonetheless, I will analyze them in reference to the Manifesto’s “Principles.” I will also demonstrate how easy it is to apply the Manifesto to non-software work if you ignore the word “software,” or replace it with whatever you deliver.


Our highest priority is to satisfy the customer through early and continuous delivery…” Given the sanctions DOE was threatening, we needed to show satisfactory progress quickly. We delivered a first draft of a new manual, completely rewritten, redesigned, and pared down to around 300 less-dense pages, in just four months. (That still sounds big, but remember this was a highly regulated nuclear weapons site with classified equipment, and the manual included policies, standard operating procedures, and example forms.) After that we delivered completed individual chapters for final approval rather than waiting until the whole book was finished.

Welcome changing requirements even late in development…” By calling, meeting with, and submitting drafts to DOE reps often, we gave them ample opportunity to tell us we were off course or add new requirements for the system as it was evolving.

Deliver… frequently, from a couple of weeks to a couple of months…” Each team met twice a week initially, and then dropped to once a week after we were running smoothly. At each meeting, the “Old Issues” section of our agenda included all tasks that had come due since the previous meeting. At the meeting, the responsible person had to report on the results, or admit they hadn’t done it, ask for help if needed, and provide a new due date. (Sound like a “standup” to you, though weekly in this case?) Those results were reported out to managers and made available to the auditors in a “public” version of the meeting notes. An unredacted version was kept on a server only team members could access. Specific requests made by those stakeholders were thus usually delivered within a week or two of the request. Also, on average I think we delivered a manual chapter every couple of months.

Business people and (team members) must work together daily throughout the project.” Never a day passed that the PSs and PAs were not in contact with PMs and others like the Export Team. We regularly touched base with those people or their representative teams, described below, to get input on proposed policy changes. I think these interactions were the keys to the rapid adoption of the new system.

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.” Managers agreed not to attend the team meetings except by invitation. They could only insert requirements through the current facilitator, and learned not to assign tasks to individuals except in emergencies. They were also very responsive to team requests for information or political support.

The most efficient and effective method of conveying information to and within a… team is face-to-face conversation.” All of the PSs and PAs were collocated, with parts of each team sharing offices. The culture of the place was such that everyone walked over to talk to people when they needed information or help, unless one or the other was in the middle of something or didn’t need a quick answer. We also held regular training and feedback-gathering meetings with the PMs and fleet managers.

Working (deliverables are) the primary measure of progress.” There was zero tracking of what any individual was doing on a weekly, much less hourly basis. I doubt any managers even looked at the team spreadsheets of the tasks for which people had volunteered. They tracked progress primarily through the meeting notes and our outputs, both daily tasks and deliverables from the system improvement project.

Agile processes promote sustainable development…” I recall no one regularly working more than 40 hours a week except by choice. Letting people volunteer for project tasks and declare their own deadlines ensured the improvement project did not overtax anyone given their daily duties. There were occasional exceptions, notably when everyone pitched in to help the PMs inventory all of the Lab’s tracked equipment—even me! That took three weeks of extra hours, but was an exception that proved the rule and a great example of teams helping each other.

Continuous attention to technical excellence and good design enhances agility.” These were effectively the goals of the project: We needed the systems to be well-designed and technically easy to use, because otherwise the Lab’s scientists and engineers would not use them. Once we were showing progress there were no more hard deadlines. The teams were just told to make DOE happy, and thus were free to focus on doing the job right.

Simplicity—the art of maximizing the amount of work not done—is essential.” A question that became a team mantra was, “Why do we do it that way?” To the great credit of these people, they eagerly dropped anything for which the answer was something like, “We’ve always done it that way.” They understood the value of getting rid of unnecessary tasks.

To illustrate the value of this principle, I am going to brag a little about personally saving the taxpayers at least $100,000 in 1995 dollars. When the auditors said we had to put labels on a category of equipment we had not tracked before, I felt they were misreading their own regulations. One line could be interpreted as they had, but it did not explicitly say this had to be done. They eventually accepted that argument, and we were spared the expense of creating, applying, and recording the labels on thousands of items.

The best architectures, requirements, and designs emerge from self-organizing teams.” The PM and Fleet Manager teams were representative bodies elected by those larger groups. All of the teams created charters with their own team rules, meeting rules, and internal team procedures. There were no team leaders: Every member of each team served in the “facilitator” role on a rotating basis.

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.” The weekly team meetings were for this purpose also. One of the policies all the teams adopted was that once the team chose a practice, members agreed to follow that practice between meetings. But anyone could add concerns that arose as a “New Issue” in the next team meeting, as well as proactive changes. The team then decided whether to change the affected part of its charter accordingly.


The results? These teams raised the auditor ratings of the system from “Failing” to “Outstanding” in two years. They also found records accounting for all but $250,000-worth of the “missing” billion dollars of equipment, an astonishing 99.99975% success rate. The sanction threats ceased.

There are several Agile truths this story tells. One is that the principles underlying Agile not only predate the Manifesto, but were well-enough established by 1994 that I took canned training on implementing them. So don’t let any doubters dismiss the Manifesto as a fad or merely theoretical. The second is that Agile is applicable to virtually any type of team, regardless of team deliverables or member backgrounds. The third is that you don’t need an Agile Coach or full-time Scrum Masters to implement them, just the will to continuously improve and managers willing to drop their egos.

Subscribe

Please share this post at the bottom of the page.


Agilists Still Face Tesla’s Enemies

Nikola Tesla readingAs you read this, give thanks to Nikola Tesla. An extraordinary genius whose career spanned the turn of the previous century, his inventions are the basis for the systems powering your device and transmitting this post wirelessly. He had so critical a role in the technology undergirding our life today, the author of a 2018 biography titled it, Tesla: Inventor of the Modern.

Tesla was born in today’s Croatia in 1856. His first significant engineering work was for Thomas Edison’s company in Europe, helping make Paris the “City of Lights.” But after he came to America and got a job working directly for the inventor, they eventually had a falling out that became the “War of the Currents.” Edison was dedicated to widespread adoption of the form of electricity now found primarily in batteries, direct current (DC). Tesla was a proponent of the alternating current (AC) form now used in everything else. They fought bitter campaigns against each other for years before Tesla’s view won out, helped by his magic-like stage demonstrations, in which he passed high-voltage AC through his body to prove it was safe.

There are entire Web sites dedicated to the man and his genius, to which I refer you for details, or to that fascinating biography, the source of this post’s quotations. I bring him up to argue that he faced resistance to change for many of the same reasons Agilists do today.

For instance, many managers resist improvements on the grounds they will cost too much, regardless of the long-term payoff. Around 1882, Tesla outlined for some fellow Edison workers an AC motor that could work with different levels of current. “’My idea,’ he said, ‘was that the more wires I used the more perfect would be the action of the motor.’ Yet the pragmatic and cost-conscious Edison engineers scoffed at the scientist’s ideal, complaining copper wires were the most expensive part of an electrical distribution system; they wanted less, not more, wiring,” author Richard Munson says.

For all his success, Edison might have accomplished much more had he not practiced the willful blindness I explored earlier, “bragging about not appreciating scientific theories. ‘At the time I experimented on the incandescent lamp I did not understand Ohm’s law,’ admitted Edison. ‘Moreover, I do not want to understand Ohm’s law. It would prevent me from experimenting.’” This position hurt him, Munson says. “Understanding Ohm’s law, however, would have revealed a core problem with Edison’s direct-current approach…” Loosely, that law says the less pressure there is or the more resistance from the wires, the shorter a distance electricity can travel. DC cannot create high enough pressure to send electricity very far. By not acknowledging that, Edison wasted years experimenting on and promoting technology that was doomed. Edison’s approach required many generation points in any location, as opposed to our modern AC model envisioned by Tesla of a central generation point sending electricity hundreds of miles to many locations.

Tesla became a celebrity through showmanship, but worried about people growing indifferent to an innovation’s value when the thrill wears off. I saw this happen with self-directed work teams, and am seeing it again with Agile. While some managers practice willful blindness, others get excited by each new fad and never implement the underlying principles. Now DevOps is the shiny bauble, though it only exists because it fills a gap in the adoption of Agile.

After he and Edison fell out, Tesla found an ally and employer in George Westinghouse— that Westinghouse—but their goals ultimately did not align. The “War of the Currents” partly came down to whether electricity was only for rich people and big business, or would be democratized to help everyone, as Tesla dreamed. “Smart money favored small and dispersed generators for mansions over central power stations for the masses. Even with the advance of alternating current and the ability for larger generators to transmit electricity over longer distances, both General Electric (after merger with Edison’s company) and Westinghouse preferred the immediate profits from selling isolated generators over the uncertainties of marketing electricity from centralized generators.” Focusing on short-term gains with a few customers instead of investing in a bigger vision for an expanded market is something I’ve witnessed first-hand in many of my employers. As an Agile Coach, furthermore, I’ve often fought attempts by product managers to derail planned efforts that would help all customers in order to meet one customer’s immediate demands.

“Commenting on other barriers to innovation, (Tesla) complained that people, particularly investors, resisted radical ideas, even ideas that improved the status quo. He also grumbled that entrenched interests opposed change, saying, ‘Perhaps the greatest impediment (to invention) is encountered in the prejudicial opinions created in the minds of experts by organized opposition.’” This remains too true of many in the project management community, and ironically is already becoming true of Agile “leaders” trying to protect their brands. The Agile-Industrial Complex takes advantage to make more money pushing self-serving concepts.

Though Tesla had the grand vision, he did not apply himself to commercializing those ideas as Edison had. Another engineer by the name of Samuel Insull had to prove centralized generation profitable, for example. Insull took a job paying 1/3 of what he could have earned at GE Edison in favor of leading a small Chicago firm. There he took innovative steps to bring down his rates for AC power, greatly increasing demand, in turn allowing him to operate his plants “regularly and efficiently” and reduce his costs and further lower rates. In short, instead of fighting the new idea, he proved how to make money off it. Insull said, “’Every home, every factory, and every transportation line will obtain its energy from one common source, for the simple reason that that will be the cheapest way to produce and distribute it.’”

Tesla described radar, alternative energy sources, and vertical takeoff airplanes decades before their invention. In 1908 he predicted the Web, video calls, news sites, music streaming, podcasts, and smartwatches in a single quote: “An inexpensive instrument, not bigger than a watch, will enable its bearer to hear anywhere, on sea or land, music or song, the speech of a political leader… or the sermon of an eloquent clergyman, delivered in some other place, however distant. In the same manner, any picture, character, drawing or print can be transferred from one to another place.” He was so often right about technology then and now, it’s sometimes lost that he was very wrong on some topics. Tesla dismissed Einstein’s relativity theory and thought airplanes could not “soar” until Charles Lindberg proved him wrong. Most tragically he was wrong about his greatest obsession, the wireless transmission of messages and power through the earth, due to misunderstanding the planet’s internal structure. I take from this a reminder to constantly question my Agile beliefs.

Tesla and Edison had diametrically opposed approaches to invention. Tesla did most of his work in his head, claiming with some exaggeration that he could envision every detail of an invention to the degree that he could see it break down in operation. (In reality, he did some prototyping and experimentation as well.) In contrast, “Edison worked best when he cooperated with a team” and progressed primarily though trial-and-error. The approach is exemplified in his famous statement quoted in the book, “’Genius is 1 percent inspiration and 99 percent perspiration.’” Imagine what they would have achieved had they dropped enough ego to truly team up—if Einstein had learned more theory, and Tesla had been a more dedicated experimenter. Managers, take heed.

Like Tesla with AC, I am pushing a cheaper and more democratic form of Agile based on scientific knowledge against “prejudicial opinions created in the minds of experts” by the Agile-Industrial Complex. Like Insull, I am foregoing greater personal wealth and power in the fight, but I am trying to follow his example by proving this approach more profitable. Like Edison, I experiment constantly with changes in a careful way that proves them effective (or not), assuring what I teach works in the real world.

When Westinghouse’s company was under attack from the Robber Barons of the era, who were trying to monopolize electricity the way they had railroads and sugar, Westinghouse asked to renegotiate Tesla’s lucrative contract. It included royalties from every Tesla-designed motor Westinghouse sold ($2.50 per horsepower, worth untold amounts today at that same price). Instead Tesla made the grand gesture of tearing the contract up, eventually ruining his finances while that company and others made billions off his ideas. Here’s hoping I don’t end up as poor and isolated as Tesla.

If I do, I will take comfort in one more quote from him: “The scientific man does not aim at an immediate result. He does not expect that his advanced ideas will be readily taken up. His work is like that of a planter—for the future. His duty is to lay the foundation for those who are to come, and point the way.”

Subscribe

Please share this post at the bottom of the page.


Source: Munson, R. (2018), Tesla: Inventor of the Modern. W.W. Norton & Company: New York.

Half-a-Dozen Proven Ways to Innovate

Illustration of ideas
Vector illustration credit: www.vecteezy.com

For those of you trying to decide right now the best way to get creative about your products, processes, or problems, a new report is worth a look. Nesta, “The Innovation Foundation” in the United Kingdom, recently released A Compendium of Innovation Methods. It was written by 24 contributors including researchers, leaders of innovation organizations, and business leaders. In the report’s introduction, Kirsten Bound, Nesta’s Executive Director of Research Analysis and Policy, and Chief Executive Geoff Mulgan explain the compendium’s raison d’etre: “Much of 20th century innovation, from transistors and integrated circuits to polycarbonate and neoprene, was driven by big laboratories in governments and big firms…” Today, though, digital technology, new collaboration methods, and big data have enabled broader sources.

“For the last decade or more, Nesta has been searching for ways to support innovations that exploit these new trends, aiming to spread the capacity for innovation and apply it to the problems and communities most in need. Featured in this compendium are just some of the innovation methods we have analysed, developed, tested and spread…” The report says this approach allows them to promote methods based on experimentation, not just somebody’s claims, and to express caution when claims become overblown.

Each of the 13 sections describes a process for increasing innovation, written in a standard format that: explains how the process works; says what Nesta has done with it; has one or more case studies; and provides references for more information. Some of the processes require government involvement, like “anticipatory regulation” to stay ahead of technology changes, and grants for social innovation. Others are ways to provide funding to innovative companies. But six could easily be implemented by a single enterprise, either for its own people or to attract ideas from outside. These I’ll summarize for you below in hopes you’ll download the free report for details and examples.

The section titles in bold and the quotations are directly from the report.

Accelerator programs. “Accelerators provide intensive and time-limited business support for cohorts of startups, aiming to get them ready for investment more quickly than traditional incubators.” These have been sponsored by governments, venture capital firms, and large corporations. For a limited period of several months to a year, they provide mentoring or training to a group of selected startups and “encourage a high degree of peer-to-peer learning…” Setting up an accelerator might be a way for your company to outsource a big problem to startups investing only your internal experts’ time. Or you could create one for teams of junior employees to work on an issue with mentoring from internal experts, setting aside a set amount of time each week.

Challenge prizes. “Challenge prizes offer a reward to whoever can first or most effectively meet a defined challenge… The formula is simple: offer a financial reward for the first or best solution to a problem, attract the best innovators, and give them the support they need to compete.” These work best for problems that can be clearly defined, and likely solved with limited cost and time. Such prizes date back centuries, to competitions helping ships determine their longitude and the military to preserve food, and include recent DARPA challenges. Nesta offers a free how-to guide. Its Open Up Challenge resulted in a number of products and services around open banking. This might help you outsource a smaller solution without taking as much of your people’s time, or stimulate “skunk works” (informal self-organized teams) among employees.

Experimentation. “An experiment is a way of trying something new while putting in place the necessary structures to find out if it works.” Different types are listed:

  • “Randomised controlled trials (RCTs)… use a control group to compare people who have received an intervention with similar people who didn’t. To make sure the groups being compared aren’t biased, they are randomly assigned.”
  • A form of RCT, A/B tests, “randomly assign a particular design or communications strategy—the wording of a letter or the header of a website—to different groups of users. We can then compare response rates to see what was most effective.”
  • “Rapid experiments enable teams and organisations to work together to solve problems, collaborating to design, implement and test small changes in short loops that provide data on whether innovations are producing better results than ‘business-as-usual.’”
  • “Design exploration and experimentation takes an experimental approach to developing or testing innovations… At the earliest stages when new ideas are being explored, trying out new and different frames helps develop ideas and reframe them as hypotheses.”

Nesta ran an RTC to see if investment vouchers given to randomly selected small and medium firms in one town would lead to innovative marketing efforts that boosted sales. In the first six months, yes. But after 12 months, there was no significant difference between recipients and nonrecipients. This is an example of neutral results being as valuable as positive ones, because these prevented Nesta from wasting money on a program with no long-term value.

People Powered Results: the 100 day challenge. Though intended for citizens and cross-agency teams working to improve public services, this intrigued me as a way to solve cross-functional issues within a larger organization as well. This line leapt out: “Frontline practitioners and people who use public services have unrivalled expertise in how the system operates, but often have little influence or ownership over change.” This is exactly my experience with line employees and customers at my many clients or employers over 25 years.

In this process, leaders “break down longer-term strategies into challenges with measurable objectives. Frontline practitioners and citizens (then) set ambitious goals and develop and test creative solutions in real conditions.” This progresses in three phases: 1) setting up the support structure such as data tools and facilitators; 2) convening the teams to kick off 100 days of self-directed experimentation, with a check-in gathering at the 50-day mark; and 3), a session to share results and “shape their sustainability and scaling plans.”

Nesta led an effort for the U.K. health care system that “generated new ideas about how to improve care for older people, reduce unplanned hospital admissions, improve discharge from hospitals and even helped develop a new preventative care process for people at risk.”

Prototyping is familiar to those in product development, but Nesta says the concept is spreading to many fields. “A model version of a product or service elicits feedback and remodelling before extensive resources are committed to implementation.” It is not intended to replace pilot testing, the report says, but can help shape the pilot or even indicate a pilot won’t work before spending money on one. If you use an iterative development process like Scrum, you prototype automatically: Each sprint produces a workable product (or at least a customer-presentable design) for feedback.

Standards of Evidence “provide a structure… for thinking about whether you are making a positive difference. Typically, the lower levels show that there is only limited evidence (that you are); higher levels show that more evidence is available.” In short, these push you to look for measurable proof that your innovations are doing what you hoped they would, instead of relying on people’s perceptions. If you have heard of “maturity models,” think of this as one for rational decision-making about innovations. Here is Nesta’s model from the report:

The report explains that the “first step of that journey asks you to describe what you plan to do, in a way that is coherent, clear and convincing. This is often referred to as a ‘Theory of Change’ and… helps you to be explicit about your goals—and how you’ll achieve those goals,” before trying the change. As the organization matures, it learns to gather data in experiments or pilot studies, preferably comparing results to similar test groups that don’t participate in the change. Eventually you try repeating the results (Level 4) before standardizing the change through policies and procedures.

This approach seems obvious. But I have often seen companies and agencies do only Level 1 before skipping to Level 5. Some partially did levels 2 and 3 in the form of limited data gathering, and possibly a pilot test, but without formally eliminating other possible causes for the results or trying to replicate them.

If you want to try a tested practice to generates new ideas about an issue you face, download the report and also visit DIY: Development Impact & You. This site includes free modules and tools for innovation you can take into your next team meeting, like a “Fast Idea Generator.”

As video gamers might say, it’s time to level up.

Subscribe

Please share this post at the bottom of the page.


Source: Nesta (2019), A Compendium of Innovation Methods. Nesta: London.

Free download: https://media.nesta.org.uk/documents/Compendium-of-Innovation-Methods-March-2019.pdf

Don’t Call the Job “Agile Project Manager”

A Self-Contradicting Job Title

Project ScheduleThough a free-lancer these days, I keep a few job alerts running just in case something fitting my exacting requirements pops up. Plus, it is a way to monitor what is happening in the real world of Agile versus the theoretical one of blogs and books. One of the biggest indicators that the doers lag the thinkers is the continuing appearance of the job title “Agile project manager.” This is an oxymoron, a phrase that contradicts itself. In a truly Agile organization, all of the tasks of a project manager:

  • Are handled by the process or tools
  • Fall to one of the Agile roles like Product Owner
  • Are left to the team’s self-organization, or
  • Simply disappear.

Before you accuse me of being an Agile ideologue, be aware that I maintain my PMP and would recommend waterfall for certain types of projects. Project management is absolutely crucial; if you aren’t Agile, you need PMs. My argument is that Agile methods—or more specifically Scrum, almost always mentioned in these ads—provide all that is needed.

Start with the fact that much of the PM role, as practiced, centers on trying to make and meet scope, budget, and schedule estimates. I’ve already detailed why this is overemphasized even in waterfall for R&D and NPD[1] projects, and irrelevant in Agile (see “Melt Down the Agile Triangle”). When the job description says a candidate needs a track record of meeting those estimates, the job is not Agile.

Managing the PM Tasks

Let us repair to the latest Project Management Body of Knowledge (PMBOK)[2] for further evidence. I’ll compare the categories of tasks a project manager does, quoted from it, to my approach to scaled Scrum.

Project Scope Management. Includes the processes required to ensure the project includes all the work required, and only the work required, to complete the project successfully.” In Scrum the scope is the Product Backlog, a listing of the currently known requirements in rank order. The Product Owner (PO) creates this, either with, or as the representative of, the customer(s). Other requirements, like technical work that has to be done to meet the business requirements, the team itself creates as part of self-organization. The customer ensures “only the work required” is done, by approving the top of the backlog through the PO before every Planning Ceremony. There is no “scope creep,” because we welcome changes.

Project Schedule Management. Includes the processes required to manage the timely completion of the project.” A customer as involved as called for in the Agile Manifesto is effectively in charge of “timely completion.” The team works as efficiently as possible to deliver as much finished work as it can in the course of each sprint without overwork. Eventually this creates a velocity figure of how many stories or story points the team usually delivers. If the project team(s) deliver 11 stories per sprint, and the must-have stories number 54, nobody needs a PM to tell them those will likely be delivered in five sprints. If the customer adds scope, they can do the math themselves to determine the impact, thus automatically approving a schedule and cost change.

Project Cost Management. Includes the processes involved in planning, estimating, budgeting, financing, funding, managing, and controlling costs so the project can be completed within the approved budget.” This, too, becomes simple math. Say the team costs $12,000 per week and is using three-week sprints. Continuing the prior example, five sprints equals 15 weeks and costs $180,000. If that’s more than the customer wants to spend, they know they must remove 11 stories for each $36,000 they want to remove.

Since in Agile late changes are embraced, and plans and documentation are de-emphasized, we don’t waste time on long-term schedule and cost planning. Customer contracts are based on periodic costs instead (see “Agile Contracts”). Regarding “financing” and “funding,” in every firm I’ve served, the project sponsor handled these, not the PM.

Project Quality Management. Includes the processes for incorporating the organization’s quality policy regarding planning, managing, and controlling project and product quality requirements, in order to meet stakeholders’ expectations.” One of the Manifesto principles states, “The best architectures, requirements, and designs emerge from self-organizing teams,” while another calls for, “Continuous attention to technical excellence and good design…” The responsibility for quality thus is delegated to the team. The customer and PO will include specific quality requirements in related epics or stories just as a PM would in work packages. The team needs to be aware of corporate or certification quality standards, but a PM is rarely the technical subject-matter expert on those.

Project Resource Management. Includes the processes to identify, acquire, and manage the resources needed for the successful completion of the project.” Truly self-organizing teams by definition assign themselves to work. These should have stable membership, as recommended by most Agile sources I’ve read (and teamwork research). When a new project comes up, the project sponsor and PO create its initial epics with the customer, and in larger organizations they negotiate the project priority with their peers and stakeholders. When a team finishes a project, it goes to the list and takes epics from the top-ranked project for which it has the skills, perhaps negotiating with another team to fill in skill gaps. If more gaps exist, a self-organizing team would be allowed to set aside time for training and/or do its own hiring using the same organizational processes a PM would.

Project Communications Management. Includes the processes required to ensure timely and appropriate planning, collection, creation, distribution, storage, retrieval, management, control, monitoring, and ultimate disposition of project information.” Pay for a full-featured Agile work management package like Agile Central or Version One and, if needed, a document-sharing tool. The Scrum ceremonies take care of almost everything else in this category, including specific communications that can become tasks under a user story. As for “planning,” a “Communications Plan” can be created by the team (use link for instructions and template).

Project Risk Management. Includes the processes of conducting risk management planning, identification, analysis, response planning, response implementation, and monitoring risk on a project.” Evidence suggests high-uncertainty projects fitting Agile benefit little from this.[3] I train teams to identify risks only during epic and story grooming. They get into the habit of asking themselves two questions for each: What could prevent this story (or epic) from getting completed within one sprint (release)? And what could we blow up if we do this story successfully? Any answers are managed easily by the team using PM practices (see “Projects are a Risky Business“).

Because we don’t care about meeting scope, schedule, and cost estimates, risks related to those disappear.

Project Procurement Management. Includes the processes necessary to purchase or acquire products, services, or results needed from outside the project team.” In almost every project I dealt with, these processes were managed by a procurement person and/or standard corporate procedures, and senior team members had deep experience using them. I teach teams needing to go through the procurement or vendor system to add those actions, such as creating a Purchase Order, to sprints as stories or tasks.

Project Stakeholder Management. Includes the processes required to identify the people, groups, or organizations that could impact or be impacted by the project, to analyze stakeholder expectations and their impact on the project, and to develop appropriate management strategies for effectively engaging stakeholders in project decisions and execution.” My Agile transformation process identifies all potentially affected stakeholders and incorporates them into the change process. Existing Agile teams can also do so by creating a Communications Plan. Stakeholders are invited to Demonstration Ceremonies at least, and in my Full Stack Scrum™ (FuSS™), can observe anything except Retrospectives. They can be given access to the Agile tracker to get their own status updates, and in FuSS can become “Agile Liaisons,” treated as additional customers. The teams get into the habit of interacting with them proactively for a very simple reason: If a Liaison disapproves a story/epic relevant to their area, it cannot be accepted.

Project Integration Management. Includes the processes and activities to identify, define, combine, unify, and coordinate the various processes and project management activities within the Project Management Process Groups” (this is listed first in the PMBOK). Since all of those are covered by the same tools, roles, and processes, they are automatically integrated.

Creating the PM Outputs

Taking an alternate tack, we can look at the most-common project artifacts to compare who does them or the covered work. In practice, waterfall PMs are not always responsible for some of these, and many are omitted, but we’ll assume a high level of PM documentation:

Artifact Traditional PM Agile
Project Charter PM or sponsor PO or sponsor
Project Scope Product manager, business analyst, PM PO, team
Cost Estimate
Schedule Estimate
PM None (per-sprint costs are provided to customers)
Scope Management Plan
Cost Management Plan
Schedule Management Plan
PM None (scope, cost, and schedule can be updated each sprint)
Resource Plan PM Team self-assignment and hiring
Risk Register
Risk Management Plan
PM None (team manages on epic/story basis)
Procurement Management Plan PM (based on organizational procedures) None (team uses organizational procedures)
Quality Management Plan PM (with relevant stakeholders) Team (with relevant stakeholders)
Communications Plan
Stakeholder Management Plan
PM Team, process (grooming and ceremonies), tracking and documentation tools

Getting Honest about the Job

What’s left for a PM to do? Nothing, in a truly Agile organization. Diehards will propose details and edge cases left out of this post to keep it short, but I have taught self-directed teams to cover everything. Where executives are stuck in waterfall project governance, they will require all kinds of unnecessary data, so maybe you need a project administrator to translate the Agile information. But that person isn’t being Agile or a project manager.

So, hiring managers, get real: If you are “being Agile,” you don’t need a PM. Otherwise, stop fooling yourselves and drop “Agile” from the title, because you aren’t.

Subscribe

Please share this post at the bottom of the page.


[1] “Research and development” and “new product development.”

[2] Project Management Institute (2017), A Guide to the Project Management Body of Knowledge (PMBOK® Guide)—Sixth Edition. Project Management Institute: Newton Square, PA.

[3] Sommer, Svenja C., Christoph H. Loch, and Jing Dong. “Managing Complexity and Unforeseeable Uncertainty in Startup Companies: An Empirical Study.” Organization Science 20, no. 1 (February 2009): 118–33. https://doi.org/10.1287/orsc.1080.0369.

Find Good Management Evidence on Your Own

Free Sources of Management Studies

Woman scientist with test tubesThose who have read my earlier posts on evidence-based management have learned about “The Management Knowledge/Practice Gap”; discovered one reason for it is that “Good Evidence is Hard to Find”; and become skeptical readers about science because “Studies Say, Question Articles about Studies.” Now comes your chance to close the gap by finding useful studies on your own. You have to invest some time and effort, but that investment will save you time and effort down the road. Other than maybe some gas costs, though, you don’t have to invest money.

There are at least four ways to read scientific studies for free, but each has plusses and minuses. Three of them—Google Scholar, your local library system, and open-access research databases like SSRN—are free via the Web. However, these cover a small portion of all studies done on a topic. Google Scholar and your library system will mostly provide abstracts or articles about studies, rather than the studies themselves. Full-text journal articles in Scholar tend to be older ones, not the most current information. The research databases have full articles, but many are earlier drafts that may contain errors or have not gone through the peer-review process.

Fortunately, many college libraries provide guest access to large databases that allow you to download or e-mail complete journal articles. You must physically go to the library unless you are a student, staff, or faculty member, and access may be limited to specific computers and/or short lengths of time. But you can easily walk out with dozens of studies relevant to a decision you are trying to make.

At U.S. public universities, I’ve never had a problem getting the same kind of access the students have. Well, that’s not entirely true. One clerk hassled me at a library where I’d had no trouble before, so I complained at the administrative office. A VP of the library system met with me, apologized, escorted me to the guest computers, and gave me his business card to wave if I was ever bothered again. Taxpayers partially fund those universities, so in my opinion we have paid for limited use of their resources. Every other librarian I’ve interacted with was eager to get me access.

They may be required to ask why you want it, in which case the safe answer is, “I am researching [your topic].” Some universities technically limit access to “researchers,” and this answer gets them off the hook because the term is broadly defined.

I’m not going to discuss local library and open-database searches further, because once you have logged into them, the search is similar to that for university libraries.

How to Find Studies

Name Your Terms

I use Google Scholar first because it is a great way to develop a set of search terms that bring up relevant results. This saves time when you are at the college library. It works pretty much like regular Google. However, the “Advanced Search” option from the menu works more like the university search, and retrieves more specific results.

Think of a few two- to three-word phrases that describe the topic you are interested in, like you would for any Web search. But remember that scientists may use fancier words to describe a concept. It is even more important in database searches than in regular Web searches to try different combinations of synonyms for each word in your phrases. Consider creating “Boolean searches,” too. This is a technique for more specifically combining and filtering out terms.

The rest of these steps may look long and complicated, but that’s only because I include a lot of detail for first-time researchers. Once you’ve used them a couple of times, you won’t need them anymore. Note that these are specific to larger U.S. universities. Smaller public or private schools may provide access, but I’ve never tried them.

Go Back to School

  1. Use the Web to identify the main library at your nearest public university, as branches may not have guest computers.
  2. Go to the library and ask at the information desk for “guest access to the research databases.”
    Note: They may ask for your ID to record, and give you a code to use for that visit only. Don’t hesitate to ask for help with any of these steps.
  3. Once at the guest computer, plug in a memory stick if desired, and log in if required.
  4. Open any browser and find the library’s catalog page.
  5. Go to the “Online Research” page.
    Note: The site might use other terms such as  “eResearch,” “Magazines and Journals,” “Articles+,” or “Online Resources.”
  6. Click: “Advanced Search” (or a similar term).
    Note: This opens a page that allows you to combine search terms and filter results.

Enter Search Phrases

For each of your phrases:

  1. Enter the phrase, one word per search field, specifying whether to search only in the “Title” or the entire text.
    Tip: If you get few relevant results below, or too many irrelevant ones, go back to this page and adjust the search locations.
  2. If the phrases “Peer-reviewed” or “Academic journals” appear somewhere on the page, select that option.
    Note: Otherwise one of those should appear with the results list.
  3. Click the “Go” or similar button.

Gather Evidence

For at least the first 20 results:

  1. Click any title that seems relevant.
  2. Read the “Abstract” (you may have to click that word somewhere on the page).
  3. Look for phrases like “results,” “implications for practitioners,” or “advice.”
  4. Write down relevant information or download the abstract.

Optional: Download Full Article

If you want more details than the abstract provides:

  1. Click the term “Full Text” wherever it appears on the page, or in Google Scholar, the link to the right of the study title.
    Note: A link appears there only if the full article is available.
  2. Download the PDF to a memory stick or e-mail it to yourself.

Later you can review each article as explained under “Follow the Map” in the “Studies Say…” post.

An Example Search

For my recent post on seagull consultants, I first did a Google Scholar search using that and related phrases and found almost nothing except actual seagulls. Then I drove to Davis Library, the main branch at the Univ. of North Carolina at Chapel Hill. At the front desk I asked for “guest access,” and handed over my driver’s license. The librarian pulled out a binder, noted my name and maybe the license number, and after a few seconds on his computer handed me a shard of paper with a short code. No doubt he would have led me to the guest computers, but I knew where they were, a cluster of three on the main floor. I was presented with a log-in window on which I clicked, “Guest Access.” I entered my code number and was informed I had one hour, more than enough time.

I clicked a browser and it opened to the library’s catalog page. Under “Research Tools” I clicked “Articles+.” On that page I clicked “Advanced Search.” In the first drop-down where it said “All Fields,” I selected “Title.” I entered “seagull.” In the next row, I left “AND” selected, chose “Title” again, and entered “consultant.” I left “Publication Date” open, and under “Content” I made sure “Journal Article” was checked. Under “Limit to,” I clicked two boxes: “Items with full text online,” and, “Scholarly materials, including peer-reviewed.”

Taken together, I was searching for all scholarly journal articles I could download that had both of the words “seagull” and “consultant” in the titles. Nothing exactly like that came up. The closest were two studies on “seagull managers” in the nursing field, which I downloaded just in case. One indeed proved useful.

After several phrases presented few or no relevant studies, it occurred to me to look at general success factors for consulting, in case the amount of client-consultant interaction was covered. I did a “Title” search on “success AND factors AND consulting” to no avail, so tried again with “All Fields.” Bingo. A flood of potential results appeared. Here is the winning search screen, with apologies for the poor quality from a smartphone pic of a computer screen:

If the title in a result seemed right, I downloaded immediately, and with others I skimmed the abstract first. Because the system accesses different databases, the download process varied. With each new one I had to search the page for a link or tab labelled either “PDF” or “Full Text.” Some sites automatically downloaded to a folder on the computer, the location of which I found by clicking the browser’s download icon, while others asked at the bottom of the screen what to do with the file. I moved or saved the file to the memory stick I’d plugged into a USB port on back of the monitor. In the past I have also e-mailed files to myself. Then I sometimes had to close a separate window before clicking my way back to the search results. Occasionally I didn’t notice the database opened in the same window, mistakenly closed it, and had to start again! I ended up with 20-plus studies, 16 of which I read entirely, quoting and citing 6 in addition to other sources.

Go for It

Now it’s your turn. Pick a topic and change your interest in evidence-based management into action. Once you’ve seen the value, I predict you’ll be hooked, and your employees and customers will be better off for it.

Subscribe


 

Please Don’t Feed the Seagulls

People and Seagulls SilhouetteNearly every week, I am contacted by a recruiter wanting me to become a consultant who guides clients in Agile transformation on a traveling basis. It feels like every major IT consulting/staffing agency is getting into the game.

Too bad the only evidence this approach works comes from drop-in consultants and their agencies. A look at the objective evidence paints a darker picture.

The “seagull consultant” has been hated so long as to be a business cliché. The worst-case examples are “well-known international firms that fly in specialists from overseas who briefly hover around the client organisation before dropping a large report and flying off home.”[1] I include anyone who drops in once a week or less. More than one bolder wag has described it like “dropping their s–t and flying away, then flying back to drop more.”

I will expose my personal bias by telling you my every experience with seagulls relative to organizational change has been negative. In fact, at least twice I was hired in part to clean up messes made by drop-in-then-remote coaches. Even if the seagull’s advice was decent, people didn’t know how to apply it in their absence as daily complexities arose. While change leaders and workers became more frustrated, resistance to change snowballed because the consultant wasn’t there to address it. These are among the reasons every sports team on the planet has a full-time coach.

The seagull trend should matter to managers because the success rate for organizational change is miserably low.[2] In a study of consulting efforts specifically, “respondents in both the interviews/focus group sessions and the web-based survey agreed that the achievement of best practice tended to range from low to partial… In addition, 30 percent of the respondents disagreed or strongly disagreed that best practice was achieved.”[3]

However, I have changed my opinions when I found better evidence, so I hit a research database to see what scientists have learned about seagull coaches for business process changes like an Agile transformation. The search was a bit of a challenge because it returned many studies about actual seagulls! I found none focused on seagull consultants, though they were often mentioned in the ones I found, and two discussed “seagull managers” in nursing, negatively.

Gripes about this lack of hard evidence in 2005 were still being repeated in 2016. The earlier book said, “At present, there are a multitude of relatively untested proprietary coaching models in the marketplace… The lack of critical evaluation and testing means that claims are made about what can be achieved via coaching using particular models, which, while often in good faith, are ultimately supported by little more than anecdotal evidence, personal conviction or blind optimism.”[4]

And in 2016, a study summarized the continuing situation: “there is a wealth of practical field work (experience-based reports, case studies, etc.) which, however, lacks the desired scientific validity. Consequently, there are only (a) few sound empirical studies and thus data about management consulting.”[5]

Fortunately, among those are some addressing the factors related to consulting client satisfaction. I plumbed those for clues that might suggest drop-in consulting helps, or at least doesn’t hurt, results. For instance, I found a survey by professors of 189 top executives in Australia about consulting success. However, of the top three “strategic capabilities” according to clients, two would be impeded by seagulling: #1, “Ability to listen/comprehend the client,” which 77% rated as “Very Important,” and #3, “Client-consultant communication” (65%).[6]

A survey of the Federal Association of German Management Consultants came up with similar factors: “intensity of collaboration has the strongest impact (0.30) on perceived management consulting success, directly followed by common vision (0.27).”[7] The first directly contradicts seagulling, and the second would be easier to achieve with higher interaction. A companion study of firms that had used consultants named “consultant experience” as the factor most related to success. But the two factors from the earlier survey were more highly rated by these clients than by those consultants, with correlations around 0.35 to perceived success.[8]

People who had used project management consultants (PMCs) in Malaysia said through a survey that the “PMC’s interaction skills” comprised the top factor in engagement success, followed by, “Efficient management of information.” The biggest contributor to that efficiency was “prompt communication of information with regards to instructions and decisions made to a project (team)… Frequent communication between team members and a PMC also helps ensure accurate information flow and facilitates a PMC in monitoring the work of team members.”[9] That seems applicable to Agile transformation projects. (A caveat is that this survey had a low response rate of 11%.)

Lower and middle managers of Bell Canada were surveyed by a business professor and a company employee about their experiences with consultants. Among the authors’ recommendations based on the responses was, “recognition that implementation planning and execution must be part of every consulting mandate.”[10] Seagull consultants do not execute the change. Unfortunately the study report did not provide a complete table of correlations between the different variables, so I can’t say whether there was a correlation between consultant availability and success.

A couple of scientific sources reinforced the value of using the right coaching method for the situation. In one, a consultant talked about being asked to review management pay for a subsidiary company. “The consultant needed to do little more than briefly investigate existing practices and provide a report recommending that the client make use of programs from other parts of the enterprise. The consultant observed, ‘They wanted a snapshot and to be provided with some advice. So that’s what we did.’”[11] Seagulling probably worked there, but that did not involve major organizational change.

Indeed, I am not totally “anti-seagull”; I have been one, for example when providing monthly management coaching to an individual. However, one person can only tackle one or two behavior changes at a time. When you are changing a group, you have to multiply that number by every person in the group.

Hence the problem is not as simple as “seagulls are bad.” The question I raise is whether it is useful in Agile coaching. Unfortunately clients underestimate the work required to make an organizational change. A researcher said, in the 2005 book mentioned earlier, “the most effective coaching for developing leadership skills and enhancing performance… takes place in interventions involving not only the individual leader-managers but also team members and other stake-holders in organisational contexts. Leadership is a social phenomenon; it is about how people influence each other and the outcomes of that. Sometimes large numbers of organisational members need to be involved in coaching if the leadership culture is to be changed and blocks to developing sustainable leadership removed.”[12] Please read that again, but this time replace the word “leadership” with “Agile.” This becomes a description of a change to Agile management.

Our themes from this evidence are that consulting success usually requires intense communication, deep understanding of the client’s world, involvement in implementing the recommendations, and interaction with all stakeholders. Seagull-style Agile consultants provide none of that. Therefore I argue that an Agile change is far too complex to implement on a drop-in/remote basis. The only possible utility of a seagull’s advice would be providing an expert outsider’s perspective to a full-time internal expert leading the change daily. An analogue would be a famous, retired basketball coach advising current coaches.

Do yourself a favor: Don’t feed the seagulls. If you are contemplating or already struggling with an Agile transformation, hire an expert to lead your change full-time until your organization is “being Agile.” The claims of value made by those firms who want to take your money do not stand up to the evidence from all these researchers who have nothing to gain from you.

Subscribe

Please share this post at the bottom of the page.


[1] Jim Kitay and Christopher Wright, “Take the Money and Run? Organisational Boundaries and Consultants’ Roles,” The Service Industries Journal 24, no. 3 (May 2004): 1–18, https://doi.org/10.1080/0264206042000247731.

[2] Detailed with citations in my paper, “An Evidence-Based Model for Agile Organizational Change.”

[3] Alan Simon, Peter Schoeman, and Amrik S. Sohal, “Prioritised Best Practices in a Ratified Consulting Services Maturity Model for ERP Consulting,” Journal of Enterprise Information Management 23, no. 1 (January 5, 2010): 100–124, https://doi.org/10.1108/17410391011008923.

[4] Michael Cavanagh, ed., “Introduction,” in Evidence-Based Coaching, Vol. 1: Theory, Research and Practice from the Behavioural Sciences, Evidence-Based Coaching (Bowen Hills, Qld: Austalian Academic Press, 2005).

[5] Matias Bronnenmayer, Bernd W. Wirtz, and Vincent Göttel, “Determinants of Perceived Success in Management Consulting: An Empirical Investigation from the Consultant Perspective,” Management Research Review 39, no. 6 (June 20, 2016): 706–38, https://doi.org/10.1108/MRR-06-2014-0145.

[6] Alan Simon and Vanya Kumar, “Clients’ Views on Strategic Capabilities Which Lead to Management Consulting Success,” Management Decision 39, no. 5 (June 2001): 362–72, https://doi.org/10.1108/EUM0000000005472.

[7] Bronnenmayer, Wirtz, and Göttel, “Determinants of Perceived Success in Management Consulting.”

[8] Matias Bronnenmayer, Bernd W. Wirtz, and Vincent Göttel, “Success Factors of Management Consulting,” Review of Managerial Science 10, no. 1 (January 2016): 1–34, https://doi.org/10.1007/s11846-014-0137-5.

[9] Pollaphat Nitithamyong and Zijin Tan, “Determinants for Effective Performance of External Project Management Consultants in Malaysia,” Engineering, Construction and Architectural Management 14, no. 5 (September 11, 2007): 463–78, https://doi.org/10.1108/09699980710780764.

[10] Steven H. Appelbaum and Anthony J. Steed, “The Critical Success Factors in the Client‐consulting Relationship,” Journal of Management Development 24, no. 1 (January 2005): 68–93, https://doi.org/10.1108/02621710510572362.

[11] Kitay and Wright, “Take the Money and Run?”

[12] Ray Elliott, “The Parameters of Specialist Professional Leadership Coaching,” in Evidence-Based Coaching, Vol. 1: Theory, Research and Practice from the Behavioural Sciences, ed. Michael Cavanagh, Evidence-Based Coaching (Bowen Hills, Qld: Austalian Academic Press, 2005).

Studies Say, Question Articles about Studies

1931 Science and Mechanics Magazine CoverStart with the Source

In previous posts I talked about attempts to close the management knowledge-practice gap through evidence-based management (EBM), and the problems with the information sources most managers use. Science is only one source in EBM, but it is the least understood, so in this post I will give you the tools to decide how much faith to put in a single study you read or hear about. The key is to notice what sources your sources are using.

Articles published in “peer-reviewed” scientific journals have been critiqued by other scientists regarding the methods or results, and edited by skeptical experts in the field. They also provide details that let you spot potential problems and complete lists of their sources, which allow you to check the authors’ conclusions. In contrast, fact-checkers at magazines, and even at publications that sound scientific like Harvard Business Review, don’t have that depth of knowledge, and their editors can’t be experts in every subject covered. Few newspapers today have science reporters. Blog posts and business conference presentations usually aren’t reviewed or edited at all, at least by a knowledgeable third party. Popular periodicals and blogs often use sensational headlines more interested in capturing your attention than being strictly accurate, a problem if that’s all you see.

Despite what general writers reporting on studies seem to believe, most studies are only pieces of a large puzzle. Those studies always point this out, but the warnings rarely make it into articles about them. (In the rest of this post, I will use the word “study” to mean both the research and the journal article reporting it, and “article” to refer to press and blog reports or nonacademic presentations.)

All Studies are Not Born Equal

As you read about a study, look for clues indicating what kind it was and whom was studied. There are two types most likely to apply to your workplace. A “meta-analysis” takes the data from all relevant studies the researcher found and runs the numbers as if they were one big study. This filters out many of the limitations present in any one study, discussed further below. Also, a “structured literature review” (versus “narrative” reviews) goes through the texts of previous studies using techniques meant to reduce bias from the researcher. For example, it might use an algorithm, or at least several humans who must agree on ratings, to analyze the words used.

With other studies, give more weight to those with more of the following characteristics:

  • Larger numbers of people and/or organizations in the “sample.”
  • Samples selected by the scientists, as opposed to letting anyone choose to respond like a typical online survey.
  • “Longitudinal” or “panel” studies, which are more likely to prove what caused an outcome.
  • Studies that compare the tested group to a “control” group—a similar group that did not do whatever is being tested. Otherwise the result might have happened even if nothing different was done.
  • Combinations of laboratory and “field” studies, giving a better indication that the lab results, which can better eliminate other explanations for those results, also apply to the “real world.”
  • Authors who are or who worked with professors, and thus are less likely to interpret results to support something they are selling.
  • Studies done in organizations like yours (same industry, country, type of organization, type of customers, etc.) or crossing those lines, and thus more likely to apply to your situation.
  • Studies “replicating” or “extending” prior knowledge, rather than being the first to look into the specific topic.

Next pay attention to the claims made by the author about the study’s findings. News outlets, consulting firms, and professional groups often over-generalize. I frequently see the phrase “workers believe” even though the sample was skewed toward white males working for large tech corporations who found the survey and cared enough about the topic to fill it out. If you manage a diverse blue-collar crew in a small family owned company, those survey results could be the exact opposite of what your “workers believe.” The article should also note questions raised by the researchers about their own findings, meaning things they don’t know yet, so that the results don’t seem definitive.

Notice the size of any correlations reported and whether they were “statistically significant,” meaning the math indicates they weren’t from random luck. Correlations, showing whether two factors moved together, range from –1.0 to +1.0. (A –1.0 means Factor A went up the same amount Factor B went down.) A correlation of 0.15 might be statistically significant but is pretty weak. And again, correlation doesn’t prove causation. Factor B might have gone down because A went up; or A up because B went down; or other factors might have caused both changes. In articles, I often see phrases like “B went down when A went up,” suggesting A caused B, even when the study can’t have said that given its design.

Be wary, too, of scientific-sounding titles. Almost every headline I have read on the general Web starting “The Science of…” was effectively a lie. A company reporting on its own data is not science.

Even big-name companies can produce crappy results. Google claimed its internal research proved teams perform better if they have team leaders, but their reported methods could not have proved that. The company did not, apparently, set up leaderless teams and compare them to similar leader-led teams, or use objective performance metrics. All their “study” proved was that teams who like their managers and those managers think their teams are more productive—as best I can tell, given that Google did not detail its methodology.[1]

Even the best reporters might leave out important details or misunderstand something. I once prompted a change in a Today Show online article by pointing out the word the reporter was using (“nice”) was not the one used in the study (“agreeable”) and that meant something different to researchers.[2]

When an article names a study, paste the study title into a search engine. Sometimes a link to the full-text version will appear. Abstracts usually do, which might be enough to make you question the article.

Follow the Map

Though studies can be a dull read, you may be surprised at how easy most are to follow. It helps that they tend to use a standard format, often with the exact section names I present here:

  • Abstract—A short summary of the entire study. Frequently you can find what you need here. The problem is, depending on how terms were defined and measured, the results may not mean what they seem to. If you can download the whole article, it helps to skim the sections below to make sure your understanding matches the researchers’.
  • Literature Review and Hypotheses—In developing the hypotheses they want to test, scientists read as many relevant studies as they can find. They outline those in this section to justify their hypotheses, and the study itself. Reading these narrative “lit reviews” in a couple of relatively recent studies on a topic will provide you with the current scientific consensus, if there is one. In effect, the authors have done your research for you!
  • Methodology—This section details how they conducted the study. Most of this you can skip, but note: the type of study, for the reasons covered above; who the data was gathered from (the “sample,” “subjects,” “participants,” etc.); and how the key terms were defined and measured. For example, was “product success” measured using objective standards like financial benefits to the firm or customer satisfaction surveys? Or were the survey respondents free to rate their own success in their own ways?
  • Analysis and Results—Here appear detailed explanations of the math done and the specific answers found. Most readers can skip to the next section and backtrack here if that one raises questions for you.
  • Discussion—This is a narrative explanation of the findings, and will be the most useful to managers. The scientists tell you what they learned.
  • Advice for Managers—Sometimes the researchers make recommendations about how managers might apply the results in the workplace. This part could be a subsection under “Discussion,” or just a few sentences instead of a separate section.
  • Limitations—Important to skim through before taking the scientists’ advice, this section lays out reasons the study results might not apply to your workplace.

Once you’ve read through a couple of studies, it gets much easier to find the good stuff. Hopefully then you’ll feel brave enough to proactively search out studies related to questions you have, as explained in the next and last post of this EBM series.

Subscribe

Please share this post at the bottom of the page.


[1] Google, “Guide: Identify What Makes a Great Manager,” 2018, https://rework.withgoogle.com/guides/managers-identify-what-makes-a-great-manager/steps/learn-about-googles-manager-research/.

[2] See: “The Birth of a Myth: Niceness Does Not Pay?