Business/Industry Offers Dynamic Environment for Statisticians

Of course, the pharmaceutical industry has hired large numbers of statisticians in the private sector since the 1960s. This broad arena warrants its own article, however.

As more financial transactions became digitized, portfolios became larger, and derivatives (complex financial instruments such as collateralized debt obligations) became commonplace, the need to develop complex and numerically challenging risk models grew.

While not the focus of this article, some speculate one reason for the financial collapse in late 2008 is that the complexity of the financial instruments exceeded analysts’ ability to understand and model them. In any case, the market for quantitative scientists (“quants”) who could model the risk of complex portfolios grew and numerous disciplines with mathematical foundations stepped in to fill the void—physics, quantitative finance, applied mathematics, and, yes, statistics.

Significant new leadership opportunities exist for those who can extend beyond the limitations of traditional roles.

In his book Competing on Analytics: The New Science of Winning, Thomas Davenport provided a convincing argument that the ability to effectively analyze and use information would be a deciding factor, separating winners and losers in business. While the financial markets are down at the time of this writing, financial services are critical to the global economy, and it is clearly just a matter of time before they come back.

Six Sigma was launched in the 1980s when Motorola was having its lunch eaten by foreign competition in the electronics market. It needed to dramatically improve the quality of its products and significantly reduce the costs just to stay in business. There were too many data-oriented process improvement problems to attack using solely professional statisticians, so Motorola decided to develop a systematic methodology for process improvement using primarily statistical methods and teach it to large numbers of employees.

However, rather than simply teaching people various statistical methods and hoping they figured out how to effectively use them, Motorola developed a generic four-phase methodology (measure, analyze, improve, control—or MAIC) that linked and sequenced the tools, creating an overall approach to problemsolving that could be applied to just about anything. It worked.

GE and other companies subsequently adopted the methodology and enhanced it. GE added a “define” stage at the beginning to ensure problems were properly identified and scoped, and others began to add productivity approaches from lean manufacturing. This resulted in the current Lean Six Sigma, which primarily uses the DMAIC approach (D for define).

Billions of dollars of economic benefit have been documented, and while perhaps no longer in its heyday, Lean Six Sigma is still alive and well. An interesting question to ask is why Six Sigma has succeeded for so long, especially given that it has invented no new statistical techniques. I believe the answer provides a glimpse into the dynamic environment for business and industrial statistics now and in the future.

Our Dynamic Environment Brings New Challenges and Opportunities

I think Six Sigma has been successful due to the following:

• It attempts to get “everyone in the game,” rather than relying solely on professional statisticians.
• It is led from the top and supported by organizational infrastructure (budgets, managerial review, defined responsibilities, etc.). In other words, it is embedded into the business system, rather than being a stand-alone, purely technical initiative.
• Through the DMAIC process, it integrates and sequences various statistical tools into an overall approach to improvement, rather than simply providing a laundry list of tools. In practice, few complex problems can be solved with one statistical tool used in isolation.
• It focuses on process improvement—root causes—and reducing variation. These are key elements in statistical thinking, which provides an overall philosophical context for use of statistical methods.

Interestingly, I think these attributes tie in well with some of the changes that have taken place in the environment of most statisticians working in the private sector. The major changes include the following:

• Explosion of the Internet and computer science in general, leading to radical changes to the global economic environment
• Proliferation of low-cost, or even free (R), easy-to-use statistical software
• Extension of longstanding global competition (We have recently seen the emergence of a low-cost market for providing statistical services. This market is often located in developing countries with lower cost structures, but which also have good educational systems for mathematical sciences and the English language.)
• Expansion of Six Sigma “black belts,” finance quants, data miners, and others who can, or at least claim they can, perform high-level statistical analyses
• Growing consensus that there needs to be a greater emphasis on statistical thinking—the philosophy of why and where statistics should be used to make an impact—to augment our historical emphasis on the methods, themselves

The net result of these changes has been the “democratization” of statistics, whereby anyone with a computer connected to the Internet has access to data and software that allows him or her to do statistical work. It therefore has become virtually impossible for us, as professional statisticians, to “own statistics.”

While some statisticians may look upon these changes as the glass being half-empty, I tend to look at them as the glass being half-full. If others are able to perform basic statistical tasks for themselves, professional statisticians can be freed to some extent to do higher-level statistical work, allowing us to provide true leadership to our organizations. Of course, we will need to make significant changes in the way we do our work, as well as what we view as our work, to take advantage of this situation; inaction on our part will likely result in less influence.

For example, rather than being confined to consultative roles—passively providing advice on other people’s problems—or doing narrow technical tasks that may eventually be outsourced to others, we have the opportunity to help shape our organization’s use of statistical thinking and methods. We also can assume leadership in identifying innovative ways to use statistics and then lead the resulting projects. We have the opportunity to focus on the big, unsolved problems facing our organizations as equal collaborators on teams, rather than being bogged down on routine analyses that have been farmed out to us. Six Sigma took advantage of the democratization of statistics to achieve unparalleled results, and I think our profession can, too. In my opinion, such opportunities make our profession more exciting, not less.

Opportunity Is Knocking

We inherit a rich legacy of statistics affecting business and industry, one that goes back at least to the 1920s. While many of the specific problems have changed, important challenges remain. In particular, the environment for statistics in the private sector today is dynamic, requiring real change on our part. Significant new leadership opportunities exist for those who can extend beyond the limitations of traditional roles. If you are looking for a challenge and would like to have the opportunity to develop and apply your leadership skills, consider business and industry as a career option. Opportunity is knocking.