Steve Lane, Adeniyi Adewale, and Allison Florance

Gary Sullivan

John Scott

Eric Gibson

Janet Wittes

The 2018 ASA Biopharmaceutical Section Regulatory-Industry Statistics Workshop held in September featured a session titled “Statistical Leadership: Its Importance, Imperatives, and Impacts.”

The session—organized by Adeniyi Adewale and Allison Florance of Novartis and chaired by Steve Lane of BMS—consisted of two presentations representing industry and regulatory perspectives on statistical leadership by Gary Sullivan from Espirer Consulting and John Scott of the FDA, respectively.

After the presentations, panelists Eric Gibson of Novartis, Gary Rosner of The Johns Hopkins University, and Janet Wittes of Statistics Collaborative corroborated the presentations with their own experiences. Here are summaries of the presentations and an excerpt of the discussion that followed.

Gary Sullivan

Sullivan defined leadership as the ability to deliver value consistently to an organization or cause by inspiring people to take a specific direction when they truly have the choice to do otherwise.

He used examples from his experience in nonclinical statistics to underscore the following three key messages:

1. The skills and insights that come easily to statisticians likely do not come as easily to colleagues in other functions, so don’t hesitate to use those skills to add value. In his early days as a nonclinical statistician and while engaging with a team on an issue dealing with variability of tablet potency, Sullivan collated data and did an analysis that provided important insight. He initially hesitated to share the analysis because it seemed “simple and obvious,” but he soon realized his simple analysis held the key to resolving the issue.
2. Statisticians have a perspective unlike anyone else on their teams and can use it to add value in a unique way. While Sullivan was providing statistical support for a bio-product development group, he realized he was likely the only person in the department who worked closely with every group or function—analytical, molecular biology, microbiology, fermentation, engineering, purification, and pilot plant operations. This gave him a broad and deep understanding of the business, credibility with the entire organization, and a perspective no one else had, which allowed him to provide influence across the organization tactically and strategically.
3. As statisticians advance to more senior levels and gain influence, they need to have more of an ownership mindset and contribute at higher levels. Sullivan shared a situation in which he learned from a business partner that manufacturing deviations were creating a drain on time and resources. Many of these deviations were caused by out-of-control signals from control charts of product characteristics. Sullivan realized the control-charting, although well-intended, was creating cost and not providing insights or value to the business. He went through the process of shutting down most of the control-charting, thereby saving valuable time and resources that could be re-focused on troubleshooting and improvements.

Sullivan concluded his presentation with the following three points:

1. Statisticians have a strong foundation for leadership (problem solving, critical thinking, information processing, integrity), but must improve on decision-making, business judgment, and acting on ideas to be more effective.
2. Statisticians spend a significant amount of time improving their technical skills and relatively little time honing their soft skills, including leadership. Investing in the latter would serve them well.
3. More senior statisticians need to actively play a role in advancing the soft skills of less experienced statisticians.

John Scott

Scott opened his presentation by noting that any statistician can provide statistical leadership, the goal of which is to drive better decision-making. He underscored what leadership is about by borrowing the catchphrase, “Management is getting people to do what you want. Leadership is getting people to want what you want.”

To drive better decision-making, statisticians must invest time into thinking about the problems at hand. Statisticians must refrain from reflexively doing what collaborators think they want and instead think about what the real scientific questions are and how they can be addressed with the resources and tools at hand.

He advised statisticians to give collaborators genuine help in generating, summarizing, and analyzing information and relayed an example of an engagement in which a laboratory scientist was interested in comparing antibody levels between three conditions and the differences were found not to be statistically significant. The scientist decided to collect more data and returned with the question, “Is it significant now?” Rather than address the question of statistical significance, Scott was able to structure the problem in a Bayesian framework, synthesizing the previous and new data. He noted that the write-up was fully transparent about the experiment and the sequential nature of data collection. The results were presented using descriptive statistics and credible intervals, not p-values or binary conclusions about significance.

The scientist did not get exactly what he asked for, but what he got addressed the real question. Statisticians must avoid being boxed in by methodology in approaching problems.

Scott disavowed the notion that “statistics is a fortress to protect us from marauding commercial interests” or “statistics is a way to say ‘no,’ never a way to say ‘yes’.” Physicians sometimes look for statistical justifications to establish preconceived notions.

To drive home his point, Scott shared an experience in which a regulatory clinical team objected to a sponsor’s proposal regarding a noninferiority study design. The sponsor proposed to use a noninferiority margin on an absolute difference scale while the regulatory clinical reviewers were insisting on a risk ratio scale. The FDA statistician was asked to help provide statistical justifications to support a risk ratio, which had lower power and, according to John, is generally less interpretable.

By going beneath the surface and probing to understand the clinicians’ perspectives, it turned out the real issue for the FDA was the concern that sample size resulting from the sponsor’s proposal was considered inadequate to characterize the safety profile of the compound. The clinicians and statisticians then agreed to accept the sponsor’s proposal of an absolute difference scale, along with a stipulated minimum sample size for adequate safety evaluation.

From Scott’s perspective, technical knowledge, disease-area knowledge, and good communication tools are imperatives for influencing wise decision-making.

Finally, Scott advised that statisticians must be assertive, speak up, and not be afraid their opinion may be wrong or challenged. They must value their own expertise and literally sit at the table. Sitting at the back of the room conveys the message the statistician has no important contribution to the discussion.

Panel Discussion

Janet – Both Gary and John’s talks addressed what leadership ought to be. Beware, however, of negative leadership. I once worked for two aggressive competing managers who both wanted me to fail so they could use my failure as a weapon against each other. In such counter-productive set-up, removing yourself from the environment (for example, quitting) is always an option. As you establish yourself as a leader, your role should be to encourage and help others succeed—that will increase the chance of success of the project on which you are all working.

Eric – Leadership is about influence without authority based on the relationship capital you have built over time working with people. One of our biggest responsibilities is cultivating the next generation of leaders by opening doors and creating opportunities for others. Don’t make the mistake of evaluating future talent through the lens of your own experience; they are readier for opportunities than you may think.

Gary R. – There is a difference between statistical leadership and leadership in the broader sense. Everyone should take advantage of ASA leadership training, as not all places of employment offer formal leadership training. Relationship with clinicians can become confrontational because they come with preconceived opinion. It is our job to question and come to a mutually agreeable solution.

Watch the YouTube cartoon about consultation with a basic scientist.

How have you been able to shift someone’s opinion?

John – Be aware of the personalities you are dealing with, speak with authority, and don’t be intimidated.

Steve – It can be a simple act of engagement and conversation.

Janet – Statisticians really need to know where the individual expressing an opinion is coming from. Do not be afraid to say, “I don’t understand.” The scientist to whom you are talking wants to teach; take advantage of that!

Gary S. – Work on building the relationships and establishing trust. Meet with individuals to understand what they are doing and what challenges they are facing. Most physicians and scientists like to talk about what they are doing, so you shouldn’t have much trouble learning about the problems they are working on and what their interests are. Start by contributing, helping, and adding value to what they want before trying to redirect them or their work. This will demonstrate competence and build credibility. With patience and time, you will be able to interject, have your opinions be heard, and have influence with them and their work.

Eric – Be mindful of your biases and starting point. Do not approach a discussion with a mindset that your clinical partners do not want to listen to you; they are actually thirsty for your ideas and input. Step out of your comfort zone and talk to people. Statistics is also about helping scientists find the right question, and that takes legwork. Build relationships, ensure two-way conversation. Influence is process and not a single event, and it is depends on the network of relationships you develop with those you are seeking to influence.

Gary R. – Working together among ourselves and with junior-level statisticians to develop courage and confidence. Lead by example.

Audience: How can we breach the difference between academic training that emphasizes independent contribution and the industry reality of collaboration and interdependence?

Gary R. – Academic statisticians must continue to advocate for elements that are missing across academic programs. Some schools have started offering courses useful for statistical leadership. Students should be encouraged to seek internship opportunities.

Eric – Going back to my experience in graduate school, it was helpful to be able to shadow someone in academic consulting engagement. Also, in industry, bringing junior team members to the table is important for their development so they can see the dynamics of the decision-making process.

Janet – As Eric stated, senior statisticians should provide opportunities for junior statisticians to shadow them. Statisticians should not be afraid of saying something stupid. Watch others; they are not afraid of dumb comments. We often keep silent because we fear looking ignorant.

Gary S. – Teaching leadership courses in graduate school is one thing that will help. There has been a course taught at UNC since 2010 that was co-developed by Lisa LaVange. I am talking with Purdue and may be teaching a leadership course there in 2019.

John – Formal training in soft skills is also important for leadership development. Coursework or a practicum in teaching or even in making PowerPoint slides could be more important in the long run than a lot of the technical coursework currently in the curriculum.

Audience: Physicians desire to make decisions, while statisticians think they should only present the facts. How do we define the scope of statistical leadership?

Gary S. – It’s important to begin to set expectations for statisticians early in their careers to see themselves as leaders. Just as they will continue to invest in developing their technical skills, they also need to invest in their leadership skills.

Eric – It is exciting to work in an organization in which the CEO places value on statistics and quantitative sciences. In an organization like that, statisticians should continue to think about where they can add value to the enterprise. Constantly ponder the question, “How can we continue to engage and be broadly relevant to the problems confronting the organization?” We have an example in Novartis in which statisticians have collaborated with other functions in the organization to develop tools to facilitate quantitative decision-making on drug development portfolios. Statisticians should speak up about their ideas and not be afraid of some ideas not succeeding.

Audience: How do we fight the perception that statistics is a narrow discipline?

John – Statisticians should take the opportunity to engage in the subject matter in which they are working. It is a privilege to be in a narrow field that can affect everyone else’s discipline. I encourage people to read about the medical area they are working in; even starting with a Wikipedia article can be a big help.

Janet – It is not bad to be an expert in a narrow area. One can be an expert in a narrow area but have broad interests and participate in a broader multidisciplinary space.

Gary S. – Statisticians should leverage the insights of supervisors and superiors on development opportunities. If you have an interest in advancing administratively or technically, you need to have the conversation with your supervisor or your functional lead. Ask questions such as “What skills do I need to develop to be considered for that position or promotion?” “Can you help me find an opportunity to gain pertinent experience?” It is the responsibility of a supervisor to have those discussions with you and provide you appropriate feedback and direction.