Home » Community Analytics

Supporting Community-Engaged Research Ethical Challenges and Plausible Responses in Statistics, Data Science Practice

1 September 2022 371 views One Comment

Rochelle Tractenberg is a tenured professor in the Georgetown University departments of neurology; rehabilitation medicine; and biostatistics, bioinformatics, and biomathematics. She is an ASA Accredited Professional Statistician and a fellow of the ASA and AAAS. She earned a PhD in cognitive sciences from the University of California, Irvine and a PhD in measurement, statistics, and evaluation and doctoral-level certificate in gerontology from the University of Maryland, College Park. She chaired the ASA Committee on Professional Ethics from 2017–2020 and the working groups on revising the ASA Ethical Guidelines for Statistical Practice in 2016 and 2018 (co-chair in 2021). She is the author of two forthcoming books about ethical reasoning and its application with the ASA ethical guidelines: Ethical Reasoning for a Data-Centered World and Ethical Practice of Statistics and Data Science

The American Statistical Association first published its Ethical Guidelines for Statistical Practice in 1995. Since 2016, when the first formal revision effort was approved by the ASA Board, they have been slated for review and revision every five years.

While the ASA Committee on Professional Ethics maintains and disseminates the guidelines, their actual application has been a challenge for the committee and entire ASA community.

Critically, the guidelines are described as promoting ethical decision-making. The guidelines specify that “statistical practice” includes activities such as designing the collection of, summarizing, processing, analyzing, interpreting, or presenting data, as well as model or algorithm development and deployment. They also explicitly point out that, irrespective of job title, level, or field of degree, the guidelines apply whenever the individual engages in statistical practice.

Ethical reasoning is a process that can be learned and improved. The six knowledge, skills, and abilities of ethical reasoning based on a mastery rubric by Kevin FitzGerald and me are the following:

    1. Determining your prerequisite knowledge
    2. Identifying decision-making frameworks
    3. Recognizing an ethical issue
    4. Identifying and evaluating alternative actions

    5. Making and justifying decisions
    6. Reflecting on the decision

With a bit of luck, the ethical statistical practitioner will need only #1 and #2 for 95 percent of their work. How to use the ASA Ethical Guidelines to practice ethically is the topic of most of my book Ethical Reasoning for a Data Centered World. Here, we treat the ASA Ethical Guidelines as the “prerequisite knowledge” needed to identify when ethical challenges arise. Memorizing the guidelines is not the focus; knowing their role and the general organization is enough to start with.

The ASA Ethical Guidelines for Statistical Practice includes the following eight core principles and an appendix for organizations and institutions with 72 specific elements:

  • Professional integrity and accountability (12)
  • Integrity of data and methods (7)
  • Responsibilities to stakeholders (8)
  • Responsibilities to research subjects, data subjects, or those directly affected by statistical practices (11)
  • Responsibilities to members of multidisciplinary teams (4)
  • Responsibilities to fellow statistical practitioners and the profession (5)
  • Responsibilities of leaders, supervisors, and mentors in statistical practice (5)
  • Responsibilities regarding potential misconduct (8)
  • Appendix: Responsibilities of organizations/institutions (12)

In terms of ethical decision-making frameworks (#2 on the list), there are two that can be helpful in accomplishing #3, recognizing an ethical issue. These are “utilitarian” and “virtue” perspectives. The virtue ethics perspective can generally be summarized as, “what would the (ideal) ethical practitioner do in this situation?” Since the ASA Ethical Guidelines elements have the stem, “the ethical statistical practitioner (does x),” it is clear the guidelines feature the virtue perspective.

However, the utilitarian perspective can generally be summarized as, “how can benefits be maximized while harms are minimized in this situation?” The guidelines can also be used to check whether there may be harms when responsibilities outlined in the guidelines are ignored.

Thus, we see that not only are the ASA Ethical Guidelines part of prerequisite knowledge, but the actual event or behavior also must be clearly described.

Many ethics cases are purposefully vague to help encourage discussion. However, to determine what is going on and how best to respond when unethical behaviors occur or are observed, it is important to be able to clearly describe exactly what is going on/happened.

The most straightforward ways to identify an ethical challenge in any given situation are the following:

1. Determine if, once the actual event or behavior is clearly described, it is inconsistent with one or more of the ASA Ethical Guidelines principles or elements.

    • If one or more principle or element is violated (or threatened), there is a problem and you’ve moved closer to identifying how to address it.

2. List the harms—both actual and potential—as well as any benefits of the event or behavior for all stakeholders.

    • If there are more harms (actual or potential) than benefits, harms are more serious than benefits, or harms accrue to vulnerable or at-risk populations (especially if benefits do not accrue to these populations), then you’ve identified the problem. The stakeholder analysis does not move you closer to identifying how to address it, but you will have evidence that your solution was successful if the harms you identified are mitigated or eliminated.

A stakeholder is defined as “one who is involved in or affected by a course of action” by Merriam-Webster. In the context of ethical case analysis, the stakeholder is simply any individual or group that might be affected by the outcome of the event or behavior. Stakeholders include yourself (and your professional reputation), your boss, your employer/organization, your nonstatistical practitioner colleagues, the profession (of statistics/data science), and the public. Note that you can use a stakeholder analysis to evaluate your alternative actions and determine if an event or behavior may constitute an ethical challenge.

At this point, we can see the guidelines are important for ethical reasoning steps #1 and #3; they can be important in #2 (to support a virtue ethics approach), but can also be helpful in identifying stakeholders and harms/benefits that accrue to each. Note that Principle C outlines responsibilities to stakeholders, while principles A (you), D (“anyone directly affected by statistical practices”), E (team members), and F (other practitioners and the profession) are each explicit about responsibilities to different stakeholders.

Once an ethical challenge has been identified, there are always three alternative actions to take or decisions that can be made:

  1. Do nothing (ignore the unethical behavior), ignore the request to do (the unethical thing), or agree to do (what was asked, even if that conflicts with the guidelines or creates more harms than benefits for any stakeholder).
  2. Consult with a peer or supervisor using the professional guidelines or other resources.
  3. Refuse to do what was asked (if it was unethical) and/or report violations of policy, procedure, ethical guidelines, or law.

The ASA Ethical Guidelines exist to help practitioners identify and avoid—or prevent/put a stop to—unethical statistical practices. Many elements of the guidelines explicitly state the ethical practitioner avoids, avoids condoning, and avoids appearing to condone unethical behavior. Thus, it is never ethical to “do nothing” when faced with an ethical challenge. That is a decision and a “plausible alternative,” but it is never ethical.

To consult a peer or supervisor will be much more straightforward if you have completed the ethical reasoning steps 1–3 as outlined here. That is, when you determine who would be a good person to consult with, you simply use your reasoning steps 1-3: “Hi, colleague, I’m in a bit of a situation. I was directed to do/observed X, and that’s contrary to the Ethical Guidelines [list]. What do you reckon I should do?”

Moreover, if you have identified exactly what the ethical challenge is and what elements or principles in the guidelines are—or would be—violated by the behavior you observed or were asked to do, your and your conferee’s decisions about what to do will be clearer. For example, “stop and do.” And “share the guidelines with X so they stop asking the statistics practitioners to do.”

Finally, making a decision (Step #5) requires both the decision and its justification. This is clearly supported by the effort you put into steps 1–3, and since you’ll never choose alternative 1 (do nothing/ignore/fulfill the unethical request), there are two clear options to tailor to your situation (2, confer, or 3, report).

Ethical reasoning and the 2022 ASA Ethical Guidelines are specifically formulated for all practitioners at all levels. Ethical practice standards are not “moral principles,” but specific descriptions of what constitutes ethical practice. The implication is that if a practitioner does not follow the guidelines/code, then they are not doing their job ethically.

The guidelines support professional and scholarly and scientific work in, and with, statistics and data science. When practitioners do not follow ethical practice standards, all those who make decisions on the basis of the results of quantitative practice may find their decisions or scientific or scholarly work undermined.

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...

One Comment »

  • Stan Young said:

    A statistician notices that a subject matter expert and thought leader computes individual p-values correctly for 61 questions. Each is covariate-adjusted, etc. So far so good.
    1. Should the expert be alerted to the multiple testing problem?
    Suppose the expert does not respond, or the expert responds saying no adjustment is necessary.
    2. Dr. Tractenberg gives three options:
    a. Do nothing. (Using unadjusted p-values is not an ethical issue; p-hacking is widely done and buyer beware).
    b. Consult with peers, which is what I am doing.
    c. Report violation; Is selectively reporting small p-values from many a violation?