The Data Management Standard Operating Procedures (DMSOP) Survey conducted among American Statistical Association (ASA) members aimed to investigate data management approaches and practices within the ASA community. The survey’s objective was to discern their needs for services and support, with the ultimate goal of utilizing the survey results to develop data management tools that enhance support for researchers funded by the Public Health Service (PHS).
Survey Questionnaire
Carried out by the research team at the University of Maryland through a cooperative agreement with the Office of Research Integrity within the US Department of Health and Human Services, a thorough review of publications and reports was conducted to construct the initial questionnaire. This questionnaire initially comprised 80 items deemed applicable in the biomedical and health professions. After a rigorous evaluation by a panel with expertise in data management from academia and industry, the final questionnaire, consisting of 53 items, obtained approval from the United States Office of Budget at HHS (OMB No. 0990-0486).
Survey Samples
To align with the project’s budget constraints and objectives, obtaining 200 completed surveys from ASA members appeared to have sufficient statistical power for the study’s intended purpose. Email addresses, acquired through a Data Use Agreement with ASA, were gathered while ensuring participant anonymity and confidentiality. As a gesture of gratitude, participants were rewarded with a $50 Amazon gift certificate upon the completion of the survey.
Data Collection Procedures
Randomly selected ASA members were sent an email that included an introductory recruitment letter outlining the study’s purpose, survey details, and guarantees of anonymity and confidentiality. Upon reading and consenting to the provided consent form, participants proceeded to complete the survey. All data were securely stored in the Qualtrics database, with participant emails excluded to maintain anonymity. Participants shared their emails separately to receive the promised $50 gift certificate. The data collection process concluded upon reaching 202 completed surveys.
Key Results
Response Rate Calculation
The response rate for the survey was calculated at 56.2%. The denominator for this calculation encompasses all individuals who received the email invitation and accessed the consent form. This approach includes both participants who agreed to participate in the survey and those who declined to do so.
Survey Participant Roles
The data reveals that among the 202 participants, 40.6% identified as principal investigators (PI), 50.0% as co-investigators (Co-I), 65.3% as collaborators, and 47.5% as consultants. Additionally, 42.1% of participants selected the “other” category. The higher representation of co-investigators, collaborators, and consultants suggests that a substantial portion of the survey group played collaborative roles rather than occupying decision-making roles in their respective research projects. Please note that participants were able to select more than one role, which may result in the total percentages exceeding 100% due to overlap.
Involvement in Data Management (DM) Tasks
It is important to note that a significant number of ASA survey participants are engaged in tasks related to data management (DM). The top two tasks reported by participants include data processing (85.1%) and data reporting (74.8%). The prevalence of these tasks underscores a distinct demand for robust and effective data management practices.
Motivations for Implementing a DM Procedure
When asked about the motivations behind implementing data management procedures, about two-thirds of participants (67.3%) identified “good research practice” as the primary reason for establishing such procedures. Furthermore, the acknowledgment of other motivations, including institutional requirements (35.1%), mandates from the funding agency (29.7%), and guidance from supervisors or research leaders (24.8%), highlights a commitment to compliance and accountability in data management practices.
Maintenance of a Metadata Document
The breakdown of responses regarding the creation of metadata for research data provides a comprehensive view. Approximately 19.8% stated they do it “often,” while 22.2% consistently reported engagement in this practice. Interestingly, varying percentages reported less frequent involvement, with 18.8% indicating “sometimes,” 16.8% “rarely,” and 7.4% claiming to have never participated in this practice.
Recordkeeping Practices
The breakdown of when participants incorporated recordkeeping practices across different research phases offers valuable insights:
- Organizing research projects: 60.9% of participants implemented recordkeeping practices at this stage.
- Planning experiments: 33.7% established recordkeeping practices during the planning phase of experiments.
- Recording data: Slightly over half, at 54.0%, reported developing recordkeeping practices during the actual recording of data.
- Analyzing results: A substantial 79.7% implemented recordkeeping practices during the analysis of results.
- Storing records for future reference: Another 57.4% reported established recordkeeping practices when storing records for future reference.
This indicates a widespread recognition of the importance of recordkeeping practices across various phases of the research process.
Organization of Data Files
When participants were asked about their data organization practices, the top four choices emerged as follows:
- Self-explanatory file names: 72.8% of participants preferred this method to ensure that data files could be easily located.
- Logically organized file folders: Almost an equal percentage of participants (77.2%) indicated a preference for organizing file folders in a logical manner, such as chronologically at each level.
- Creating a data dictionary to describe data (73.3%) to keep their data organized.
Using data validation to avoid data entry errors (63.9%) to keep their data clean.
These practices reflect a strong commitment to maintaining a systematic and efficient organization of research data.
Duration of Data Maintenance
When participants were asked about the length of time that they typically maintained older data from their research projects:
- 44.1% of participants indicated 5 or more years.
- 34.2% of participants indicated indefinitely.
- 21.8% of participants indicated under 5 years.
This sheds light on the length of data maintenance practices among the participants.
Data Retention on Multiple Storage Media
In response to the question “Do you always retain data on at least two different types of storage media?” 46.5% of participants answered “Yes,” while 23.8% indicated “Sometimes.” Notably, 28.2% indicated “No,” prompting questions about whether this group has ever experienced any negative consequences. Further investigation into this matter may be warranted.
Primary Backup Solution
Institution-managed backup storage emerged as the predominant choice for the primary backup solution for digital research data. Results showed that it constitutes more than half (55.4%) of all backup solutions. This indicates a reliance on institutional infrastructure and centralized storage systems, underscoring a commitment to ensuring the security and accessibility of research data.
Incidents of Data Loss
Survey findings indicated that 11.9% of participants reported that they experienced data loss in their research. While relatively small, this is not inconsequential and highlights a potential challenge in data management practices. On a positive note, it is reassuring that the majority, 88.1%, reported that they had not encountered data loss. Addressing issues related to data loss is crucial for ensuring the integrity and reliability of research findings.
Inclusion in Data Management Standard Operating Procedures (DMSOP)
When participants were asked about the content that a DM procedure should encompass, they emphasized the importance of the following elements:
- Type of data (82.7%)
- Data security (82.2%)
- Data distribution policy (75.2%)
- Data collection information (67.8%)
- Metadata information (65.3%)
- Software details (52.5%)
- Samples information (41.1%)
These results suggest an overall commitment to providing detailed information about the research data, which is essential for ensuring transparency and reproducibility.
Adherence to Data Management Standard Operating Procedures (DMSOP)
The responses regarding adherence to data management procedures present a mixed picture. While a good portion follows guidelines, the presence of “sometimes” and “no” responses suggests that there may be room for improvement in promoting consistent adherence to data management practices.
- 55.0% of participants indicated that they consistently follow data management guidelines.
- 34.2% indicated that they follow them on occasion.
- 10.9% stated that they do not adhere to data management guidelines.
This indicates a need for targeted efforts to achieve consistent adherence to data management procedures.
Data Description Practices
The participants’ varied approaches to describing their research data indicate a thorough approach to data documentation. The highlighted areas include:
- How the data were generated (88.1%)
- The format of the data (86.1%)
- Number of records and the number of files (83.3%)
- Stages the data pass through (e.g., raw, processed) (71.3%)
- What software tools were used to collect data (68.3%)
- What hardware tools were used to collect data (33.7%)
This finding reflects a commitment to providing detailed information about the research data, which is essential for ensuring transparency and reproducibility.
Willingness to Share Research Data
A notable number of participants expressed openness to publicly sharing research data, with 39.1% indicating that they have already done so and an additional 23.8% planning to share data publicly in the future. This reflects a positive attitude towards open data-sharing practices, fostering greater accessibility to research findings within the broader scientific community. However, approximately one-third of participants who indicated “Not sure” or “No” may warrant further investigation to understand the reasons behind their stance.
Institutional Support of Data Management Practices
In the survey, 62.4% of participants indicated that their institution provides a protocol on procedures, standards, and guidelines for Data Management (DM), which suggests that there was about a quarter (26.2%) of participants whose institutions have not established frameworks related to data management practices.
Interest in Online DMSOP Toolkit
It is encouraging to observe that a majority of participants, constituting 61.9%, expressed interest in an online Data Management Standard Operating Procedures (DMSOP) toolkit. The University of Maryland, under a cooperative agreement with the U.S. Department of Health and Human Services Office of Research Integrity, will create a toolkit that will proactively support and address the needs and interests of researchers, research institutions, and research administrators.
Summary
The DMSOP Survey results, conducted among ASA members, offer unique insights into data management approaches and practices within the ASA community. This presents an opportunity for ASA members to benchmark their data management practices against the broader ASA community. These findings serve as a valuable reference for self-assessment and may also provide actionable insights for individuals seeking to enhance their data management practices.
These survey results hold particular significance for the research team at the University of Maryland, engaged in a cooperative agreement with the Office of Research Integrity within the US Department of Health and Human Services. The insights gleaned from the survey will inform the development of advanced data management tools. These tools aim to bolster support for researchers funded by the Public Health Service, contributing to more efficient and effective data management practices within the research community.
For more information about this survey, contact Min Qi Wang.
Leave a Reply