QDET2 Chair Talks About Her Experiences in the Field

Amanda Wilmot is a senior study director at Westat with 30 years of professional experience in survey research. She is part of Westat’s Instrument Design, Evaluation, and Analysis (IDEA) Services group. During her career, Amanda has managed large-scale government surveys covering a range of social policy issues and specialized in the development of data collection instruments. She currently chairs the organizing committee for the 2016 International Conference on Questionnaire Design, Development, Testing, and Evaluation (QDET2), which will take place in Miami, Florida, in November of this year.

Tell us about your research background.

My working life has provided me with a series of opportunities that have led me in directions I could not have foreseen. I began my survey research career at the U.K. Office for National Statistics (ONS) in London, joining their Social Survey Division (SSD) after leaving university. It was in SSD that I learned about survey research from renowned experts in the field and went on to manage large-scale surveys in the U.K., including the National Travel Survey, Family Resources Survey, and the National Diet and Nutrition Survey of adults. When I first started work, we were still using mainframe computers and a response rate of anything less than 80% was considered poor! The world of survey research has certainly become much more complicated since then, chasing ever-declining response rates and finding new and innovative ways to motivate the time-strapped public to take part.

In 2002, I was given the opportunity to attend the first-ever Conference on Questionnaire Development, Evaluation, and Testing (QDET) in Charleston, South Carolina, chaired by Jennifer Rothgeb from the U.S. Census Bureau. This was a real turning point in my life. I had always had a particular interest and flair for the front-end survey data-collection phase, and attending the conference meant I was able to learn from and network with others from around the world with the same interest. I could never have known that, more than 10 years later, I would be asked to take Jennifer’s role and chair the organizing committee for the second QDET conference, which will take place in November of this year in Miami, Florida.

Soon after the 2002 conference, I was given another opportunity by ONS to head up a team of quantitative and qualitative methodologists specializing in the development of data collection instruments for social surveys. Through the use of various survey pretesting methods, I was able to experience and understand the true meaning of language and how it evolves from the survey respondent’s perspective. It was during this time that I led the team in developing a harmonized question on sexual identity that could be used on social surveys and for equality-monitoring purposes. At that time, [it was] something that had not been undertaken on quite that scale.

When the ONS moved its headquarters from London to South Wales, another opportunity arose for me to pursue my growing interest in cross-national questionnaire design. I took a position at the University of Leicester working with my longtime mentor, professor Howard Meltzer, to develop the European Health and Social Integration Survey questionnaire. Professor Meltzer was an adviser to the UN [United Nations], WHO [World Health Organization], OECD [Organisation for Economic Co-operation and Development], and EC [European Commission] committees on disability issues. It was his wish to see Europe measure disability from the perspective of the barriers to social integration as reported by the respondent, rather than using the classic medical model in which disability is simply the consequence of a health problem. Sadly, professor Meltzer passed away in 2013, but it was my great privilege to be able to help support the survey through its implementation in 28 European countries. In fact, I will be presenting on that work at QDET2.

Moving to the United States was another amazing opportunity for me to work with some of the world’s leading survey methodologists, and this was when I truly learned the meaning of the phrase “two nations divided by a common language.” In the U.S., pavement is apparently something you drive on, not walk on! A trolley is something to convey passengers, not to move groceries around the grocery store. And I should have asked for potted plants in the local garden center and not pot plants, which is the British-English term!

What interested you about your current position at Westat?

It was the chance to work with a group of like-minded and highly skilled individuals specializing in questionnaire design and pretesting who really care about data quality. We get to work on a range of different topics and apply cutting-edge methodologies, so life is never boring. It is a very stimulating environment in which I feel privileged to be a part of.

What is QDET2, and what do you hope participants will gain from attending the conference?

QDET2 will be the first international conference of its kind in 14 years specifically devoted to survey questionnaire design and the methods used for questionnaire development, evaluation, and testing. The vision from the start was to bring together questionnaire design specialists from across the world (from both the public and private sectors) to discuss methodology and new developments in pretesting and evaluation of survey questionnaires. The conference aims to set the benchmark for best practices in the field, as well as provide a forum for discussing areas for future research. Participants will hear from industry thought leaders, learn about real-world case studies from survey agencies, and, of course, have the opportunity to network with international peers and colleagues.

What are some of the challenges you have faced organizing the conference?

Obviously, a conference of this scale is going to be a lot of work and require a huge time commitment—I probably underestimated that aspect. But we have a strong group of volunteers on the various committees taking responsibility for different aspects of the conference, including the program, publications, and training committees—and not forgetting the all-important events committee! A big help to me at the start was being able to talk with Jennifer Rothgeb and learn from her experiences organizing the first QDET. Jennifer will be joining us at the conference and will speak about what came out of the 2002 QDET. I would also like to mention that our ASA conference planner, Naomi Friedman, has been amazing in guiding us throughout the process.

What part of the conference are you most excited about?

That is a difficult question to answer because we already have an incredible line-up of invited and contributed presentations, preconference short-courses given by leading experts in the field, and, of course, not one—but two—keynote speeches to reflect the overall aims of the conference. Gordon Willis from the National Institutes of Health and the guru of cognitive testing will kick off day one with his keynote titled “Questionnaire Design, Development, Evaluation, and Testing: Where in the World Are We?” And Mario Callegaro, from Google UK and a leading expert in web survey methodology, will follow on day two with “Questionnaire Design, Development, Evaluation, and Testing: What’s the Future Look Like?” There will also be a conference book published by Wiley showcasing the best papers. We have tried to vary the learning experience for conference attendees, as well. The program is interspersed with workshops, demonstrations, and roundtable discussions. So, on reflection, the biggest reason to be excited is the potential for facilitating knowledge exchange at the international level.

What does the future hold for questionnaire designers?

You will need to come to the conference to find out!

Do you have an example of a ‘bad’ survey question?

Not so much a ‘bad’ survey question, but an example of how language and meaning evolves. I have always been interested in social history. Census forms are really interesting in this regard. The disability question from the 1901 U.K. Census form, for example, still collected information about whether anyone in the household was a lunatic, imbecile, or feebleminded—probably a question we would want to cognitively test these days!

What advice can you give to someone who would like to pursue a career in questionnaire design as a survey methodologist?

You really need to have a passionate interest in people and their use of language. I would suggest that if you are interested in specializing in the design and development of survey data collection instruments, then a good grounding in survey methods and experience of applying them in a real-world context is crucial.