The Study Plan and Data Collection Tools are ready. Now, as we begin recruiting our heroes (the study participants), we must follow strict Principles of Data Collection to ensure the information we gather is flawless. This is the most crucial phase, as poor data collection invalidates everything that follows.
1. The Goal: Reliability and Accuracy
The two primary attributes that define good data quality are Reliability and Accuracy.
| Attribute | Definition | Simple Idea |
| Reliability | Repeatability and Precision. The information is stable and consistent over time, even if repeated by different investigators. | If we ask the same child the same question today and next week, we should get a similar answer. |
| Accuracy | The ability of the measurement to be correct and true. | The answer we recorded is actually the true answer. |
Scenario Check: A scale that always measures weight 5 pounds too high is Reliable (it gives the same result repeatedly) but Not Accurate (it’s systematically wrong). An ideal study is both Reliable and Accurate.
2. Six Principles of High-Quality Data Collection
To achieve high reliability and accuracy, we must follow these principles:
A. Develop a Question-by-Question (Q-by-Q) Guide
This is a simple, essential document that acts as a road map for every investigator.
- Purpose: Ensures uniformity and consistency by ensuring all investigators understand and ask the questions in the exact same way.
- Content: A short explanation under each question on how to ask it, what the possible responses are, and any necessary probes (follow-up questions) or skip patterns.
- Maintenance: If difficulties arise in the field, the guide should be revised, and all changes must be documented immediately.
B. Conduct Thorough Staff Training
Choosing the right people is the first step, followed by rigorous training:
- Selection: Recruit investigators suitable for the study (e.g., familiar with local language and culture for a field study, or clinical terms for a hospital study).
- Classroom Session: Introduce the study objectives, basic definitions, and walk them through the Q-by-Q Guide. Use role-playing (investigator acts as interviewer, another as respondent) to simulate the process.
- Pilot Testing: Before the main study, conduct interviews in a similar setting (not the exact study site) to practice. This uncovers unexpected difficulties and clarifies doubts.
C. Ensure On-Site Data Quality
Once data collection begins, the following must be maintained:
- Supervision: A supervisor should be present, even periodically, to address queries immediately (phone/messages) and verify study forms daily in the field.
- PI Visits: The Principal Investigator must conduct periodic on-site visits to ensure the protocol is being followed.
- Time Management: Avoid pressuring staff with tight deadlines (e.g., “ten questionnaires every day”), as this dilutes quality.
D. Do Periodic Data Reviews
Regularly check the collected forms to ensure high quality and completeness:
- Field Checks: The supervisor checks forms daily for completeness (no blanks) and readability.
- Review Checks: The Principal Investigator checks for consistency. Do the answers make logical sense, or is there a systematic error?
- Review Meetings: Hold regular meetings (weekly/monthly) to clarify queries, discuss unanticipated field issues, and make (and document) any needed changes to the questionnaire or Q-by-Q Guide.
E. Institute Debriefing and Supportive Supervision
- Investigators must feel comfortable sharing difficulties they encounter. Supportive supervision and a focus on teamwork (not individual fault-finding) is key to solving problems before they damage the data.
F. Validate the Collected Data
Validation ensures the data collected is actually true:
- Sub-Sample Re-interview: Select a small sub-sample (even $5\%$) of the participants. An independent, second interview is conducted by a senior member or a different team.
- Comparison: Compare the results of the first interview with the second to find discrepancies or repetitive errors, either by an individual or across the whole team. This step ensures the final study results are valid.
Supportive supervision and teamwork are key to successful, high-quality data collection.
Leave a Reply