DOING A PILOT STUDY: WHY IS IT ESSENTIAL?
The questionnaire was divided into 3 sections that related to a) socio-demographic characteristics of respondents; b) sleep, with items seeking information on the sleep pattern, impact of inadequate sleep on daytime function and psychological health; and c) the Hospital Anxiety and Depression scale (HADS) 7 to screen for symptoms of anxiety and depression. The HADS has been translated and validated in Malaysia8. The questionnaire was produced in English and Bahasa Malaysia versions and was piloted 4 times, using 10-12 subjects each time.
Issues that were observed among patients in the pilot of the questionnaire included:
- Ability to comprehend the instructions in the covering letter
- Understanding of questionnaire items, the terms used, the sequence of questions and the flow of statements
- The format, including the font and lay out
- Length of the questionnaire (i.e. the time taken to complete the questionnaire)
- Other comments by patients.
All comments were taken into consideration and errors amended and re-piloted until no further changes were considered necessary.
4. Data entry and analysis
Data from the 20 subjects were entered into a computer by a research assistant who was not involved in data collection. Entry was directly into the Statistical Package for Social Sciences (SPSS) program, with specific codes used for each questionnaire item. Data were then analysed using SPSS version 11.5 .
The analysis was mainly descriptive, with data about age, total hours of sleep and total hours of time spent in bed, plus other items, being analysed for their mean, median, modal, minimum and maximum values.
To compare the categorical variables such as gender, ethnic group and employment status with insomnia symptoms, a c2 test of association or a Fisher exact test were used. To compare ordinal data such as age group, marital status and educational level with insomnia symptoms, a c2 test for trend was used. To compare the difference in means between the insomnia and non-insomnia groups, independent t tests were used. Statistical significance was set at p< 0.05.
1. The feasibility of the study protocol
The time taken by the doctor to explain the research project and obtain consent from the participants was about 3-5 minutes. The study procedures for data collection encountered some initial problems, with the research assistants unsure how to guide and assist the subjects due to a lack of familiarity with the questions. This difficulty was overcome after 3 or 4 subjects.
In view of the number of subjects to be recruited for the full study, it appeared that each centre required two research assistants, with each spending an average of 15 minutes to guide a respondent. Two research assistants were deemed necessary because, they were the nurses working in the respective clinics, which have 2 working shifts (9am till 3pm and 3pm till 9pm).
Note that a pilot study should preferably be done using subjects from a population that is different from those recruited for the main study. Experience gained by subjects in the pilot study may bias the results of the main study if the same subjects are included.
2. Recruitment of subjects
All patients invited agreed to participate in the study. They filled in the questionnaires in the waiting room while waiting for their medications.
3. Testing the measurement instrument
On average, the respondents took about 10-15 minutes to complete the questionnaires. While they all attempted to respond to all questions, there were some items that they missed. This appeared to be because the questions were spaced too close to each other, causing some of the participants to miss a line.
There was considerable discrepancy in the answers to some of the items, either because they were too vague or due to a language barrier, resulting in participants not understanding the questions properly. This was observed with the items asking about sleep pattern and sleep efficiency.For some of the items, the boxes were placed too close together which made it difficult for the research assist to identify which one the respondent had ticked. Reformatting was done to overcome this problem. There were also a few typographical errors noted and some of the coding for the HADS items was wrong. HADS has been widely used and validated in primary care and from this pilot study respondents encountered no difficulty understanding the items (in both languages).