Survey Methods
This section contains an overview of the survey instrument, sample, and data collection methods, including site selection and recruitment. A discussion of the measures used and the multivariate analysis of the nursing data are also included. An earlier article by Marshall et al. (2013) provides a more detailed account of the methods, as well as the descriptive findings of the full study of physicians, residents, and nurses.
Survey Instrument
In the web-based survey, health professionals were given the following instructions: "Think of an occasion in the last 6 months when you looked for information resources for patient care (beyond what is available in the patient record, electronic medical record [EMR] system, or lab results). Please answer the following questions regarding this occasion." Respondents were then asked about the patient's diagnosis; type of information needed; information resources used, information access points; quality and completeness of information retrieved; as well as the patient care outcomes of using the information obtained. The survey also included questions related to the health professional's role, including factors such as how many years the respondent had worked as a health professional and educational attainment. Each web page contained the study name, "Value and Impact of Library and Information Services in Patient Care Study," so that respondents were reminded about the focus of the research.
Sample
Study sites were recruited by the National Network of Libraries of Medicine, Middle Atlantic Region (NN/LM MAR) and by members of the study planning group through an invitation to all the libraries on their email list. Libraries did not have to be a member of NN/LM MAR to participate in the study. Once a site's eligibility was determined by the researchers at the University of North Carolina (UNC), the library director was asked to complete an online profile that described the library's users, staffing, budget, collections, and services. In order to participate, the libraries had to provide services to health professionals who were involved in direct patient care. Librarians also had to agree to make the study invitation available to physicians, residents, and nurses in their setting and to promote participation. Special efforts were made to recruit different types and sizes of hospitals. These methods resulted in a sample of 56 library sites that served health professionals at 118 hospitals. The UNC Institutional Review Board approved the study after reviewing all study procedures. The survey was anonymous and no personally identifying information was gathered. Participation was voluntary. Informed consent was implied when the health professional chose to respond to the survey.
Individual respondents at each site were recruited through listservs or the institution's intranet, depending on the typical means used to communicate with all physicians, residents, and nurses at that institution. The survey achieved a response rate of 10%. This is likely an underestimated rate because only those healthcare providers who were involved in patient care or clinical research, and who had used the library and information services for seeking information related to patient care in the previous six months were eligible to participate. Those who began the survey and were found to be ineligible because they were not involved in patient care were removed from the denominator (which was based on the total number of physicians, residents, and nurses at the hospital). However, there may be others who did not start the survey at all because they had noted earlier the criteria for inclusion. Another factor that should be taken into account when interpreting the response rate is that the validity of CIT studies is judged primarily by the ability of the researchers to obtain a sufficiently large number of critical incidents to determine consistent trends in the data.
Of the 19,234 responses, 2,290 were deemed ineligible as the responders were not involved in either patient care or clinical research; and 826 people did not complete enough of the survey to be included. Of the remaining eligible cases, 6,788 nurses, who identified patient care as at least part of their current job role, and who had used the library for seeking information related to patient care, completed the survey. This analysis is restricted to those nurse respondents. Full details of the survey sample, site and respondent recruitment, data collection methods, and descriptive results have been reported elsewhere by Marshall, et al. (2013).
Data Collection
A pilot study was conducted to ascertain the effectiveness of survey content, structure, and delivery method. The pilot test was conducted at seven sites that were members of the NN/LM MAR. A call for participation by sites outside NN/LM MAR was sent via relevant health library listservs. Additional recruitment efforts included a webinar and presentations at regional meetings. One hundred and twenty library sites initially expressed interest in the study. Of these interested sites, four were ineligible because they did not provide library services for clinicians and eight cases were duplicates (i.e., multiple staff members at a library had expressed an interest). Forty-nine libraries participated in this pilot phase of the study. Data from the pilot and the full launch were combined in the data analysis as the survey did not require major revisions between the two waves of data collection.
Measures
The survey questions and outcome measures from the original Rochester study (Marshall, 1992) were reviewed by the study planning group and revised to reflect the current information access and healthcare environments. Since the time of the original study, electronic access to library resources had steadily increased, with databases and full text articles being available electronically in healthcare settings in both clinical and research areas.
In the analysis reported here, we identified four key outcomes relevant to nursing and patient care to provide an assessment of the impact of the use of the library and information resources. The first outcome was created to predict the amount of time saved as a result of the information obtained in the respondents' searches. This variable was created from a survey question that read, "Recalling the [clinical situation], please indicate whether you agree or disagree with the following: Having the information saved me time." If respondents agreed with this statement they were asked how much time they had saved in minutes and hours. Time saved that was reported in minutes was converted to hours using decimal points as needed.
The second outcome was a summary variable measuring the number of reported adverse events avoided as a result of the information found in the respondent's search. The survey question read, "Recalling the same [patient care situation], did any of the following change in a positive way as a result of the information?" The respondents were then presented with a list of 12 possible adverse events: hospital admission, hospital readmission, patient mortality, language or cultural misunderstanding, patient misunderstanding of disease, hospital acquired infection, surgery, regulatory non-compliance, additional tests or procedures, medication error, adverse drug reaction or interaction, and misdiagnosis. An option was given to list another adverse event not on the list. The researchers counted the number of adverse events that respondents reported avoiding, and then they computed a summary variable from that total.
Finally, the researchers identified two additional outcomes that are particularly relevant to nursing care: whether the nurses had changed the advice they gave to the patient and whether they had handled the patient care situation differently. These outcomes were measured using the following question: Recalling the [patient care situation], please indicate whether you agree or disagree with the following statements about the information you used. As one respondent said, "I was able to give better advice to the patient because I was better informed." Another respondent noted, "the [library] information helped us prescribe the right medication – or, more importantly, prevented prescribing the wrong one." The respondent added, "I think it is mostly a patient safety thing, so [the library information] makes me deliver safer care."
The following control variables were tested in initial analyses but were excluded from the final multivariate model because they did not contribute significantly to the explanatory power of the model: age, gender, location of hospital (urban, suburban, or rural), annual expenditures on electronic resources, annual expenditures of the library, and whether the library staff offered instructional programs. It should be noted that because budgets are often affected by the consortium arrangements used by libraries to reduce costs, the actual cost of information resources may not show up in individual hospital library budgets. Also, almost all libraries offered instructional programs.
How the respondents accessed information was operationalized using four access methods: (a) whether they asked a librarian for help with the information search; (b) whether they performed the search in a physical library; (c) whether they used the library's website; and (d) whether they used the institution's intranet. At some sites, health professionals accessed the library's information resources via a separate library website. At other sites, the library resources were accessed via the institution's intranet.
The information resources and services available at each site were operationalized using the following measures: (a) the number of full-time equivalent (FTE) library staff available at the hospital; (b) the number of FTE professional master's degree librarians; (c) the number of electronic databases and other types of information resources available; and (d) the number of library-provided information resources respondents reported using for their search, such as Micromedex, PubMed MEDLINE, online books and journals, UpToDate, Cumulative Index to Nursing and Allied Health Literature (CINAHL), and Nursing Reference Center. The nurses reported using an average of 2.8 (S.D. 2.2) different library resources in their search. The nurses selected the resources from a list of library resources included in the survey; however, an option was given to add additional sources that were not on the list.
Respondents were also asked how they accessed resources. While a third of the nurses reported using Google to access at least one resource, almost 90% reported using the library website or the institution's intranet. It is common for Google searches to be used as a quick way to identify a relevant publication. The point-of-care information resource, UpToDate, was also used frequently as a 'first place to look.' UpToDate is a commercially available database that is marketed as an 'evidence-based, clinical-decision-support resource.' It contains peer reviewed, referenced summaries on a wide range of clinical topics, as well as patient education articles and other materials. The use of such tools is often followed by a search of additional library resources to access the full text of referenced articles or other relevant materials on the topic in the library collection. Many journals make their full text articles available only by subscription, which makes use of the library essential. Library staff members also provide assistance in navigating the increasingly complex world of electronic information and locating items that are difficult to find.
Level of education and tenure as a healthcare professional were used as control variables, as were key characteristics of hospitals where the nurses were employed. These variables included dichotomous indicators of whether the hospital had Magnet®status, was a member of the Council of Teaching Hospitals, and bed size. See Table 1 for further elaboration of the dependent variables and the descriptive statistics.
Multivariate Analysis
The analytic strategy was designed to assess the relative impact of using the physical library and library-provided electronic resources for information seeking related to patient care on key outcome measures. Four separate models, two using Ordinary Least Squares (OLS) regression for the continuous variables, and two models using logistic regression for the dichotomous variables, formed the basis for the analysis. These types of multivariate analyses allow the researcher to assess the strength and direction of the impact of independent variables while controlling for the other variables in the model. These model types were chosen as appropriate to the level of analysis of the dependent measures.
Ordinary least squares (OLS) regression is appropriate for use with continuous variables. These measures are counts that approximate a normal distribution given their range and descriptive characteristics. These were chosen over logistic regression for the first two outcome measures (time saved and number of adverse events avoided), as dichotomizing the outcomes would have resulted in the loss of valuable information on how much impact could be seen on the dependent variable. The other two models (changed advice given to the patient and handled the situation differently) were based on dichotomous 'agree' or 'disagree' responses; therefore logistic region was used.
The researchers also identified workplace and respondent characteristics that might potentially influence information-seeking behavior as control variables. By holding these variables constant, we could assess the impact of the independent variables of interest. The control variables included: how long the respondent had worked as a healthcare professional; level of nursing education; whether they were affiliated with a Magnet® hospital; whether their hospital was a member of the Council of Teaching Hospitals (COTH); the bed size of their largest affiliate hospital; and which access method was used to conduct their information search. Holding these characteristics constant, the authors operationalized the following concepts: (a) how nurses accessed the information, and (b) the number of information resources that the library made available to nurses in their workplace.