Measuring an Academic Library’s Performance and Service: A Case Study in Georgia Southern University Library

This paper shares the Georgia Southern University Library’s experience of LibQUAL in user-centered and service-oriented academic learning environment. On the basis of reviewing the library literature on LibQUAL, this study presents the process of data analysis and data visualization for academic library assessments. Using the 2016 LibQUAL Survey at the Georgia Southern University Library as a sample, this study illustrates basic methods of analyzing and interpreting the LibQUAL Survey Raw Data and User Response Raw Data saved in Excel files. Also mentioned in this study are other common statistical tools and a set of general procedures, including data analysis, data validation, data migration, data mining, and data visualization, for academic library assessments to identify patron needs and satisfactory factors. To cite this article: Li, L. (2017). Measuring an academic library’s performance and service: A case study in Georgia Southern University Library. International Journal of Librarianship, 2(1), 53-65. https://doi.org/10.23974/ijol.2017.vol2.1.26 To submit your article to this journal: Go to http://ojs.calaijol.org/index.php/ijol/about/submissions INTERNATIONAL JOURNAL OF LIBRARIANSHIP, 2(1), 53-65


INTRODUCTION
The Internet and the World Wide Web (WWW) surge in the 1990s changed ways of information delivery and dissemination in the modern information society. The rapid advance of information technologies has boosted dynamic demands and expectations from faculty and students. Due to the uncertainty of the global economic outlook, fiscal retrenchments year after year require that many academic libraries offer the best information resources and services within stringent operating budgets. Many academic library administrators, executives, and librarians have realized that measurements of institutional effectiveness are more necessary than ever. Library administrators need to present sufficient proofs how their library resources and services are planned and delivered in ways that maximize positive impacts on accreditation of academic programs, teaching and learning activities and research projects in colleges and universities.
In the current external environment, libraries are challenged to be nimble, innovative, responsive, proactive and, most of all, able to demonstrate their value. Libraries must be able to measure their outcomes and systematically make technology, budget allocation, service, and policy decisions based on a range of dataneeds assessment data, customer evaluation data, stakeholder expectation data, and internal process and organizational effectiveness data. Pressure to offer value-added service is mounting in intensity, and the rate of change is relentless. (Lakos & Phipps, 2004, p. 346-347) In ever-changing academic learning environments, an academic library' operations, sources, supports, and values must be associated with student learning outcomes promoted by a college or a university's mission. Therefore, this paper intends to share the Georgia Southern University Library's (GSUL) experience of LibQUAL; specifically, how an academic library performance quality and service improvement measurement could be improved via a process of collecting and analyzing LibQUAL survey data to identify patron needs and satisfaction.

LITERATURE REVIEW
In the service-oriented and user-centered academic learning environment, library assessment is a required process to measure a library's performance quality and service improvement while supporting the missions and the needs of an academic institution of higher education. What is library assessment? Defined by the online encyclopedia Wikipedia, a library assessment is "a process undertaken by libraries to learn about the needs of users (and non-users) and to evaluate how well they support these needs, in order to improve library facilities, services and resources" (Library Assessment, 2017, para. 1). In his book titled Library Assessment in Higher Education (2 nd ed.), Joseph R. Matthews (2015) indicates that ranges and scopes of an academic library assessment have been expanded to include student learning outcomes (i.e. student's achievements, experiences, and retention), teaching effectiveness, research environment, library as a place, and impacts on a college or a university's reputation. The purpose of his book is to provide "a broad perspective about assessment activities that occur in colleges and universities as a preface to investigating library assessment and evaluation options aimed at determining the library contribution to the 'holistic' success of an academic organization's mission and vision" (p. 3).
Among many library assessment tools, such as ClimateQUAL, LibQUAL+, LibValue, MINES for Libraries, StatsQUAL, etc., LibQUAL+ (http://www.libqual.org) has become a benchmark most widely used by many academic libraries in the world to measure their patron needs and satisfaction. Confirmed by the Lib QUAL.org, "Since 2000, more than 1,300 libraries in 33 countries have participated in LibQUAL+, including college and university, community college, health sciences, and academic law libraries." (Association of Research Libraries, n.d., para. 2). Apparently, this achievement has proven the global reputation of the LibQUAL+.
During 2007, LibQUAL+ is on track to be used to collect data from the 1,000,000th library user from the 1,000th institution! LibQUAL+ now has been used in the United States, Canada, Australia, New Zealand, the United Kingdom (England, Scotland, and Wales), France, Ireland, the Netherlands, Switzerland, Germany, Denmark, Finland, Norway, Sweden, Egypt, the United Arab Emirates, and South Africa. A Chinese version is being implemented in Hong Kong in the fall of 2007. Currently, the system supports 12 languages: Afrikaans, American English, British English, Chinese (Traditional), Danish, Dutch, Finnish, French (Canadian), French (European), German, Norwegian, and Swedish. The various editions of LibQUAL+ have been used over a period of eight years. (Thompson,n.d.,para. 10) Since 2000, thousands of research papers have been published regarding library assessments via LibQUAL surveys. Generally speaking, those primary published research papers on LibQUAL can be divided into the following five different categories: 1. The first category tracks the evolution of LibQUAL and other emerging library assessment methods and tools (such as Thompson, Cook, & Heath, 2000;Kalb, 2010;Roy, Khare, Liu, Hawkes, & Swiatek-Kelley, 2012;Savage, Piotrowski, & Massengale, 2017).
2. The second category explores the LibQUAL's factor structure, reliability, validity and unidimensionality associated with library service quality (such as Thompson , Cook, & Thompson, 2002;Thompson, Cook, & Kyrillidou., 2005;Morales, Ladhari, Reynoso, Toro, & Sepulveda, 2012;Fagan, 2014 Library literature using LibQUAL data supports academic library administrators in making critical decisions for enhancing and integrating their library resources and services. "The library literature regarding assessment tools, and individual assessment activities is vast, but library literature analyzing programmatic, effective assessment is not. Systematic assessment within libraries is gaining momentum but is still a nascent area of librarianship" (Lewin & Passonneau, 2012). Although there are a large number of research papers available focusing on academic library assessments via LibQUAL, but only few library literatures directly illustrate the process of customizing the LibQUAL Discipline Analysis and User Subgroup Analysis.
Therefore, the purpose of this study is to illustrate how to initialize and implement a process of information visualization based on the LibQUAL Survey Raw Data and User Response Data saved in Microsoft Excel files. The importance of this study is to assist other academic librarians to know how to use Microsoft Excel to analyze LibQUAL Survey Raw Data and User Response Data, if they do not have other content analysis software available.

METHODOLOGY
One of the strengths of LibQUAL+ is information visualization. Information visualization is a visual process of transferring abstract data to two-dimensional or three-dimensional graphs or objects to improve human recognition. Explained by Hsuanwei Michelle Chen (2017), "The views of information and visualization bring up two important aspects to consider: (1) information visualization is used to discover new insights and knowledge from abstract data through graphical means; and (2) information visualization can be considered a representation of data that amplifies cognition" (p. 6).
As we all know, LibQUAL+ is a web-based library survey to assess a library's performance quality and service improvement. A LibQUAL Survey focuses patron comments from three aspects: Information Control, Library As a Place, and Affect of Service. At the cost of US$3,200 for the registration fee, LibQUAL provides a registered institution with a standard LibQUAL+ Analysis Report, which includes final summary data, representativeness information, raw data files in .csv format, SPSS syntax files, survey comments, and Results Notebook. If library administrators and librarians intend to view more detailed information, however, LibQUAL will charge additional US$3,200 for Customized Discipline Analysis and US$3,200 for User Subgroup Analysis separately. Obviously, it is a heavy burden for academic libraries if they do not have additional operating budgets available. Measured by the LibQUAL's 22 survey questions and 3 scales (desired, minimum, and perceived), a well-known LibQUAL Radar Chart presented in this section indicates where the three dimensions of the GSUL's service quality have met or not met patron desires and expectations in 2016. From the perspectives of most of the GSUL librarians and staff, the LibQUAL User Response Data Report is more practical for them to understand how and where to improve their library service quality.
Georgia Southern University Library (http://library.georgiasouthern.edu/) participated in the LibQUAL Surveys in 2003, 2006, 2010, 2013, and 2016. Due to a stringent operating budget, the university library decided to use its own technical forces to engage in the User Subgroup Analysis, instead of paying an additional US$3,200 to the LibQUAL.org. After downloading the 2016 LibQUAL Survey Raw Data and User Response Raw Data from the website of LibQUAL.org, the university librarians would use the Microsoft Excel software to complete the following tasks: 1. Data Analysis: Data analysis refers to a general interpretation process of analyzing and summarizing the large amount of data collected by the appropriate statistical analysis method, extracting the useful information and forming the conclusion, and summarizing the data in detail. Data analysis can be divided into several different phases, including data visualization, data filtering, data mining, data visualization, etc.
2. Data Validation: As a part of data analysis, data validation is a process of ensuring that users enter qualified and useful data required by default business rules and regulations. In Microsoft Excel, for instance, data validation can be done by selecting "Data Validation" under the Data menu on the top, which will prevent users from entering an invalid input. Before a data analysis, data validation can also be done by checking and removing invalid data cells in a data set.
3. Data Migration: Data migration is one of the common business operations to transfer data from one system to another while changing data formats, data types, or data storage systems, if it is required. For academic librarians, data migration often occurs when they have to outsource Microsoft Excel files to other statistical software to generate interactive data visualization, such as Gephi, SAS, SPSS, Tableau, etc.
4. Data Mining: Data mining is a process for knowledge discovery to turn raw data into useful information. The purpose of data mining is to identify established and/or specific associations, patterns, structures, etc. in a large statistical data set. From the perspective of academic library assessments, data mining means to dig out hidden meanings from the collected raw data sets through the LibQUAL surveys.
5. Data Filtering: Data filtering is a way of controlling and processing data. With the assistance of specific computer software, data filtering only presents those data rows that satisfy the specified criteria and hide those rows that do not need to be displayed. After passing the process of data filtering, this new set of data can be copied, edited, formatted, and printed to create new charts, graphics, tables, etc. 6. Data Visualization: As a part of information visualization, data visualization refers to a technical process of converting abstract data to 2-D or 3-D graphics or objects. In various scenarios of academic library assessments, data visualization is a technical process of presenting statistical data in any graphical or pictorial formats, such as bars, charts, circles, columns, curves, hexagons, lines, pies, rectangles, squares, triangles, etc. This is a required step to assist academic administrators, executives, librarians, library staff, and other stakeholders to "see" data analysis in a different visual way.
Academic librarians and IT specialists who have knowledge and skills in Microsoft Excel, particularly with filters and pivot tables, charts, and dashboards, as well as Relational Database Management System (RDBMS) and SQL (Structure Query Language) can complete these tasks successfully.
To complete quantitative content analysis and qualitative content analysis, furthermore, three data analytics tools -Excel, SPSS (Statistical Package for the Social Science), and SAS (Statistical Analysis System)are widely used in academic library assessments. Many studies have been done to compare strengths and weaknesses among these conventional statistical software products. As Business Intelligence Specialist at Bell Canada, for instance, Angela Ju shared her insights while comparing Excel, SPSS, and SAS from the aspects of user friendliness, charts and graphics, analysis capability, and popularity. Based on her study, Angela Ju recommends that Excel spreadsheet is easy to learn and friendly to use. It has a clear advantage in basic descriptive analysis. SPSS is all you need if you wanted to minimize your data management efforts without formulating complex software programs. SAS is a likely choice for a person who manages complex data analytics on a continuous basis. Choosing analytics tool is a tradeoff between benefits and costs: if you only opened a statistic program three times a week, it is not worthy to spend hundreds of hours to learn SAS language; but for people who use SAS several hours a day, the steep learning curve and complex command structure are only a small price to pay for its extraordinary power. (Ju, 2015, para. 15)

RESULTS AND DISCUSSION
"All Roads Lead to Roma" is the way to interpret the hidden values from the collected LibQUAL raw data. Based on the methods we discussed previously, it is not difficult to assemble statistical data to present in 2-D or 3-D graphical ways. As you can see, the following is the LibQUAL Survey Raw Data downloaded for the Georgia Southern University Library. To help other librarians and library staff understand the related Excel column values, some reference notes are inserted on the top of the Excel file. After a process of data validation, rows with null values have been deleted. The row numbers in this Excel file range from Row Number 1 to Row Number 2290. How many Georgia Southern faculty and students participated in the LibQUAL 2016 survey held by the university library? To answer this question, we may use the COUNT function in Excel to get the number of entries in a number field that is in a range or array of numbers. So, we can type =COUNT(C29:C2290) to calculate the number of users who participated in the 2016 LibQUAL survey at Georgia Southern University. The result is: 2,262 (Note: the COUNT function must count from Row 29 under the Column C, since Row 1-28 under the Column C has no UserID). In addition, three columns in the above-mentioned raw data file have important meanings. DiscLOCA (Column H) refers to different colleges in Georgia Southern University. UGroupID (Column I) means user categories for Georgia Southern faculty and students. RoleID (Column J) are user ranks. Using the same COUNT function in Excel, we can calculate the number of faculty and the number of students separately. At the same time, we need to add data filters for DiscLOCA (Column H), UGroupID (Column I), and RoleID (Column J) separately, if we intend to view how many faculty members or undergraduates or graduates from which departments participated in this LibQUAL survey. The COUNT function will also be used to count the filtered data values. Because of the limit of the length of this paper, it is not possible to present each graph for the GSUL's 2016 LibQUAL survey. Except for the 2016 LibQUAL Survey Raw Data Report, many of the GSUL librarians and staff are more interested in the 2016 LibQUAL User Response Raw Data. Compared to those abstract statistical data, user comments from the real world will help librarians and library staff see many differences between the library service quality and the user satisfaction. Of course, some users may not understand that an academic library can only provide the best customer services at an affordable operating budget. Desired customer services will not exist all the time.
For librarians and library staff who are engaged in academic library assessment, they need to dig out the hidden values from the collected raw data. After a process of data validation, rows without any proper comments have been deleted. There are only 409 rows in this file. How do we identify user responses from this Excel file? The COUNTIF function in Excel will help us count text in Excel. For example, we can type =COUNTIF(E2:E409, "Business") to count how many responses we received from the business school. The number is 66. To illustrate the method how to interpret the related user comments, the GSUL's 2016 LibQUAL User Response Raw Data is listed here for your review: However, it is a little tricky to interpret user comments under the section of "textResponse" from the downloaded LibQUAL User Response Raw Data file. The COUNTIF function in Excel cannot count a specific text in a range of paragraphs (multiple strings in multiple cells). To solve this puzzle, one solution is to insert a new column (Column I) to the right side of the section of "textResponse" (Column H in this case). Then, using different keywords, such as "Complaint", "ILL", "Library", "Parking Spaces", "Praise", "Study Rooms", "Web Page", etc., to categorize different user responses. Next, using the COUNTIF function in Excel to count the data. By the way, the COUNTIF function in Excel is case sensitive. To convert abstract statistical data, the final process is to present the collected data in a graphic ways: Generally speaking, the library performance quality is still recognized. The library services offered by librarians and library staff are acceptable to faculty and students at Georgia Southern University. Their praises have formed a strong contrast to a few complaints regarding various library scenarios. Of course, there are some major issues waiting to be solved to further improve the library service quality. First of all, study rooms are the top concerns from Georgia Southern students. The library has to review this issue from mechanisms of making reservations, including policies and regulations regarding the scheduling conflicts, staff miscommunication, public service trainings, etc. Under an overbooked scenario, vacant library classrooms can be used as a backup for students who need presentation rooms or study rooms to prepare their group discussions or group projects. As a trial, the library has placed two Brody Private Study Stations to enhance students' library experience. Also, the library administration has been collaborating with Department of Custodial Services regarding the size of garbage cans and the noisy sound generated from carpet vacuum cleaners. As core functions to an academic library, at the same time, library collections and instructions have to be reviewed to match dynamic user needs. As for the size of the parking space, it will take time for the university library to negotiate a solution with multiple university departments, since it depends on additional funding support and the university's strategic planning.
Additionally, the LibQUAL Survey Raw Data indicates that 2,262 users participated the 2016 LibQUAL survey organized by the Georgia Southern University Library. However, the LibQUAL User Response Raw Data suggests that 408 Georgia Southern faculty and students, which are only 18% of the 2016 LibQUAL Survey participants, made their comments about the library information resources and services. Georgia Southern University has over 1,500 faculty and staff as well as 20,000 students. It is still a great challenge for the university library to encourage more faculty and students to share their experience and opinions in the future LibQUAL surveys.

CONCLUSION
The essential summary of academic library assessment is to support the library administration's strategic decisions. Through a visual process of transferring abstract data to graphic patterns, it is assessment librarians' primary duties to uncover the hidden truth in statistical data. In addition to Excel, SPSS, and SAS, other data visualization software like Gephi (https://gephi.org/) and Tableau (https://www.tableau.com/) are also useful to promote business intelligence and data visualization for academic library assessments. All in all, an academic library assessment is only a means to boost institutional effectiveness in ever-changing academic learning environment.