Home

Run Reports

Import Statistics

Manage Profiles & Lists

Reference

Survey Results w/ Data

Survey Results Summaries with RAW DATA

This survey solicited responses from the ERIL listserv (Electronic Resources in Libraries) of 1,567 subscribers. The survey was not conducted in a formal manner and no conclusions should be drawn or inferred from the summary of results that follow. The purpose of the survey was anecdotal - to provide an introduction to the current practices of some electronic resources librarians working with e-resource usage statistics.

29 survey responses were received between 14 July and 6 August 2004 and are included in the following summary, except where noted. When percentages appear, they reflect portions of the total number of respondents to that question only.

For further information, please see the main ERUS page, or contact Caryn Anderson at caryn.anderson@simmons.edu.

Quick Links to Survey Results

This page contains summaries with raw data of the results for each question. For results summaries only, please view the ERUS Survey Results Summaries page.

Executive Summary (View Summary)
Demographic Data
- 1. Organization Size (View Summary with Data)
- 2. Carnegie Classification (View Summary with Data)
Data Collection
- 3. Who collects e-resource usage data? (View Summary with Data)
- 4. How is it gathered? (View Summary with Data)
- 5. When is the data collected? (View Summary with Data)
- 6. Challenges with data collection? (View Summary with Data)
- 7. Ideal data collection procedure? (View Summary with Data)
Data Analysis
- 8. Additional analysis beyond vendors? (View Summary with Data)
- 9. More than COUNTER standard? (View Summary with Data)
- 10. What time frames do you address? (View Summary with Data)
- 11. Who are the summary statistics provided to? (View Summary with Data)
- 12. Challenges with analysis? (View Summary with Data)
- 13. Ideal analysis options? (View Summary with Data)
Utilization
- 14. What are the statistics used for? (View Summary with Data)
- 15. Decisions/actions affected by statistics? (View Summary with Data)
- 16. Challenges utilizing statistics? (View Summary with Data)
- 17. Ideal utilization of statistics? (View Summary with Data)
ERUS Model Review
- 18. What do you like about the ERUS model? (View Summary with Data)
- 19. What would you not be likely to use from the ERUS model? (View Summary with Data)
- 20. What is missing? What else would you like the ERUS model to do? (View Summary with Data)

Executive Summary

The Survey

This survey solicited responses from the ERIL listserv (Electronic Resources in Libraries) of 1,567 subscribers in support of a project to develop an integrated, user-friendly, web-accessible data repository for electronic resources usage statistics. 29 survey responses were received between 14 July and 6 August 2004 and and while the survey was not conducted in a formal manner and no conclusions should be drawn or inferred from the summary of results that follow, the responses provide a useful introduction to the current practices of some electronic resources librarians working with e-resource usage statistics.

The Respondents

The 29 respondents represented primarily academic institutions, though 2 corporate and 1 public library were also included. The academic institutions ranged from Associate Colleges and Specialized Institutions to Masters Colleges and Universities (I) and Doctoral/Research Universities (Extensive). The most responses came from these last two. The average size of the institutions represented is 13,654 (FTEs). Questions were asked about Data Collection, Data Analysis, Utilization of Statistics and feedback was solicited on a potential model for an integrated usage statistics system.

The Problems

The general sense from the results of this survey is that these 29 respondents are frustrated with the lack of standards in electronic resources usage statistics provision - both in the calculation of data and in the delivery of the reports. This is the most irritating problem, because it prohibits real comparisons of usages statistics across vendors. It also contributes to challenges in the extensive time and effort in collecting data as well as trying to explain the statistics to colleagues or subordinates.

The respondents are equally exasperated with the electronic resources vendors. They claim that vendors are not consistent in calculating their data, there are not a lot of easy to use interfaces, their data is not all COUNTER compliant (even when they say they are), the exported data formats are not consistent and often it is easier to just re-enter data manually, and the customization options for reports are not flexible enough to accommodate the needs of users who are importing the data into their own local applications which are necessary for interacting with other institutional applications (e.g. ILS, financial systems). In addition, vendors frequently change the way they calculate statistics and the data becomes inconsistent with itself over time. With some changes, data is not archived, and is therefore lost, and customer service has been poor for many respondents.

All of the above contribute to the other largest lament - the amount of time and complexity involved in gathering and analyzing usage statistics data. Because all the systems are so different, and because virtually all the data available needs some level of manipulation, and because the differences in data calculation methods and data units available requires alert and informed attention on the part of the data collector, the entire processing is deeply time-consuming. Add to this the combination of a lack of standards and questionable calculation methods by vendors and the emotional strain of constantly questioning the reliability of the data creates a stressful and somewhat unstable atmosphere around the whole endeavor.

To top it all off, once usage statistics have been collected, analyzed and presented to various personnel, many respondents report significant challenges in explaining the statistics to colleagues and getting them to actually believe the results.

The Solutions

The good news is that most of the respondents are pretty clear about what they want to see in an electronic resouces usage statistics system. They want standardization (compliance by all vendors), customization in small data units for easy integration with their local applications, automated data collection (or at least significantly reduced human intervention from the current environment) and delivery. They do not want to have to go and collect it all themselves.

As far as analysis goes, most of the respondents do not need/want much more than they types of data that the COUNTER standard recommends. But many already conduct additional cost-related analysis (e.g. cost per search) and if they don't, they would like to. They are also interested in having the option for more journal/title specific details and subject analysis of the statistics.

Virtually every respondent uses their statistics for making subscription decisions, while about half use them to improve marketing, promotion and library instruction. The statistics also provide a supportive role for budget justification in the case of half the respondents.

Most of the respondents seemed interested in the ERUS model and thought that it would serve a lot of purposes for them. Some were concerned about the ability to integrate such a system with their existing ILS and were wary about learning a new tool unless it was able to eliminate other programs they currently use. But overall, the concept of a single repository for all electronic resources usage statistics was attractive, and the added value of providing cost-related analysis, comparisons to peer institutions and subject analysis were interesting to many.

While this survey was not formal, it seems fairly clear that any usage statistics system ought to primarily focus on automating the data collection process and storing data in the smallest reasonable data units so that customized reporting is more feasible. Standardization, and compliance with the standards, are problems that can only be solved by advocacy with vendors and support of standardization efforts, but it seems possible that a tool can be developed to assist electronic resources librarians in managing usage statistics in the current, chaotic environment.

We are deeply grateful to those 29 who took the time and energy to complete the survey. Summaries of the responses to each question are listed below, with links to view the raw data. For further information, to be kept updated on the progress of the ERUS Project and/or to be included in plans for a pilot project, please e-mail Caryn Anderson at caryn.anderson@simmons.edu.

Top

Question Summaries

Demographic Data

1. Total number of students enrolled at your institution in FY04 (FTEs (full time equivalents), both undergraduate and graduate)

Raw Data

Respondent		Institution FTEs
1		3,500
2		24,249
3		8,976
4		15,000
5		4,500
6		27,384
7		6,057
8		400
9		21,756
10		25,000
11		6,000
12		35,000
13		15,090
14		7,880
15		19,659
16		8,090
17		1,100
18		11,113
19		12,418
20		8,388
21		2,000
23		19,237
24		12,589
25		10,852
26		17,345
27		16,000
28		35,694
29		7,035

Summary

28 respondents provided data for this question. Range and average includes two non-academic institutions that listed their organization size.

Range: 400 – 34,694. Average size: 13,654

Top

2. Your institution’s Carnegie Classification.

Raw Data

Respondent		Carnegie Classification
1		masters II
2		Doctoral/Extensive
3		Master Colleges and Universties I
4		associate's college
5		Doctoral/Intensive
6		Tier 1
7		Master's Colleges and Universities I
8		Medical schools and medical centers
9		Doctoral/Research Universities—Extensive
10		Doctoral/Research Universities—Extensive
11		Baccalaureate Colleges—General
12		Doctoral/Research Universities—Extensive
13		research extensive
14		Master’s (Comprehensive) Colleges and Universities
15		Doctoral/Research Universities—Extensive
16		Master's Colleges and Universities I
17		baccalaureate college - liberal arts
18		Baccalaureate/Associate's Colleges
19		Master's I
20		Baccalaureate College-Lib Arts
21		Specialized Institutions
23		Doctoral Research--Intensive
24		Master's II
25		Master's Colleges and Universities II:
26		D/R-Extensive
27		corporate
28		Doctoral Extensive
29		Master's I

Summary

Two respondents were not academic institutions and one respondent did not provide data. The remaining 26 were distributed among Carnegie classifications as follows:

Carnegie Classification	ERUS Total	ERUS Percent	U.S. Total	U.S. Percent
Doctoral/Research Universities – Extensive	8	30.8%	151	3.8%
Doctoral/Research Universities – Intensive	2	7.7%	110	2.8%
Master’s Colleges and Universities I	6	23.1%	496	12.6%
Master’s Colleges and Universities II	3	11.5%	115	2.9%
Baccalaureate Colleges – Liberal Arts	2	7.7%	228	5.8%
Baccalaureate Colleges – General	1	3.8%	321	8.1%
Baccalaureate/Associate Colleges	1	3.8%	57	1.4%
Associate’s Colleges	1	3.8%	1669	42.3%
Specialized Institutions (Theological, Medical, Engineering/Technology, Business/Management, Art/Music/Design, Law, Teachers, Other)	2	7.7%	766	19.4%
Tribal Colleges and Universities	0	0%	28	.7%
TOTAL	26	100%	3941	100%

There may be a variety of reasons why the distribution of respondents does not parallel the distribution of institutions in the United States as a whole, but no conclusions can be drawn here. The distribution, however, does provide a useful context to keep in mind while reviewing the remainder of the results.

Top

Data Collection

3. Who collects e-resource usage data? (e.g. on-site personnel? student worker? contract/temporary worker?)

Raw Data

Respondent	Data Collection Personnel
1	e-resources librarian
2	clerical staff, librarians, students
3	Associate Director of the Library
4	electronic resources librarian
5	On-site professional -- head of collections
6	Director, Electronic Reosurces Program, Assistant electronic Reosurces Librarian and Web-Tech in Systems Department for site hits for resources that do not provide usage information.
7	E-Resources librarian
8	on-site personnel; me.
9	This varies, but is primarily a student worker and a full-time staff member
10	Student worker (library school student) under direct supervision of a full-time librarian (the Coordinator of Electronic Collections).
11	The Automation Librarian
12	Assisstant librarians
13	a. part are collected through the state consortium which purchases many online resources. b. rest are collected by on site personnel (it's way too complicated for a student worker)
14	Head of Reference,Library Services Coordinator and Systems Administrator
15	Temporary worker
16	Electronic Resources Librarian and Periodicals/Acquisition Librarian
17	I do (I am head of technical services)
18	an on-site library technician, doing it as one of the duties.
19	Faculty/professional librarian
20	on-site personnel; electronic resources librarian
21	Electronic Services Librarian, Reference Library Assistant
22	Librarian III in charge of Acquisitions and Collection Development
23	students, faculty on Electronic Resources Reviews Committee
24	librarian on-site
25	Electronic Services Librarian
26	Assistant Director of Collection Management (Librarian)
27	on-site non-professional staff
28	On-site personnel and student worker
29	A full-time librarian with the title Online Services Director who is based in the Reference Department.

Summary

Of the 29 responses to this question:

Electronic Resources Librarians (11 / 37.9 %) - Eleven respondents indicated that data was collected by individuals specifically identified as e-resource librarians (various titles), of whom two were additionally associated with technical services.

Professional Library Staff Other (9 / 31.0%) - Nine respondents noted a variety of library staff as assuming responsibility for collecting e-resources usage data including professionals from Acquisitions, Collection Management, Library Services, Reference, Systems, Technical Services, Periodicals and at least one Assistant Library Director.

Non-professional Staff (7 / 24.2%) - Seven responses state that primary responsibility for data collection fell to non-professional staff including student, temporary or clerical workers in cooperation with, or under the supervision of, other library staff.

Unclear, but amusing (2 / 6.9%) - The nature of the roles of two of the respondents was unclear as they responded with "me" and "The Automation Librarian."

One respondent indicated that data collection was also coordinated with the consortium they were members of.

Top

4. How is it gathered? (e.g. downloaded from the sites? Notes taken from sites or e-mailes? Imported into spreadsheet or dateabase?)

Raw Data

Respondent	Data Collection Procedures
1	copied from website into spreadsheet
2	downloaded, emailed; imported into spreadsheets
3	Downloaded from websites, automated email from vendors, and specific email requests by the Associate Director
4	downloaded from vendor site
5	Frankly, a bit of a mish-mash; some are downloaded, some are e-mailed
6	All of the above: downloads, email, taken from sites and loaded into a MS Excel spreadsheet.
7	downloaded from sites, gotten off of emails, put into spreadsheets
8	Notes taken from sites
9	Downloaded from web sites and/or received as emails. HTML and .csv files are imported into Excel. In most cases, we related full-text data using a pivot table for analysis elsewhere in the spreadsheet file. We also collect our own click-through statistics when a user tries to access a product from our research database locator.
10	Whatever it takes. Our end product is a series of fairly standardized spreadsheets, but it takes a lot of human intervention to import the data into the appropriate cells. Even when the vendor provides the capability to import from a Web page or download as a spreadsheet, we still need to do copying and pasting to get it formatted to meet our needs. In many cases, it is simpler to print out the vendor-supplied data and enter it by hand.
11	downloaded from sites and imported into a spreadsheet
12	Imported into excel spreadsheet
13	Every vendor has a different mode of providing stats: downloadable databases, web based stats, email message sent on a regular basis, email messages sent occasionally, email messages sent only after a request
14	Downloaded from sites or from e-mail received
15	downloaded, via email, whatever way possible. Imported into a database
16	Downloaded from vendor sites and received via e-mail. Data is then put into a spreadsheet.
17	Primarily I retrieve statistics from the publisher's site. I print them and have a student enter the data into a master spreasheet I maintain that also includes circulation and in-house use stats for print journals.
18	downloading, imported into spreadsheet and database programs from vendor sites.
19	Gathered from vendor sites and entered into spreadsheets.
20	Stats are downloaded from sites or in some cases emailed directly to the electronic resources librarians. Stats are then entered into a spreadsheet.
21	Downloaded from sites or from vendor-supplied e-mails. Imported into spreadsheet or database
22	Downloaded or via email.
23	Downloaded from sites and emails; optimally input into Excel spreadsheets
24	notes taken from emails, faxes, and websites and then input into in-house database
25	Usually it is emailed to me, however there are several vendors that require me to login with a username/password
26	Downloaded from sites and imported into a spreadsheet
27	downloaded, copied, email, however we can get them preferred to download directly into excel
28	Downloaded from sites, e-mails and imported into spreadsheets.
29	We use a Quatro-Pro spreadsheet. But the data is not imported into the spreadsheet. It is collected by hand from the various admin modules and then keyed in.

Summary

Of the 29 respondents to this question, most identified data collection procedures as a "mish-mash" of downloading from vendor sites and copying from e-mails into spreadsheets and/or databases.

Some specifically noted the time and effort necessary to manipulate the data even when available in .csv formats. Two respondents said that they have found it easier to just print out reports from the vendor web sites and enter the data into their systems manually rather than fuss with the downloading and manipulation process. Two others mentioned integrating the vendor-provided data with statistics collected locally.

Top

5. When is the data collected? (e.g. annually, each semester, when requested)

Raw Data

Respondent	Collection Frequency
1	few times per year
2	on request for some; regularly for most A and I databases
3	Monthly
4	annually or as needed
5	As it becomes available; monthly in some cases, quarterly or annually in others.
6	monthly
7	when requested, when databases are under question, annually
8	when requested
9	Monthly. Ugh.
10	Monthly, wherever possible. SOme vendors do not supply monthly updates, of course, and in a few cases (with extensive title specific data on full-text journal collections, for example) we may only tabulate semi-annually.
11	monthly and annual basis
12	Each month
13	monthly by preference. We do an annual project to note missing stats and try to acquire them then
14	monthly or semiannually
15	Ongoing, but generally reported annually
16	Annually and when requested or when needed to review subscription status (e.g. to renew or not to renew...)
17	As they are made available by the publisher, though I am rethinking this frequency.
18	monthly or on demand from vendor sites.
19	Monthly
20	Monthly
21	Every few months
22	My goals state monthly; in practice quarterly.
23	when requested; now beginning to do so on a monthly basis
24	supposed to be monthly; more often it's closer to quarterly
25	Monthly
26	When available by vendor whether monthly, quarterly and annually
27	on a quarterly basis
28	Monthly
29	Annually. After the end of our fiscal year (which is June). Unfortunately, we sometimes fall behind and don't collect it together always in July.

Summary

Primary Data Collection Frequency:

48.3% (14 respondents) - Monthly
17.2% (5 respondents) - Intermittently (2-6 times a year)
13.8% (4 respondents) - Upon Request
10.3% (3 respondents) - Annually
10.3% (3 respondents) - As Available (M, Q, or A as vendor provides)

While nearly half of the 29 respondents collect their usage data monthly, many of these indicated that actual reporting may only be annually, or that the monthly collection is only for certain types of resources (e.g. indexing and abstracting databases). Of those that collect data between 2-6 times a year (the next largest group), two admitted that they were supposed to be doing it monthly but just have not been able to manage it.

Over 20% collect data annually or follow a data collection schedule uniquely conformed to that of each different vendor. Most of these respondents indicated that they also collect data whenever it is requested for specific purposes. There was also a small group that primarily collected data only on an as needed basis.

Top

6. What challenges do you encounter with data collection of usage statistics?

Raw Data

Respondent	Challenges with Data Collection
1	one problem is the ability for users to search multiple databases from one vendor at the same time; does that vendor count a search for each of the databases searched?
2	massive amounts of data; difficulty explaining differences in resources to students, clerks; (need differing data, get differing data, depending on resource)
3	1) Some sites provide data for the previous month on the first of the next month; others are months behind. 2)I have to maintain a list of instructions to myself for accessing statistics for each vendor. 3) Every vendor counts differently (in spite of insisting that they are COUNTER-compliant). 4) Terminology and definitions vary from vendor to vendor. 5) Vendors are constantly changing access methods, what is counted, definitions of what is counted, etc. 6) Slow email technical support--responses come weeks after I send email. (Telephone support is generally hopeless.)
4	comparisons between databases are difficult because each vendor uses different terminology, data points, etc
5	Variation of definitions amongst vendors (i.e., is a "search" in one database a "session" in another?), unreliability by some vendors, not enough data from others.
6	searches are counted for A and I resources, full-text retrievals for full-text reosurces, hits are ocunted when statistics are not given by provider. Keeping this ino straight in people's counsciousness is often hard. With the advent of htings like LOCKSS, Institutional Repositories, publisher-archiving it becomes increasingly difficult to synthesize a reliable track of access.
7	Different formats of presentation (emails, web files, admininstrator sites), differences in stats given (searches vs. logins, for example)
8	I never know how 'real' the hits are; or how trustworthy-IOW if the publishers are fudging.
9	1) Getting access to the data. We still have to nag many vendors just to find out how they make it available to us (or even IF it's available); 2) We have about 80 data sources at this time; that's a lot to track; 3) I inherited a number of spreadsheets from my predecessor which had been well maintained, but which were not consistently designed. I've been too preoccupied elsewhere to go back and clean this up.
10	Every vendor is still wildly different in their data tabulation and presentation methods, including those who are now COUNTER compliant. Also, we still get a fair number of vendors who have gaps in their data because of unexpected computer crashes at their end.
11	Sometimes it is hard to understand the statistical report
12	Registration process to get admin username. Obviously different editors offer different statistical data, e.g. IEEE gives us IEEE ASPP collection together with IEEE POP and we would like to get them separateley. Namely, ACM gives NONE!!!
13	Endless, endless challenges. Vendors don't provide; they aren't provided in increments which fit our fiscal year; they go back and change the data from years before; they change their definitions; they don't explain the basis of their reports (ESPECIALLY when they change all the figures!!); they forget to send; they change access to the web site where the figures reside and forget to tell you; they have technological issues relating to access which no one ever resolves; etc, etc.
14	Differing terms used that indicate the data is not standardized from vendor to vendor
15	inconsistent data and definitions
16	Many! A few of the biggest: 1. Lack of standards makes it impossible to compare use of different databases 2. No standards, guarantee, and lack of vendor disclosure about how use statistics are being captured. We have checked use stats one month, then checked the back a few months later and noticed that the previous months' numbers had changed.
17	It's time consuming. I question the accuracy of the numbers, especially since some publishers are considering implementation of use-based subscription charges.
18	change of format, change of collection method by vendors. lack of adherence to standards by vendors and inconsistencies of application of standards, preventing meaningful analysis and comparisons. format of statistical reports downloading not formattable.
19	Every interface is different, each has its own username and passwords, definitions. Not all resources provide stats. Some are way behind (2 months or more).
20	Many vendors use different terminology to indicate "searches." Counter is helping to make this more consistent.
21	Lack of statistics for some resources, unhelpful statistics (not detailed enough), statistics don't contain ISSN's making it hard to match on titles in database, many sites to visit (or e-mails to download) to gather statistics
22	One third of my databases report on the 1st of the Month; next few by the 8th of the Month; another group by the 10th of the Month; My Section's Statistics are due by the 7th of the Month.
23	translating the data,where possible, into useful, comparative information
24	not being able to have a student do it...too many differences among the labels, types, etc. Also, many sites include admin info with the usage stats module, and we don't give access to most librarians, let alone students, to admin modules.
25	Getting each vendor to give me the statistics in one usuable form, some are html, comma delimited text, excel spread sheets, or just plain text.
26	That they are not consistent across the vendors
27	statistics difficult to retrieve, difficult to compile because of different sources, different formats, different statistical units
28	Unreliable and/or irregular receipt, data elements not defined, different methods of counting from different vendors, missing data because of vendor problems, some vendors only supply quarterly or annual stats, some vendor don't supply any usage stats.
29	Vendors will upgrade their system and not always archive older data. If we're behind in collecting it, then we sometimes lose some of it altogether.

Summary

The volume of challenges identified was quite large as most respondents mentioned more than one. In order to reflect the fullest scope of the issues discussed, each separate issue identified by a respondent was assigned a category. Some compound issues were assigned to two categories.

Number of challenges listed by 29 respondents (grouped by issue, ordered by most prevalent):

Challenges	Category	Description

30	Vendor	Vendor challenges include complaints about vendors changing their systems (for data calculation, presentation, or even access to the statistical reports) too frequently, without advising clients effectively and/or without archiving old data; poor customer service response; statistical reports that are confusing; interfaces that are difficult to use; and general issues of trustworthiness (not presenting COUNTER compliant data although claiming to do so, and doubts about honesty of calculation methods).

17	Definitions	Definitions challenges include complaints about the sheer diversity of ways that data is calculated under different and similar terminology. The challenges counted here reflect the "what do you really mean by search?" types of questions.

7	Comparability	Challenges counted here include those where the respondent specifically mentioned the inability to compare statistics between vendors. If a respondent mentioned terminology or definition problems but did not identify comparability their issues was included in the Definitions category only.

4	Timing	Timing challenges were noted by those respondents for whom vendors do not provide data in time for the respondents scheduled reporting on statistics for their institution. (e.g. "My Section's statistics are due on the 7th of the month.")

4	Formatting	Formatting challenges include issues of poor formatting options, formatting options not working correctly, or the fact that different vendors don't all have the same options available.

4	Substance	Substance challenges include complaints about not enough statistics, not enough detail to the statistics or issues related to the cross-counting of resources available from multiple sources.

4	Procedures	Procedures challenges counted here reflect those respondents that complained about the complexity of remembering all the various instructions for accessing, selecting, running, downloading and manipulating data.

4	Explanation	Explanation challenges include complaints about how hard it is to explain all the diversity in statistics to colleagues or subordinates (e.g. "not being able to have a student do it..."), and also the poor descriptions by vendors of the precise nature of the data they are providing (some of these types of complaints were also counted as a vendor challenge).

3	Amount	Amount challenges counted here include complaints about the sheer volume of work involved in collecting, manipulating, analyzing and reporting on e-resource usage statistics.

The results above highlight the well-known dissatisfaction with the lack of adherence to standards for defining, calculating and presenting data (Definitions) and the resulting inability to compare statistics between vendors (Comparability). The data also reflect an apparently deep frustration with the behavior of vendors (Vendors).

Top

7. How would you like to see the process improved? What do you envision the ideal data collection procedure would be?

Raw Data

Respondent	Ideal Data Collection
1	downloadable COUNTER stats into an application which could extract and interpret
2	COUNTER compliance by all would be nice.; Ideally our data would be archived and available via the web from all vendors.
3	I'd like to have statistics available for the previous month by the fifth of the current month. I'd like them to be on a passworded web page. I'd like historical statistics maintained at least three years. I'd like standard terminology and definitions for search/query, full-text/PDF/HTML/, etc. I'd like to see changes in the statistics gathering and reporting process implemented--if not quickly--then at the beginning of the calendar, fiscal, or academic year.
4	standardization of terminology and data points
5	For ourselves, I think we'd have the easiest time with modifiable emails; in other words, if we could define on the vendor sites what information we want, how frequently we want it, and to what address we want them mailed, that'd be ideal.
6	There is no ideal collection procedure when you are attempting to gather statistics form resources that are inherently dissimliar!! At best, the process can be improved for each segment but not as an overall process. At best, this owuld become part of an ILS that allows to pull this information together with other needs bits of info such as price and access points.
7	I wish there was consistency among vendors in labeling and collecting of data. Also, consistency of distribution of data. I need stats broken down for on and off-campus use (not just one lump number).
8	ISO standard for publishers
9	My blue sky: 1. Vendors would use XML protocols SOAP and WSDL to enable automated downloading of our usage data 2. We would immediately import this data into a database, where we could start to use the data, rather than spend time and effort just formatting it 3. All vendors would agree what a "search" or a "session" is. Of course, as long as some vendors allow 1500 downloads of the same article without batting an eyelash and others freak and shut off access if they see 500 downloads from the same IP (even for different journal titles in a collection) from their website in an hour by a voracious reader who's simply gathering info for a long weekend, we won't really be able to compare products across vendors and platforms. But we might have a better sense about usage overall between vendors, which might be nice.
10	The ability to go to a vendor Web site and customize output for a spreadsheet to conform to your specific needs. By this I mean not only to specify the time period and frequency covered (as many vendors now do), but to secify which data elements you want to download and which you choose to ignore--as opposed to accepted their standardized formats.
11	The Counter project is good but very complicated. Standardization of statistical reports is essential. Simplifying those standards is very important
12	Same data gathering so that we can add the full text downloads from different vendors.
13	Regularity; predictability; conformance to the standards ICOLC has been trying to establish. Ideal: have good, detailed websites with downloadable excell databases which also have keys defining each stat
14	I would like to see all of the vendors use the same terms with standardized meanings to those terms so that all of the measurements would be consistent.
15	Easily downloadable in excel or comma delimited
16	Standards development for how use statistics are captured and reported.
17	"Ideal" to me would be a central repository where my institution's stats would be fed by all publishers. The stats would be imported into the centarl repository at the moment they became available. The stats could be downloaded from the repository into my local ERM, which could run reports against financials.
18	standardization of input and offering of various formats for downloading. Making it available to customers the option to select the ouput reports from a list of elements based on standards. the vendors should be more transparent with their change in the method of collecting usages, resulting in disparity of title usages presented by itself and being broken down within different databases.
19	Less time consuming.
20	I would like for all terminology to be consistent and all vendors to supply the same level of statistics. In addition, having a means to automatically harvest the statistics from all vendors each month would be a plus. Some institutions may already have created homegrown systems that accomplish this task, but our library does not have the adequate staff or expertise to do this at the moment. Having the flexibility to manipulate data within such a system for comparative purposes would also be a plus.
21	More statistics available; more detailed statistics, ISSNs for all e-journal statistics, fewer sites to visit
22	Automatic reporting that I can put into an excel spreadsheet format. Information on how many of the twelve databases have reported in.
24	One report from each vendor which lists number of searches and browses by type (basic, advanced, etc), number of article accesses by type (html, pdf), number of sessions, number of turnaways for each database to which we subscribe. I'd prefer these reports to be emailed to me so they all arrive at once, rather than me having to go to the website, put in my access info, find the necessary reports, gather the data...way too many steps when I have over 30 vendors! I wouldn't even care about the exporting if I could just receive all the data I need from all the vendors at the same time, preferably with identical data labels so I could pass it along to someone else to do the data input.
25	Pick one format (probably comma delimited text or excel) and have a consistent method for delivery - always emailed.
26	If they were all Counter compliant
27	ideal: all vendors using COUNTER standard statistics, downloadable into Excel, sent directly to staff person.
28	Ideally, vendor data mapped to local database without human intervention.
29	If there were a truly standard spreadsheet, then the data could conform to this standard and be imported with ease.

Summary

As might be expected, the ideal systems for 28 respondents are too diverse to effectively group. Themes that emerge include a combination of standardization, customization, and a delivery vs. retrieval model for data collection.

Standardization dreams include compliance by all vendors, simplification of the COUNTER standard, standardization of the delivery format as well as the presentation format (e.g. While the COUNTER report may appear the same on the screen, the delivery form is not always the same. For example, some vendors send .csv files as attachments that can be easily opened in Excel, while others send the ".csv" data in the text of an e-mail which cannot be effectively imported into Excel. Or the .csv version is displayed in a browser and must be saved from there.)

Customization of output beyond time period and frequency covered is desired. Respondents wish to be able to select data elements desired and ignore those unwanted, and/or to select in distinct data units so as to avoid transposing data and further manipulation in the local target application. A note on Integration: It seems important to recognize that standardization of reporting is only good up to a point as many respondents must use local applications for managing usage statistics because they ultimately need/want to integrate the statistics with other local institutional applications (e.g. "become a part of ILS" "run reports against financials").

Delivery vs. retrieval model of data provision refers to the desire to reduce the amount of time accessing vendor web sites to retrieve data. In combination with the key themes above, the simultaneous delivery of all data from all vendors in a standard and customizable format that is ready for direct import into a local application seems to capture the ideal.

Top

Data Analysis

8. What additional analysis do you do beyond what the vendors provide? (e.g. do you group the statistics? how?)

Raw Data

Respondent	Additional Analysis
1	cost per search, yearly change
2	We look at cost per hit for some resources. We evaluate turnaways relative to seats purchased.
3	We compare usage by individual database to determine if it is cost effective to continue to subscribe.
5	Minimal
6	For the 2 resources we have package deals, we review how the package is utilizied overall and whether the package deal works for us in this instance or not. We also get statistics from our electronic journal management system: TDNet and are look at aggregator overlap and which aggregatos are used most often when a title appears in more than aggregated service.
7	I look at trends of database use over a period of years, cost vs. number of searches, on vs. off-campus use.
8	weighting of numbers
9	We group the stats using ARL e-metrics if possible. We also compare click-through data with vendor-reported session data. Or we use click-through data when vendors don't supply any usage data
10	Where possible, we do an annual comparison based on two or three key measures, but of course those mesures are not available or defined in the same way from all vendors.
11	we do the following calculation: Highly used database; Lowest used database; cost/search search/user; cost/search/user; and many more
12	I wish we could. No time for this.
13	by vendor; by subject area; by year
14	I list the information by month in an annual report and list for each database what its percentage is as a part of the library's total database usage.
15	group by provider and by year
16	N/A
17	None, though I would if tehre was a way to associate use with particular volumes or years. For instance, if Elsevier reported that in calendar year 2003 Virology had 14 article downloads, I would like to know from which volumes these downloads occured. This would help me calculate more accurate cost-per-use figures.
18	make comparisons of usage of databases between vendors.; make sense of relationships between searches and results.; analyze title usage within each database and its relevance to the focus of that particular database.
19	Cost per use, use per major. Gather title level stats as much as possible and combine them to see which titles are getting used the most across all databases.
20	I calculate cost per search and rank the databases according to number of searches and number of documents downloaded.
21	Load statistics into database so e-journals from differen vendors can be compared; use spreadsheet to generate graphs
22	I've tried a few ideas:percentage of money spent for each database;comparison of average daily usage;comparison of average montly usage.
23	none
24	Our in-house database allows me to select which databases I want in each report, what time frame, etc. We also set up the database to put in cost data and can run per-search and per-article costs (including our per-search database costs in FirstSearch), based on database selections, time frames and/or certain criteria (subsidized by state, supported by consortia, etc)
25	Here I break these statistics down into my own excel spreadsheet. One each month has its own column. At the end is a total across. I also include the price that we are paying for this and breaking it down by month, I can determine how much each search costs us. The lower the price per search - the happier we are.
26	Nothing at the moment
27	We calculate price per article/chapter/record viewed, and create graphical displays.
28	Annual comparisons; cost per use
29	# of searches; total dollars spent; average price per search; average price per FT doc; ranking databases by number of searches and by cost; each database montly and as a percentage of the whole; comparison with previous years' totals; broken out by which ones are consortial and which ones we pay for directly.

Summary

Of the 28 respondents to this question, over 20% are not currently doing any analysis beyond the COUNTER level, mostly due to time constraints. Of the other 80%, there is a broad range of analyses being conducted, but three types of analysis are by far the most common:

* Ranking databases and other e-resources by volume of searches or other indices
* Annual comparisons to track change over time for each vendor
* Cost-related analysis including cost per search (most common), cost per article/hit, cost per download, search per user, cost per search per user

Other types of analysis reported by the respondents include measuring turnaways per "seats," on- and off-campus access comparisons, reviewing titles, ranking and comparing by subject and/or major, comparing vendor-provided and locally-collected click-through stats, ARL E-metrics and weighting.

Top

9. Do you summarize more than the sessions, session turnaways, searches, and items retrieved that the COUNTER standard recommends (e.g. do you track to deeper levels (journals), do you track stats for unique types of items retrieved when available?)

Raw Data

Respondent	Beyond COUNTER?
1	no
2	On request, we look at individual titles.
3	Once or twice a year we look at the full-text of journal articles used from all sources title by title, looking for duplication among databases and with our print subscriptions with the goal of cancelling to save money.
4	journal title-level stats for e.g. aggregator databases are useful for collection development. The in-house vs remote access stats are useful as well
5	On demand, but not regularly
6	We do track Journals for independent collection management studies twice yearly. We also do this for the print and compare with print usage.
7	I do not do this because I have no time to do this.
8	do you track stats for unique types of items retrieved when available? Yes.
9	We do try to gather number of downloads of full-text articles from ejournal collections
10	Depends on the data available and in some cases the requirements of our librarians based on unique characteristics of a product. For the most part, we focus on sessions, turnaways, searches, and articles and/or records viewed. If we purchase a product via a consortium or other buying group, and if the vendor allows, we like to compare our institutional usage to the group's--highest use, median use, etc. I especially ike JSTOR's capability of comparing our usage to peer groups (defined by institutional size).
11	No. But we look into the type of searches, ie, basic search vs. advanced search
12	Yes, tracking to journal level is essential for our institution. The "Pareto Law" (80/20) occurs therefore it is important to know which titles are at the top.
13	Another dept in this library is tasked with tracking journal stats; the ref dept tracks research database usage. I only am interested in number of searches, which seems to me a good common denominator which indicates actual use of the database. Number of sessions is not of interest - somebody might sit down and do one search in a session and a faculty member in the field might sit down for one session and do 25 searches.
14	No
15	we track "browse," "Circ" (meaning downloaded or viewed article or abstract level), and turn aways when possible
16	When available, we track unique types of items retreived, and always look at stats to see if there is some kind of unique/different snapshot of use available.
17	I track at the journal level.
18	track different modes of delivery, e.g. email, online display, search mode, formats, etc.
19	Journal title level when available.
20	No.
21	We track individual journals (where this data is available) in addition to summary information
22	no.
23	occasionally by journal title
24	I want to keep journal level stats, but we don't yet have the staff time to set that up, because if I'm going to do that then I want subject access, which adds a lot more complexity to the project.
25	No. Right now we are mostly interested in total number of searches. About once a year we may have an interest to see which journals are accessed.
26	We have basically done nothing with the statistics that we have received
27	Mostly we're just interested in number of log ins, number of searches, and especially number of articles/chapters/records viewed.
28	No; not usually, We only track individual journal usage when reviewing titles for cancellation.
29	No, because our Periodicals librarian gathers stats for journal titles and our acquisitions librarian handles stats for e-books.

Summary

The responses to this question appear to indicate that either the COUNTER project is on the right track in what it is choosing to track and/or it is helping to shape habits of e-resource librarians. Of the 29 respondents, over 40% do not track any statistics beyond those covered by the basic COUNTER reports. An additional one third say their institution tracks journal related statistics at least periodically, although sometimes it is another department that does it or cooperates. For the most part then, the COUNTER standard serves over 75% of these respondents well.

Additional types of statistics collected and analyzed by the remaining six respondents include detailed item retrieved information, basic v. advanced search, comparisons to peer groups where available (JSTOR), and modes of delivery.

Top

10. What time frames do you address? (e.g. monthly? daily? hourly?)

Raw Data

Respondent	Time Frames Analyzed
1	month
2	Varies. At renewal time we tend to look at the past year in total.
3	Monthly
4	monthly and annual
5	Monthly, more often than any other
6	monthly or a 2 week period
7	I am concerned with monthly stats.
8	annually
9	Monthly
10	We try to collect the data on a monthly basis and report it in monthly intervals, plus an annual summary.
11	only monthly
12	Monthly
13	monthly is my preferred, since it gives me flexibility in establishing a variety of annualized looks (fiscal year, but also the unexpected Board of Trustees reports, and so on)
14	monthly or semi-annually
15	monthly, then combined into an annual look
16	Usually monthly and then daily.
17	Monthly.
18	monthly, by semester and annually.
19	Monthly.
20	Monthly
21	Mainly monthly and yearly
22	monthly
23	monthly, daily
24	monthly is the base; I can run reports, though, based on any number of months or years
25	Monthly
26	Whatever is available from the vendor
27	quarterly
28	Monthly
29	monthly

Summary

Virtually all respondents answered "Monthly."

Top

11. Who are the summary statistics provided to? (e.g. library director? committee? senior institution personnel?)

Raw Data

Respondent	Statistics Recipients
1	library director; annual report
2	Director, electronic resource selectors, liaisons. Sometimes faculty request.
3	A summary goes to the director and more detailed information goes to our electronic services/collection development librarians
4	library director and administrators upon request
5	Library director, reference librarians (upon request), occasionally faculty liaisons to the library, consortium office.
6	Summary information goes to subjectl ibrarians and collection management people primarily. Some info is used on ARL stats packages but it is not reported up the chain unless asked for.
7	The stats are provided to other librarians as needs and yearly they are given to the library chair, library dean, and upper level administration.
8	intra-library Committee
9	Subject liaisons; the committee which selects products, the library director, ARL. Some numbers are also reported in annual report, etc., but to my knowledge this is just a sample; no one outside the library has interest in the full range of data we collect.
10	Available on an Intranet for all staff to view, though few do so. Primarily used by a committee to evaluate such decisions as renewal, cancellation, and simultaneous user license upgrades. We have a few librarians who make specific requests based on what the vendor provides, for example, data by type of search conducted.
11	The automation librarian provides the summary to the library director who also provides them to the VPAA on a semestriel and annual basis.
12	Serials librarian and it is made public on the university web every month.
13	Reported in the ref department annual report; goes to the library director; stats are often also used for reports to the university admin and to the board of trustees; stats are also reported to the bibliographers so they can keep track of discipline usage trends
14	library director
15	Any personnel who need them -- we have a web interface that is linked from our Intranet. Librarians may view as HTML or in excel.
16	Electronic Resources Librarian, Periodicals/Acquisitions Librarian, and Collection Development Librarian first. Then, the Collection Development Forum (an open committee). The Library Director when stats are requested. Also, stats are shared across our state-wide university system libraries.
17	Statistics collecting agencies and senior staff as needed/requested.
18	subject librarians,library director and higher levels.
19	Annual report, library personnel when useful. Mainly used for collection management.
20	Library Director for annual report purposes and the Electronic Resources Committee for decision-making purposes.
21	Some are displayed on the library's intranet and some distributed to selected staff in Reference and Electronic Services
22	1. Division Librarian 2. Library Director 3. Collection Dev. Committee
23	committee and interested librarians
24	library dean and others as needed/requested - except the cost stats. Only the dean knows and sees those. Too many would focus on that rather than the content needed to support all our programs.
25	We have a Dream Team (Digital REsources mAnageMent) that consists of librarians and administrators across our multicampus environment.
26	They are available to the entire library staff in a MS Outlook Mail public folder
27	collection development team, library management team and all library staff
28	Senior institution personnel and collection managers
29	director, all librarians, teaching faculty serving on the Library Committee, anyone who is interested. The spreadsheets are saved as HTML and uploaded to our website.

Summary

Percentage (and number) of respondents who have indicated that usage statistic data are provided to the various types of personnel or media listed below:

65.5% (19 respondents) - Library Director or Dean
58.6% (17 respondents) - Other library staff (reference, subject librarians, etc.)
34.5% (10 respondents) - Committee (within library and/or across institution)
34.5% (10 respondents) - Annual Report and/or senior institution administration
31.0% (9 respondents) - Collections department
24.1% (7 respondents) - Available on intranet or other public internal location
10.3% (3 respondents) - Faculty

Top

12. What challenges do you encounter with the analysis of usage statistics?

Raw Data

Respondent	Challenges with Analysis
1	most databases/resources will be used by users from more than one department/major; although I am frequently how much are we spending for e-resources for this department the answer is almost impossible to give except for very specific resources
2	Differences in output from different vendors. Differences in the resource itself.
3	Comparing apples to oranges is always problematic
4	Again, the inability to compare usage across similar types of databases is a problem
5	Hard to say, yet -- there isn't a long-enough time period to look over usage and see patterns, etc.
6	There is always a concern on reliability but that existed in print stat gathering is well. Just because something moved off a shelf doesn't necessarily meant it was used or looked-at. You have to make some basic assumptions. The hardest thing isp ulling together the inforamtion in the discrete packages that are often needed. This is getting easier with COUNTER.
7	Trying to do analysis of stats over a period of time is a nightmare in some database systems because of the methods the vendors use in distributing stats. It is impossible to do a comprehensive analysis of on vs. off-campus use because these stats are not available.
8	apples and oranges effect
9	In my case, time to do the analysis. At this point real analysis seems to be an afterthought. I can usually count on no more than 1/2 hour between interruptions.
10	One is never sure whether data are comparable, even when definitions are provided. Also, we have never had great success coming up with standardized ratios or measures that can aid in decision making, such as cost per use.
11	see question 6 (from 6:Sometimes it is hard to understand the statistical report)
13	The main challenge is getting everything. If I had the responsibility of tracking the journal stats I think I would have more analysis challenges. I resolved my analysis challenges at the start, have refined them over time, use them, and so far, haven't seen the need to go any deeper. Cost of the database's use is easy to figure if you have the actual cost of the database available - that might be consortial information, or might be a result of specific negotiating within each separate library
14	I feel like we are comparing unlike figures
16	Many, please refer to answers for question 6. (from 6: comparability, reliability of vendor calculation methods (changing)
17	Problem reported in #8 -- inability to associate use with year. Inability of determining a use of necessary versus a use of convenience. Also my suspicions that stats are or easily can be inflated to suit the purposes of publishers.
18	the unrealistic high number of hits which seem to be impossible to display during the sessions, e.g. Ebsco. the illogical timing of changes to the reporting methods from vendors during a conventional reporting period, e.g. one year.
19	Definitions. Comparison across disciplines or types of resources.
20	Figuring out how reliable the numbers alone are. In addition, I find it difficult to objectively compare quantitative usage statistics from different vendors because of the different terminology and definitions. Once again, COUNTER is helping to alleviate this problem.
21	Length of time needed to gather and process statistics. Lack of statistics or lack of title-level statistics
22	the time it takes to access all of the websites.Gale emails my statistics monthly. this is heavenly.There is still a lack of standard statistics...although it is much improved with COUNTER initiative.
23	finding time
24	time; assigning a resource type
25	Some vendors do not provide statistics so we end up using our proxy statistics as a benchmark instead.
26	Have not really analyzed them at this point
27	It's difficult to convey the info to a group
28	Comparing apples and oranges; changes by vendors in methods of counting make analysis less than valid.
29	Outlyer databases often need to be factored out of totals to give a more accurate result. For example, partial year totals, combined search databases, client-based software (SCIFINDER), extremely low-use databases needed just for one course, data lost by vendors, data not grouped monthly by vendors. This really should be a bigger box on your survey form.

Summary

The primary challenges for analysis for the 27 who responded to this question are similar to those for collecting the data:

Comparability Across Vendors - over 40% mentioned the "apples and oranges" effect. They complained of the fact that it is fruitless to try to compare data across vendors which are collected in different ways under different definitions for terms and processes.
Vendor Internal Inconsistencies - nearly 30% noted problems within resources. The most common problem mentioned was vendors changing their methods of calculating data units such that numbers are not consistent over time and therefore not comparable even within the same database. In addition, it damages the perception of vendor's reliability.
Time - Over 20% complained of not having enough time to do analysis at all.

Other challenges include comparing on- and off-campus statistics, understanding and/or explaining the reports, inability to verify type of use (e.g. "Just because something moved off a shelf doesn't necessarily meant it was used or looked-at."), and the difficulty/impossibility in disaggregating statistics to identify subject/academic department usage.

Top

13. What additional kinds of analysis do you wish you could easily do?

Raw Data

Respondent	Ideal Analysis
2	Cost per hit.
3	Hmmm?
5	I'd like to really be able to drill down to specifics -- which journals (and maybe even which articles) are people using?
6	This is som much easier in the e-realm than in the print realm. we can spot trends of use easier and have a better comprehension on what is used in ways we never could with print volumes. All statistics are suspect--we're just trying to get snapshots of usage really.
7	see above question. (from 11:Trying to do analysis of stats over a period of time is a nightmare in some database systems because of the methods the vendors use in distributing stats. It is impossible to do a comprehensive analysis of on vs. off-campus use because these stats are not available.)
8	ergonomical and cost/benefit. I know that's not very germaine, but I really would like to see how many resources are used or not, or struggled with/abandoned with respect to interface difficulties. I'd also like to know if the resources are truly worth the price.
9	cost per search, cost per session, cost per item retrieved. I'm sure others would occur to me if I could clear that hurdle.
10	We would like to create ratios that would compare internal user data, such as usage per number of enrolled majors, etc., but we've never been able to come up with a methodology that makes sense.
11	Some type of a system which could generate statistics of the databases in total instead of doing it one by one
13	see 12 above (from 12:The main challenge is getting everything. If I had the responsibility of tracking the journal stats I think I would have more analysis challenges. I resolved my analysis challenges at the start, have refined them over time, use them, and so far, haven't seen the need to go any deeper. Cost of the database's use is easy to figure if you have the actual cost of the database available - that might be consortial information, or might be a result of specific negotiating within each separate library)
14	As some journals are included in several databases, I would like to be able to know which databases are accessed the most for those journals.
15	Would like to tie to invoicing information to track costs over a multi-year period
16	Compare use stats from different databases, and have some comfort that the use stats received are accurate.
19	More subject-oriented things, as pulling out full-text journal use for accreditation reports.
20	I wish that those databases which are abstract or citation only would provide an equivalent to the documents downloaded (in other words, track the number of times a user clicks into a record in order to see the abstract). If you are measuring the potential usefulness of a database or relevancy of results to the user, in the case of a non-fulltext database, I think this number would be useful. In addition, I wish it were possible to see the number of users that come in through our proxy IP address and search. I know that we could track this at a very rudimentary level through the proxy server, but this only provides a minimal level of information.
21	Currently in the process of adding price information to database so cost-per-use info can be calculated for each journal
22	a better analysis of the percentage of money spent for each database
23	any kinds, to be honest
24	subject analysis
25	Mostly the price per search.
26	N/A at the moment
29	benchmarking obviously. Otherwise, I pretty much figure out how to do what people ask for.

Summary

Only 22 individuals responded to this question. As with question 7 (ideal data collection) the desires are varied. The most common types of analysis respondents wished they could do easily and do more of were:

Financial Analysis - Linking usage statistics to financial data is clearly done by some respondents as evidenced by question 8, but it remains the most popular item for additional analysis desired (27.2% listed it). Cost per search, per item, per session, per user - all were mentioned.
More Journal/Title Specific Analysis - Interests ranged from wanting to isolate which database is most used for which journals, to simply more detail on usage of individual journals. In the case of journals, this survey was not specific enough about soliciting feedback about the differences between statistics for journal usage through full-text indexing and abstracting databases and e-journal subscriptions (often tracked separately by the serials department). This is a topic for further exploration.
Subject Analysis - Interests included distribution of usage by subject generally and also a desire to track by major or academic department.

Other desires mentioned include better information on abstract only databases to match the tracking of full text databases, getting more detailed data from proxy server/click throughs, and, once again, dreaming of a more automated system for collecting and analyzing electronic resources usage statistics.

Top

Utilization

14. What are the statistics used for (e.g. deciding whether to drop/continue subscribing to a resource? Identifying resources to promote/market more? Comparing with in-house stats? scheduling support services/staff? making budget requests to administration?)

Raw Data

Respondent	Use of Statistics
1	need to market databases/effectiveness of marketing; decideding to drop subscription
2	Renewal decisions, promotion of resource decisions.
3	Deciding to continue subscribing to a database, determining how well a given subject is covered, noting user preferences, making PR decisions, etc.
4	stats are used for collection development decisions and for budget decisions
5	Recently, they've been used most to decide on subscriptions; if a journal is heavily used, we might keep the print volume…
6	Used in part ot flag resources to be marketed more, to review for cancellation (usage is never a sole factor), used in comparison to print usage, used to show that one platform is used mroe than another.
7	I look at stats to decide whether we should continue with a subscription, whether we should begin a subscription (in the case of FirstSearch Basic Collection), to prove that a department/subject area is active in searching databases (try to get more resources for that area), to try to support database usage by marketing these resources, and requesting funds for e-resources.
8	deciding whether to drop/continue subscribing to a resource identifying resources to promote/market more comparing with in-house stats scheduling support services/staff making budget requests to administration
9	Everything you've listed, with the exception of scheduling support services and staff
10	Primarily the first two. Except for the real outlyers (the most heavily used product vs. those used not much at all, it is difficult to draw conclusions. We have a huge number of products whose usage falls into that vague middle area. Also, realistically speaking, stats are often used after the fact to justify a decsion we have already made for other reasons. We have a few librarians that ask for very specific data, where available, for products important to them. For example, if a vendor provides stats with specific information on the most frequent keywords used in a search, or the top ten searches conducted during that interval, some librarians find that helpful for providing suggestions for library instruction.
11	deciding whether to drop/continue subscribing to a resource? and making budget requests to administration
12	for continuing subscription
13	is cost of resource justified; what are use trends and priorities; where are areas we might concentrate with library instruction, publicity, workshops, or other marketing; budget request support
14	Making budget requests and giving input to our statewide consortia from whom we receive most of our databases, preparing other reports
15	used to determine renewal, promotion of underutilized resources, budgeting, etc.
16	As one piece of information used when deciding whether to continue a subscription or not, budgeting, identifying which disciplines appear to use subject-specific databases most, and determining where to focus PR efforts - low use = more PR and training for a year.
17	Our stats have been used for collection development purposes, but this is rare. However, as more of the big deals become unbundled, I think stats will be increasingly more useful as a measure -- not the only one and certainly not the most important one -- to help us make decisions. We have also had occasion, as mentioned above, to report use stats to statistics collecting agencies.
18	evaluate the usefulness and relevance of databases. monitoring of key titles to various disciplines. complementing collection development work (add/drop) on print collection.
19	Collection management - deciding to keep/drop, or adjusting budget to more electronic in general.
20	Statistics are used to decide whether or not to keep a database (of course, we do not rely on statistics alone). We also use statistics to target resources we should market more heavily (especially if we know there is a need for the database, but people are not using it).
21	Just getting this process started but plan to use statistics to decide whether to drop or continue subscription or upgrade statistics (e.g. additional backfiles). Promote heavily used or underused resources
22	1. deciding whether to drop/continue subscribing to a resource. 2. identifying resources to promote/market more. 3. making budget requests to administration.
23	determind which subscriptions to maintain or to focus our advertising on; looking at possible print subscriptions to cancel; requesting budget support
24	right now, mostly for deciding to continue/drop and promotion efforts as well as using data for funding requests from the state for our consortia
25	The statist are used to see if the database is worth the money that is being spent on it. If it is a big ticket item that doesn't get much use then we may strongly consider dumping it. They are also used to show administrators that library resources are being used and to not cut our budget further.
26	1) To drop/continue subscribing 2) To add/drop number of simultaneous/concurrent users
27	deciding to cancel or renew, deciding whether resource needs more promotion.
28	deciding whether to drop/continue subscribing to a resource, making budget requests to administration
29	Primary -- identifying resources to promote/market more Secondary -- deciding whether to drop/continue subscribing to a resource Tertiary -- making budget requests to administration Quartiary -- accrediation reports

Summary

It seems very clear what the statistics are being used for. Virtually every respondent uses the statistics for making subscription decisions. These decisions include dropping or adding electronic subscriptions as well as print ones. Approximately half use statistics to uncover needs for increased marketing, promotion and library instruction. Half also use the statistics for budget justification.

Other uses include identifying subject coverage strengths and weaknesses and comparing with locally collected statistics. One respondent also uses statistics to help in scheduling library staff.

Top

15. What kinds of decisions/actions (if any) can you point to in recent years that were directly affected by your statistics?

Raw Data

N/A

Summary

Due to an error, unfortunately no responses were captured for this question. Some independent responses, however, indicated that usage statistics are never the sole determining factor in any decision, which makes it therefore difficult to identify specific results.

Top

16. What challenges do you encounter with utilizing your usage statistics?

Raw Data

Respondent	Challenges in Utilization
3	I'd like to be able to complete my monthly report on database usage in an afternoon with data for all databases. Instead I have to keep checking the availability of the data and usually end up omitting data for the current month for some vendors.
5	Faculty understanding of the importance.
6	see above (from 6: searches are counted for AandI resources, full-text retrievals for full-text reosurces, hits are ocunted when statistics are not given by provider. Keeping this ino straight in people's counsciousness is often hard. With the advent of htings like LOCKSS, Institutional Repositories, publisher-archiving it becomes increasingly difficult to synthesize a reliable track of access.
7	Wrestling them into a usable format. The vendors send a mix of different types of figures.
8	weighting results
9	Well, some staff simply don't believe them. At one meeting, a liaison made the (valid, in my view) point that nearly every set of stats needed an asterisk next to it.
10	See #14. I think the biggest challenge is detrmining what the numbers really mean and whether we can be confident that they provide useful data that can contribute to the decision making process.
11	see question 6 (from 6:Sometimes it is hard to understand the statistical report)
13	There are a lot of other factors which go into these decisions. One thing which always make utilization somewhat difficult is that so many vendors just go back and completely change whatever they reported as use levels for previous years! I have seen this happen with major vendors (Ovid, for example) and with lots of little ones who don't necessarily have a lot of technical expertise in all this tracking in the first place
14	I wonder how reliable these figures are.
15	none
16	Cannot compare stats between databases, user interfaces for retrieving web stats are not usually very good. See additional info under question 6. (from 6:Many! A few of the biggest: 1. Lack of standards makes it impossible to compare use of different databases 2. No standards, guarantee, and lack of vendor disclosure about how use statistics are being captured. We have checked use stats one month, then checked the back a few months later and noticed that the previous months' numbers had changed. )
18	inability to determine one set of statistics is comparable with another based on consistent definitions and method of collection.
19	Lack of consistency or authority - if they don't jibe with what our reference folks think, they don't want to believe the stats.
20	Collecting usage statistics and putting them into a meaningful format can be very time consuming.
21	Lack of statistics for some resources, unhelpful statistics (not detailed enough), keeping statistics current is hard when they take so much time process
22	Time it takes to consolidate. Explaining differences in statistics to frontline staff.
23	explaining the language and limitations of use
24	politics; having time to collect and analyze them
25	I cannot think of one.
26	Have not really used them to answer this question
27	difficult to convey info to a group (management team, coll development team). Detailed info on all resources in our collection (>20 excluding e-journals) is overwhelming. Inconsistent statistical units between resources makes comparing difficult.
29	Finding the time to do them.

Summary

23 responses were received for this question. While some respondents noted the now familiar refrain of problems of comparability and vendor behaviors like changing methods and general reliability, there were two issues that stode out noticably.

Time - Over 39% of the respondents indicated that time and the complicated procedures for collecting, analyzing and distributing the electronic resources usage statistics were their chief challenge in utilizing the statistics.
Explaining Usage Statistics to the Frontline - This issue turns out to be the most significant one. Over 43.4% outlined this problem in a variety of ways. Some mentioned general politics, some discussed difficulties in getting faculty to understand what the e-resources statistics say about their programs. Many mentioned general difficulties in explaining the statistical reporting of different vendors with all of their quirks. In some cases, the staff simply refuse to believe the statistics. This apparently occurs for at least two reasons: 1. the plethora of special conditions surrounding the statistics damages the credibility (e.g. "a liaison made the (valid, in my view) point that nearly every set of stats needed an asterisk next to it."); 2. inconsistencies (e.g. "if they don't jibe with what our reference folks think, they don't want to believe the stats.")

Top

17. How do you wish your usage statistics could be utilized?

Raw Data

Respondent	Ideal Utilization
6	I just wish subject librarians would pay more attention to usage.
7	I wish, esp. at the end of the year, that the stats from the different vendors were funneled into a central web distribution point automatically, differences between vendors accounted for, and the stats were available in this central web distribution point. So, I could request stats on a couple of databases - or all databases, and a report could be generated. Tables and graphs could be generated from these reports - Automatically - without having to import data into excel.
8	I wish it could be used automatically. Like a running score with machines querying and modifying and summarizing.
10	I would like to see more creative use of the data by our front lines librarians, especially to give the a better idea of how the products are actually being used (or misused). To put it another way, we tend to focus on the stats for budgetary and cancellation purposes. I would like to see them used more to improve library instruction. See answer #14.
11	To disseminate those statistics to faculty members and dept. deans and chairs
13	I think we are on the right track. Increased reliability would be a help, but some of this is teaching administrators (library or university) how / why to use such reports. This is a longer education process.
14	I wish had reliable figures from all of the databases so that a truer picture could be presented of our use
15	again, need to tie to invoicing information
16	To compare usage between all databases, cost per search info, determining heavy use periods, which databases are used most for which subject areas - the possibilities are endless!
17	I wish they were an accurate measure of curricular need.
18	wish that the guesswork could be taken out the exercise.
20	I would like to continue using the usage statistics as we have been, however I would like to be assured that what we are collecting is an accurate reflection of what is really happening with our users.
21	Plan to track cost-per-use info and use statistics to decide whether to drop or continue subscription or upgrade statistics (e.g. additional backfiles).
22	To do the following faster, better and more accurately:1. decision to focus marketing efforts 2. decision to increase simultaneous user level 3. decision to keep current databases
23	to gain more support staff
24	would definitely like subject analysis
25	I cannot think of one.
26	To be uniform/conssitent and to compare with our print usage statistics
29	To justify doubling our budget.

Summary

Only 19 responses were recorded for this question. Four respondents (21.0%) wished that faculty and staff would take more of an interest in how the statistics can help them improve their own work. Others longed for greater data reliability (15.7%), better integration for financial analysis and budget justification (15.7%), and the possibility of real subject analysis (15.7%).

Top

ERUS Model

The following outline of the proposed ERUS model was reviewed by respondents in order to answer questions 18-20:

ERUS Model

Administration

Each institution will use a simple web form to input their initial profile, detailing institutional demographics, selecting resource subscriptions, etc. (similar to the set-up for a product such as Serials Solutions or SFX). An institutional representative can update the profile as appropriate (when demographics or contracts change).

The ERUS project will maintain a central e-resource statistics depository to enable review of products and statistics:

Indexed by subject – subjects also linked to degree programs to enable program specific queries
Indexed by resource types (e.g. bibliographic citation database, e-books, online reference (encyclopedia), and ARL E-metrics categories)

While the primary purpose of ERUS is to manage statistics, institutions can also choose to simplify subscription management by storing some subscription details in this central, web-based database (including contacts, URL, admin passwords, and basic contract info including price, usage restrictions, renewal dates, dbase updates, etc.).

Data

Usage statistics will be updated on a regular basis depending on the practices of each specific vendor. The following data will be available for analysis on a monthly basis, tracked to the database/journal/book level when possible:

Sessions
Session Turnaways
Searches
Items Retrieved

These statistics comply with the COUNTER standard and will be made available at the degree to which the various vendors provide the information (regardless of whether it is provided in the COUNTER report format or some other way). For vendors that offer further detailed statistics than the COUNTER standard, links to the appropriate sites, or e-mail contact information, will be provided.

Analysis

The above data categories can be analyzed by:

Time period
Subject (and/or academic program)
Resource type
Cost per search
Comparison to average of peer institutions

Response to model:

18. What do you like about the ERUS model?

Raw Data

Respondent	ERUS Model Elements Interesting
2	Not clear whether the usage restrictions area would be useful. It needs detailed data on: ILL; Ereserves; Printing, downloading, copying restrictions; Proxy information (list of servers?)
3	This all looks very interesting.
4	Does this mean that someone on your end will collect our statistics? If so, and if they're standardized, that would save me time. cost per search would be nice too.
5	We use something similar to this in our consortium, annually. Frankly, I don't think we look at it closely enough.
6	This is way too ambitious...good luck!!
7	This system sounds great. Almost like my dream system.
8	That it might be integrative.
9	The overall concept.
10	Analysis by subject and resource type. Comparison to peer institutions.
11	Comparison to average of peer institutions; Cost per search
13	We already have the means to do the first 4 and through our state consortium can do the last for state institution comparisons. I would really like the possiblity of being able to select our national peer institutions (by name) and using their stats as comparisons as well.
14	I think it looks good.
15	looks good
16	Sounds like a fantastic start.
17	I like the repository model, and the idea of vendors delivering the stats to the repository without library mediation.
19	Sounds good but awfully tech-heavy - I don't envy whoever's going to do the work to put all this together!
20	I think that this would be an excellent tool that the entire library community could benefit from. I especially like the ability to compare your data with that of peer institutions. The subject area analysis will also be helpful.
21	Idea of one place to go to for e-resource statistics
22	Analysis by subject [could this be DEWEY]
23	All of it, especially the cost per search and comparison to peer institutions. I also like the data provided on journal usage and the linking to data supporting programs as we're trying to support social sciences and humanities more (when possible).
24	comparison to peer institutions would be interesting
25	Data. In analysis the idea of comparison to average of peer institutions sounds really good.
26	Everything
27	I love the centralized location, and simplified data. Sounds great! We'd pay for it, as we currently spend about 15 hours per month on use statistics.
28	Analysis by subject, analysis by resource type (we don't do that now), all of your suggestions seem useful.
29	comparison to peer institutions

Summary

Many of the 26 respondents were generally pleased with the overall idea. In addition, a few identified specific things they liked.

9 (34.6%) liked the comparison to peer institutions
5 (19.2%) were interested in the subject analysis and resource type identification component
3 (11.5%) specifically mentioned the cost per search functions, and
3 (11.5%) were most pleased with the whole repository concept

Others were also intrigued by the integrative nature of the project and the possibility that they might be relieved of some of their data collection responsibilities.

Top

19. What would you not be likely to use from the ERUS model?

Raw Data

Respondent	ERUS Model Elements Uninteresting
2	We ideally need a system integrated with our ILS. Why enter cost data twice?
6	You need to count full-text access not just session access.
7	I would use most all of the features. Possibly I wouldn't use the comparison to average of peer institutions..
8	anything about it that costs money
9	Well, as we will be working on ERM ourselves this year, I'm not sure I could find time to manage yet another tool. I would be curious what the analyses looked like, even if we haven't used them in the past or recent past
10	Nothing I can think of.
11	Resource type
13	I might use such a database but we already have the means to do the first 4 and through our state consortium can do the last for state institution comparisons. If this is easier and less cumbersome in collection for us, I would be glad to use all elements.
14	I think we could use all of these capablities.
15	all would be useful
16	Probably time period due to a user's tendancy to walk away from a station with a session still active.
17	I think I'd give it all a try. Nothing seems useless to me.
19	I could see using all of it at different times.
20	I think that we would use all of this data.
21	I know products like Serials Solutions are sometimes restrictive in limiting which resources can be handled by the product and how much the data can be customized. I would want to see what the restrictions were on this product.
22	nothing
25	Subject, resource type, items retrieved, For the most part we aren't interested in Indexed by subject or resource types.
26	I would use all of it
27	Some of our vendors are "non-traditional" such as ChemSystems/Nexant and SRI. Would the vendor lists be limited?
29	storing passwords

Summary

Only 20 responded. This may mean that the other nine would have used all of it, or simply chose not to respond. Of those that did respond, nine (45%) indicated that they would use all of it. The other 11 were concerned with whether it would integrate with their ILS or if the vendor lists would be restricted? And some were wary of learning and managing a new tool when they are managing with their own systems at an acceptable level - the ERUS system would have to do everything they are doing already and better.

Top

20. What is missing? What else would you like the ERUS model to do?

Raw Data

Respondent	ERUS Model Elements Missing
2	Field for consortial purchasing info; Field for tech support contact; Field for Acquistions notes (invoice approved, renewal agreement sent, license in negotiation, etc.); Ticklers or alerts for soon to expire resources, otherwise we would have to continue to maintain a separate spreadsheet
6	You need to focus just on usage not include access stuff...you're over-extending into the realms of an ILS and I don't think you want to re-create an ILS, do you?
7	I still would like to be able to make graphs, charts without having to use excel. I would hope that it's easier to set up than SFX.
8	the word free
9	It would be really nice if this didn't replicate the mistake lots of vendors make by not allowing access to multiple staff. As it is, I have to collect data because usually the username/password to retrieve stats is the same as the admin interface for the product. I'd like to give an interested liaison direct access to as much data as they can stand (much like SerialsSolutions), rather than feeling like a prison guard. Can I assume you'd do this using open source software, made available using an open source license, so you could share the code without having to face licensing restrictions? Or are you considering this to be a pilot for a commerical service?
10	We'd love to have a demographic component--who is using by category: sfaculty vs. grad student vs. undergrad; off campus vs. on campus, etc. But I don't see how that is possible with your model.
11	search per user; cost per search per user
13	specific institution comparisons
14	I would like it to display remote usage versus usage within the library
15	interlibrary loan abilities; tied to my integrated library system; alerting ability to let me know that I should be expecting a renewal notice or invoice
16	A statement of standards to help with comparing, collecting, and analyzing stats.
17	I'd like the system to be open and provide for easy download/import of files so spreadsheets, local databsaes, and commercial ERMs can get at the data.
21	I would want to be able to download locally collected statistics and have them match on data in the system
22	1. An element in "Subscription Details" that lets you document HOW your subscription price is calculated....by Population, FTE, searches, Size of Institutions, etc... 2. "Institutional Demographics" section at the top of the survey does NOT allow for Public Library use. We use population size instead of FTE's, AND we do not have an equivalent of the Carnegie Classification. 3. Please explain how secure our passwords, etc would be on this site.4. There should also be space in the Administration page for multiple contacts. sometimes Vendors have multiple contacts, addresses, phone numbers, etc...For ordering, maintenance, etc..
25	Unless the data would automatically be dumped into a spreadsheet by month and automatically calculate cost per use. Also, if it could automatically be formatted to show the data broken by fiscal year and calendar year that would be even better!
26	Not able to comment at this time
27	graphical features would be nice
29	A useful distinction for us is what it paid for by our statewide consortium and what we pay for ourselves. Another distinction is between subscriptions and the dwindling number for which we pay by the search.

Summary

"the word free" said one respondent. "I would hope that it's easier to set up than SFX," said another.

Two mentioned graphical features. Two asked for more administrative area for contacts, passwords, etc. while one suggested that less administrative functions would be better so as not to duplicate an ILS. Two wanted to ensure integration with existing ILS. Two wanted a breakout of various pricing strutures.

Other items and components the respondents would like the system to do include: created in open source technology, including a demographic component, tracking remote vs. in-library usage, including a statement of standards, and options for tracking consortium subscriptions separately.

Top

For further information, please see the main ERUS page, or contact Caryn Anderson at caryn.anderson@simmons.edu.