Posted on

Data Mining and OPAC Usage Data

I’m working with someone in our IT department to look at our consortial OPAC usage data. This is really at the edge of my abilities so I’ll be rather blindly documenting that process here – hopefully it will be interesting and helpful to someone but I’m certainly not an expert in this area – at least not yet!

Tentative Process

  1. Download logs from CARLI.
  2. Creator Decoder.
  3. Design database
  4. Write parser thing
  5. Do basic analysis in MySQL.
  6. Do more advanced analysis in Weka.

We are going to use the program Weka to do this bit of data mining and I’ve installed this on my local machine. It was pretty easy to install and download – so that was nice. Here is the link: http://www.cs.waikato.ac.nz/ml/index.html

 Research Questions

These are the basic questions that are currently guiding my database design:

  • Basic numbers for the searches – how many?
  • How does VuFind compare to Classic Search?
  • Types of searching (title, keyword, author)
  • Search terms – what are the most popular? What are common misspellings?
  • Platform and OS?
  • What formats/filters/facets are applied?
  • Time of day?
  • Mobile vs. Desktop?
  • Look at Curriculum Center headings (start with a heading and see if it is used).
  • Number of search terms (how many terms in a kw search, for example)
  • Spelling Errors
  • Failed searches (no results)
  • Clicked vs. type subjects (i.e. what happens when someone clicks a subject heading?)
Posted on

E-Resource Workflows in CORAL

A while back, I read the book “The Checklist Manifesto.” It was a very worthwhile read that was very relevant to libraries and librarians. That has been in the back of my mind as I craft workflows for our electronic resources. The books main premise, as summarized by Steven Levitt is simply this:

No matter how expert you may be, well-designed check lists can improve outcomes.

Two small details from that book have stuck with me.

First, useable workflows are short. Long checklists seem doomed to fail – it is better to break those up into smaller, more manageable lists. Many of the books examples are drawn from the airline and health care industries; it would be crazy to craft comprehensive workflows in those situations. So they have checklists for particular situations that, if need be, direct you to other checklists. I’ve adopted and adapted that idea here.

Second, checklists can either be READ-DO lists or DO-CONFIRM lists. READ-DO checklists are like recipes; you read something and then do something. DO-CONFIRM checklists reverse the order; do the things that need to be done and then confirm they are done. I think too many library checklists are READ-DO checklists when I believe that DO-CONFIRM lists are better suited to this part of library resources. Why? Because the diversity of library resources sometimes demand different things and because, often, the order of steps isn’t vitally important. I could be swayed on this – but right now I’ve been happy with my DO-CONFIRM checklist.

In CORAL, workflows are set for Acquisition Type, Format, and (optionally) Type. All of these fields can be defined in the admin setting of the resources tab. I’ve decided not to really discriminate based on resource type or resource format. So, essentially, we have four basic workflows for electronic resources. They are as follows:

Active Trial/Electronic/[Any]

This is the starting place for all new electronic resources and is the basic workflow when setting up a trial.

  1. Resource information is complete in CORAL.
  2. Organization information is complete in CORAL.
  3. Collection decision has been made.
  4. Appropriate Access Points have been determined and recorded.
  5. Pay invoice; gather license; change Acquisition Type in CORAL.

Paid/Electronic/[Any]

If after the trial period we decide to purchase  resource, I will change the acquisition type to “Paid” – this will trigger a number of different CORAL workflows based on the resource type. Here is the basic and general outline.

  1. Resource and Organization information is complete in CORAL.
  2. A copy of the invoice is saved in CORAL.
  3. A copy of the license is saved in CORAL and has been parsed for ILL rights.
  4. Appropriate access points have been created.
  5. Access has been checked from both on-campus and off-campus.
  6. SUSHI (or alternative statistic measuring tool) is in place.

Free/Electronic/[Any]

I’ve decided to add all of the free resources that North Park provides access to in CORAL for a number of reasons. First, I’d like CORAL to be a centralized and complete inventory of all our electronic resources. Second, if we are providing access we need to commit to supporting that access. Third, because so many valuable information resources are moving forward open access models.

  1. Title information is relatively complete in CORAL.
  2. Applicable access points (Catalog, Database pages, Libguides, SFX) have been created.
  3. Check access points.

Additionally, a few of these resources are actually cancelled print+online titles that guarentee perpetual access. Because the workflows are identical, I’d added these to this category with notes detailing the acquisition history. They are identifiable by having a resource type of “ejournal (single title)”. NOTE: If we start adding many open access titles to our list, I might have to specify what sort of “free” we are talking about; free because of a past subscription or free because it’s open access.

Cancelled/Electronic/[Any]

Right now, our “Historical” Acquisition type is for cancelled titles are trials that we decided not to purchase. In addition to changing the status as “archives” we have a one step workflow for these resources.

  1. All Access Points have been removed, disabled, or deleted.

Additionally, I note why the product was cancelled and/or why we opted not to purchase the product.

So that’s it. I have other workflows for the more detailed processes (adding a title to SFX, setting up SUSHI, etc.) but those exist best as separate workflows.

Posted on

An Archon (and data) manifesto

Last Saturday I was asked to give a short overview of Archon to the Preservation Committee of the Swedish American Historical Society. It was mostly a hands-on demonstration of how to use Archon and not a theoretical exercise about the transformative role of technology in archival metadata. However, the basic question I was trying to answer was “Why does this matter?” or perhaps “Why is this important?” and those seemed like questions worth answering in a compelling way.

Structured Data > Unstructured Data

By this I mean simply that data that is organized into formal structures (whether MARC, Dublin Core, EAD, whatever) has clear and distinct advantages over unorganized and unstructured data. This project began with loosely organized and structured data – paper based finding aids – but migrated that into the more formal structure of Archon. This had several advantages:

  • Ability to standardize language and make the data more uniform.
  • Ability to set clear metadata standards and systematically improve and add metadata.
  • Ability to migrate to new and differnt systems.

The main advantage of this structure vs. unstructured data is so important to me that it deserves it’s own heading

Linked Data > Unlinked Data

Structuring metadata in an online system allows for a paradigm-shifting revolution – that data can be linked to other pieces of data. I’ve been thinking about this for a while – and will likely continue to think about this – but this is the power of the paradigm shift. Data is linked to other data – thereby linking thoughts, ideas, people, etc. Linked data is fundamentally different than unlinked data.

Typing a subject heading onto a card catalog is fundamentally different than typing a subject heading into a linked database. The question when typing a card catalog seemed to be “what will describe this item to our users?” whereas the fundamental question when created linked data is “how can I link this item to similar items in ways that will be helpful and meaningful?” and that is a huge distinction in my mind. Thinks like controlled vocabularies, conforming to national standards, etc. all make infinitely more sense when linked data is viewed from this perspective.

More Access > Less Access

This perhaps seems obvious, but it’s worth mentioning. More access is better than less access. Connecting people to information resources is the most important work of the library and of archives and providing more access seems nearly undeniably better than providing less access.

I will almost certainly add to this later on (as my views on technology and metadata continue to evolve) but I wanted to take a moment to document this particular moment in time,  capture my current thoughts on the subject, and rely a bit of what I wanted to convey to the Preservation Committee of the Swedish American Historical Society.

 

 

 

Posted on

Getting Started with Archon, part 1

I’ve finally started uploading out finding aids to our CARLI hosted install of Archon and am really enjoying this whole process. Simply recreating the finding aids we currently have (as word documents) in Archon has gone pretty well.

I’m manually creating the Collection group from the current finding and then uploading the Collection Content (Box and Folder list) in small batches in CSV form via the Database Management tab.

One thing I’m noticing is that most of these finding aids have a long (very long!) arrangement section and no scope or description as such. It’s very rich metadata that I don’t want to lose, but it would be nice if there was a shorter description and scope note to aid researchers. Archon prioritizes these fields by having them display on the main landing page and that makes it important enough to consider. I’ll add that to the “next step” for Archon that I have planned.

One small by interesting question was how to format abbreviations, etc. in Archon. Migrating and populating a new database (especially by batch loading and manipulating data in Excel) allowed for some relatively global changes. So I’ve decided to implement some principles of RDA! Exciting, I know.

RDA and Archon

Looking to the future, I’ve decided to implement the following changes to our descriptive metadata following the recommendations of RDA:

  • The abbreviations “c.” and “ca.” will be spelled out as “approximately” for the box and folder lists.
  • The abbreviation “n.d.” will be written out as “Date not identified” when standing alone and “undated” when part of a list.
  • Months will be spelled out in full form; “August” instead of “Aug.” and so forth.

 Next Steps for Archon

Reviewing my previous post on Archon, my updated “next steps” now include:

  • Write short Scope and Abstract statements for each collection.
  • Apply language headings for the material.
  • Create and maintain creator records – doing authority control and supplying minimal descriptive information.
  • Choose and apply Library of Congress Subject Headings.
Posted on

Managing multiple Zotero libraries

I love Zotero. I use it all the time and have been for just over 4 years now. I organize all of my research there and just love using it. I’ve become a bit of a Zotero evangelist too – also happy to tell others about the joys and wonders of Zotero!

In fact, I’m relying on Zotero to help me compile my research for my Master’s thesis project on the history and theology of Covenant Publications. “Covenant” refers to the Evangelical Covenant Church – my denominational home and the parent institution of North Park University and Seminary. However, for the sake of this project, I wanted to keep my personal Zotero library separate from the new library I was creating with Zotero. How do I manage multiple Zotero libraries?

Firefox Profiles

After a quick google search, I found this old thread on the Zotero forums. After that, it was smooth sailing and has allowed me to do things I didn’t even think to dream of! The official Firefox support page is (as always) very helpful and I won’t bother clarifying or adding to it. Here is the link!

Now, when I open Firefox, I just have to select what profile I want to work with. And it’s awesome. My library work profile is separate from my seminary research profile and my personal library is separate from my thesis related research.

Zotero Evangelism

[This has less to do with multiple accounts and more to do with my love of Zotero]

One think I’ve loved as I’ve poured lots and time and energy into my thesis is Zotero’s syncing ability and the fact that all my information is stored in the cloud. It has meant that I’m always working with the most current text – not having to deal with multiple drafts, etc. – and that my information is accessible from anywhere with an internet connection.

Posted on

Omeka updates

In the very little spare time I manage to find, I still try and update my personal family archive project. Again, it’s been a great experience working with a wonderful (and very personal collection!) and I’ve learned so much. I feel like I should write a little about that project here to show what I’ve learned.

People – This was surprisingly difficult and I dithered on this for quite sometime. First, I created a controlled vocabulary of sorts using the person’s official birth (therefore maiden) name. Then I decided to use the tagging system within Omeka to describe the people in the resource. For the intended audience, this is perhaps the most important piece of metadata and I wanted it featured prominently.

Date – trying hard to use the international standard yyyy-mm-dd consistently through out this project. Inferred dates are entered in square brackets [] and partial dates are left incomplete.

Language – for the few items I’m including that have a language component, I’m using a controlled vocabulary (via the Simple Vocab plugin) to describe the language of these resources. Right now it’s just English and Dutch.

Posted on

Regular Expressions in the Library – 2

Imagine you’ve just witnessed a hit and run accident. A car hit a biker and sped away. Despite the shock, you managed to remember the license plate number and promptly report it to the police. Great! You think the license plate number was 123ABC. They search their system and – yes! – it’s a match and justice was served.

This (obviously fictional) story is my way to introduce the idea of regular expressions. Theoretically, regular expressions could enable the police to find this car without perfect search terms.

Regular Expressions Solve Crime

Let’s change the memory. Instead of remembering the whole license plate number, you remember the first five characters –  123AB – and that the license plate had six characters total. You could do multiple searches like this:

123AB1
123AB2
 …
123ABZ

That would work…but it would be much easier to use the search term:

123AB.

In regular expressions, the dot character represents a single character. Searching “123AB.” would be functionally equivalent to searching all of the examples listed above (“123AB1“, “123AB2“, etc.). Again, the dot metacharacter matches only single items. Searching for “123AB.” would not match a license with 7 characters – if the license plate number was 123ABCD, they would get away!

Better Searching with Brackets

Let’s say that searching 123AB. worked, but it gave too many results. And, even though you can’t remember it exactly, you are pretty sure the last character was a letter and not a number (or perhaps this is a requirement for valid license plate numbers). The police could tighten up their search by using the following search term:

123AB[A-Z]

This expression uses two new characters: it uses square brackets and the hyphen symbol. Defined quickly, square brackets “identifies a user-defined class of characters, any of which will match” and the hyphen will “identifies a range of characters to match” (Source). The hyphen requires that items appear in their natural order – searching for 123AB[Z-A] would not return any matches. Other examples:

  • If you knew the last character wasn’t an A 0r B = 123AB[C-Z]
  • If you thought the last character was a vowel = 123AB[AEIOU]
  • If you thought the last character was a number = 123AB[1-9]

Summary

Three metacharacters in regular expressions could help solve this crime:

  • The dot characters matches any single character
  • Square brackets define allowable characters
  • Hyphens can be used to define an allowable range of characters.
Posted on

Regular Expressions in the Library – 1

In several library contexts (MarcEdit, for example) , I’ve read or been told “have you tried doing that using Regular Expressions?”. Having no idea what this meant or referred to, I did a little wikipedia-ing and found out a little more about this method of matching text strings. Initial thoughts:

  • Yes, I think this can accomplish some important library-related tasks.
  • No, I don’t know anything about them.

I’ll chronicle my attempts to understand and explore Regular Expressions on this blog under this heading. I’m doing so with the intended context of MarcEdit, but hope to think of other ways Regular Expressions could be used to do library work.

First, a definition via Wikipedia:

In computing, a regular expression provides a concise and flexible means to “match” (specify and recognize) strings of text, such as particular characters, words, or patterns of characters

I’ll jump right in and discuss some special characters that made regular expressions so powerful. I like examples and getting messy, so here we go! I’ve decided to adopt a crime show format genre for my initial explorations. Enjoy!