We are still talking a lot of data at North Park – in particular Chicago data. So I’m going to start getting my hands dirty working with this data to build capacity for future partnerships with faculty and students. So here is the first in what I hope to be many installments of the “Working with Chicago Data” series.
Mapping Chicago’s Grocery Stores
First step: Download data from the Chicago Data Portal (https://data.cityofchicago.org/). I’m using the Grocery Store 2013 dataset for this example.
The data itself seems pretty clean and well formatted. I’m going to use Tableau for this example because that’s the tool I’m learning right now. I opened Tableau and imported the spreadsheet from the Chicago Data Portal. I ended up creating 4 different visualizations based on this data.
The first is a map of grocery store locations. It uses the latitude and longitude from the dataset to create points. Pretty standard and vanilla.
These next map is much more interesting. It takes into account the size of the store (measured in square footage) and codes that as size and color. Larger stores have larger, darker circles.
The last two maps were just variations on the second map. One version filtered out “small stores” that were less than 10,000 square feet. The other filtered out stores with the work “liquor” in the title. On a technical levels, these filters were easy to apply. However, I’m completely aware of the cultural assumptions I’m bringing to bear here. When I (white, affluent, middle class) think about a grocery store I think about a large store that doesn’t have the word “liquor” in the title.
That’s that! It was pretty easy to get this data and put it to use in the form of a map. I used Tableau here but I could also use Excel (with the power map add) or a more specialized tool like ArcGIS.
In terms of next steps or extensions:
It would be interesting to compare results using a different tool. Might be good to showcase the basic steps for using each tool.
It would be very interesting to add neighborhood boundaries and/or other information such as demographic information and/or economic status. I’ll have to look at ways to incorporate this data.
It would also be very interesting to combine this data with user feedback like Yelp reviews.
In addition to providing the raw data to our campus community, I think the library can take a leadership role in providing the tools and expertise to mine this data into something usable and useful. However, many of the tools that are used to transform data are highly specialized and have a pretty steep learning curve. So I’m going to work to provide an overview of the tools available and focus on those that would be useful in the context of undergraduate education. Continue reading “Tools for Data Analysis”
We are talking a lot about data, data literacy, and how North Park University can use Chicago data in the classroom. There are already a lot of courses using data in instruction and research so part of my work is figuring out what is already happening. Continue reading “Chicago Data for Undergraduate Research”
I’m working on improving the interlibrary loan services at North Park as well as improving my skills in statistics and data visualization. I’ve combined these two interests to look at analyzing and visualizing our interlibrary loan data using Tableau. Continue reading “Interlibrary Loan Data Analysis and Visualization”
I recently revised CV to include the following line:
Manage interlibrary loan systems; increased the local fulfillment rate from 59% to 80% while decreasing average turnaround time
I thought it would be good to provide a little additional context for this claim by supplying some data, look at visualizing that improvement, talk about how we accomplished this change here at North Park, as well as what I learned from looking at the data. Continue reading “Report on Interlibrary Loan Improvements”
I’m taking the Foundations of Data Curation class at GSLIS and just finished a progress report for the MODIS Snow Frequency data set. Here is a link to the report.
I’m working on updating and uploading my thesis to an new, online home. I’m particularly interested in the metadata and how moving from analog to digital offers new opportunities and challenges. I’ve been working on this project for a while now and think it’s time to get somewhat more serious about it. Sometimes my idealist and dreams get in the way of actual, productive work so I wanted to address them both head on here and now.
Ideals
Ideally, I’d like to use my thesis as the basis of an exploration in linked data and the semantic web. My grandest visions involve creating a “Wikipedia of Covenant history” that allows raw data (names, dates, etc.) to be incorporated into meaningful narratives as well as stand alone as definitive data sets. The narratives I have in mind would be analogous to the text of my thesis – interpreting the bare facts of the history – and the data sets would be somewhat analogous to the bibliography.
The data sources I have in mind are ambitious; they include:
Books. Given my interest, this should be obvious. But I’d also love to include other information resources: tracts, pamphlets, videos, and blog posts. Perhaps even tweets and news stories.
People. Obviously all authors and editors but also all ordained Covenant clergy, including basic biographical information. Former Covenant archivist Timothy Johnson compiled a wonderful resource that I’ve used many times – I’d love to transition that data online and mix it with other datasets
Churches. A list of all Covenant churches of all time. Ideally, this would include information such as location, date of founding, etc.
Dates. This is pretty easy – link all of these events to a standard calendar.
Events. Catalog covenant events like CHIC, Midwinter, the Annual Meetings, etc.
These would be the different data sets – at least the ones I’m currently imagining – and they would be linked by different connections. I’ve done a little dreaming and research about different taxonomies and schemas but am having a hard time (1) determining what schema/structure makes the most sense and (2) what the technical challenges would be in implementing such a thing.
Additionally, there is the question about how I would generate and manage these different datasets. I have a relatively complete lists of books…but in that process I realized how incomplete that list was and how arbitrary certain distinctions seemed in practice. For example, certain articles in The Covenant Quarterly were much more influential to Covenant life than certain books. Yet cataloging all the periodical – Covenant Quarterly, Covenant Companion, weekly newspapers! – seems like an overwhelming and daunting task but ultimately one that would be very beneficial.
I’m going to spend sometime in the next few days exploring other examples of linked data set so that I get a better sense of how these things can related, what relationships already exist, and what tweaks I would need to make locally.
I’m imagining both a site where these different entities are linked to each other (much like my current sites in Omeka) but also where the user could browse and filter so as to ask relatively complex questions and get back hard data in response. For example, a list of pastors that served at a given church, list of churches started in a certain decade, books written by North Park grads, etc. I think this falls squarely in the “digital humanities” ball park and I’m excited to continue thinking about it.
Used under CC license – https://flic.kr/p/8agsb6
Practical
I’m going to start building an omeka site that will import the Zotero for my thesis and create items for each book. This will also allow me to generate: a small set of names to begin thinking about people and some items to plot on a timeline (to see what features are available there). I think it will also be nice to get the data into a “better” format – i.e. a format where a limited amount of metadata is structured in such a way that it can move moved to a different system.
At this point, I’m thinking of these datasets much like tables in a database but I think there are alternatives (XML, JSON, etc.) that are worth exploring and a sample set of this size – and a project with no real consequences! – is a perfect test run.
I also think practically about how I will generate the datasets that I need to complete this project. That is a bit daunting but the raw data is available in the Covenant Yearbooks. Ideally, I’d build on existing works but that’s a start.
I’m working with someone in our IT department to look at our consortial OPAC usage data. This is really at the edge of my abilities so I’ll be rather blindly documenting that process here – hopefully it will be interesting and helpful to someone but I’m certainly not an expert in this area – at least not yet!
Tentative Process
Download logs from CARLI.
Creator Decoder.
Design database
Write parser thing
Do basic analysis in MySQL.
Do more advanced analysis in Weka.
We are going to use the program Weka to do this bit of data mining and I’ve installed this on my local machine. It was pretty easy to install and download – so that was nice. Here is the link: http://www.cs.waikato.ac.nz/ml/index.html
Research Questions
These are the basic questions that are currently guiding my database design:
Basic numbers for the searches – how many?
How does VuFind compare to Classic Search?
Types of searching (title, keyword, author)
Search terms – what are the most popular? What are common misspellings?
Platform and OS?
What formats/filters/facets are applied?
Time of day?
Mobile vs. Desktop?
Look at Curriculum Center headings (start with a heading and see if it is used).
Number of search terms (how many terms in a kw search, for example)
Spelling Errors
Failed searches (no results)
Clicked vs. type subjects (i.e. what happens when someone clicks a subject heading?)
Last week, I watched the webinar entitled “What is a data-driven academic library?” hosted by Library Journal and really, really enjoyed it. Like, best webinar ever.
I was too engrossed in the content (and busy tweeting the event) to take comprehensive notes. But here are some of my notes from the webinar as well as later reflections.
To be data driven means to use datato drive change.
I feel like this is the core of the presentation. To be data driven means to use the data libraries and library systems collect to drive decision making. She proposed a five part method to accomplish this goal.
1. Create Questions
All questions are good – but bigger questions might need to be broken down into smaller chucks. Good questions are ones that are answerable based on data. To me, that meant questions like “are the nursing databases meeting current needs” must be translated into questions about usage and coverage – and that we should be honest and clear about that work. She also advised to not be overly reductive and to pay attention to other factors. Lastly, she advised people to let these questions guide your research – don’t get lost in the data!
2. Create a Plan
Along with crafting questions, Tudesco suggesting creating a plan. Essentially, what data will answer your question and how will you get that data? Refine your questions and create a plan or timetable for this project.
3. Collect and Manage Data
Honestly, I feel like I usually begin my data-driven projects at this step – and my projects almost always suffer because of this! My workflow usually begins with “What can I do with this data?” rather than the more important “How can this data help me make a decision?”. Allowing the question and project drive the data needs – and not the other way around – is a very valuable lesson I will take from this webinar.
She was also clear that sometimes the data you want/need doesn’t exist and that perhaps you need to create a tool to gather that data. Whether than means a survey or structured observations will be guided by the question
4. Analyze the Data
Again, this methodology focused on using the data to make a decision. This data analysis will bring together a lot of information – perhaps from different sources and with different nuances – and this will require interpretation and analysis to make sense of this. More on this later.
5. Make a Decision
This is where all that data matters! It’s very basic but it’s worth repeating – Data Driven libraries use data to drive decisions.
What Skills and Tools are Needed?
Tudesco mentioned three core skills library staff need to truly embrace the data movement and mentioned several tools under each area. This section also provided my favorite quote – one near and dear to my heart!
I’m trying to remember what she presented the best I can while also taking the liberty to add my own skills and tools to this list. This is a work in progress and will likely add more in the future.
Overall, this was a wonderful and timely webinar. I will do my best to take the lessons and tips to heart and structure all future data project in light of what this presentation taught me.
Here is the link to the webinar – I believe that if you register you will get archived access to webinar so you can watch it yourself!
Last Saturday I was asked to give a short overview of Archon to the Preservation Committee of the Swedish American Historical Society. It was mostly a hands-on demonstration of how to use Archon and not a theoretical exercise about the transformative role of technology in archival metadata. However, the basic question I was trying to answer was “Why does this matter?” or perhaps “Why is this important?” and those seemed like questions worth answering in a compelling way.
Structured Data > Unstructured Data
By this I mean simply that data that is organized into formal structures (whether MARC, Dublin Core, EAD, whatever) has clear and distinct advantages over unorganized and unstructured data. This project began with loosely organized and structured data – paper based finding aids – but migrated that into the more formal structure of Archon. This had several advantages:
Ability to standardize language and make the data more uniform.
Ability to set clear metadata standards and systematically improve and add metadata.
Ability to migrate to new and differnt systems.
The main advantage of this structure vs. unstructured data is so important to me that it deserves it’s own heading
Linked Data > Unlinked Data
Structuring metadata in an online system allows for a paradigm-shifting revolution – that data can be linked to other pieces of data. I’ve been thinking about this for a while – and will likely continue to think about this – but this is the power of the paradigm shift. Data is linked to other data – thereby linking thoughts, ideas, people, etc. Linked data is fundamentally different than unlinked data.
Typing a subject heading onto a card catalog is fundamentally different than typing a subject heading into a linked database. The question when typing a card catalog seemed to be “what will describe this item to our users?” whereas the fundamental question when created linked data is “how can I link this item to similar items in ways that will be helpful and meaningful?” and that is a huge distinction in my mind. Thinks like controlled vocabularies, conforming to national standards, etc. all make infinitely more sense when linked data is viewed from this perspective.
More Access > Less Access
This perhaps seems obvious, but it’s worth mentioning. More access is better than less access. Connecting people to information resources is the most important work of the library and of archives and providing more access seems nearly undeniably better than providing less access.
I will almost certainly add to this later on (as my views on technology and metadata continue to evolve) but I wanted to take a moment to document this particular moment in time, capture my current thoughts on the subject, and rely a bit of what I wanted to convey to the Preservation Committee of the Swedish American Historical Society.
