Addition of digitized newspapers to the Arnprior Archives’ search interface

by Kathy Bryce Friday, June 22, 2018 8:54 AM

Andornot has recently completed work for the Arnprior & McNab/Braeside Archives to add the newly digitized versions of their newspapers up to 1937 to their searchable collections. The majority of issues are from the Arnprior Chronicle starting in 1885.  We also created a Finding Aid allowing researchers to see what issues are available for each of the 16 newspapers with the ability to browse each individually. 

Funding for this project was provided by the Ottawa Branch of the Ontario Genealogical Society, and will be a wonderful new option for genealogical research as well as providing a window into the coverage of historical events. Individual names can be searched, and search words or parts of words are highlighted on the newspaper pages, as in the screenshot below:

image

A search on a general term such as “sawmill” pulls results from several data sources and allows users to easily narrow down their results.

image

As well as providing new search capabilities for this important set of documents, this initiative removes the need to consult the now very fragile originals.

The digitization itself was handled by a local vendor and Andornot scripted the OCR’ing to create a searchable layer in the PDF’s.  When funding permits, the aim is to enhance the search option further by matching up the newspaper issues with an index to births, marriages and deaths created by the Archives. 

If you are considering a similar digitization project, or have databases or other material that you would like to make searchable, contact us for a chat to discuss options!

Arctic Health Upgrades Search Engine for Easier Access by Researchers

by Jonathan Jacobsen Monday, June 11, 2018 7:46 AM

Arctic Health, intended for students, researchers, and anyone with an interest in health aspects of the Arctic, is a central source for information on diverse aspects of the Arctic environment and the health of northern peoples. The Arctic Health website provides access to a database of over 280,000 evaluated publications and resources on these topics. To improve access to this collection, a new search engine has just been launched at https://arctichealth.org

Search results in Arctic Health include published and unpublished articles, reports, data, and links to organizations pertinent to Arctic health, as well as out-of-print publications and information from special collections at the University of Alaska. Resources come from hundreds of local, state, national, and international agencies, as well as from professional societies, tribal groups, and universities.

Arctic Health is managed by the Alaska Medical Library at the University of Alaska Anchorage, by Prof. Kathy Murray and a team of staff. Andornot has worked with this group since 2005 and designed several previous search interfaces using Inmagic WebPublisher PRO and dtSearch.

Prof. Murray approached Andornot last year with several updates in mind, such as to ensure the search results are accessible on mobile devices, not just desktops. Rather than simply adjust the existing site, this precipitated a complete review of the current system, including data entry workflow and the actual content to be included, as well as discussions on a more modern search engine.  

As we do with many projects, Andornot began this challenge by separating out the user groups and functions. Library staff need a system to manage and upload records, with features for adding, editing, converting and validating data. Researchers and health care practitioners, on the other hand, need an easy to use, robust system for searching the vast archive of resources. With such a large number of records, a sophisticated search engine is needed to float the most relevant results to the top of any search.

For the back-end, Andornot developed a web application that uses Inmagic DB/TextWorks for data storage, and Inmagic WebPublisher PRO as a middle layer. We were able to update and re-use an XSLT we'd previously developed that UAA uses to import records in XML format from PubMed. This hybrid approach of using existing commercial software and a custom-developed web application provided the features needed by library staff at a more economical cost than a completely custom written system. 

For the public search interface, we used our Andornot Discovery Interface (AnDI). AnDI is a modern search engine based on the popular Apache Solr system, with features such as:

  • Excellent keyword search engine and relevancy-ranked search results.
  • Automatic spelling corrections and “did you mean?” search suggestions.
  • Full text indexing of linked documents.
  • Facets, such as subjects, authors, places, dates, and material types, to allow users to quickly and simply refine their search.
  • A selection list allows users to mark items of interest as they search, then view, print or email the list.

AnDI helps users quickly find relevant materials from the large collection at Arctic Health and is a significant improvement over the previous search options.

Both systems in this solution are hosted by Andornot as part of our Managed Hosting Service.

Check out the new iteration of the Arctic Health resource database at https://arctichealth.org, and contact Andornot for help with your project.

Automated Sitemap Generator Added to Andornot Discovery Interface

by Jonathan Jacobsen Friday, June 08, 2018 11:36 AM

Andornot believes strongly that it’s not enough for an archive or museum to simply have a fascinating collection and excellent software for managing it and making it publicly accessible. Drawing the public to these resources is equally important, something larger museums and some archives do well of course. For smaller organizations, that means the curator or archivist has to put on a marketing hat from time to time. However, this need not be a painful experience.

For example, a couple of months ago we wrote a blog post about using Wikipedia as a means of increasing the exposure of your organizations and your collections. This can be a quick, easy and fun afternoon task.

And today we're announcing a new feature in our Andornot Discovery Interface (AnDI) to also help attract the public: an automatic site map generator.

A site map is an XML file placed within your website, listing all available pages or resources, to help search engines such as Google and Bing index as much of your content as possible. While search engines will crawl links they find, such as on your home page, to help them discover records, this site map file can be provided to guide them to the full set.

e.g.

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url><loc>http://search.yoursite.org/Permalink/descriptions281616</loc><changefreq>weekly</changefreq></url>
<url><loc>http://search.yoursite.org/Permalink/descriptions281617</loc><changefreq>weekly</changefreq></url>
<url><loc>http://search.yoursite.org/Permalink/descriptions281618</loc><changefreq>weekly</changefreq></url>
<url><loc>http://search.yoursite.org/Permalink/descriptions281619</loc><changefreq>weekly</changefreq></url>
<url><loc>http://search.yoursite.org/Permalink/descriptions281620</loc><changefreq>weekly</changefreq></url>

 

Within AnDI, the sitemap lists all available records that can be found in the search engine, using the permalink URL.

This file is not seen by the public and has no impact on the site, but will be used by Google and others to index more of the records in an AnDI site. And thus, when people search by keyword in Google for records that happen to be in that collection, especially ones with unique names, places and words, these records are more likely to appear in their Google search results, drawing more traffic to the site.

This feature has been rolled out to all the clients who participate in our Managed Hosting service, and is available to our other AnDI clients (just send us an email to request it).

There are many ways to spread the word online about your collections and resources, some requiring very little effort. Stay tuned to our blog and newsletter for more!

Converting DB/TextWorks Data to MARC: Not as Easy as it Sounds!

by Jonathan Jacobsen Wednesday, April 25, 2018 4:55 PM

We recently had a client ask us if we could help them convert their DB/TextWorks library catalogue database to MARC format, for submission to another search system.

Our initial reaction was "Sure, easy-peasy." After all, we're librarians, we know MARC well, we're experts in DB/TextWorks, and we've done this before. How hard could it be?

Ha! Famous last words, and all that.

To be fair, there are a few wrinkles:

  1. The conversion is not a one-time affair, but rather something the client would like automated and running on a regular basis.
  2. Not all records in the database are to be converted.
  3. MARC is not the simplest of formats. Whether MARC or MARCXML, there are a fairly rigid set of rules that must be followed to create 100% valid, standards-compliant MARC records.

Nonetheless, since MARC has been around rather a long time, there are a plethora of tools available for creating MARC records from other data sources. There's Inmagic's own MARC Transformer, which works directly with DB/TextWorks databases, as well as various free or open source tools from the U.S. Library of Congress and other agencies, MARCEdit, Balboa Software's Data Magician, and others.

We also have our Andornot Data Extraction Utility for automatically exporting data from a DB/TextWorks database and manipulating it into a variety of formats.

Ever optimistic, we figured some combination of these tools could be strung together in sequence without too much effort to create a solution for this client. We wanted to avoid developing a DB/TextWorks-to-MARC conversion program from scratch as it would be quite time consuming, mostly due to the MARC format requirements themselves.

Several tools, upon closer investigation, proved to be too ancient to run reliably on a modern Windows server, or couldn't work with the current version of DB/TextWorks. Others proved almost impossible to use in an automated fashion. They could be useful in a one-time manual conversion to MARC format, but not in the hands-off, automated workflow we needed.

The exploration of this issue was an interesting exercise in seeing how old data formats and old programs age and become harder to work with.

The recipe that baked the cake in the end was:

  1. Use the Andornot Data Extraction Utility and the Inmagic ODBC driver to extract data from the DB/TextWorks database to a pseudo-MARC plain text format. 
  2. Use a custom-developed PowerShell script to manipulate the records in this file to handle some of the quirks of the ODBC output and to more closely adhere to the MARC format.
  3. Use the command line interface to MARCEdit to convert the pseudo-MARC to MARC Communications Format files.
  4. Upload the MARC files over FTP to the destination server.
  5. Manage all the moving parts through a PowerShell script and log the steps and results to a file for easy troubleshooting in case of problems.
  6. Run the script nightly as a scheduled task on a Windows server.

When written like that, in hindsight, it sounds so simple. And in the end, it was, and works well. But the journey to arrive at this solution was one of the more challenging small projects we've undertaken, considering how simple the task sounds at first.

We hope this will help you if you have a similar project, but don't hesitate to ask us for help, now that we've worked through this.

Andornot Professional Development Grant for 2018 Awarded to Gayle Graham

by Jonathan Jacobsen Wednesday, March 28, 2018 10:35 PM

We are very pleased to announce a recipient for the Andornot Professional Development Grant for 2018: Gayle Graham of the Nova Scotia Health Authority.

Gayle is relatively new to health sciences librarianship, and has never attended a conference specifically about health libraries. She will use the grant to attend the Canadian Health Libraries Association (CHLA) Conference in St. John’s this June. Gayle notes that “as a new organization, we would really benefit from an update on what's happening in the wider community of health sciences libraries.”

Andornot strongly believes in the value of attending conferences to foster professional development. We attend events across Canada and the United States all year long to learn about new trends and technologies, meet with clients, and share our expertise with like-minded folks.

We inaugurated this grant last year, awarding it to Mark Goodwin of the BC Cancer Agency. We were delighted to be able to offer this grant again this year, and only wish we could send everyone who applied to the conference of their choice. 

We hope that everyone who applied, and all of you, will also be able to attend a conference this year. Check out the list of ones we’ll be at and drop by to say hi if you can.

Tags: events | funding

Month List