Arctic Health Upgrades Search Engine for Easier Access by Researchers

by Jonathan Jacobsen Monday, June 11, 2018 7:46 AM

Arctic Health, intended for students, researchers, and anyone with an interest in health aspects of the Arctic, is a central source for information on diverse aspects of the Arctic environment and the health of northern peoples. The Arctic Health website provides access to a database of over 280,000 evaluated publications and resources on these topics. To improve access to this collection, a new search engine has just been launched at https://arctichealth.org

Search results in Arctic Health include published and unpublished articles, reports, data, and links to organizations pertinent to Arctic health, as well as out-of-print publications and information from special collections at the University of Alaska. Resources come from hundreds of local, state, national, and international agencies, as well as from professional societies, tribal groups, and universities.

Arctic Health is managed by the Alaska Medical Library at the University of Alaska Anchorage, by Prof. Kathy Murray and a team of staff. Andornot has worked with this group since 2005 and designed several previous search interfaces using Inmagic WebPublisher PRO and dtSearch.

Prof. Murray approached Andornot last year with several updates in mind, such as to ensure the search results are accessible on mobile devices, not just desktops. Rather than simply adjust the existing site, this precipitated a complete review of the current system, including data entry workflow and the actual content to be included, as well as discussions on a more modern search engine.  

As we do with many projects, Andornot began this challenge by separating out the user groups and functions. Library staff need a system to manage and upload records, with features for adding, editing, converting and validating data. Researchers and health care practitioners, on the other hand, need an easy to use, robust system for searching the vast archive of resources. With such a large number of records, a sophisticated search engine is needed to float the most relevant results to the top of any search.

For the back-end, Andornot developed a web application that uses Inmagic DB/TextWorks for data storage, and Inmagic WebPublisher PRO as a middle layer. We were able to update and re-use an XSLT we'd previously developed that UAA uses to import records in XML format from PubMed. This hybrid approach of using existing commercial software and a custom-developed web application provided the features needed by library staff at a more economical cost than a completely custom written system. 

For the public search interface, we used our Andornot Discovery Interface (AnDI). AnDI is a modern search engine based on the popular Apache Solr system, with features such as:

  • Excellent keyword search engine and relevancy-ranked search results.
  • Automatic spelling corrections and “did you mean?” search suggestions.
  • Full text indexing of linked documents.
  • Facets, such as subjects, authors, places, dates, and material types, to allow users to quickly and simply refine their search.
  • A selection list allows users to mark items of interest as they search, then view, print or email the list.

AnDI helps users quickly find relevant materials from the large collection at Arctic Health and is a significant improvement over the previous search options.

Both systems in this solution are hosted by Andornot as part of our Managed Hosting Service.

Check out the new iteration of the Arctic Health resource database at https://arctichealth.org, and contact Andornot for help with your project.

Automated Sitemap Generator Added to Andornot Discovery Interface

by Jonathan Jacobsen Friday, June 08, 2018 11:36 AM

Andornot believes strongly that it’s not enough for an archive or museum to simply have a fascinating collection and excellent software for managing it and making it publicly accessible. Drawing the public to these resources is equally important, something larger museums and some archives do well of course. For smaller organizations, that means the curator or archivist has to put on a marketing hat from time to time. However, this need not be a painful experience.

For example, a couple of months ago we wrote a blog post about using Wikipedia as a means of increasing the exposure of your organizations and your collections. This can be a quick, easy and fun afternoon task.

And today we're announcing a new feature in our Andornot Discovery Interface (AnDI) to also help attract the public: an automatic site map generator.

A site map is an XML file placed within your website, listing all available pages or resources, to help search engines such as Google and Bing index as much of your content as possible. While search engines will crawl links they find, such as on your home page, to help them discover records, this site map file can be provided to guide them to the full set.

e.g.

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url><loc>http://search.yoursite.org/Permalink/descriptions281616</loc><changefreq>weekly</changefreq></url>
<url><loc>http://search.yoursite.org/Permalink/descriptions281617</loc><changefreq>weekly</changefreq></url>
<url><loc>http://search.yoursite.org/Permalink/descriptions281618</loc><changefreq>weekly</changefreq></url>
<url><loc>http://search.yoursite.org/Permalink/descriptions281619</loc><changefreq>weekly</changefreq></url>
<url><loc>http://search.yoursite.org/Permalink/descriptions281620</loc><changefreq>weekly</changefreq></url>

 

Within AnDI, the sitemap lists all available records that can be found in the search engine, using the permalink URL.

This file is not seen by the public and has no impact on the site, but will be used by Google and others to index more of the records in an AnDI site. And thus, when people search by keyword in Google for records that happen to be in that collection, especially ones with unique names, places and words, these records are more likely to appear in their Google search results, drawing more traffic to the site.

This feature has been rolled out to all the clients who participate in our Managed Hosting service, and is available to our other AnDI clients (just send us an email to request it).

There are many ways to spread the word online about your collections and resources, some requiring very little effort. Stay tuned to our blog and newsletter for more!

Converting DB/TextWorks Data to MARC: Not as Easy as it Sounds!

by Jonathan Jacobsen Wednesday, April 25, 2018 4:55 PM

We recently had a client ask us if we could help them convert their DB/TextWorks library catalogue database to MARC format, for submission to another search system.

Our initial reaction was "Sure, easy-peasy." After all, we're librarians, we know MARC well, we're experts in DB/TextWorks, and we've done this before. How hard could it be?

Ha! Famous last words, and all that.

To be fair, there are a few wrinkles:

  1. The conversion is not a one-time affair, but rather something the client would like automated and running on a regular basis.
  2. Not all records in the database are to be converted.
  3. MARC is not the simplest of formats. Whether MARC or MARCXML, there are a fairly rigid set of rules that must be followed to create 100% valid, standards-compliant MARC records.

Nonetheless, since MARC has been around rather a long time, there are a plethora of tools available for creating MARC records from other data sources. There's Inmagic's own MARC Transformer, which works directly with DB/TextWorks databases, as well as various free or open source tools from the U.S. Library of Congress and other agencies, MARCEdit, Balboa Software's Data Magician, and others.

We also have our Andornot Data Extraction Utility for automatically exporting data from a DB/TextWorks database and manipulating it into a variety of formats.

Ever optimistic, we figured some combination of these tools could be strung together in sequence without too much effort to create a solution for this client. We wanted to avoid developing a DB/TextWorks-to-MARC conversion program from scratch as it would be quite time consuming, mostly due to the MARC format requirements themselves.

Several tools, upon closer investigation, proved to be too ancient to run reliably on a modern Windows server, or couldn't work with the current version of DB/TextWorks. Others proved almost impossible to use in an automated fashion. They could be useful in a one-time manual conversion to MARC format, but not in the hands-off, automated workflow we needed.

The exploration of this issue was an interesting exercise in seeing how old data formats and old programs age and become harder to work with.

The recipe that baked the cake in the end was:

  1. Use the Andornot Data Extraction Utility and the Inmagic ODBC driver to extract data from the DB/TextWorks database to a pseudo-MARC plain text format. 
  2. Use a custom-developed PowerShell script to manipulate the records in this file to handle some of the quirks of the ODBC output and to more closely adhere to the MARC format.
  3. Use the command line interface to MARCEdit to convert the pseudo-MARC to MARC Communications Format files.
  4. Upload the MARC files over FTP to the destination server.
  5. Manage all the moving parts through a PowerShell script and log the steps and results to a file for easy troubleshooting in case of problems.
  6. Run the script nightly as a scheduled task on a Windows server.

When written like that, in hindsight, it sounds so simple. And in the end, it was, and works well. But the journey to arrive at this solution was one of the more challenging small projects we've undertaken, considering how simple the task sounds at first.

We hope this will help you if you have a similar project, but don't hesitate to ask us for help, now that we've worked through this.

Andornot Professional Development Grant for 2018 Awarded to Gayle Graham

by Jonathan Jacobsen Wednesday, March 28, 2018 10:35 PM

We are very pleased to announce a recipient for the Andornot Professional Development Grant for 2018: Gayle Graham of the Nova Scotia Health Authority.

Gayle is relatively new to health sciences librarianship, and has never attended a conference specifically about health libraries. She will use the grant to attend the Canadian Health Libraries Association (CHLA) Conference in St. John’s this June. Gayle notes that “as a new organization, we would really benefit from an update on what's happening in the wider community of health sciences libraries.”

Andornot strongly believes in the value of attending conferences to foster professional development. We attend events across Canada and the United States all year long to learn about new trends and technologies, meet with clients, and share our expertise with like-minded folks.

We inaugurated this grant last year, awarding it to Mark Goodwin of the BC Cancer Agency. We were delighted to be able to offer this grant again this year, and only wish we could send everyone who applied to the conference of their choice. 

We hope that everyone who applied, and all of you, will also be able to attend a conference this year. Check out the list of ones we’ll be at and drop by to say hi if you can.

Tags: events | funding

Library and Archives Canada announces launch of 2018 funding cycle for Documentary Heritage Communities Program

by Jonathan Jacobsen Sunday, December 03, 2017 4:45 PM

Library and Archives Canada has announced the launch of the 2018 funding cycle for the Documentary Heritage Communities Program (DHCP). This is the fourth year of a planned 5 year program, with $1.5 million available this year, as in previous rounds.

The DHCP provides financial assistance to the Canadian documentary heritage community for activities that:

  • increase access to, and awareness of, Canada’s local documentary heritage institutions and their holdings; and
  • increase the capacity of local documentary heritage institutions to better sustain and preserve Canada’s documentary heritage.

The deadline for submitting completed application packages is February 7, 2018. 

This program is a great opportunity for archives, museums, historical societies and other cultural institutions to digitize their collections, develop search engines and virtual exhibits, and other activities that preserve and promote their valuable resources.

The program is aimed at non-governmental organizations specifically, including:

  • Archives; 
  • Privately funded libraries; 
  • Historical societies;              
  • Genealogical organizations/societies;  
  • Professional Associations; and 
  • Museums with an archival component.

Businesses, government and government institution (including municipal governments and Crown Corporations), museums without archives, and universities and colleges are not eligible.

Types of projects which would be considered for funding include:

  • Conversion and digitization for access purposes; 
  • Conservation and preservation treatment; 
  • The development (research, design and production) of virtual and physical exhibitions, including travelling exhibits; 
  • Conversion and digitization for preservation purposes; 
  • Increased digital preservation capacity (excluding digital infrastructure related to day-to-day activities); 
  • Training and workshops that improve competencies and build capacity; and 
  • Development of standards, performance and other measurement activities. 
  • Collection, cataloguing and access based management; and 
  • Commemorative projects.

Further program details, requirements  and application procedures are available at http://www.bac-lac.gc.ca/eng/services/documentary-heritage-communities-program/Pages/dhcp-portal.aspx

How can Andornot help?

Many Andornot clients have obtained DHCP grants in previous rounds, and Andornot has worked on many other projects which would qualify for this grant. Some examples are detailed in these blog posts:

We have extensive experience with digitizing documents, books and audio and video materials, and developing systems to manage those collections and make them searchable or presented in virtual exhibits.

Contact us to discuss collections you have and ideas for proposals. We'll do our best to help you obtain funding from the DHCP program!

Month List