Converting DB/TextWorks Data to MARC: Not as Easy as it Sounds!

by Jonathan Jacobsen Wednesday, April 25, 2018 4:55 PM

We recently had a client ask us if we could help them convert their DB/TextWorks library catalogue database to MARC format, for submission to another search system.

Our initial reaction was "Sure, easy-peasy." After all, we're librarians, we know MARC well, we're experts in DB/TextWorks, and we've done this before. How hard could it be?

Ha! Famous last words, and all that.

To be fair, there are a few wrinkles:

  1. The conversion is not a one-time affair, but rather something the client would like automated and running on a regular basis.
  2. Not all records in the database are to be converted.
  3. MARC is not the simplest of formats. Whether MARC or MARCXML, there are a fairly rigid set of rules that must be followed to create 100% valid, standards-compliant MARC records.

Nonetheless, since MARC has been around rather a long time, there are a plethora of tools available for creating MARC records from other data sources. There's Inmagic's own MARC Transformer, which works directly with DB/TextWorks databases, as well as various free or open source tools from the U.S. Library of Congress and other agencies, MARCEdit, Balboa Software's Data Magician, and others.

We also have our Andornot Data Extraction Utility for automatically exporting data from a DB/TextWorks database and manipulating it into a variety of formats.

Ever optimistic, we figured some combination of these tools could be strung together in sequence without too much effort to create a solution for this client. We wanted to avoid developing a DB/TextWorks-to-MARC conversion program from scratch as it would be quite time consuming, mostly due to the MARC format requirements themselves.

Several tools, upon closer investigation, proved to be too ancient to run reliably on a modern Windows server, or couldn't work with the current version of DB/TextWorks. Others proved almost impossible to use in an automated fashion. They could be useful in a one-time manual conversion to MARC format, but not in the hands-off, automated workflow we needed.

The exploration of this issue was an interesting exercise in seeing how old data formats and old programs age and become harder to work with.

The recipe that baked the cake in the end was:

  1. Use the Andornot Data Extraction Utility and the Inmagic ODBC driver to extract data from the DB/TextWorks database to a pseudo-MARC plain text format. 
  2. Use a custom-developed PowerShell script to manipulate the records in this file to handle some of the quirks of the ODBC output and to more closely adhere to the MARC format.
  3. Use the command line interface to MARCEdit to convert the pseudo-MARC to MARC Communications Format files.
  4. Upload the MARC files over FTP to the destination server.
  5. Manage all the moving parts through a PowerShell script and log the steps and results to a file for easy troubleshooting in case of problems.
  6. Run the script nightly as a scheduled task on a Windows server.

When written like that, in hindsight, it sounds so simple. And in the end, it was, and works well. But the journey to arrive at this solution was one of the more challenging small projects we've undertaken, considering how simple the task sounds at first.

We hope this will help you if you have a similar project, but don't hesitate to ask us for help, now that we've worked through this.

Inmagic releases version 15 of DB/TextWorks, WebPublisher PRO and DB/Text for SQL

by Jonathan Jacobsen Saturday, May 02, 2015 10:42 AM

Inmagic has released version 15 of DB/TextWorks, WebPublisher PRO and DB/Text for SQL

Highlights of this release include:

DB/TextWorks

  • New Field Type: Rich Text. The Rich Text field type permits you to change fonts (font name, size, color, bolding, italics, etc.) within a single field, align text, create bulleted lists, and so on. 
  • New Field Type: Call Number. When records are sorted by a field using the Call Number field type, Call Numbers are sorted the correct order.
  • Email Enhancements: Email dialogue more closely resembles sending mail from an email client, and includes fields for additional message text and a signature.
  • Limits on the number of fields, characters in field names, linked textbases, forms, query screens, and many other features have all been increased.
  • Substitution Lists: Additional Field Types now support Substitution Lists: Call Number, Code, Image, Link, and Number.
  • Import Document. Improved IFilter text extraction for PDF files, especially those with text in the margin. Also, line breaks are added after each period-space (". ") to improve readability.
  • HTML, XML and Text Encoding enhancements for increased compatibility.
  • As well as features for publishing to the web, assorted small fixes, and a few older, less-used features removed.

More information is available in the DB/TextWorks Version 15 Read Me.

WebPublisher PRO

  • Issues addressed with the InmagicBrowse app, and new features added for publishing to the web from DB/TextWorks.

More information is available in the WebPublisher PRO Version 15 Read Me, as well as the DB/TextWorks Version 15 Read Me and the DB/Text for SQL Version 15 Read Me.

DB/Text for SQL

  • In addition to the features added to DB/TextWorks and WebPublisher (see above), DB/Text for SQL is now supported on SQL Server 2014.

More information is available in the DB/Text for SQL Version 15 Read Me.

 

All clients with a current Inmagic maintenance subscription will be receiving emails from Inmagic with download instructions for this release.

This release also includes a new version of the PowerPack (Importer, Checker and Updater apps), popular for automating data import and full-text extraction and indexing, as well as the search-only DB/SearchWorks version of the software.

A new version of the ODBC driver is expected within the next couple of weeks.

 

Important Notice

Upgrading the Inmagic software to version 15 requires a change to textbase structures. Once made, only version 15 of DB/TextWorks and WebPublisher PRO will be able to access the databases. Therefore, plan to upgrade all servers, workstations and databases at once.

Clients using the ODBC driver should not upgrade to version 15 of DB/TextWorks, nor upgrade their textbases, until version 15 of the ODBC driver is released.

 

Please consult the system requirements and supported OSs and web browsers for DB/TextWorks, DB/Text for SQL and WebPublisher PRO,  available on the Inmagic Extranet.

 

If you have a current Inmagic maintenance subscription but have not received a notification email in the next week or so, please email advantage@inmagic.com with your serial number and email address so it can be resent. Please also remember to let us know if your contact information has changed so we can update our records and pass this on to Inmagic.

As always, contact us with questions, for assistance with this upgrade, or to renew an expired Inmagic maintenance subscription.

Inmagic usage at MDA – maximizing the ROI of your software

by Kathy Bryce Friday, June 25, 2010 5:41 PM

It's enlightening to visit with clients and find out how they are using Inmagic software, especially when they are not typical applications.  We are always suggesting maximizing your return on investment by using the software for anything and everything, so here is a great example of a client doing just that.

Last week I was in Toronto and met with Dave Hook at MDA, a long time client and user of Inmagic software.   Dave has some very cool applications with over 30 different DB/TextWorks databases populating much of their intranet content.  He showed me a few of these such as their home page which displays the latest news, plus their system for listing the latest training courses and how users can register for these online.  Following registration, the user’s supervisor is notified by email, the number of remaining available spaces updates until the class is full and then an attendee list can be printed out by the instructor.  Behind the scenes multiple DB/TextWorks databases are in play.  The web interface including all data entry for many of the databases is handled exclusively by ODBC calls into the databases using standard SQL queries written into each HTML page.  This results in pages that don't look like they were generated from a database but fit seamlessly into their intranet interface.

Dave also uses DB/TextWorks for quick prototyping of database ideas for various projects.  As it is so easy to add fields and functionality, he is able to build these prototypes and have users critique and suggest modifications in a very short time frame.  If applicable, the prototype can then be finalized as an Inmagic application, or used by their IT group as a basis for building a system in SQL or other software.  This prototyping process is very much in sync with our philosophy of launching fast, and refining later. 

So, just like Dave at MDA, don’t be restricted to just traditional uses for the software! Let us know if you have ideas for new databases that you would like us to help you with?

How to make and use file DSNs for Inmagic ODBC

by Peter Tyrrell Sunday, January 25, 2009 12:24 PM

There are three ways to set up a DSN for ODBC connections: System DSN, User DSN, and File DSN. The first two store the connection settings in the registry, while the third stores all the connection info in a text file. The latter can be easily copied, filed with its parent project, put in source control, and modified by a build process. Plus just having that file there is a reminder that an ODBC connection is an application requirement. (Oh, I *have* forgotten.)

Create the file DSN

You can create a file DSN by opening the ODBC Data Source Administrator dialog (listed as "Data Sources (ODBC)" from the Windows Start Menu), but I have encountered problems doing it that way.* It's easy to do from scratch, anyway.

  1. Create a text file with a .dsn extension: MyFileDsn.dsn
  2. Fill in the contents as follows:
   1: [ODBC]
   2: DRIVER=Inmagic DB/Text Driver (*.tba)
   3: READONLY=1
   4: SERVER=NotTheServer
   5: DBQ=c:\project\textbases\
   6: PWD=INMAGIC
   7: UID=

Notes:

  • Each key-value pair is on its own line separated by a carriage return. Ignore the fact that the view you're looking at on the blog may be wrapping.
  • The Inmagic ODBC driver must be installed on any machine using this connection.
  • ReadOnly can be set to 1 for readonly access. READONLY=0 means write access as well.
  • PWD can be blank if the textbase(s) don't use passwords.
  • UID is always blank.

* E.g. password not being written to the file DSN, and the potential for confusion between 64bit and 32bit ODBC Data Sources, since a 64bit machine has both and has separate Admin dialogs.

Use the file DSN

Pass a connection string to your application which points to the file you created:

   1: // C# file DSN example
   2: string connectionString = "filedsn=c:\MyFileDsn.dsn";

 

Tags: Inmagic ODBC

El Thicko

by Peter Tyrrell Friday, August 11, 2006 4:11 PM

I just spent 20 minutes letting head bonk sharply and repeatedly on desk over Inmagic ODBC's "unable to open table" error, because I FORGOT that the ASP.NET worker account on Windows XP is aspnet, not network service, like it is with Windows Server 2003. I kept upping the permissions on network service to no avail. It was like I'd strapped bananas to my feet instead of shoes for the Summer Fun 10K of Broken Glass Run, and then, when things went badly, tied on bigger and bigger bananas.

Tags: Inmagic ODBC

Month List