Know thy users, or at least their searching behaviours

by Jonathan Jacobsen Wednesday, July 14, 2010 10:00 PM

You've built this great online database of resources and you're as proud of it as a mama bear and her cubs. You just know everyone must be using it to find all sorts of great materials. But… how can you be sure? How do you know if in fact just 'cause you built it, they will come. Well, it is possible to learn quite a bit about who, how, when and how often users are accessing your site. There are two main sources of useful information: web server logs and the Inmagic WebPublisher Query logs.

Web Server Logs

Web servers typically log all activity, including serving up pages from your online database.

Software packages such as AWStats and SmarterStats analyze these log files and allow you to generate reports summarizing site activity. These two packages are installed on a server (could be the same web server as your online database, or a separate one) and are accessed through your web browser. They typically require IT involvement to install and locate the web server log files.

An alternative to log analysis software you install is Google Analytics. To use this service, you register with Google and place a small bit of code in your pages, wait for Google to gather some data, and visit your Google Analytics account to learn about your site. This approach works well for publicly-accessible sites.

At Andornot we use SmarterStats to provide statistics to all our hosted clients, and also Google Analytics on other sites such as our Genie demo.

So, once you have your logs under analysis, what can you learn from them?


You can learn how many total and unique visitors your site gets, per day, week, month, etc. If your audience is known (e.g. your site is not public, but is only available to users within your organization, a known quantity) then you have a measure of what percentage of potential users are accessing your site. If your IT department can give you a list of computer IP addresses (which is what’s stored and reported on in these logs) and the corresponding names of staff who use those computers, you’ll really know who in your organization is accessing the site.


You can see when users are most likely to access your site and then schedule work around those times. For example, you could post new materials before the peak periods so there’s something fresh for users, and avoid updates or maintenance during those times.

How Much and How Often

You can learn whether users tend to come once and never again, or are repeat browsers. Do they come daily, weekly or monthly? Do they linger? Do they access many pages in your site, or just a few? Which ones are the most popular? What path do they follow through the site? Depending on how your site is organized, you may be able to get answers to all of these questions, then think about pushing most-accessed content to the home page, or just one link away.

Inmagic WebPublisher Pro Query Logs

You can configure Inmagic WebPublisher PRO to log all searches. To do so, add WebLogQueries=1 to the [WebPublisher] section of the DBTWPub.INI file (or to the [ICSWeb] section of ICSWeb.INI file for DB/Text for SQL). Once configured, each online search is logged in a CSV-formatted file named query.LOG, located in the WebPublisher QSETS folder. If your site receives a lot of activity, this file can grow quickly, so we recommend either only enabling this for a specific duration (e.g. one month) or periodically archiving the file and allowing a new one to start (e.g. on the first of each month). The archiving can be done using a Scheduled Task so you only need to set the process up once.

Important Note: You should only enable query logging in version 12 or later of WebPublisher Pro. Do not enable query logging in version 11.

Once you have a log file of searches, you can use the QueryLog database included with the Andornot Starter Kit to analyze it. The log file can be imported into the textbase using the Import command. You can then search for all records where the search produced no results to see, most critically, what users are searching for and not finding. Maybe your database doesn’t have the type of resources they are looking for, or maybe they’re searching with different words than you used to describe the resource. Either way, this is valuable information! You can also learn what they are searching for and finding, and as with the information from the web server logs, who, when and how often they are searching.

One Andornot client, the Alberta Energy Resources Conservation Board loads and analyses their WebPublisher query log every week to see what people are searching for, then edits records in their databases to include any new search terms favoured by users. They were even able to identify an unexpected usage of one of their databases. It turned out that their responsiveness in quickly adding entries for search terms that had returned zero results meant that their database was more up to date than other internal sources.

Combined, web server log analysis and query log analysis can give you a pretty good picture of who is using your great online database, when, how often, and what they're finding, or not. You might supplement this with the occasional online survey, asking users for their opinion of your resources.

Need help installing and using any of this software, designing a survey, or making sense of the information gathered? We're always happy to help clients learn more about their users.

A necrology – a unique use for DB/Textworks

by Jonathan Jacobsen Wednesday, July 07, 2010 8:06 AM

One of the more enjoyable aspects of life at Andornot is the diversity of the projects. We recently had an interesting request from a group of archivists representing the Canadian Federation of the Sisters of St. Joseph, a group of six congregations of Sisters of St. Joseph in Ontario. Several of these congregational archives maintain Inmagic databases for their archival arrangement and description. The archivists were interested in developing a biographical database as well as designing a printer-friendly report that they could use to easily display information on their deceased sisters and organize by the anniversary of their death. We adjusted and re-indexed the databases to ensure that separate fields were used to sort in this manner. We then designed a report from an MS Word mockup and merged four databases into a single one. We thought this was an excellent and unique use for DB/Textworks.

"Kathy and Jonathan are just wonderful. They have provided the Archives Taskforce with the expertise needed to bring this project to life. We are so pleased to be able to produce a necrology that will be shared and used by so many women religious." (Linda Wicks)

You can learn more about these congregations at their websites:

How to solve 401.2 errors related to Windows Authentication and the BUILTIN\Administrators role in IIS 7

by Peter Tyrrell Tuesday, July 06, 2010 3:28 PM


An ASP.NET web application set to use Windows authentication where the web.config allows/denies authorization based on Windows roles, which references the BUILTIN\Administrators role specifically.

Your account is in the BUILTIN\Administrators group. The web application is running on a Windows OS with User Account Control [UAC] enabled. (hint HINT.)

Here's an example taken straight from official Microsoft Patterns and Practices: How To: use Windows Authentication in ASP.NET 2.0.



Even though your account is in the BUILTIN\Administrators group - you checked three times, pinching yourself in case you were dreaming - you cannot authenticate to get in to the web application. Instead, you get 401.2 errors.


You know your credentials are correct, because if you deliberately screw up your password, you get a 401.1 error. If you allow your specific account (DOMAIN\frustratedguy) in web.config, you can authenticate as normal.

It's as if the app doesn't think you are a member of the BUILTIN\Administrators role at all...


In fact, as far as the app is concerned, you are not a member of BUILTIN\Administrators. UAC has cast you out from that divine realm, and you no longer hold the keys to heaven.

using System.Web.Security

Listing roles might get you a list like:

None,Everyone,HomeUsers,BUILTIN\Users,NT AUTHORITY\NETWORK,NT AUTHORITY\Authenticated Users,NT AUTHORITY\This Organization,NT AUTHORITY\NTLM Authentication

No BUILTIN\Administrators. Allow another of those roles in web.config and your credentials will authenticate.

Myself, I didn't know how UAC actually worked. I knew that it didn't let me operate in the context of Administrator unless explicitly told to do so, but I didn't realize that meant that a role check would fail to find me in the Administrators group. Live and learn.


Using a text editor to modify DB/TextWorks forms and query screens

by Kathy Bryce Monday, July 05, 2010 10:52 PM

If you ever need to change multiple Inmagic DB/TextWorks forms or query screens, this is a useful little trick.  For example, say you want to change a field label for a box and it has been used on multiple forms. Use Maintain > Manage Textbase Elements to select all the affected forms or query screens and click on Export.  The resulting .xpf can be opened in a text editor like Notepad or Notepad ++ and carefully modified.  The parameters for each form and for each box etc. on each form are included in this file. For example, you  could replace a box label, by searching the text file for “label=Subject Headings” and replacing with “label=Subjects”.

[13] box_type=R obj_type=F class_type=Q
    anchor=L abox=12 left=5 top=5 width=732 min=0 max=5 tab_order=13 tab_skip=2
    label=Subject Headings
    border scrollbar show_label label_loc=T label_font=8 bkg_color=14613758
  (1) item_field=Subject Headings

We always recommend including the parameter such as label= as well as the actual text to be sure you are replacing exactly what you intend. In the example above the words “Subject Headings” are also used to indicate the field to be searched.  Once you have finished editing the forms, save the .xpf file under a different name, and use Maintain > Manage Textbase Elements to re-import the forms.  You can either delete the existing forms first as long as you still have the backup, or just go ahead and import the forms which will then be named formname_1.  You can then check the modification worked as expected before deleting each old form and renaming each revised one to remove the _1.

This trick works to globally change a whole range of parameters such as box background colors, or the file paths to servers embedded in web forms etc.   Contact us if you need training or any other assistance with forms design.

Month List