Know thy users, or at least their searching behaviours

by Jonathan Jacobsen Wednesday, July 14, 2010 10:00 PM

You've built this great online database of resources and you're as proud of it as a mama bear and her cubs. You just know everyone must be using it to find all sorts of great materials. But… how can you be sure? How do you know if in fact just 'cause you built it, they will come. Well, it is possible to learn quite a bit about who, how, when and how often users are accessing your site. There are two main sources of useful information: web server logs and the Inmagic WebPublisher Query logs.

Web Server Logs

Web servers typically log all activity, including serving up pages from your online database.

Software packages such as AWStats and SmarterStats analyze these log files and allow you to generate reports summarizing site activity. These two packages are installed on a server (could be the same web server as your online database, or a separate one) and are accessed through your web browser. They typically require IT involvement to install and locate the web server log files.

An alternative to log analysis software you install is Google Analytics. To use this service, you register with Google and place a small bit of code in your pages, wait for Google to gather some data, and visit your Google Analytics account to learn about your site. This approach works well for publicly-accessible sites.

At Andornot we use SmarterStats to provide statistics to all our hosted clients, and also Google Analytics on other sites such as our Genie demo.

So, once you have your logs under analysis, what can you learn from them?

Who

You can learn how many total and unique visitors your site gets, per day, week, month, etc. If your audience is known (e.g. your site is not public, but is only available to users within your organization, a known quantity) then you have a measure of what percentage of potential users are accessing your site. If your IT department can give you a list of computer IP addresses (which is what’s stored and reported on in these logs) and the corresponding names of staff who use those computers, you’ll really know who in your organization is accessing the site.

When

You can see when users are most likely to access your site and then schedule work around those times. For example, you could post new materials before the peak periods so there’s something fresh for users, and avoid updates or maintenance during those times.

How Much and How Often

You can learn whether users tend to come once and never again, or are repeat browsers. Do they come daily, weekly or monthly? Do they linger? Do they access many pages in your site, or just a few? Which ones are the most popular? What path do they follow through the site? Depending on how your site is organized, you may be able to get answers to all of these questions, then think about pushing most-accessed content to the home page, or just one link away.

Inmagic WebPublisher Pro Query Logs

You can configure Inmagic WebPublisher PRO to log all searches. To do so, add WebLogQueries=1 to the [WebPublisher] section of the DBTWPub.INI file (or to the [ICSWeb] section of ICSWeb.INI file for DB/Text for SQL). Once configured, each online search is logged in a CSV-formatted file named query.LOG, located in the WebPublisher QSETS folder. If your site receives a lot of activity, this file can grow quickly, so we recommend either only enabling this for a specific duration (e.g. one month) or periodically archiving the file and allowing a new one to start (e.g. on the first of each month). The archiving can be done using a Scheduled Task so you only need to set the process up once.

Important Note: You should only enable query logging in version 12 or later of WebPublisher Pro. Do not enable query logging in version 11.

Once you have a log file of searches, you can use the QueryLog database included with the Andornot Starter Kit to analyze it. The log file can be imported into the textbase using the Import command. You can then search for all records where the search produced no results to see, most critically, what users are searching for and not finding. Maybe your database doesn’t have the type of resources they are looking for, or maybe they’re searching with different words than you used to describe the resource. Either way, this is valuable information! You can also learn what they are searching for and finding, and as with the information from the web server logs, who, when and how often they are searching.

One Andornot client, the Alberta Energy Resources Conservation Board loads and analyses their WebPublisher query log every week to see what people are searching for, then edits records in their databases to include any new search terms favoured by users. They were even able to identify an unexpected usage of one of their databases. It turned out that their responsiveness in quickly adding entries for search terms that had returned zero results meant that their database was more up to date than other internal sources.

Combined, web server log analysis and query log analysis can give you a pretty good picture of who is using your great online database, when, how often, and what they're finding, or not. You might supplement this with the occasional online survey, asking users for their opinion of your resources.

Need help installing and using any of this software, designing a survey, or making sense of the information gathered? We're always happy to help clients learn more about their users.

blog comments powered by Disqus

Month List