Andornot Consulting
Wednesday, June 09, 2010 12:15 PM

Content Server (aka "DB/Text for SQL") Backup Script Utility

by Ted Jardine

If you are using Content Server (now known as "DB/Text for SQL"), it goes without saying that you should have regular automatic backups in place: regularly verified and regularly "test" restored. You DO have such a plan in place, right? Right?!?

Dilbert.com

For backing up DB/Text for SQL, it's a touch more involved than a typical SQL Server backup, simply because there's data both in and out of SQL Server and both need to be backed up at the same time. Moreover, SQL Server databases are not able to be backed up by typical backup mechanisms (to oversimplify, the actual files are always locked).

Fortunately, a manual backup of CS is pretty easy: just open up your CS Admin and follow the prompts to back up everything in one fell swoop into one handy .dat backup file. The bad news with that however, is that manual backups are a half-baked solution; backups need to be automated or they just won't get done often enough (if at all).

Options for automated CS backups:

  1. Back up the SQL Server aspects of CS as part of your regular automated SQL Server backups. And ensure that the external files are backed up at the same time. This solution will likely require the involvement of your IT staff. Moreover, automated SQL Server backups require SQL Server Agent which is not included with SQL Server Express (a requirement at least for a backup process that isn't the digital equivalent of the hokey-pokey dance crossed with a waltz). See your Administrator's Guide for details (for backup instructions, not dance tips).
  2. Use the built-in CS Admin backup capabilities, but automate them by plunking some CS-specific backup scripts in a batch file and scheduling them with Windows Task Scheduler.

Aha, you SQL-Server-Express-loving-person! Option two sounds great! Let's do it! Not so fast...have you seen those scripts? If you've got any more than one or two textbases, you'll go blind trying to accurately create and maintain those batch files. And that's where our new handy-dandy CS Backup Script Generator comes in. With it, you can quickly and easily generate accurate backup scripts contained in an automatically generated batch file that can be automatically run with a scheduled task. Did I mention automatically!?

Backups + Automation = Sweetness.

CS Backup Script Generator

 

  1. Download the Andornot CS Backup Script Generator here (yes, it's priceless so it's free) and install. Note that you can install the utility on any Windows machine as it does not need to be on your CS server.
  2. If you don't already have the Microsoft .NET Framework 4.0 installed, download it here and then install.
  3. Run the utility (should be in your start menu with a link on your desktop) and fill in three fields:
    1. SQL Server Instance: the name of your SQL Server instance, such as MachineName\SQLEXPRESS (if you don't know it, you can find it easily in your CS Admin).
    2. Backup directory: where you wish to have all backup files saved to (such as "D:\Backups\").
    3. Textbase Information Text: the text from the "List Textbases" summary within CS Admin (just select all the text in the summary, making sure to include the entire textbase listing such as the following:

      ...
      Textbase: D:\data\Barcodes
        SQL Database: '_InmTB_18'
      Textbase: D:\data\Borrower
        SQL Database: '_InmTB_19'
      Textbase: D:\data\Catalog
        SQL Database: '_InmTB_20'
      Textbase: D:\data\Contacts
        SQL Database: '_InmTB_21'
      Textbase: D:\data\Loans
        SQL Database: '_InmTB_22'
      ...

      The utility is smart enough to pick up the list of textbases from all the text in the summary, so you can paste in as much or as little of the summary as you like, as long as it includes the textbase listing. Get to the listing in CS Admin via the "Manage Textbases" > "List Textbases" menu item.
    4. Hit "Preview" to take a look at your new masterpiece (and shudder to think of doing that manually especially if you've got 10+ textbases) and/or "Batch it!" to save a .bat file to the location and name of your choosing.

      CS Backup Script Generator Preview
    5. Test the generated batch file by double-clicking it. After a few minutes (or less depending on the size of your database), there should now be a  .log and .dat file for each of your textbases in your specified backup directory.
    6. Review the .log file for any errors or warnings.
    7. Schedule the batch file to run daily during an off-peak time.
    8. Regularly review the generated log files for any errors or warnings and verify that backups are taking place as required.
    9. Ensure that your backup directory itself is backed up off-server and off-site for disaster recovery. The generated .dat files can be simply x-copied (copied and pasted) to another location, or better yet, automatically backed up with your server's backup software as part of your server's regular backup routine.
    10. Every so often, test out restoring your .dat backups via the CS Admin.

Remember, "hell hath no fury like data scorned." Please love your data and back it up.

Disclaimer: this backup utility is very much "beta" quality. I'm not responsible for anything it may or may not do to your system (bad or otherwise). Use at your own risk. Having said that, the setup program simply places the .exe with some support DLLs on your system, and when the utility is run, you can easily review the batch files it generates before running them (which you could generate manually yourself if you prefer).

Wednesday, May 12, 2010 10:10 AM

Adding Google Analytics without touching your site

by Ted Jardine

I had a problem: Kathy and Denise wanted Google Analytics configured for our demo Genie site that can be found at http://genie.andornot.com. However, Genie doesn't provide an easy way to centrally add the couple lines of Google Analytics code to every page served up by the application. For that matter, neither does any WebPublisher PRO site right out-of-the-box (easy-peasy though if we're using ASK and our WebPubResults control).

You might also have the same problem: you recognize that having multiple means of tracking and analyzing your site's traffic is no longer optional. However, while you've got server log file analysis handled (we use SmarterStats for all our hosted clients) your site doesn't have a single central place (such as site-wide central templates) to easily integrate a javascript-based page tagging solution such as Google Analytics.

So what do we do? Instead of updating 10s if not 100s or 1000s of files manually one-by-one (crossing my fingers that I could find them all) and hoping my code wouldn't need to change anytime soon, I came up with a modified version of the Web Analytics Tracking Module available on Microsoft's IIS.net site. Now, with a little elbow grease for initial server setup (seriously, not very much elbow grease at all), turning on site tracking for any site on our servers is simply a matter of placing a DLL in our application's bin directory, and adding a couple lines to our site's Web.config. Sweet.

Why not just use the original IIS.net Web Analytics Module?

Before jumping in and explaining how to set everything up, I should explain that I needed create my own custom build of the module because the module on IIS.net only works under IIS 7's integrated pipeline, not under IIS 7's classic pipeline (which Genie needs to run under) or IIS 6. The required changes ended up being relatively minor:

  1. Modified the ReadModuleConfiguration method in WebAnalyticsHttpModule.cs to the following:
  2. /// <summary>
    
    /// Reads the module specific configuration properties
    
    /// </summary>
    
    /// <param name="context"></param>
    
    /// <returns>Boolean indicating the success/failure</returns>
    
    private bool ReadModuleConfiguration(HttpContext context)
    
    {
    
         try
    
         {
    
             ConfigurationSection section = null;
    
             if (HttpRuntime.UsingIntegratedPipeline)
    
                section = WebConfigurationManager.GetSection(context, "system.webServer/webAnalytics", typeof(WebAnalyticsSection));
    
             else
    
                section = WebConfigurationManager.GetSection(context, "system.web/webAnalytics", typeof(WebAnalyticsSection));
    
              if (section != null)
    
                 _webAnalyticsModuleConfig = (WebAnalyticsSection)section;
    
         }
    
         catch (Exception)
    
         {
    
             return false;
    
         }
    
           return _webAnalyticsModuleConfig != null;
    
        }
    
    }
  3. Add the classic pipeline <system.web>/<webAnalytics> schema to WebAnalytics_schema.xml:
  4. <sectionSchema name="system.web/webAnalytics">
    
        <attribute name="trackingEnabled" type="bool" defautlValue="false"></attribute>
    
        <attribute name="trackingScript" type="string"  defautlValue="This is a default text"></attribute>
    
        <attribute name="insertionPoint" type="enum" defaultValue="body">
    
            <enum name="head" value="0" />
    
            <enum name="body" value="1" />
    
        </attribute>
    
    </sectionSchema>
    
  5. To make sure there's no versioning conflicts, I updated the Assembly's version to v1.1 in AssemblyInfo.cs (note the v1.1 references in the instructions below as opposed to v1.0):
  6. [assembly: AssemblyVersion("1.1.0.0")]
    
    [assembly: AssemblyFileVersion("1.1.0.0")]

 

Configure your server (one time only)

  1. Register the WebAnalyticsModule.dll in the GAC (for shiny stuff in the IIS 7 Manager GUI).

    gacutil -if WebAnalyticsModule.dll
  2. Copy the WebAnalytics_schema.xml to "%windir%\system32\inetsrv\config\schema" folder.
  3. Add the following section definition to the "%windir%\system32\inetsrv\config\applicationhost.config" file in the sectionGroup for "system.webServer"
  4. <section name="webAnalytics" overrideModeDefault="Allow" />
  5. Add the module to the IIS Manager configuration by adding to two collections in the "%windir%\system32\inetsrv\config\administration.config" file:
    • Add the following to the moduleProviders collection:
    • <add name="WebAnalytics" type="WebAnalyticsModule.WebAnalyticsProvider, WebAnalyticsModule, Version=1.1.0.0, Culture=neutral, PublicKeyToken=c6b7132bcfe43312" /> 
      
    • Add the following to the modules collection:
    • <add name="WebAnalytics" />


web-analytics-tracking-in-iis
Web Analytics Tracking now directly available in IIS 7 Manager (at least for sites running under Integrated Pipeline).

 

Configure your site/s

For each site you wish to have Google Analytics enabled on (or any other script/text/html you wish to have run on every page in the site), enable it according to the following instructions (slightly different for different flavours of IIS/pipeline).

IIS 7 Integrated Pipeline

  1. Place the same WebAnalyticsModule.dll in your application's bin directory or reference the GAC version via the Web.config's <system.web>/<compilation>/<assemblies> element.
  2. Add the module to the <modules> element in your Web.config. This can be done through the IIS Manager, but it's easier to simply add it directly in the Web.config:
  3. <modules>
    
      ...
    
      <add name="WebAnalytics" type="WebAnalyticsModule.WebAnalyticsHttpModule, WebAnalyticsModule, Version=1.1.0.0, Culture=neutral, PublicKeyToken=c6b7132bcfe43312" />
    
    </modules>
  4. Specify the script/text you wish to have inserted and the location to insert it at. Again, this can be added directly in the Web.config. However, because the actual script/text needs to be encoded in order to keep your config file from going boink, it's a lot easier to just use the GUI.

  5. web-analytics-tracking-in-iis-configuration
    Easy configuration within the IIS 7 Manager GUI, including properly encoding the script for the Web.config (safe for XML).

    Enabling the above adds a <webAnalytics> element to the <system.webServer> section, looking something like (note the encoding):
    <webAnalytics trackingEnabled="true" trackingScript="&lt;script type=&quot;text/javascript&quot;>&#xD;&#xA;  var _gaq = _gaq || [];&#xD;&#xA;  _gaq.push(['_setAccount', 'UA-494411-1']);&#xD;&#xA;  _gaq.push(['_setDomainName', '.andornot.com']);&#xD;&#xA;  _gaq.push(['_trackPageview']);&#xD;&#xA;  (function() {&#xD;&#xA;    var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;&#xD;&#xA;    ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';&#xD;&#xA;    var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);&#xD;&#xA;  })();&#xD;&#xA;&lt;/script>" insertionPoint="head" />

 

IIS 7 Classic Pipeline

Exactly the same as the IIS 7 integrated pipeline instructions above, with the following changes:

  1. Add the module reference to the classic pipeline specific <system.web>/<httpModules> section instead of the <system.webServer>/<modules> section.
  2. Make it easy for yourself, and add the script-specific settings (as in step 3 above) via the GUI in order to make sure it's all encoded properly, but then move/copy the resulting <webAnalytics> element from <system.webServer> (IIS 7 integrated pipeline specific) to <system.web>.
  3. Add the following to the <configuration>/<configSections> element in your Web.config (there should be a better way to add another sub-section to the already defined <system.web> declaration in the <configSections> element, so let me know in the comments if you know what it is):
  4. <sectionGroup name="system.web" type="System.Web.Configuration.SystemWebSectionGroup, System.Web, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"> 
    
        <section name="webAnalytics" type="WebAnalyticsModule.WebAnalyticsSection, WebAnalyticsModule" /> 
    
    </sectionGroup> 
    

Caveat: as your app is running under classic pipeline, not all text/html resources are running through ASP.NET, and therefore, if you have for i.e. .html and/or .asp pages you want the script automatically added to, you will need to map those extensions to also route through the ASP.NET runtime. Pretty standard classic pipeline stuff: without doing so, only pages with the .aspx extension will have the script added.

 

IIS 6 (Classic Pipeline)

I haven't tested it, but I see no reason why it won't work with the same set up as IIS 7 Classic Pipeline, with the same caveats.

Notes:

  1. As you might have guessed, you can automatically insert *anything* using this module, as long as you don't mind it inserting just before the closing </head> tag or the closing </body> tag. Copyright statements, survey scripts, "Ted wuz here" alerts for every page...if you think of it, you can do it!
  2. Again, the fancy GUI stuff doesn't help you in classic pipeline scenarios (other than an XML-encoding aid as described above).
  3. Typically you want to register scripts as close to the end of your page as possible (various technical performance and usability reasons). However, Google Analytics' latest scripts have a "push" functionality making this a moot point: so register the latest scripts right before the closing </head> tag. See the "Asynchronous Tracking Usage Guide" for more information.

Different web analytics packages have different strengths and weaknesses, and it's only with multiple different perspectives on your site traffic that you even begin to get a clear picture of what is really happening with your site. Using this web analytics module makes it even easier to add and maintain a page tagging solution on your site.

 

Download

I'd like to just directly link to my compiled DLL and schema file of the module, but unfortunately, it's not at all clear what licensing model is associated with the original module (although it's clear from his article explaining the module, that Microsoft's Ruslan Yakushev intends for us to use it). Therefore, I'll be requesting clarification on this issue, but in the meantime, just download the source code from the original article and make the quick modifications as described above. If you have problems doing so, or you want help deploying it for your site, drop us a line.

Wednesday, March 31, 2010 12:08 PM

Outlook 2010: HTML skills from the 1990s still required

by Ted Jardine

Way back in 2006/2007, we were shocked/disgusted/plain-out-flabbergasted to find out that the then about-to-be-released Outlook 2007 was going to stop using Internet Explorer to render HTML emails, and instead use the Microsoft Word rendering engine. Not a big deal you say? No background images, no support for essential CSS elements such as float or position, horrible box model support, requirements for inline styles, and the list goes on. In other words, the leading email client at the time (75-80% of the corporate email market) deliberately disregarded several years of significant progress in Web standards and single-handedly completely stunted any hope of progress in email design for at least another five years. In an email I sent to other Andonotters at the time, I stated something along the lines of "at least HTML skills I mastered in the late 1990s won't go to waste for many more years to come."

A brouhaha, at least amongst developers, ensued, with Microsoft apparently caught completely off-guard by the response. Anyhow, fast forward three years with Outlook 2010 set to release on June 15, 2010. and unfortunately, yet again we're going to be stuck with Word's rendering engine. Microsoft's rationale can be found in their official Outlook Blog, revealingly entitled "The Power of Word in Outlook":

"There is no widely-recognized consensus in the industry about what subset of HTML is appropriate for use in e-mail for interoperability. The “Email Standards Project” does not represent a sanctioned standard or an industry consensus in this area. Should such a consensus arise, we will of course work with other e-mail vendors to provide rich support in our products."

I know "yes, there are email standards: they're called Web standards" is an overly simplistic retort; nevertheless, it is the only approach that makes sense. I have trouble comprehending why Microsoft continues to perpetuate a bastardized version of email standardization that is about as counter-productive to industry consensus as possible, especially when they could enable Outlook customers to "write professional-looking and visually stunning email messages" in a standards-compliant way while still using the "rich tools that our Word customers have enjoyed for over 25 years" (and no, I'm not being naive here).

At least there's a glimmer of a possible, just maybe, once-in-a-blue moon reason to be optimistic about future versions of Outlook: yesterday, David Greiner of CampaignMonitor quoted the following from Dev Balasubramanian of the Outlook team:

"At this point, our plans for email authoring and rendering in Outlook 2010 are unchanged. However, I can tell you that this is a significant topic of discussion as we plan our business going forward, and something we will definitely be thinking about for future releases of Outlook."

...but don't get your hopes up.

In the meantime, I'll retweet fixoutlook.org, keep my <table> skills sharp, and dumb down our email templates all the while muttering under my breath "what a waste".

Wednesday, March 31, 2010 12:07 PM

If it's not on Google Maps, does it really exist?

by Ted Jardine

The Golden Ears Bridge opened June 16, 2009. And while it can't be seen from space, it's a fairly significant bit of road infrastructure (you know, the six-lane bridge kind of road infrastructure). However, on both Google Maps and Bing Maps, almost a year later it's, ummm...still kind of missing:

golden-ears-on-bing-maps

golden-ears-on-google-maps

To say nothing of the significant additions all around the bridge.

I finally investigated what to do about this, and got around to submitting it to Tele Atlas and NAVTEQ, the main suppliers of the actual maps to Google. And lo and behold, I received an email this morning:

Thank you for contacting Tele Atlas. Based on a review of your report, we can now confirm that the change you suggested has been made. It will go out in the next release of our map database. This report is now closed.

Tele Atlas supplies maps to the companies that make devices or applications, not directly to the people who use them. We update the map we supply so these companies can incorporate the map update in their own systems. When this process is complete, your change will be made available to you. It may be possible to purchase an updated map; please contact your device manufacturer or application provider for further information.

Thanks again for your willingness to help keep Tele Atlas maps up-to-date and accurate!

Please do not reply as this is an automated email.

The Tele Atlas Map Insight Support Team

Now we'll just see when the updates propagate to Google and Bing (apparently it's now up on MapQuest). And even better, when the Google Street View car/bike takes a trip over it (I wonder if it will have to pay the toll?).

golden-ears-teleatlas

http://mapinsight.teleatlas.com/mapfeedback/ReportStatus/782730a8-2ccf-102d-bcb7-7841d97eed29/en

UPDATE: And no, while I'd like to think it was *my* submission that led to the update, I'm well aware that it was probably just part of the critical mass. I do wonder however, how many submissions were actually made.

 

UPDATE 2: Google now has it (May 2010)! But Bing Maps still doesn't.

 

Friday, November 20, 2009 10:21 AM

IE6 gzip bug solved using IIS7's URL Rewrite Module

by Ted Jardine

“IE6, that shambling wreck that clings with nightmare strength to its living death.”
Peter Tyrrell, Literary genius nonpareil

When Internet Explorer 6 was first released way back in 2001 it quickly became the predominant browser. It was much better than everything else out there. But now NINE years later, working with IE6 is truly one of the most frustrating things a Web developer has to deal with. In 2009, it has countless issues: problems with transparent PNGs, flakey caching, esoteric rendering, lack of support for modern Web standards, and the list goes on and on.

robot-jonny-ie6
Image courtesy of RobotJonny

This post is about yet another IE6 hair-puller: gzip compression. Compressing HTML, CSS, javascript, etc. allows content to be scrunched up so that it’s served up way faster while reducing bandwidth costs. It’s a great way to increase the performance of your site…except when IE6 is involved that is: IE6 does not correctly handle data that has been compressed using the gzip algorithm. This would be frustrating to deal with, but nevertheless acceptable, except for the fact that IE6 problems with compressed content only pop up sometimes, on certain machines, and intermittently. There doesn’t seem to be any rhyme or reason to it. The worst kind of bugs are the ones you cannot faithfully replicate.

With IIS6 we use Port80 Software’s excellent httpZip module to compress site content (compression is available natively in IIS6, but it’s a pain to administrate, and doesn’t work well with older browsers). One of httpZip’s big selling features is that it accurately deals with browser idiosyncrasies; therefore, until now we haven’t had to worry about IE6 and compression.

However, we have recently started transitioning more sites from being hosted on Windows Server 2003 (with IIS6) over to Windows Server 2008 (IIS7) and Windows Server 2008 R2 (IIS7.5).  With IIS7, gzip compression can be easily and correctly implemented right out of the box. But we’ve discovered that in certain scenarios with static and dynamic compression enabled, IIS7 doesn’t safely handle IE6’s idiot tendencies concerning gzip compression. The immediate solution is to turn off dynamic compression altogether, and this seems to solve the problem, but that doesn’t sit right with me: do yet another regression for IE6 users that punishes non-IE6 users.

Therefore, on to saving the world from IE6, one workaround at a time! Seb Duggan presents a solution involving ISAPI_Rewrite which essentially sniffs out whether or not IE 5 or 6 is doing the request and turns off compression for CSS and JS files if that is the case. Sounds good (although he does a sniff for pre-SP2 IE6 only, which I believe is erroneous). However, we already use IIS7’s URL Rewrite Module and I would prefer to avoid bringing another cook into the kitchen. There’s a couple of obstacles to getting this working however:

  1. The current version of the URL Rewrite Module v1.1 does not support rewriting server variables.
  2. The IIS7 URL Rewrite Module has very different syntax.

So how did I do it?

  • Download and install the URL Rewrite Module 2.0 RC (Release Candidate) or wait until the RTM is released (don’t know when). This version supports rewriting request server variables so we can fake IIS7 into thinking the requesting browser does not support gzip encoding (because for all intents and purposes, it DOESN’T – at least for CSS and javascript files that is).
  • Brute force hacking and documentation reading resulted in the following (the handy “Import Rules”, which can be used to translate Apache mod_rewrite rules, wasn’t up to the task). The important thing to remember is that each dash is replaced with an underscore and each server variable is made all caps and prefixed with “HTTP_”. Oh, and looking at the RFC specs, “0” is valid for setting the encoding to nothing (“.*” didn’t work for some reason).
    <system.webServer>
        …
    <rewrite>
            <rules>
                <rule name="IE56 Do not gzip js and css" stopProcessing="false">
                    <match url="\.(css|js)" /> <!— Match all .css and .js requests -->
                    <conditions>
                        <add input="{HTTP_USER_AGENT}" pattern="MSIE\ [56]" />
    <!— Where the User Agent includes MSIE 5 or MSIE 6 -->
                    </conditions>
                    <action type="None" />
    <!— Don’t do any redirects, rewrites, etc. -->
                    <serverVariables>
                        <set name="HTTP_ACCEPT_ENCODING" value="0" />
    <!—Make it so the request’s Accept-Encoding variable is
    set to nothing instead of gzip,decompress -->
                    </serverVariables>
                </rule>
            </rules>
        </rewrite>

    </system.webServer>
  • Explicitly allow the Accept-Encoding server variable to be overwritten by specifying the following in the applicationHost.config file (required for security reasons). If you don’t do this part, you’ll get an "HTTP/1.1 500 URL Rewrite Module Error".

    <rewrite>
        <allowedServerVariables>
            <add name="HTTP_ACCEPT_ENCODING" />
        </allowedServerVariables>
         …
    </rewrite>
  • You can implement the above on a site-by-site basis or server-wide (rules in site’s Web.config or location specific in applicationHost.config or site-wide in applicationHost.config).
  • ie6-gzip
    Fiddler showing IE6 with compression off for .js and .css files (while still compressing all other applicable files such as .htm).

    ie8-gzip
    Fiddler showing IE8 with compression on for all applicable resource types.

    The usual caveats apply to implementing the above: we don’t guarantee any of it. If it kills your cat, I know nothing. If you lose your job because of it, it’s your fault, not mine. On the other hand, if it helps you out or if you figure out a refined implementation, please let us know.