As I was working on the recap of my recent trip to The Netherlands, I discovered that the text of our 2001 trip to St. Thomas and Puerto Rico had vanished from Witsdom.  The index page was still there, as were the photographs, video and music; just the text page was missing.  I looked everywhere I could think of to recover it, but it was nowhere to be found.  This was a bit surprising since I actually had an archived backup of the entire web content of the server from 2003.

Fortunately, I still had the hard copy notes I had written while we were on the trip, so I re-typed them and recreated the page.

Not being one to be able to leave well enough alone, the fact that the audio controls on the page weren't working was more than I could bear.  The fact that I spent quite a bit of time reworking them so they would operate correctly on Internet Explorer or Firefox or Safari is silly since probably no one will ever view the page, but, at least I know it is right now.

Even though I kludged / brute forced the solution, I am pretty proud of it, as I wasn't able to find another reference that provided a solution for playing a randomly selected music file on a web page, in the background, that provided functioning controls in multiple browsers.

In case you're interested, here's the code:

<SCRIPT LANGUAGE="JavaScript">
   1:  
   2: <!------------HIDING
   3:  
   4: /*Script by Scott R Harrod  
   5:                 7/18/1997
   6:  http://members.aol.com/kirrak9/cyberbase.html
   7:  modified by David Sanders 11/21/1999 and 12/27/2007 with snippets from F. Permandi
   8:  http://www.permadi.com/tutorial/flashVars/
   9: */
  10:  
  11:    Play=new Array                                                                               
  12:    Play[0]="music/feelhi2.mid"         
  13:    Play[1]="music/hothothot.mid"
  14:    Play[2]="music/jama2.mid"
  15:    Play[3]="music/redredSD.mid"
  16:    var now=new Date()
  17:    var music=now.getSeconds()%4                                                
  18: // DONE HIDING-------->
  19:  
  20: document.write(
  21: '<OBJECT ID=Crescendo CLASSID="clsid:0FC6BF2B-E16A-11CF-AB2E-0080AD08A326" WIDTH="200" HEIGHT="55">\n'+
  22: '    <PARAM name="src"        VALUE="' + Play[music] + '">\n'+
  23: '    <PARAM name="autoplay"   VALUE="true">\n'+
  24: '    <PARAM name="controller" VALUE="true">\n'+
  25: '    <PARAM NAME="volume"     VALUE="100">\n'+
  26: '    <PARAM NAME="type"       VALUE="audio/x-mid">\n'+
  27: '<EMBED SRC="' + Play[music] + '"\n'+
  28: '    TYPE="audio/x-mid"\n'+
  29: '    WIDTH="200"\n'+
  30: '    HEIGHT="55"\n'+
  31: '    VOLUME="100"\n'+
  32: '    AUTOPLAY="true"\n'+ 
  33: '    CONTROLLER="true"\n'+
  34: '    ></EMBED>\n'+
  35: '</OBJECT>\n');
  36:  
</SCRIPT>

The first section (above the DONE HIDING) creates the array of possible selections, then chooses one.  The second section creates the code to generate an HTML object and an Embed statement using the selected file from above to activate the player, play the selected song, and display a small control console.  Both the Object and Embed are needed to provide cross-browser functionality.  Since a variable is being used in the Object and Embed statements, both statements must be generated using the javascript document.write function.

A couple of times, I have encountered an issue with Windows Live Writer related to the handling of special characters like in Crème Brûlée.

The issue was that in WLW everything appears fine, but when posted to the blog, it displayed as Crème Brûlée.

The only way I could get the text to display properly was to enter the special characters as HTML entities like this:

Crème Brûlée

I should note that in the above initial example, I did not enter the characters directly into WLW, but instead copied them from a web page then pasted them into the WLW editor.

I posted an item on the Windows Live Writer MSN Group and got a very quick response.

The net of the solution was:

Ah, here's the problem. The encoding is simply declared incorrectly in your blog template. If you change this:

<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />

to this:

<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

everything will work fine. You can confirm this by forcing your browser to display your homepage in UTF-8 (in IE7, right-click on the page and select Encoding | UTF-8; in Firefox, it's on the View menu).

Here is a sample of Crème Brûlée that was copied from another page and pasted into WLW so you can see the difference between the display with iso-8859-1 and UTF-8 encodings (you'll have to force the display as noted in the solution above).

So, one option is to edit each of the master templates in Movable Type to use the charset-utf-8 directive and then rebuild the site.  Since I don't use special characters that often, I am going to hold off on doing that since I am concerned that the change might break something else.

The other option is just to convert the special characters manually to HTML entities.  Of course, to do this, one would need a list of the characters and their corresponding Entity Names.  Fortunately, there are many pages out there with this information concisely captured and nicely formatted including:

http://www.w3schools.com/tags/ref_entities.asp

and

http://www.cookwood.com/html/extras/entities.html

just for starters.  For a longer list, Google HTML Entities.

Unfortunately, there is what I consider to be a bug in WLW.  If I enter the HTML entities into WLW in the HTML Code View, then switch to any of the other views, when I switch back to the HTML Code View, the HTML entities are converted into their character equivalents, effectively overriding the HTML code that was entered.  The only way I have identified to work around this is to not switch back to a WYSIWYG view prior to publishing. Of course, if you have want to edit the entry at a later date, you will have to remember to switch to the HTML Code View prior to opening the entry, or the HTML will get munged.

One of the great values that Blogdom provides to me is that it serves as a repository for helpful tips and tricks that I have discovered along the way.  Unfortunately, my mind usually isn't sharp enough to remember the tip or trick, but fortunately, at least sometimes, it is sharp enough to remember that I had encountered the challenge previously, came up with a solution, and maybe even recorded the solution for posterity in a post.

Such was the case today.  I had received a Lotus Notes email at work with an animated GIF as an inline (or embedded) image.  To my knowledge, there is no easy way to save such an image from Notes as a GIF file - even saving it as a BMP is a bit of a challenge.  But, I figured I could forward the email to my personal email account, then extract the image from Outlook.  Ultimately, it turned out I was right, but I needed the trick captured in this post from 2004 to accomplish it.

For quite some time, I had noticed that the calendar didn't display "quite right."

The problem was with the shaded background on the right hand side - it wasn't even.

I finally got around to playing with the CSS template, and now have it looking "right," at least in Firefox and IE6.

It still doesn't display correctly in Safari or IE7.

I believe that the problem is related to the fact that both Safari and IE7 use a slightly larger font as the default, and the size of the shaded background is fixed.  At least in Safari, if I go to View, Make Text Smaller, the calendar displays properly.  My guess is that if I were to increase the size of the area just a bit more, I'd get closer to a solution - although it would be a brute force solution rather than an elegant solution, and then I'd be wasting space when displaying in Firefox for IE6.  Since my main browsers are Firefox and IE6, and no one else has complained, I think I'll consider this fix "good enough."

For those interested, the details of the CSS section in question are:

The main change was to adjust the width: from 120 to 180.  I also changed the right: from 5 to 1.

While I was at it, I made one other tweak. I modified the display when you hover the cursor over a date that has an entry in the calendar so that the reverse display unreverses.  This makes it more obvious than the old behavior that just changed the color of the underlying text from blue to greenish.

Here's the code:

PS. notice the sleazy way I am presenting the code snippets?  They are screen shots, rather than text.  Why?  Saves me from having to edit a bunch of characters to keep the browser from interpreting the code rather than just displaying it.

I just can't help myself...

It had always bugged me that when I am leafing through the monthly archives,  I had to scroll back to the top of the page to get to the navigator to be able to go to the next month.

Notice the operative word had in the sentence above.

One quick change to the "Date-Based Archive" template - which involved simply copying the navigation code at the top of the page and pasting it in at the bottom - and now I don't have to scroll back to the top to navigate.

When the Linux server was revived, one of the things that I didn't get working was the UPS signaling.  I kept getting a message that the getty was respawning too rapidly and I couldn't figure out how to fix it, so I just disabled the entry in /etc/inittab.

I decided to play with it a bit today, and I partially fixed it - at least it's back to functioning the way it used to, which wasn't quite right, but close enough.  The not quite right part is that if I call powstatd as part of the boot process (i.e. from /etc/rc.d/rc3.d) as soon as I log in, the sytem shuts down as if the UPS were sending a low battery signal.  However, if I run powstatd manually after logging in, it runs fine.  My guess is that this is a bug in powstatd, but I don't have the interest to install the latest version right now, and it really doesn't matter, since once the system shuts down if the battery goes low, I have to manually power it back on, so I just restart the UPS daemon after I do so.

As to the specifics of the problem, the initial symptom was that the console would generate the following message:

INIT: Id "S0" respawning too fast: disabled for 5 minutes

This appeared to be a problem with how the serial port was configured, so I tried a whole bunch of different settings for S0 in /etc/inittab with no luck.  I searched through /etc/gettydefs and tried calling different stanzas in /etc/inittab with no luck.

I thought maybe there was a problem with the physical port not reporting its UART correctly, so I tried running the setserial command with various parameters with no luck.

Somehow, it finally occurred to me to see if maybe the problem was that even though there is only one serial port on the back of the system, that it was configured as Com1 rather than Com0, and maybe I should change the line in /etc/inittab from:

S0:2345:respawn:/sbin/uugetty ttyS0 F38400

to

S1:2345:respawn:/sbin/uugetty ttyS1 F38400

That fixed the respawning message, but still didn't get the powstatd running correctly.

I did some more digging, and discovered that I also had to modify /etc/powstatd.conf from:

# Watch /dev/ttyS0
watch ttyS0

to

# Watch /dev/ttyS1
watch ttyS1

I still haven't fixed the locale error message when httpd starts, nor have I fixed Xwindows, but I am happy now that the UPS signaling is working again.

This makes the 1006th entry for Blogdom...

I flew right past 1000 while posting all of those pent up entries.

Whoda thunk?

So, I changed my mind...

I posted one old post (Dead Sleep) as a recent entry, then decided it made more sense to keep the chronological integrity of the entries, so I posted them using their original creation dates.

So, on the one hand, scrolling back through the archives will (mostly) correctly reflect the order that the entries were created in.  On the other hand, the fact that Blogdom was off-line for six months will not be quite as readily apparent.

I just installed Beta 2 of WLW...

I have a number of entries that I created in Windows Live Writer that never got posted since the server was dead and thus there was no where to post them to.

I had debated whether to post them using their original creation dates to keep the blog chronologically correct, but I've decided to just post them using the current date as I get around to it.

Maybe I'll get around to it this weekend.

It's been almost six months, but we're finally back up!

That title could (should?) also be viewed as advice...

I ended up purchasing a replacement motherboard on eBay.  It seemed a bit silly to purchase a six year old motherboard with a 400Mhz AMD K6-2 processor, but I didn't want to take a chance that there would be a problem with the OS loading.

I didn't count on one of the drives dying in the interim.

The good news was that I had a backup.  The bad news was that it was two years old.

Fortunately, the server had two drives - one for the system directories, and one for the two primary user directories.  The drive that died was the system drive.

So, I was able to get the system back up with the Mondo Rescue bootable backup CDs, then install the old "data" drive.

It took some tinkering to get things working - subtle things that drive you crazy like the fact that the backup's network configuration had a different default gateway than the router I now use.

One of these days I'll need to break down and buy a new server and build a new software environment from scratch, but, for the time being, we're back online!

Amazing what a change disabling the comments and trackbacks have made...

Since I made the change:

• There has not been one new comment or trackback spam posted to the blog.
• No nusiance emails in my inbox alterting me to comment or trackback spam
• Average utilization on the server is down to below 3%

And, unsurprisingly, not even one email asking me to post a comment that the reader couldn't enter directly.

They've totally won.

I've had all I can stands, and I can't stands no more!

Well over 99% of the comments and trackback pings posted are spam, and I'm tired of having to clean up the garbage every day, so I've removed the ability to create comments or trackbacks.  My guess is that the maybe three people who actually intentionally read this blog don't really care.  Even they didn't regularly leave comments,  and they need to email me more often to let me know how they're doing anyway.

To disable the features, I modified the comment and individual entry templates to remove the comment entry form.

I tried renaming the mt-comments.cgi and mt-tb.cgi files in the cgi directory on the server, but this caused all sorts of errors in the blog, so, since I don't have the time right now to figure out how to do it "right" I just changed the blacklist in MT-Blacklist to have an entry for each individual letter of the alphabet.  Since the blacklist is now MUCH shorter, I am hoping it reduces the load on the server.

If you desperately want to leave a comment, you can send an email to:

comments at blogdom dot org

replacing the at above with @ and the dot above with .

The bad guys continue to do bad things.

"They" have now started using my witsdom.com domain to forge the "from" address on spam e-mails they are sending.  This results in two major hassles:

1. My in-box gets filled with non-delivery notices because many of the addresses the spammers send to are invalid.  Even though the "from" address is not a valid user in my domain, because I am the administrator, I see them all.
2. As more and more spam goes out and gets caught by automated anti-spam agents or gets reported by users, the entire domain gets blocked by more and more mail servers.  Eventually, I'll no longer be able to receive valid email sent from valid senders because the servers will block the mail due to the domain being viewed as a spam initiator.

I have searched the Internet trying to find a way to overcome this challenge, but so far, I have come up with nothing.  Any suggestions would be greatly appreciated.

Suggestions along the lines of: just give up and shut down your sites and domains and servers will also be seriously considered.

Sometimes, I really don't quite understand why I keep changing the structural elements of the blog.

Sometimes, I really don't quite understand why I keep writing new entries, but that's a different issue for another day.

Anyway, this idea of social bookmarking has me intrigued. So, I decided to add a couple of easy links for a couple of popular sites, digg and del.icio.us

It wasn't as hard as I had anticipated.  I created a new MT Template Module that contains the actual code to generate the links.  The key was to use the <$MTEntryPermalink$> and <$MTEntryTitle$> tags to construct the references.  I got put on the right track from an entry which gives specific examples for Movable Type on Dave Taylor's great site Ask Dave Taylor.

As an example, my code for digg is:

<!-- Start Digg -->
<img src="http://www.blogdom.org/images/digg.png" alt="Digg It!" /><a href="http://digg.com/submit?phase=2&url=<$MTEntryPermalink$>&title=<$MTEntryTitle$>" title="Digg It!" target="_blank">&nbsp; Digg It! &nbsp;</a>
<!-- end Digg-->

I then simply had to add calls to the new module in the entrybody module and the Individual Entry Archive module and then rebuild the site.

I'm still not super happy with the asthetics, but, it'll do for now.

I doubt I'll add any more, as I doubt I'll ever have an entry actually submitted to either, but, it was an interesting academic exercise.

The last two entries were old entries I had started, but never finished, much less posted.

I have four more waiting in the wings.

What I sometimes do when I don't have a lot of time is start an entry and outline it, with the intention of fleshing it out in the near future, and then getting it posted.

Of course, we know what they say about (g00d) intentions...

So, I learned a few new things today.

You know those challenge dialogs which ask you to look at a distored picture of numbers/letters, decipher them, and type them into a box, something that a human can do fairly easily, but is extremely difficult for a computer to do; typically as a method to prevent automated computer entries into a web page?

Well, those things are called a CAPTCHA, which is based on something called a Turing Test.

Well, doing a Google using CAPTCHA as the search phrase yields lots of hits.  While the top rated ones seem to be related to CAPTCHAs written for .asp or .net, if you hunt, you can find ones for perl or php.  Here is a link to a php script.

Doing a search using: captcha movable yields lots of hits including a nice implementation specifically for Movable Type written by James Seng called MT-Scode.  Its installation is a bit more involved, and requires the installation of a couple of additional modules on the web server.

MT-Scode Quick Links:
- Download latest mt-scode
- You need GD for this plugin to work
- You also need GD.pm

I also found the very interesting Six Apart Guide to Comment Spam which frowns on CAPTCHAs because of their negative effect on accessibility, but offers lots of good suggestions to managing/mitigating comment/trackback spam.

The big question is my mind is...

Do I implement a CAPTCHA, which would take quite a bit of time up front but which should allow me to re-enable the subscriptions and significantly reduce the time I spend removing spam, or do I just spend the few minutes a day with MT-Blacklist to delete the spam and rebuild the blog?

Considering that I've yet to have someone complain about the removal of the subscriptions - or the fact that they didn't work for so long, at least for the moment, I think I'll stick with the latter, but at least I've now documented an alternative, should I get inspired...

I continue to be vexed by spammers exploiting the Subscribe To Comments hack by ScriptyGoddess.  Apparently, someone has designed a robot which opens a specific entry, enters an email address, selects the "subscribe" radio button, then "clicks" the submit button, adding the email address to the subscription list.  Unfortunately, the existing Subscription hack does not integrate with MT-Blacklist, so even if the actual comment gets blocked, the subscription gets through.

As a bandaid, I have blocked an entire Class B IP address range from which the vast majority of the attacks have originated.

The ultimate solution would require upgrading to the current release of Movable Type or going back and unfixing sendmail on the server for the email spam doesn't get sent.

While I was poking around yesterday trying to solve the sendmail / spam issues, I realized that one of the other blogs I host had never had MT-Blacklist installed.  Turns out that part of my server utilization problem was the massive number of comment spams going through a separate instance of MT-Comments for that blog.  Since the primary author of that blog hasn't posted in almost two years, I just disabled MT-Comment as a bandaid.  Unfortunately, there are over 20,000 comment spams, and MT-Blacklist chokes when asked to perform a mass delete on anything much over 25 comments at a time.  It really bugs me to have that many comment spams feeding search engine rankings, but, I'm not willing to spend the time it would take to clear them out 25 at a pop.

SPAMMERS SUCK!

So, the good news was I got mail working on the server again.

The bad news?

Apparently, someone has come up with a new exploit which generates copious amounts of spam email.  I'm afraid quite a bit of spam went out before I figured out what was going on.

The exploit targets the Subscribe To Comments hack by ScriptyGoddess.

Apparently, some clever person figured out that they could directly access the emailtolist.php file which generates the email notification.  Somehow, they can use this even for posts which have comments turned off.

After I figured out what was happening, I changed the name of the file, and the references to it in the comment templates.  I believe the exploit is based on the specific file name, so I am hopeful that I can retain the functionality, without having it exploited again.

Arrg!

So, another vexing problem solved!...

For whatever reason, I've been more interested in working on the mechanics of the blog lately...

Today, I fixed a bug in my blog which has perplexed me for, literally, years...

I have continued to play with Windows Live Writer, and I must say, the more I use it, the more impressed I am.

The major new "feature" I discovered?

When you modify an entry which has already been saved and published, and then (re)publish it, WLW updates the entry in Movable Type.  With SharpMT, editing an entry and posting it results in a second entry being added to the blog.

I did also notice that the last entry did not have a Trackback link.  I don't know if that is because I reconfigured the default in MT to turn Trackback off - no one ever used the feature for anything other than spam - or if because it is another MT unique feature and therefore not supported by WLW.  I am going to set the Trackback in the Properties tab to "Allow" to see what happens with this post...

I am not putting SharpMT down - it still has a lot going for it, especially the full support of all of the unique features of Movable Type, but, I am impressed with WLW.  If only it supported Extended Entry...

I now have eleven lines in my .htaccess file...

I've decided to try a more brute force approach to stopping the worst of the spammers...

I accidently deleted two comments while clearing comment spam today...

It still tickles me when I get a new (legitimate) comment...

That last entry was number 900! I guess what I lack in frequency, I make up for in duration. I've always had a goal of 1000 entries in the back of my mind. At this rate, that goal will only take maybe five more years to get realized...