Tech

Shortly after I wrote my complaint about Thunderbird's junk filter last October, I decided to try to train SpamAssassin's Bayes filter. Using procmail, PHP, and MySQL, I built a crude interface to keep copies of my incoming email and let me classify them as OK or spam. After training SpamAssassin with over 7000 messages, I have to say I'm really disappointed. SpamAssassin still regularly misses spam, and Thunderbird frequently misses the spam that SpamAssassin misses.

(I added a similar (but prettier) interface to a web-mailbox that my client uses, and unfortunately the SpamAssassin Bayes feature has been ineffective for them too.)

POPFile is the only reliable blocker I've used, but it runs on the client and therefore you have to download all your mail, spam included, before POPFile processes it. I think the method I will try next is to turn off SpamAssassin's Bayes filter and just let it find the really obvious spam, then pass the rest onto the client for POPFile to sort out. Might as well turn off Thunderbird's junk filter at the same time.

In the meantime I keep pondering allowing only whitelisted addresses send me mail. Everyone else could use my web contact form. I'd just have to be really careful that I whitelist all the websites I use.

Via Slashdot I found this detailed article regarding various rounding methods. There are a lot more ways to think about it besides just rounding up to the next highest digit on the fives.

The article reminded me of a problem I worked on last fall. PHP's standard rounding function round always rounds the fives up. This was causing an upward creep in my calculations for hours worked in time clock data. I needed a way to minimize that creep. I settled on a different method of rounding known as "banker's rounding." This method alternates the rounding of fives based on the even-/odd-ness of the preceding digit. So for example, a 3.5 rounds up to 4 and a 4.5 rounds down to 4.

I have created a PHP function to do this, and the source code is here: GPL version; BSD version. You give the bround function two parameters: first the value to round, and second how many decimal places to keep. So bround(3.55,0) produces 4 and bround(3.55,1) produces 3.6.

I hope it's helpful to someone and please don't hesitate to report bugs or a faster way to do this. (Normally I'm not a fan of using the Ternary Operator but in this case it keeps the function compact and is fairly straightforward.)

Update 2007-04-23: Someone going by the name of "Hitlers Pet Gerbil" replied to my method stating it was "slightly incorrect." I posted a reply to the comment on the PHP site, but for some reason it was deleted. Today I stumbled upon a copy of my response, which was written on October 6 of last year:

In reply to Mr. Pet Gerbil, I think you're wrong when you state "Your calculations are slightly incorrect." My calculations do take into account when the thousandth's digit is a 5. I ran your function and mine side-by-side and got the same results.

However, I found your string-based method to be interesting and it runs faster than my code in testing. It would need some adjustment to allow the user to specify the number of decimal places. My function does not assume the user wants to round to two decimal places, though that likely is what the user will want.

Update 2007-10-01: Added BSD-style licensed version (see above).

Happy Holidays everybody!

Do yourself and webmasters a favor and use anything but Internet Explorer. Try Firefox!

I do work for a few websites and while most of my work is done behind the scenes on databases and scripts, I do a little basic web design. Today I was updating a site for a client and decided that instead of trying to match the background color of my logo graphic to the table cell color, I would try to use a transparent PNG instead. This worked beautifully in Firefox, but I had forgotten about Internet Explorer. IE can't render 24-bit transparent PNG images properly...it leaves an ugly cyan background.

Fortunately there are some workarounds. Normally I avoid these but too many people still use IE and I like PNGs. This site shows how to do it with Javascript. This one uses PHP, but it's meant for people using PNGs all over the place. As I only needed to do this for one image, I adapted their solution and used this code:

<?php
// Deal with PNG transparency problem
$arySize=getimagesize($strImage);
if (preg_match('/msie\s(5\.[5-9]|[6-9]\.[0-9]*).*(win)/i',
  $_SERVER['HTTP_USER_AGENT'])) {
  echo '<img src="spacer.png" ', $arySize[3],
    ' style="filter: progid:DXImageTransform.',
    'Microsoft.AlphaImageLoader(src=\'',
    $strImage, '\', sizingMethod=\'scale\';"></a>';
} else {
  echo '<img src="', $strImage, '" $arySize[3], '></a>';
}
?>

The spacer.png is a 1x1 pixel transparent 8-bit PNG which keeps the area empty until the filter has a chance to run (you can download it from the PHP-fix link above). If you put the original image there, you'll see the buggy PNG rendering briefly until the filter runs.

Ugly, isn't it?

After running this blog for nearly two years and never receiving a meaningful trackback link, I have turned them off. Trackback spamming has been picking up, including 5 today.

I'm getting to the point where I just might stop running this blog software. There's other ways to publish content. Spammers don't seem to bother as much with custom software...they go after the thousands of people all running the familiar stuff.

The PHPBB board that I maintain recently picked up a new user account: Analina. The user signed up with the email analina@gmail.com. The user left a link to visualpicz.com, which redirects to a page full of prescription drug links.

At first I thought it was someone who signed up manually, but I think this was automated, and it bypassed my basic bot preventer. The reason I think it was automated is because if you do a Google search on "analina phpbb" you get over 37,000 results! I guess I need to improve my bot prevention scheme.

So I'd advise PHPBB admins to delete that user and ban that gmail email address. It would also help if all PHPBB board would tell search engine spider to not index the memberlist and user pages, which would render spammer links useless.

I've been using Mozilla Thunderbird for my email client for over a month now, and one thing I am disappointed with is the Junk mail/spam filter. This filter misses dozens of spam messages a week. The developers should just throw away that junk filter code and integrate (or share code) with POPFile. I run that on another PC and it rarely misses a spam message.

What I'd really like to do is offload the whole spam-determination process from my local PC to the mail server. The reason I gave Thunderbird a shot was that Outlook and the Outclass add-in were memory-intensive and slow. I run SpamAssassin on the server, but there's no nice GUI way to train the Bayes filter. If I get a spare moment, I might try to build a basic POPFile-style web interface for SpamAssassin.

I just received an email puporting to be from the Republican National Committee requesting donations to the Red Cross. The email has graphics straight from the Red Cross.

I thought it was weird since I'm not registered with any political party and I already gave to the Red Cross for Katrina relief. Then I looked at the links, which led to a web server in Asia, all set up to look just like the American Red Cross donation page. Apparently nothing is beneath internet thieves.

This blog has more info on other Katrina phishing scams.

Please make sure you go to the real Red Cross page if you donate.

I've been playing around with the PHP PEAR class HTML_QuickForm. So far using it has not been very quick, but I hope to get used to passing the form work off to other code. Of course, it also means that you have to review someone else's code if it doesn't work like you expect.

In trying to duplicate and improve a form on my client's website, I wanted to use the 'html' element to add some text to an element group. It turns out that you can't do this, as the formatting will be wrong. There is a bug in PEAR for this, but they haven't updated the documentation. Don't let it waste an hour of your time like it did mine!

Instead of this:
$aryGroup[] = &HTML_QuickForm::createElement('html', 'Choose a password.');

Use this:
$aryGroup[] = &HTML_QuickForm::createElement('static', null, null, 'Choose a password.');

I was relying on a couple lists at blackholes.us to block a lot of spam. But the server is down without any news as to why. When I can't use the blackholes.us blacklists, my spam triples.

I found a replacement list to use at okean.com, but they don't have a DNSBL, so I have to figure out how to best integrate their list with klunky sendmail. Anyone have a tutorial?

Update 2005-09-30: I recently found countries.nerd.dk and I'm trying out their DNSBL. Yesterday it blocked a couple hundred spams from the countries I specified.

Today I happened upon this website which lets you upload GPS trackline/waypoint data, and it plots the data on a Google Map, which you can zoom in/out and scroll around. Very cool. I need to start playing around with the Maps API; it would be fun to incorporate this kind of stuff right into this website.