One such thing is the occurrence of the UTF byte order mark, or BOM. The UTF-8 character for the byte order mark is U+FEFF, or rather three bytes – 0xef, 0xbb and 0xbf – that sits in the beginning of the text file. For UTF-16 it is used to indicate the byte order. For UTF-8 it is not really necessary.

But for UTF-8, especially on Windows, it has become more and more common to use it to indicate that the file is indeed UTF. Most text editors handle this well and you won’t ever see these bytes. As it should be.

The problems start when you are using PHP binary safe string functions such as strcmp() and substr(). Then these three bytes that won’t be visible even when using var_dump() can become bothersome. (You would however see that the string length output by var_dump() is correct and also counts the invisible bytes.)

So you need to detect the three bytes and remove the BOM. Below is a simplified example on how to detect and remove the three bytes.

$str = file_get_contents('file.utf8.csv');
$bom = pack("CCC", 0xef, 0xbb, 0xbf);
if (0 == strncmp($str, $bom, 3)) {
	echo "BOM detected - file is UTF-8\n";
	$str = substr($str, 3);
}

It’s as simple as that.

The bug is known and open so I didn’t need to report it. I voted for the issue to be fixed, waited two minutes and still no fix… I thought that was how open source projects worked

Seriously though, I fixed the issue locally and added what I think should be a correct fix in the comments. (Even though I think it is weird that the transfer adapter should even have the remotest connection to translation logic.)

http://framework.zend.com/issues/browse/ZF-6647

Looking at the statistics for Zend Framework issues I think it is time to step back and slow down on implementing new features. The diff between new issues and resolved issues is constantly getting bigger and bigger while the average time to fix an issue is getting longer and longer. This is natural as the project constantly have grown since it’s birth. But as I like working with ZF I’d hate to see it become a big and unruly behemoth that adds more problems than solutions.

$isochars = "\xFC\xFD\xFF";
$asciichars = "uyy";
$urlfriendly = strtr($actual_string, $isochars, $asciichars);

What struck me was that the character þ (FE in hex) was translated into y. As is correct if you look at how y, in different forms, was used as an abbreviation for the, that and so on in old English – or Anglo-Saxon. Often it is now written “Ye” as in the blog post title. (Yes – the “Y” in the title should be pronounced as “th“.

However, it is a bit odd as the Icelandic language still use the letter frequently. The sound value is more or less the equivalent of the English “th” in this or the.

The major value in SEO friendly urls is the readability. Shouldn’t it be more friendly and natural to translate þ into “th” in SEO friendly urls then?