Programming books in general

I’d better start off with telling you that I haven’t read many “pure” programming books since the mid 1990’s when I was studying at the university. I tend to stick to books on concepts and methodologies and then read online tutorials and articles when it comes to programming. Programming books are rarely worth the money as they are either reference books which gets out of date quickly (I’ve thrown away quite a few Java books over the years…) or learning by example books which you painstakingly slowly must follow line by line to build something thousands open source projects alrady built.

My thought was that this book was probably both outdated (as ZF 1.9 has been around for awhile) and probably boring in that building a web shop isn’t that interesting. I was wrong in all ends.

The book

I especially like the way Pope introduce the MVC concept in Zend Framework. In the first chapter you get a good overview on configuration, the bootstrap process, controllers, action helpers, views and error handling. Chapter two goes deeper into the specifics of the request and routing. He also briefly delves into more advanced topics as plugins and component customization. The reader quickly gets a very good grasp on how to work with MVC applications in Zend Framework.

Another thing I like is how Pope reasons about various strategies; both pros and cons as well as how and how not to do things. Good examples are chapter four where Pope gives a good explanation on the Fat Model Skinny Controller strategy and chapter five with it’s best practices regarding accessing models from views with the help of view helpers. Reasoning and explanations like this is good, especially for unexperienced developers. This – and giving optimization and testing their own chapters – makes the book better. As Pope is thowing in both Zend Tool as well as Ant into the book makes it even better still.

Summary

To sum up my opinions the structure of the book makes it easy to read at leisure as well as using it as reference later. The MVC concept and how it is used in ZF is way better explained than in the ZF documentation. (Truth be told the documentation is hopeless here.) I’ve been working with ZF a “long” time now and whether the book is better for complete beginners or programmers at least a little bit familiar with Zend Framework I really can’t tell. Remembering how I struggled to find a good introduction to various topics found in the book I belive this book is a better introduction to Zend Framework than can be found in online tutorials and articles – or even the online documentation.

However, a beginner will get more out of this book than just learning Zend Framework. He, or she, will also learn about unit tests, build procedures and best practices.

Even though free time has been somewhat scarce I have tried improving my routines when it comes to development and coding. But as the workload has been going up and down a lot I’ve mostly used the few slow periods to relax. I haven’t been to a PHP conference since 2007 and a more general software development conference since 2006. I feel I’ve been, work aside, somewhat lazy the last two years.

So I decided it is time to read a bit more on agile methologies, Zend Framework, setup a completely new laptop development environment using Ubuntu 9.10, Zend Studio 7.1 and Zend Server. I will also try out Mingle from Thoughtworks for project management. Ok the last thing can be a bit weird having only one additional team member besides myself on current projects. But I’ve been interested in trying it out ever since hearing about it by Dan North from Thoughtworks in an software development conference a couple of years back. Expensive but interesting to see if it can be used in teams trying to stay agile even though they are spread out geographically.

At the moment I’m rereading the excellent book Practices of an Agile Developer. It is a brilliant book packed with inspiration that can help you develop and improve your habits for an agile approach to software development. It is good to browse through it every once in a while to remind yourself on how you should work Conveniently enough I was also just recently shipped a copy of Zend Framework 1.8 Web Application Development which I hope will be an interesting read as well. I also feel it is about time I read a more in depth book on XP or Scrum. I’ll see what good books I can find on that topic.

So the next few weeks, when I’m not working or trying to be a good dad to my daughters, I will try not to shut down my brain in front of the TV and instead study for a bit.

In my experience, trying out new things and reading a few good books, can do wonders to your inspiration and also give you more energy to face both challenging as well as more mundane, even boring, tasks.

After a trip to Dublin I finally launched the beginning of a new social tourist Dublin guide. It is exactly the same site as both the Swedish Fuengirola guide as well as the English Fuengirola guide I launched after living there for five months. The functionality is somewhat basic as of yet but every now and then I’ll add something more.

Basing three different sites, even though they are very similar, using two different languages gives me the possibility to try out several parts of Zend Framework. It also requires a good design both in the backend and frontend to keep it maintainable. I thought maybe some people would be interested in a basic overview of the different parts needed to put everything together.

Backend

The sites aren’t really that complicated. On a basic level there is a MySQL database and a database layer using Zend_Db/Zend_Db_Table, there is a Zend Framework MVC architecture using models, controllers, views, layouts with Zend_Layout as well as a few view helpers and some HTML, CSS as well as Javascript. All commonly needed. There is more needed to make it a sitethough.

I want as little configuration per site as possible but I naturally still use Zend_Config_Ini for settings. Zend_Registry is needed to keep the global scope clean and is used to store instantiated objects that need to be available throughtout the website logic.

I have been a bit particular on using ZF as often as possible on these sites. With few exceptions. Thus I also use Zend_Form, Zend_Locale, Zend_Translate, Zend_Cache and more. Zend_Cache is really a no brainer and mainly used to speed up translations. This is extremely easy as Zend_Translate and Zend_Locale both are connected to the cache with one simple method call each. It is not complete but how simple this is to setup is illustrated by the below code.

$configuration = new Zend_Config_Ini(
    APPLICATION_PATH .'/config/app.ini',
    APPLICATION_ENVIRONMENT
);
$frontendOptions = array(
    'lifetime' => $config->cache->lifetime,
    'automatic_serialization' => true
);
$backendOptions = array('cache_dir' => $config->cache->dir);
$cache = Zend_Cache::factory(
    'Core',
    'File',
    $frontendOptions,
    $backendOptions
);

$conf_locale = $configuration->locale;
$locale = new Zend_Locale($conf_locale);
$locale->setCache($cache);

Zend_Translate::setCache($cache);
$translate = new Zend_Translate(
    'array',
    APPLICATION_PATH .'/config/translation-' . $conf_locale . '.php',
    $conf_locale
);
$translate->setLocale($conf_locale);

Zend_Form have had a few problems in many versions of ZF. In my opinion it is also a bit bloated and limiting to be used all the way. (Even though I like the automatic connection to the translation functionality in Zend_Translate and the validation through Zend_Validate) So I have settled for a simpler way where I use Zend_Form fully for validation (with Zend_Validate) and then give the view access to the form through a Zend_Form subclass to be able to print the fields individually. I think this is easier to handle than all the overloading and coding needed to fully make Zend_Form create forms as I want them. Another good thing with the form classes is that they too are locale aware and are translated automatically by connecting them to Zend_Translate through the simple line Zend_Form::setDefaultTranslator($translate);. The sub classed Form utility class looks like below. The generated elements are translated automatically. Very convenient.

class Custom_Form extends Zend_Form  {
    /**
     * Render a field
     * @param string $name The name of the form element
     */
    public function _e($name) {
        $e = $this->getElement($name);
        return ($e) ? $e->render() : "<dt>Error</dt><dd>Missing <$name> field element</dd>";
    }
}

Another thing needed for these sites are to keep the urls understandable and thus translated into the language used on the site. For this the routes, or paths, are kept in the translation file as well. The translated paths are then registered to the router through the use of Zend_Controller_Router_Route and Zend_Controller_Router_Route_Regex objects.

$frontController = Zend_Controller_Front::getInstance();
$router = $frontController->getRouter();
$router->addRoute(
    'route_review',
    new Zend_Controller_Router_Route(
        $translate->translate('route_review'),
        array('controller' => 'review', 'action' => 'index')
    )
);

For image manipulation I haven’t looked further than ImageMagick. After a bit of tweaking you get very good quality when producing different image sizes.

Search

Last but not least, on the backend that is, there’s the search engine. ZF have an implementation of Lucene through Zend_Search_Lucene (derived from the Apache Lucene project). This is the one time I haven’t gone with ZF as I very much like the open source Sphinx search engine as it is so easy to integrate with MySQL. So Sphinx get to power the search.

This is convenient for several reasons. MySQL is not a great full text search engine, Sphinx give you better weighted results. The main reason though is the performance. Sphinx is very fast in itself but for a site with heavier traffic it is as simple as moving the search backend to it’s own machine to get a performance increase.

Frontend

On the frontend there is as clean html as possible to make it easy to change the design with only CSS as well as manipulate the client side with JQuery. If it wouldn’t be for the Google map the site would actually be pretty useful even without no css style at all.

A good practice I learned just recently (remember I’m mostly a backend developer) is to base all javascript functionality on modules that are instantiated depending on element ids present in the html. This makes it a lot simpler to split the javascript functionality into manageble pieces and also makes it a lot easier to maintain.

There are functionality in the client side javascript used to display error messages and information. This makes it necessary to create one javascript file with translation stirngs for each language. The correct javascript translation file is chosen in the layout (template/view file) depending on the current locale.

The map used is (are there any alternatives) Google Maps API. I chose to completely initiate the map by scanning the actual data displayed on the page. This is possible through clean html markup and also means that if I choose to list ten reviews on a page instead of five nothing need to be changed at all in the frontend. Only a loop limit on the backend.

Conclusion

Well. No conclusion. This was just a very basic walkthrough of most of the different pieces needed to create a fairly simple Zend Framework based website. Mostly it was a walkthrough of the bootstrap… I still hope it is useful or at least interesting.

I know I find it interesting to read about other sites and the architecture and design behind them.

The bug is known and open so I didn’t need to report it. I voted for the issue to be fixed, waited two minutes and still no fix… I thought that was how open source projects worked

Seriously though, I fixed the issue locally and added what I think should be a correct fix in the comments. (Even though I think it is weird that the transfer adapter should even have the remotest connection to translation logic.)

http://framework.zend.com/issues/browse/ZF-6647

Looking at the statistics for Zend Framework issues I think it is time to step back and slow down on implementing new features. The diff between new issues and resolved issues is constantly getting bigger and bigger while the average time to fix an issue is getting longer and longer. This is natural as the project constantly have grown since it’s birth. But as I like working with ZF I’d hate to see it become a big and unruly behemoth that adds more problems than solutions.

It is important to know that floats are actually locale aware in PHP. I’m not sure if that have always been the case. The below code will actually output 10,5 as Swedish use “,” as decimal point.

setlocale(LC_ALL, 'sv_SE.UTF-8');
echo 10.5;

The test in Zend_Validate_Float looked like

$locale = localeconv();

$valueFiltered = str_replace($locale['thousands_sep'], '', $valueString);
$valueFiltered = str_replace($locale['decimal_point'], '.', $valueFiltered);

if (strval(floatval($valueFiltered)) != $valueFiltered) {
  $this->_error();
  return false;
}

Under the Swedish locale mentioned above this will result in a comparison between two strings; “10,5″ and “10.5″.

I have proposed the following solution.

$locale = localeconv();

$valueFiltered = str_replace($locale['thousands_sep'], '', $valueString);
$valueFiltered = str_replace($locale['decimal_point'], '.', $valueFiltered);

list($num, $dec) = explode('.', $valueFiltered);
if (strval(intval($num)) != $num || strval(intval($dec)) != $dec) {
  $this->_error();
  return false;
}

But I just realized that it can be further simplified.

$locale = localeconv();

$valueFiltered = str_replace($locale['thousands_sep'], '', $valueString);

list($num, $dec) = explode($locale['decimal_point'], $valueFiltered);
if (strval(intval($num)) != $num || strval(intval($dec)) != $dec) {
  $this->_error();
  return false;
}

The framework is getting better and better even though I probably never will use things like the form builder functionality. One should never say never. But it seems backwards to me and I think it makes it more difficult to produce more advanced user friendly form based applications. Maybe I should force myself to use it a few times so that I can have a better opinion on it. Maybe…

Anyways, I’m more than happy with the first few days.

Why am I not happy with emacs and the power of the command line and all the tools available a few key strokes away you might ask? Well, as being a lazy programmer is nowadays considered being a plus, I want some more help from the environment. I want to be reminded about all those FIXME’s and TODO’s scattered in the code. I want better integration with subversion, better and easier testing and reliable refactoring support for starters.

Getting started

It is easy try an IDE and quickly loose interest. You start the program, look around, get annoyed when you don’t find the keyboard shortcuts or don’t understand the concepts. I’ve done that before. I know this and decided to use Zend Studio on the biggest project I’m on at the moment. The team members are located in two different countries, use different operating systems and time is sparse. Everything needs to run smoothly. Blissfully ignoring the tingling bells warning me not to do it I imported the project into Zend Studio from subversion.

That part went really well. It seems easier to import a project from subversion than directly from a directory structure.

Using the IDE

One of the concepts I always have had trouble understanding is when the IDE keeps weird project files in the code tree. Thouse files should not be part of the project. Team members should be free to use whatever tools they find best for their productivity. Zend Studio does not do this even though it has it’s own status for files which is good. It can still be a bit annoying when Zend Studio gives me the weird error Could not open the editor: Editor could not be initialized when opening files. Huh? If you look at the java exception, available as error details, it only means the resource or file have been changed from outside of Zend Studio. To open it you need to refresh the file through F5. A minor nuisance but why not tell me the file is changed and ask if I want to reload the file? It can’t be that hard.

There are many good things though. I like having PHP Documentor just a click away. Using the refactoring feature and being able to reliably rename class methods throughout the project is a big plus. Code folding, inline language warnings and errors in the editor and easily navigating between different places in the code are also things that makes life easier. It is also something you’d expect in any modern IDE. Sometimes I miss grep on the command line but it is probably just a matter of getting used to the GUI way of searching.

So all in all I’m happy this far. The IDE gives me the support I want. But there are a few things that I think is problematic.

Subversion integration. It is easy to see the history of a file and compare against different revisions directly in the IDE. But it is actually very hard to see the small annotation icons which indicates if a file has local changes. (I have a 17″ laptop with a screen resolution of 1920×1200.) And the other day when committing a change that spanned many files in many directories not all of the changed files were selected for commit. I didn’t notice this and subsequently the when updating the test environment it broke. Easily fixed but still annoying. This problem together with the editor not handling file updates from outside of the IDE is making life a little bit harder than it should be.

Encoding support. I am in the habit of using UTF-8 for everything. This particular project was originally created on a windows machine and all the files were encoded in ISO-8859-1. I didn’t give it much thought as i’ve been on the project for some time and Emacs have always detected the encoding without bothering me. Zend Studio for Eclipse use UTF-8 as default but completely ignores the encoding in the already existing files. I didn’t notice the problem until a team member using windows called me and asked why I screwed up the files.

It is my responsibility but the stupid editor didn’t exactly help me. Setting the project encoding to ISO-8859-1 and then manually fixing the faulty files removed this issue.

Both of the above problems can be fixed by me being a bit more careful now that I know more about the IDE. It is still serious when you can’t trust the IDE fully.

The last and most annoying bit is the speed. Or rather lack of speed. The program is slow. Dead slow. I have a modern dual core laptop with 3 Gb RAM running Ubuntu 8.04.1. Zend Studio for Eclipse is a bit slow to start with. After working a few hours in the IDE the editor can’t even keep up with my writing. After a long day the laptop sometimes start swapping. I don’t know if it Eclipse, the Zend Studio part or java itself. The speed is still not acceptable.

Summary

I chose to try the Professional edition as I want, among other things, subversion and refactoring support which is not available in the community version. The price is 399€ which isn’t that much if it increase my productivity enough. The slow editor and the fact that I have to double check many things isn’t really a productivity boost though. I haven’t used all the features I thought I needed. On the other hand I have only tried it with an old, non Zend Framework project. Support for the framework, which I use for new development, is integrated into the IDE in various ways and I haven’t had the time to try it out.

The trial version is working another two weeks. Before then I have to decide if the extra features in the commercial version are worth 399€.

What I want to aggregate are Swedish web development and web entrepeneur blogs. I like the planet-planet concept that for example planet-php use. (Though I don’t think planet-php use python…) Conveniently I had this idn domain, översikt.se, lying around which I haven’t used up until now. As is normal nowadays (for me at least) I used Zend Framework to build the site. It was the first time I used Zend_Feed class. I works like a charm.

The whole website logic (both model and controller) is about 200 lines of code. And the main logic of fetching the feeds are really nice and short.

foreach($urls as $url) {
    try {
      $feed = Zend_Feed::import($url['feedurl']);
    }
    catch(PDOException $ex) {
      error_log($ex->getMessage());
      continue;
    }

    $feedTitle = $feed->title;
    $feedLink = $feed->link;
    $title = '';
    $link = '';
    $desc = '';
    $pub = '';

    foreach ($feed as $item) {
      if (is_a($feed, "Zend_Feed_Rss")) {
        $title = $item->title();
        $link = $item->link();
        $desc = $item->description();
        $publ = ($item->pubDate()) ? $item->pubDate() : $item->date();
      }
      else if(is_a($feed, "Zend_Feed_Atom")) {
        $title = $item->title();
        $link = $item->link('alternate');
        $desc = $item->content();
        $publ = ($item->issued()) ? $item->issued() : $item->published();
      }
      else {
        error_log("Feed <$feedid> is an unsupported format");
        continue;
      }

      // Parse and normalize date and check if new
      $time = strtotime($publ);
      $publ = date("c", $time);

      // Execute the previously prepared sql insert statement
      $result = $sti->execute(array(
          'feedid' => $url['feedid'],
          'title' => $title,
          'link' => $link,
          'publ' => $publ,
          'desc' => $desc));
    }
}

I skipped database exception and error handing in the above code to make it even more readable. Zend_Feed have failed me only once when trying to read and parse an old weird rss format. Wonderful.

(For those that don’t have browsers that support idn-domains I linked to the punycode variant of the domain name översikt.se, www.xn--versikt-80a.se, above. But mostly because my damn editor refuses to use the Swedish letter “ö” in the href attribute.)

It looks like PHP 4 is still going strong as a worried Clay Loveless write on his blog. Ivo Jansch have an older article, PHP 5 adoption – a summary, on the subject. It is true that the PHP 5 uptake has been slow. Many internet service providers as well as many developers are reluctant to switch. This is probably, as others have stated, a circular dependency. When lots of popular open source applications still only work with PHP 4, ISP:s are reluctant to upgrade. As ISP:s are reluctant to upgrade the popular open source projects are hesitant to upgrade. I still think the wrong people are worried. Still PHP 5 is increasing it’s share and finally the day will come when version 4 will be considered legacy. Hopefully before PHP 6 is here but not necessarily.

PHP 5 is not all about objects as some developers seem to think. New features in the language itself as well as modern frameworks like Zend Framework written completely in PHP 5 makes it easier and faster to produce quality software. Even though people – surprisingly – think that todays situation is set in stone and that things never will change they will. Popular open source applications written in PHP 4 will be overrun by new fresh applications that fully make use of all the new features. These applications are developed faster with better quality. If the service providers do not upgrade they will soon not be able to attract new customers. Eventually they will even start losing customers.

So if open source projects want to stay competetive, the project leaders – if any – would do good in thinking hard on switching before it is too late. The ISP:s would also do good in planning ahead. The new fresh applications of tomorrow (and today) won’t work on their servers. If they want the new customers they need to adopt PHP 5 soon. I think that the early adopters, that struggle today, may soon be the winners.

I decided that it was time to really make the effort and test a framework in a real project. I had a look around and decided to use the brand new Zend Framework as the base for a hosted help desk solution. I need a help desk solution myself to handle customer support in my company and fail to find something I like. It seems help desk applications  are either to complicated (packed with CRM, CM and help desk features), or just too ugly (in my opinion). So all in all a perfect project that would fill a real need.

Why Zend Framework over more mature frameworks then? It is still brand new and very much in development. Zend Framework Preview 0.2.0 was released only a couple of days ago. The reason I chose this framework over others was first of all simplicity. But no less important is the fact that it has no legacy – there are no PHP 4 backwards compatibility workarounds. This means that it can fully take advantage of PHP 5.

I have been working with the framework for a couple of weeks now and I must say that I really like the simple MVC implementation in Zend Framework. I have started using only the built in “PHP template” solution but it would be easy to switch to Smarty or any other template engine. The help desk solution is coming along nicely and should soon be ready for a public demo/beta. I just need to decide on a name and domain where I should host it.