Me and my partner in Dotvoid, a small web development and integration company, have used linux as a client OS for more than a decade. All the tools I needed for developing are there already; emacs, apache, php, MySQL, xdebug, other scripting languages, various compilers, OpenOffice and so much more. For my partner, Gimp and Inkscape have provided all the functionality for producing both designs and graphics. And all for free. Fantastic. Even so, a few months ago I suddenly decided it was time to abandon our ugly plastic laptops from Dell and Sony. I ordered two Macbook Pro. I haven’t used or even touched a Macintosh since 1995. But hey, they look good.

We soon discovered that some things aren’t as user friendly as we exptected. The multitouch trackpad with it’s gestures is wonderful. My partner still use Inkscape and Gimp under MacOS. Still excellent tools. But she needs more precision and bought the Magic mouse which she’s not very satisfied with. You can change the acceleration but not the speed. Annoying. OpenOffice also works like a charm on MacOS.

As for me I really – really – miss a decent keyboard with keys as delete, home, end, page up and page down. I didn’t expect them to be missing from the Macbook keyboard. In time I guess I’ll get use to the weird key combinations that is needed to replace them. I might even get used to the weird combinations needed to write backspace, brackets and curly braces. But the keyboard is not really designed for programmers. On the other hand the multitouch trackpad is great. Virtual desktops in MacOS is also a bit limited but it’s not a big issue. All in all I’m slowly getting used to the hardware and the way things work under MacOS.

However, MacOS is not enough. There are PHP distributions for Mac. But as I use Ubuntu in production I use Virtualbox to run a 32-bit Ubuntu in a virtual machine. After disabling the Intel VT-x CPU optimization the 32-bit Ubuntu runs as smooth as ever. For some reason Virtualbox refuse to disable VT-x for Ubuntu 64-bit which is the reason I have to use 32-bit. With the VT-x (optimization) enabled the virtual machine with Ubuntu freeze for a couple of seconds every five seconds or so. I work in many different places, in different wireless networks and often using 3G. To make it work in any network I’m connected to I needed to setup two network adapters for the guest OS. One network adapter to allow the guest OS reach the outside world and another for communication between the host OS and the guest. I also have Virtualbox running a windows installation to allow me easy testing in a windows environment without hassle.

When buying Macbooks I also decided to buy Zend Studio for MacOS. I haven’t looked back once to either PDT or emacs. It is a great feeling being able to use subversion, debug, refactor, run tests and much more in the same editor without setting things up and tweaking PDT. It just works. I’m happy with it. Let’s me focus on development. And it is not as slow as for example SpringSource Tool Suite for java development. The workspace with the projects I work on I have in a shared directory accessible to both MacOS and the Ubuntu virtual machine.

In addition I use MySQL Workbench a lot. It is also available for MacOS. It is a great tool for designing and working with MySQL databases. I still use the command line a lot as I’ve done since 1997 when I first started working with MySQL. But with just a few clicks the Workbench let me see changes between a database and the schema, export all the changes to a database in development, staging and finally production. (I always use an ssh tunnel to push schema changes to staging and production databases as I never allow ftp or anything else.) I still run the MySQL development databases on the virtual machine though.

OmniGraffle is another application I decided to buy. I use it to create diagrams and flowcharts but also to build graphical user interfaces. Especially when designing iPhone apps. Great tool and well worth the money. You can find stencils for most purposes at Graffletopia.

Even though we still use mostly open source software, compared to a linux environment, the move to Macbook cost a lot. Especially for a small company as ours. But after a few months I still think it’s worth the money. The actual hardware is great. Looks good. Silent although it does get a bit too hot underneath. The high resolution anti-glare screen is fantastic. I can even work in direct sunlight outdoors. I never want to see another crappy Sony Vaio. If only my Macbook could stop crashing every now and then I’d be really satisfied.

Same stupid debate as so often before. Just the topics are different from time to time. Found via the Swedish Utvbloggen, RWW, highscalability och myNoSQL.

Representing a tree of nodes, for example product categories, in a relational database is not hard. The most common way of doing it is to store a parentid with each node. This is also known as the adjacency list model. It is sometimes also referred to as the recursive model. Whatever the name it is easy to grasp. It is very simple to insert and move nodes around but it is not an optimal solution if you often select the whole tree from the database for displaying on a web page as you need to recursively select all children of nodes.

The nested set model

A much better solution is the nested set model. In this model trees are represented as nested sets. Instead of using a parentid you have a left and right value for each node expressing the relationship to other nodes. A tree of nodes can look like this.

        34
         |
         +--36
         |   |
         |   +--46
         |   |
         |   +--47
         |
         +--38
             |
            +--39

The same data can be viewed more graphically as nested sets. This is also the way they are stored in the database. Notice how the left and right values are numbered in order from the outermost left to the outermost right.

                            id:34
   +----------------------------------------------------+
   |             id:36                     id:38        |
   |   +------------------------+    +--------------+   |
   |   |    id:46      id:47    |    |     id:39    |   |
   |   |   +-----+    +-----+   |    |   +------+   |   |
  1|  2|  3|     |4  5|     |6  |7  8|  9|      |10 |11 |12
   |   |   +-----+    +-----+   |    |   +------+   |   |
   |   +------------------------+    +--------------+   |
   +----------------------------------------------------+

In the database this is stored as in the following table.

 +----+-----+-----+
 | id | lft | rgt |
 +----+-----+-----+
 | 34 | 1   | 12  |
 | 36 | 2   |  7  |
 | 46 | 3   |  4  |
 | 47 | 5   |  6  |
 | 38 | 8   | 11  |
 | 39 | 9   | 10  |
 +----+-----+-----+

This allow you to select the full tree with only one select statement. Normally you select the full list of nodes including the depth of each node as shown below in the small table containing id and depth. It is then easy to loop over the data in PHP to produce a nice tree in HTML/CSS.

The data below can be displayed as the the tree on right. The depth is in parenthesis.

 +----+-------+        34 (0)
 | id | depth |         |
 +----+-------+         +--36 (1)
 | 34 | 0     |         |   | 
 | 36 | 1     |         |   +--46 (2)
 | 46 | 2     |         |   |
 | 47 | 2     |         |   +--47 (2)
 | 38 | 1     |         |
 | 39 | 2     |         +--38 (1)
 +----+-------+             |
                            +--39 (2)

Here is a more thorough article on how you use MySQL with nested sets.

On to the problem

Last night I was working on an application that display a full tree where the user must be able to to drag and drop nodes to reorder the tree. The problem is that it is much more difficult to insert or reorder the tree using the nested set model.

For the actual html I produced nested ordered lists using the OL and LI html element so that it was easy to use with the excellent MooTree script based on MooTools. (It does seem to have trouble with the latest version of MooTools though.) After reordering the structure it was easy to extract the tree list including the depth of each node. I sent this to the PHP backend to be handled there. The text data I sent to the PHP backend then look like below.

 34,0
 36,1
 46,2
 47,2
 38,1
 39,2

As you can see it looks exactly like the data I selected from the database previously. So I had to create a function in PHP that took this indata and produced a list of nodes with the correct left and right values. This list is then looped over to update all the nodes in the database.

function depth2nestedset($data)
{
  $lines = explode("\n", $data);
  $rows = array();
  $stack = array();

  $lft = 0;   // Left value
  $rgt = 0;   // Right value
  $plvl = -1; // Previous node level

  foreach($lines as $line) {
    list($id, $lvl) = explode(',', $line);

    // Skip empty/faulty lines
    if (trim($id) == '') {
      continue();
    }

    if ($lvl > $plvl) {
      $lft++;
      $rgt = 0;
      array_push($stack, $id);
    }
    else if ($lvl == $plvl) {
      $pid = array_pop($stack);
      $rows[$pid][2] = $rows[$pid][1] + 1;
      $lft = $lft + 2;
      $rgt = 0;
      array_push($stack, $id);
    }
    else {
      $lft = $lft + ($plvl - $lvl) + 2;

      $diff = $plvl - $lvl + 1;
      for($n = 0; $n < $diff; $n++) {
        $pid = array_pop($stack);
        $rows[$pid][2] = $lft - $diff + $n;
      }
      array_push($stack, $id);
    }

    $rows[$id] = array($id, $lft, $rgt);
    $plvl = $lvl;
  }

  $plvl++;
  $cnt = count($rows) * 2;
  $leftovers = count($stack);

  for($n = 0; $n < $leftovers; $n++) {
    $pid = array_pop($stack);
    $rows[$pid][2] = $cnt - $plvl-- + $n;
  }

  return $rows;
}

The resulting associated array that is returned looks like below.

Array
(
    [34] => Array
        (
            [0] => 34
            [1] => 1
            [2] => 12
        )
    [36] => Array
        (
            [0] => 36
            [1] => 2
            [2] => 7
        )
    [46] => Array
        (
            [0] => 46
            [1] => 3
            [2] => 4
        )
    [47] => Array
        (
            [0] => 47
            [1] => 5
            [2] => 6
        )
    [38] => Array
        (
            [0] => 38
            [1] => 8
            [2] => 11
        )
    [39] => Array
        (
            [0] => 39
            [1] => 9
            [2] => 10
        )
)

That makes the circle complete. From left/right values in the database to a node tree based on depths that is easy to manipulate in html and then back to left/right values. Hopefully this makes sense. Do you have a better solution or more examples on applications that do this I would love to hear about it.

Searching Google I finally found a MySQL article on the nested set model which is a lot more efficient. The algorigthm basically skips the whole parent/child relationship between nodes. Instead you work with what is best described as “neighbouring” nodes. This allows you to retrieve a fully ordered hierarchical tree, including the level/depth of each node in the tree, using only one simple sql statement.

Those that are not familiar with the nested set model should definitely read the article.

Yesterday I decided it was time to get a proper PHP 4 environment configured on my new machine running the x86 version of Solaris 9. I know going outside Windows or Linux with PHP sometimes can be tricky. I haven’t used PHP on Solaris since my time at AOL in Dublin three years ago, so I decided to write down exactly how, what and why I did things. I expected it to be a bit simpler then last time as I planned to use Apache rather than IPlanet (or whatever the name is today, it seems to change at least once a week). So my plan was to install a pretty standard LAMP environment. That is without the L ofcourse.

# ln -s /usr/local/lib/gcc /usr/local/lib/gcc-lib
# cd /usr/local/libexec/gcc/i386-pc-solaris2.9/\
  3.4.1/install-tools
# ./mkheaders

# ./configure --prefix=/usr/local/mysql
# make && make install
# cp support-files/my-medium.cnf /etc/my.cnf

# cd /usr/local/mysql
# groupadd mysql
# useradd -g mysql mysql
# chown -R root .
# chown -R mysql var
# chgrp -R mysql .

# bin/mysqld_safe --user=mysql
# bin/mysqladmin -u root password XXXXXX
# bin/mysqladmin -u root -h myhost password XXXXXX

# ./configure \
  --prefix=/usr/local \
  --with-freetype=/usr/sfw \
  --with-png=/usr/sfw \
  --with-jpeg=/usr/sfw \
  --with-libiconv-prefix=/usr/local

# make && make install

# ./configure --prefix=/usr/local/apache-1.3.31 \
  --enable-module=so
# make && make install

# ./configure \
  --with-apxs=/usr/local/apache-1.3.31/bin/apxs \
  --with-mysql=/usr/local/mysql/ \
  --with-gd=/usr/local \
  --with-freetype-dir=/usr/sfw \
  --enable-gd-native-ttf \
  --with-png-dir=/usr/sfw \
  --with-zlib-dir=/usr/sfw \
  --with-jpeg-dir=/usr/sfw

# make && make install

LoadModule php4_module libexec/libphp4.so
AddType application/x-httpd-php .php

PHP5 is still far from being finished even though you can already download a beta version. The performance improvements, the better object orientation and XML support, the new PDO extension and many other small and large changes make many of us wish it was already here. After attending the PHP Conference 2003 in Frankfurt I wrote this article to give you an overview of the major changes in PHP5 from PHP4 and what to expect when porting old applications.

General improvements

Performance

PHP5 will generally be a lot faster than PHP4. Primarily, the execution model have been changed from PHP4. A php script is compiled into arrays of op codes before it is executed. PHP4 essentially use a big switch statement when executing operations from these arrays. PHP5 use function pointers and hash table lookups for function pointers are cached. Just calling PHP functions should be around 40% faster. Algorithms for comparisons have also been reworked and they should now be much more efficient.

Constants

Real constants are finally being introduced. Earlier define() was used for declaring constants. This was slow since it was actually a function call and resolved at runtime. This wouldn’t matter too much if it wasn’t for the fact that defines are used by many projects for storing translations of strings. It might even be the most common form of providing translations in PHP projects. Constants on the other hand are resolved at compile time. So as in most other programming languages the constants in the code will actually be replaced with the value it represents before execution.

XML support

Small revolution

The other area that is close to a revolution is the improved xml support in PHP5. As many other things, xml support in PHP4, was really just a big hack and squeezed in without much thought. Both the DOM and the XSLT extensions were marked as being experimental and they had many problems. SAX on the other hand worked pretty good but is a mess to work with. You have to write a lot of code to do very little xml parsing.

All the new xml extensions use libxml and libxslt which means expat and sablotron are being dropped. Libxml is one of the fastest implementations available and have very good support for the standards. The xml extensions in PHP5 are xml, dom, simplexml, xsl and soap. The xml, dom and simplexml extensions will most likely be compiled in as default which should improve xml support in hosting environments. The xml extension support loading of non wellformed html documents and most dom level 2 methods are implemented.

In PHP5 both the XSL and DOM extensions are new. The XSLT extension is called ext/xsl. The reason for it to be called xsl is to allow the old ext/xslt extension to stay for those who wish to use it. It should be noted however that libxslt is much faster than sablotron, which is used by the old xslt extension. The xsl api methods are derived from api methods in the mozilla implementation.

Problematic error handling

There is a problem with the error handling though. The dom methods throws exceptions where the dom standard so requires. In some other circumstances old type PHP errors will be produced. This mix of error handling methods is a bit confusing and hopefully this will be fixed by the time PHP5 is ready. Some dom methods are not completely ready. The method getElementsByTagName() currently returns an array which is not correct. This will change so that a real nodelist is returned instead.

Interoperability

All the xml extensions are now compeletely interoparable. They all load xml and create DOM trees in memory. This dom tree can then be shared and passed around between the different xml extensions. The best about this is that you can load an xml document into memory, use dom methods and xpath expressions to locate data and change it in memory, and then finally load a xsl template into it’s own dom tree and apply it on the original dom tree. You might not even have to load the xsl template since you could have it cached in shared memory from a previous request. The ability to cache dom trees in memory should give you a huge performance increase.

SimpleXML

This is all good and well. But the best part is yet to come – simpleXML. This new extension written by Sterling Hughes makes it so easy to work with xml data that, as he put it, any monkey with enough bananas can do it. I am inclined to agree with him. As with all the other xml extensions simpleXML integrate nicely with the others. What it do is enable you to work on xml documents as they were native PHP structures of associative arrays. A simple code example below loads a xml file with a booklist and displays the titles of the books. All in just five lines of code.

<?php
$booklist = simplexml_load_file(‘mybookfile’);
foreach ($booklist->book as $book) {
	echo $book->title .”n”;
}
?>

It can hardly be simpler. The extension allows you to convert back and forth between simple xml and a dom tree. You can instead of reading it from a file load the xml from a database and use the function simplexml_load_string(). Some design decisions made for simpleXML, to make it this simple, can make it weird to work with though. Node attributes are for example treated as just another level of nodes.

More advanced XML

Through the use of libxml PHP5 will support validation of xml documents against DTD, XML Schema and the simpler, independent standard, RelaxNG. Schema support in libxml is still not there to 100%. All basic things should be supported however. Validation of the xml is an important feature since you can be confident that when you are working on a dom tree that the data in it is syntactically correct. This is especially important when you start working with SOAP.

Unfortunately SOAP support is currently not finalized and yet to be added to PHP5. What is clear though is that SOAP will be supported natively by PHP5. Right now the options are to use either nuSOAP, the smaller but faster soap extension available through PECL or any other of the numerous PHP implementations available. NuSOAP gives you the ability to create WSDL specification programmatically through the use of a set of api methods. The PECL soap extension is however a faster choice since it does not have the overhead of loading the rather large NuSOAP class file written in PHP.

The xml extensions will also support PHP streams. Using the stream support you can register your own stream handlers. Supported out of the box are php, http, https, ftp, ftps and compress.bz2. One example of using your own stream handlers is in conjunction with xinclude to dynamically change the content of a static xml document.

Database support

The new PDO extension

This is another area that is getting a big makeover. There is a new extension called PDO, PHP Data Objects. Obviously enough for anyone is that it is derived from the name ADO. It is not, however, something that is supposed to look like ADO or work in the same fashion.

The pdo extensions give you database abstraction on the function layer and provides a uniform way of connecting to and using databases from different vendors. This is ofcourse a huge leap forward from the mess in PHP4 with all the different extensions for each database with different naming and parameter conventions.

More choice

The extension provides several new ways of sending queries to a database. It supports unbuffered queries where only a result handle is returned from the database. Unbuffered queries also support destructive reads. Using this feature means that data, as soon as you use it, will be destroyed in the engine. In other words, you cannot go back and reread the data returned in a result set. This is said to improve speed somewhere between 50 – 70% over buffered non desctructive reads.

You will still have the option of fetching rows in associative arrays or regular arrays. As associative arrays are much more expensive to create performancewise, the default returned will be numerical arrays. Several other query functions are also created to increase speed in certain circumstances. One method will allow you to send a query and directly get the whole result returned as a normal PHP structure. Yet another is the fastest choice when you only return one column.

A weird but neat feature is that you can query the database and at the same time provide a class. An object will automatically be intantiated and initialized with the values from the database.

Iterators

Using iterators, that also are new for PHP5, allow you to loop over a resultset without doing so explicitly in your PHP code. You could do this previously using arrays and a foreach loop. The foreach loop is extended to work with objects now. Since the looping is done internally in the PHP core it will be much faster.

The consensus is that with the pdo extension you will have more options than before and a greater opportunity to profile your code for speed.

Mysql support

Mysqli and PDO

Work on PDO, described above, was started quite recently. The need for better database api’s however was recognized a long time ago. MySQL have evolved during the years and is now a very mature database management system featuring transactions, sub queries, replication and many other things. Views and foreign keys are still not there but is slowly in the making. Foreign keys can even be tested in the early versions of MySQL 5.0. The mysql extension have on the other hand not evolved much. The api does not closely follow the C api and many things are missing. Because there are optional parameters to many mysql functions it is also hard to add parameters to functions without breaking backwards compatibility.

Therefore work started on a brand new mysqli extension. This api match the C api much more closely and supports the MySQL 4.1 api. It is not 100% backwards compatible. Since the old mysql extension will stick around for some time this should not be a major problem. It is more than likely that some PHP installations will have both extensions enabled at the same time, allowing both old and new applications to work.

Prepared SQL statements

New for mysqli is support for preparing SQL statements which let the MySQL engine compile them before you use them. If you use the same query several times with different data this will give you significant speed improvements over sending the full SQL statement every time. Since you supply the data together with a resource handle to the compiled query instead of a full SQL statement you will no longer have to bother about magic quotes and addslashes. The extension will take care about that for you. You should then obviously have magic_quotes_gpc set to off. You can ofcourse still send the whole statement at once exactly as before. However, when including large chunks of text or blobs that exceeds the maximum packet size allowed you should use prepared insert statements.

Currently it is not possible to store prepared SQL statements between requests. Zak Greant promised to look into this later as it would be very beneficial performancewise.

Encryption of passwords

One thing that might give some applications trouble is that MySQL have stopped using the old password crypto in favour of SHA-1. This is ofcourse much safer, providing 40 bit oneway encryption of passwords. Applications that use PHP functions to encrypt passwords using either SHA1 or md5 are not affected since this is on the MySQL side.

What now?

PHP5 have recently released a beta 2. It should probably not be considered as beta quality though. Most of the new features have been implemented already. Only five or six people are still in the business of discussing new or changing features for PHP5.

So it is not really beta quality. It is however stable enough to download, install and tinker with. For those of you who wants to be prepared it is probably time to start working on PHP5 support for your applications. There are enough changes in PHP5 that it might not be wise to trust it to be fully backwards compatible. We now know for a fact that it won’t.

At the conference both Sterling Hughes and Markus Börger answered the question – when? The optimistic Börger said March 2004 and the more pessimistic Hughes thought summer was more realistic.

Object oriented support

Hybrid language

Object orientation is one of the areas that is really changing in PHP5. Whereas there was only rudimentary support for object orientation before, PHP5 now implements most features required by OO purists. PHP will ofcourse be a hybrid language and apart from a few areas you can still ignore the object orientation and code procedural if that is your cup of tea. The fact remains that PHP is becoming a more object oriented language.

Changes in PHP5

One major change, that probably will decrease the bug count significantly in any project using classes and objects, is that objects are now sent by reference instead of by value as the default behaviour. You will no longer have to make sure you use ampersands at the right places. PHP5 have a reference count that keep track of the object usage. Objects will be freed when the reference count reaches zero, either by using unset() or by going out of scope.

A quick list of what object orientation in PHP5 introduces:

• Support for visibility, private, protected and public. You need to declare this for each property and method which is different from for example C++. If these keywords are omitted the default is public.
• Objects are now referenced by identifiers, or resource handlers, that really just are integers. You will have to use a copy constructor, or rather the __clone() method to obtain an actual copy.
• Constructor and destructor methods. The constructor can be parameterized but it seems there will be no support for several different constructors which is a big loss if you ask me. So the trick where you have several optional parameters and then count the number of parameters actually sent will still have to do – with all it’s drawbacks.
• Static members and methods. Class properties and methods that all objects of a class share. A weird behaviour right now is that you can call non static methods without instantiating an object. This will hopefully change.
• Interfaces allow classes to present themselves to the outside world and guarantees that a subclass implements all methods.
• Abstract classes, property members and methods that enforces you to implement them in subclasses.
• Final declaration allows you to stop further inheritance.
• Interceptors that allow you to initialize data when you need it. This is used in lazy initialization for example in database calls.
• Reflection API that helps you document your code.
• Default member property values. That you set even before the constructor is called.
• New PHP constants including __CLASS__ and __METHOD__. Both are evaluated at compile time. The first constant contain the class name and the latter evaluates to both the class name and the method name. If __METHOD__ is used outside of a class it evaluates to the current function.

Handling the memory

To destruct an object you call unset. The function unset only decreases the reference count and you cannot be 100% sure that the object will be destructed. PHP is a language where you should not have to care about the memory that much and it might be that destructors aren’t that a good idea in PHP.

Another big gotcha is that parent constructors and destructors must be called explicitly. This means that it is easier to make the code run faster. What might be thought of as being a bit inconsistent is that if you don’t have a constructor in the subclass the parent constructor will be called automatically.

Structured programming

Type hinting is also a step forward to a more structured programming model. This allows you to enforce that a parameter can only accept an object of a certain class. This is currently only planned to work with classes and objects – not on base types like strings and numbers.

The one new feature you cannot ignore is the introduction of try/catch error handling. The problem is that some extensions of PHP5 will use try/catch and others will stick to the old error handling where error codes are returned.

With the above new features it is possible to implement most design patterns, for example the common factory pattern and singleton pattern. If you take the singleton pattern for example. You can now ensure that only one object is created using static members and then overload the __clone() method and declare it as private.