I had trouble with PHP 5.2.x, but have a bunch of wordpress sites running on PHP 5.1.4 and a couple on 5.1.6 with no problems at all. Several of the sites use wp-cache and were fine after the wp-cache-phase2.php patch. Both servers are Apache 1.1.37 and MySQL 5.0.27. No idea if that has any effect on WP.