Maintaining Standards with PHP Code

In this article I want to talk a bit about standards. Not necessarily standards like you’d find with W3C, and not code formatting standards, but coming up with a consistent plan as you code and sticking with it. Staying close to the PHP manual and following developments with PHP so you don’t get bit in the butt later. I’ve been working with PHP for some years and have seen it go from version 2 through the current version and there have been many changes along the way.

In particular, I was thinking of two specific situations from my past which can bring down a site unexpectedly.

Shared Hosting

This is hard, but for most sites it is how they get online. You find a host, pay a monthly fee, and they take care of the rest. You just code and upload. This is fine, but the lack of control can be frustrating at times. Not only that, you never know who the person is playing SysAdmin on the other end of the wire.

In early 2000 I built a site for a company and it ran for nearly a year when I made a major international move. We were getting ready for another modification after I got settled. Amazingly enough, I was moving to one of the cities where this company had an office. When I got to my destination in early 2001 I had a horrible time getting an Internet connection. I think they were still in the dark ages.

Unfortunately, during my 6 weeks without an Internet connection (known now as my ‘blackout’ period), the shared host decided that PHP 4 was mature enough to upgrade. They sent out plenty of warnings that major changes were coming….but I didn’t get them without e-mail! One day they finally changed over and I had to go borrow a computer to find out why the site didn’t work. It turned out that PHP changed the format for notation for Apache directives. Very simple, very minute, probably not even necessary from my point of view. We got it fixed and we were back on the road and in a couple months we were working toward another redesign.

They say that waking up is hard to do

Have you experienced this? One day your site is down. You clear your schedule, visit the mailing lists and forums and spend your day (or two) finding out what went wrong. Waking up from this nightmare may leave you in a cold sweat. Keeping up with PHP or any software can be difficult. PHP isn’t bad, but you will notice some inconsistencies in the interface and over time things like that will be fixed, eventually breaking old code. What’s even worse is keeping up with shared hosts. Let’s take a look at scenario #2 from the hosting company’s point of view.

I worked for a typical Internet hosting company. We had many web sites on our machines, all with access to PHP. Although I can’t go into details, I would say their implementation of PHP was unusual. It was certainly not stable or secure.

It wasn’t unusual for the SysAdmin to come to work in the morning, read his e-mail, find out there was an upgrade for PHP released overnight and install it. That’s fine for your development server, but would you do that for the machines hosting many thousands of web sites?

Well, they did, they still do and so do many shared hosts. Not the good ones of course. The good ones I’ve been with test very thoroughly before making upgrades. Then they notify the clientele in advance of a change so they can monitor their site.

The problem is, you probably don’t know who is on the other end of your web site. What is their level of ability? Their work ethic? You probably don’t know.

The Standard

The ‘standard’ in this case simply reads “Keep up to date.” Read the PHP manual. Subscribe to the mailing lists. Get the RSS feed. Know what they are doing.

Scan through the manual frequently so you know what is going on. When you see that something is being phased out, like the HTTP_*_VARS, don’t keep developing with it!! Change immediately and start modifying your old code. That day will creep up when it’s gone and you won’t know when. Your site will go down even though you didn’t change anything. You’ll be on the phone with tech support blaming them. I’ve seen it. Worse yet, while you are on the phone with tech support, your former clients will be calling telling you their site is down.

If something is labelled as experimental in the PHP manual, be ready for changes. If you use it, don’t expect it to always work as it is now. You can even help the PHP team by submitting feedback if you use experimental features.

The Reward

The bonus reward for staying up to date is you are unlikely to be surprised. Not ever of course, things happen beyond our control. We can’t know everything, and we certainly can’t know the future. By knowing what’s coming as it is released by the PHP team you can at least get a head start, minimize the impact, and minimize the site outages, which could otherwise mean lost revenue.

If you get the opportunity to be your own SysAdmin, try it, at least on your development machines. PHP isn’t that hard to maintain, and you can see performance boosts by compiling only what you need. It is a big job to maintain a machine though, so if you aren’t up for it then make sure you know you’re hosting company well. Don’t just pick one. Peruse the online comments but don’t assume you are always getting a fair shake because hosting companies offer outrageous affiliate fees making people work hard to get referrals. Talk to friends, use tools at your disposal and make sure that the company you deal with only hires professionals in their field. If you made a bad mistake choosing a company don’t be afraid to move. You shouldn’t reward them for a bad job.

Taking these steps will make sure you are ready to go at each turn in the road. Neither rain, nor sleet, now snow can take down your site!

Now, are you ready for a slashdotting???

Bad PHP Code – Blogs and CMS

Lately people have been blaming PHP for being insecure. While most of them seem to blame little tiny issues that, when used incorrectly by the programmer, make their scripts insecure. Does that make sense? The programmer uses it insecurely, but the problem is in the language. Absurd. Guess what, you can do that with any language. You can make your house insecure too by failing to fasten the door locks. But more to the point, after examining some of the Blog and CMS software for PHP I’ve come to this conclusion – if that kind of programming is general across the board for these packages then we have a bigger problem.

Blame the Messenger

The first thing we need to know is that we are not all knowing.

Often times we get people learning programming from PHP because it is so easy. That’s what I did back in the days of PHP 2. PERL was just a little too hard for me, and in 1995 I had a real hard time scouring information on the web to figure out what CGI was. But I went from getting my feet wet in PHP to learning C to going to college, then going into full-time work. I was lucky.

When I think back to my early days of PHP I can see how we can be dangerous. We typically focus on getting something done, not really how to do it or the best way or even the most secure way. I was so proud when I built the unthinkable using PHP the first time.

The problem when we are that young in a skill is that we think, “OK, I’ve learned this skill, let’s move on.” What we are forgetting is that we know a language but we don’t know how to use it. We’ve figured out where to put the pieces to form sentences – that is, how to write code to not get errors. The problem is, we still don’t know how to have a conversation.

What I’m getting at here is that sometimes we forget that we need to consider why we write code. We need to understand security. We need to know how the web works. We need to know how a browser renders HTML or CSS or XML or whatever you are working with. We need to know the consequences of using a database system, the overhead caused by connects. We need to know how to bug track, profile and optimize for best efficiency. There is much more work to do once you your code “just works.”

Extending the Problem

That is what happens when we are young. Then we explore, then we work, then we forget. That’s me πŸ™‚

As we move along and take on big projects we need to keep on moving upward. From what I’ve experienced, it is common for people to get interested in one area and focus on that. We know programmers who don’t know a thing about hardware. They don’t know what a stack or register is even. On the other side, we have the hardware pros who think they can program because they know the basics of a language. They don’t know what a stack or register is even πŸ™‚

CMS and Blogs

How does this apply to Blog software? I really don’t know. I can’t tell what those individuals and development teams were thinking when they spec’ed out their project. IF they did that. I don’t know what they were thinking when they decided to put PURE PHP code inside template files made for HTML coders, graphics designers and common citizens. Seeing “DON’T EDIT THIS CODE” inside a template file is not acceptable in a project. I especially don’t know what they were thinking when they decided to make “index.php” a 10,000 line piece of code that did everything except the admin.

Obviously there are several CMS and Blog packages for PHP out there that are less than ideal. Some are very big and popular. I’ve seen absolutely useless database connects and queries inside a page that slow it down to a crawl. Major packages with 15+ queries on a blank template page that only returns navigation. You don’t want to install this kind of stuff on a shared host, and if you even get the kind of traffic your blog deserves you’ll be looking for a new package at a really inopportune time….and then figuring out how to migrate your data.

The End

I’m hoping soon we see the end of this kind of thing. The trouble is that PHP is such an easy thing to learn that we won’t. What we need is a crop of fantastic programmers to step up and build something we can be proud of. I have a feeling that won’t happen though. Most of those programmers are employed customizing the blog software that already exists out there for people who found the limitations πŸ™‚

Why Dynamic Page Caching in PHP is a Good Idea

Just yesterday I talked at length about the caching benefits of the Smarty Template Engine for PHP. Today I was able to see an example of why caching was such a good idea.

In that article I used an example of an e-commerce store that had a page with three ‘Top 10’ lists. The 3 variations were for 3 different time periods, and the ‘all time best sellers’ list was a killer SQL query that had to look through 6 years worth of sales records to determine the top sellers. I used Smarty’s version of template and page caching for the script which caches the output in a PHP file. I then chose to regenerate the cached file once per day resulting in only one database connection and query PER DAY instead of one query every time someone viewed the page.
Just yesterday I talked at length about the caching benefits of the Smarty Template Engine for PHP. Today I was able to see another example of why caching was such a good idea.

In that article I used an example of an e-commerce store that had a page with three ‘Top 10’ lists. The 3 variations were for 3 different time periods, and the ‘all time best sellers’ list was a killer SQL query that had to look through 6 years worth of sales records to determine the top sellers. I used Smarty’s version of template and page caching for the script which caches the output in a PHP file. I then chose to regenerate the cached file once per day resulting in only one database connection and query PER DAY instead of one query every time someone viewed the page.

As we discussed, this large query resulted in an execution time around 4 seconds, which was intolerable from my point of view. With the implementation of Smarty, that dropped the execution time down to around 0.4 seconds with nearly no work on behalf of the hardware. Win-win.

However, today I got a first hand example of how NOT using Smarty, or any caching for that matter, is a bad idea.

Changing Programmers

When I left that company, I had a brief meeting with the incoming programmer who really wanted to delve into Smarty for some time. He was quite eager and wanted to do some work. As it turned out, the site was functioning so well on it’s own that the management put him to work on a new project. It seems they wanted to separate the store from the company site so they would have two sites.

This was a good idea conceptually, but for whatever reason the programmer decided to NOT use Smarty on the new store, even though the programming was already in place. They did leave Smarty in tact on the static pages corporate web site. Smarty on the ‘static pages’ was originally put in so the data punchers in the office could maintain the mundane details of the HTML files (templates) without having to see (and break) th PHP code that runs everything. Smarty does a great job of separating code from display.

At the time I left they moved the whole she-bang off the shared servers onto a dedicated machine and I peeked in to see that their site was one of the fastest functioning sites I’ve ever seen, and even Alexa rated it as ‘Fast’ – which is rarely seen in Alexa ranks.

Then it all went downhill…

Somewhere along the way that decision bit them. I peeked into their new store today to look at the “Top 10” page. To marvel at my earlier work, and to revel at how fast Smarty caching is.

When I got there I remembered, “Oh ya, new store, new site.” I clicked the link to the best sellers page and I waited. And waited. And after 40 seconds I got bored and did something else. When I came back the page had loaded.

40+ seconds to load a page???? I thought 4 seconds was intolerable!! What could have possibly gone wrong.

I need it yesterday

Don’t we all. I hated that phrase. I can hear the management saying that right now. “I know you just came to me with the idea today, and I didn’t even know it was possible before that, but it is such a good idea that I NEED IT YESTERDAY. I can’t live without it now!” Geez. I find that rather impulsive.

That’s sometimes the way it goes, and that could be what happened here. All we know for sure is that the management wanted to move the site away from the corporate identity, and with the move came some new features to try and increase revenue. The programmer probably didn’t get his chance to work with Smarty and chose to go without it and use what he was comfortable with. In the process, completely wasting the money they’d spent on the project to that point.

Despite a move to new hardware, you have to remember there has probably been an additional two years worth of sales records to sort through in that query, and more products in the store. Couple that together with new product distribution methods, new lines, and then likely a rewrite of that ‘all time best sellers’ query by someone who may not have been comfortable with SQL and you get a 40+ seconds page execution time. Something which makes the pages virtually inaccessible and strains the load on the hardware during peak times.

The moral

Sometimes it is important for us not to become emotional when faced with things we don’t understand – like defending a programming methodology. Managers need to know that it takes time to sort out the facts – nothing you do today can transcend space-time, i.e. “I need it yesterday.” Not everyone is at the same level of understanding. Information systems can frequently be complex and even web designers need to consider they may be building software that could be in use for some time. Plan for the future.

The bottom line is, get acquainted with a good tool like Smarty now, while you can. Don’t wait until you are faced with a hardware problem, an expensive query, working with a new graphic designer or management that is in a hurry before you decide to look for those tools. Get them in your library, get the skills under your belt and start to implement them right away in the projects you are doing today.

Trust me. You’ll appreciate that caching template system the first time your system gets slashdotted πŸ™‚

Smarty for PHP – Caching

I never thought I would need to worry about caching. Hardware these days is fast. But the more I thought about efficiency the more I realized that my pages don’t change that frequently so why go through the overhead of connecting to a database on every page load? Or for that matter, why load a static file, unserialize something stored, recalculate the ‘year’ in the copyright tag. For that matter, why perform any of these tasks every time the page is loaded when they may change infrequently? Well, luckily Smarty has caching built-in and I found a use for it.

From the Smarty web site:
Here’s where I get happy. I never thought I would need to worry about caching. Hardware these days is fast. But the more I thought about efficiency the more I realized that my pages don’t change that frequently so why go through the overhead of connecting to a database on every page load? Or for that matter, why load a static file, unserialize something stored, recalculate the ‘year’ in the copyright tag. For that matter, why perform any of these tasks every time the page is loaded when they may change infrequently? Well, luckily Smarty has caching built-in and I found a use for it.

From the Smarty web site:

Caching: Smarty provides fine-grained caching features for caching all or parts of a rendered web page, or leaving parts uncached. Programmers can register template functions as cacheable or non-cachable, group cached pages into logical units for easier management, etc.

Wow. Seems like a lot is covered. I can cache the whole page if I want. I can cache the whole page EXCEPT for a dynamic element like a banner ad, or I can register a dynamic function that picks a random Amazon.com book in the midst of an otherwise static page – all while keeping to Smarty’s idea of separating the logic and presentation.

Database queries and caching with Smarty

In a recent project I had to put a full-text search form on a web site. This wasn’t easy because it was searching through HTML files – a mini Google if you will. The trouble was the function that creates the ‘headline’ clip from the page (that highlights the search terms) seemed to be very slow. After asking mailing lists and spending days in the manual I could not get the problem solved. I had a deadline to meet and could not change it.

What I did was implement the current version of the search – problem and all – and used Smarty caching to save load on the system (and save my bottom). Luckily there was an HTML list (links) of popular queries, and a mechanism to record the queries. I used a script to regenerate those queries and use Smarty to cache the results. The only time we had to connect to the DB and perform the query was the first time a search was performed. This was a lifesaver, and it dropped the time to query down to a value that was barely perceptible. Mainly because it rarely had to query the db!

Further Database Queries and Smarty

Another benefit of Smarty caching can be recognized when you have complex queries. We must all work within boundaries. Sometimes you don’t get the chance to select a better DB system, or add hardware. In other words, you work with what you have and find a way to make it better.

I once had to write a page for an e-commerce store that would display the top sellers. Not just top sellers, but three columns of top sellers. There was the top sellers this week, top sellers this month, and top sellers of all time. These changed because of frequent specials and of course, the time spent in the store made a difference in the rank. It was hard to push a product introduced a month ago and put on special this week unless it showed up in the list of top sellers for this week.

But how do you go about that? Performing the SQL queries is easy enough but getting a result back is hard. Really, there is no reason to perform this query 2000 times a day as people drift in and out of the store. The problem is, the store was on a shared host with a dedicated remote database. There was time associated in the connect of course but even more important was the 3 complex queries. Also, they chose a shared host meaning they could not just throw money at the hardware to speed up their queries.

The only problem query in this scenario was the all time best sellers top list. Of course we have no trouble with the others because indices on the right columns make those a snap. But how about sorting through sales records for 6 years and getting a rank for the top sellers? Owch. That can take a long time. Well, a long time from the perspective of a web surfer.

Four seconds to be exact. Can you imagine that? Can you imagine a few people at the same time making that query? That’s the kind of thing that makes machines grind to a halt. You’ll see that type of thing in some of the recent content management systems (CMS) that don’t use any caching. I recently tested one that could only support about 4 users at a time on modest hardware because of their extensive use of, in my opinion, many unnecessary queries.

With Smarty though, this query becomes a non-issue. I ran a script early in the morning that erased the cache for that page and requested it again which regenerated the cache page. You can do this with a simple call to wget or lynx. Problem solved. One query a day is all it took. And because the page showed top sellers for this week and this month there was no need to have it in real time. Even if it was in real time there is no reason to think a sale today would affect sales from the last 6 years in such a dramatic way that it would push an item up the list all the way from the bottom to the top. But even if you wanted the ‘this week’ top sellers list in real time, or a list for the best sellers today, you would just include a dynamic block in Smarty, and only that section would be generated on that page load, the rest of the real hard queries would be cached.

With that in mind, Smarty caching took a script that I wasn’t even able to execute due to the time involved, and turned it into a vital part of the store that helped increase visibility for the top selling items. It also took that 4 second query down to a page load time of 0.4 seconds.

Database connects

As mentioned above, the shared host used a remote database. That’s a great thing to do actually, but it can kill you on the connect. Of course the connect is what kills you on most databases as long as everything else is up to par.

But why connect to the database at all? If your content is cached you don’t need to, again saving that overhead and speeding up your display that much faster. If you do need to connect to a database to record some statistics or similar, you can do it after the page is sent to the browser by putting your database connect and your subsequent queries in the part of your script AFTER the call to $smarty->display() – meaning the page has already been sent to the browser.

Summary

Smarty has a multi-faceted approach to caching. Smarty first takes your template page and caches that as a PHP file. That means it doesn’t have to parse the template on every request. If you choose to use the other caching features, Smarty will then cache the actual output of your page, saving all of the overhead of dynamic files, but allowing you the ability to update the cache whenever you want.

Even if you don’t intend to use caching in any large way it is still a good idea to use Smarty to get acquainted with the idea of an intermediate cache. The Smarty cache will reduce the load on your hardware enabling you to enhance the end-user experience and decrease your dependence on bigger, more expensive hardware. It also allows you to grow your site(s) and traffic without having to worry about hitting the wall and taking the system down during a growth spurt, which can potentially harm revenue. In other words, a Smarty cache let’s you plan for the future. And as you’ve seen in my recent case, Smarty cache can help you get out of a bind when you are dependent on other developers and a timeline.

In short, there simply is no reason not to use a cache. Smarty allows you to implement only as much or as little caching as you like. As a professional PHP or web programmer you need to be efficient as well as effective, and part of being efficient means caching infrequently updated dynamic content.

Smarty for PHP – evil?

I was reading some PHP blogs last night just to pass the time when I came across one entitled something like “Smarty is evil.”

After reading the post I concluded that the author gave Smarty a fair shake, and indeed, acknowledged that this was his own opinion based on a first impression during a very short time examining the Smarty Templating system for PHP. In my opinion, I would consider this author to be a fair designer with logic because he is able to discern his own emotional opinion from the true facts. A trait hard to find in those who work with computers who are many times very opinionated and highly defensive. Kudos to the author for getting past that.

What disturbed me though, was the comments. A large number of comments, and the consensus of the commenter’s on the original post was that yes, Smarty is evil. For the most part, no one acknowledged having any more experience with Smarty than the author. Owch! Any one of those commentators would likely make a designer or programmer of questionable ability.

Let’s examine some of the comments (arguments against Smarty) and look at the facts of the Smarty Template system for PHP. For the record, I’ve been using Smarty since nearly it’s inception (about 7 years at the time of writing).

Using a templating languages with PHP makes no sense.

I guess that depends what type of coder you are. And what type of employee or contract worker you’d be. And how much time you enjoy spending maintaining your code.

I worked as a full-time PHP programmer for a web host. For the most part I was assigned a project and told to ‘go.’ The thing I dreaded was having to edit some of the legacy code. This following scenario is very typical working with people who think they are ‘programmers.’ Their entire public site, everything you see, the marketing literature, the sign up form, the credit card processing, the SQL queries, the mailing list…..everything….was contained in ONE FILE – the index.php file.

I’ve seen this done everywhere since then and can’t believe people actually do this. I had to stare down a 10,000 line minimally commented mega-file including PHP and HTML and try to determine where a particular element was. Some people would call this ‘job security’ for the idiots that wrote it, but most people would just balk at their abilities.

The bottom line is, this is the kind of garbage you see with PHP. People using it from the perspective of ‘personal homepage’ and applying it to ‘online business web application mega-project.’ You need a different mind set when working with a customer control panel than you do if you are just trying to display the current date on your homepage. That’s why the above comment, “Using a templating languages with PHP makes no sense” is absolute bunk.

Escaping in and out of php with the short open tags is fine for some things but in the above stated file it was sheer nonsense. We perform some logic in php, then escape out of PHP to print it, then back into PHP in the middle of the HTML to echo a variable, then out of PHP to HTML, then out of HTML to PHP to perform more logic. It is a mess, and it is eating the time of programmers who think this is the way we do business.

Smarty does the opposite. It follows a very nice model that allows you to separate your logic from your display. The way it was meant to be. I’ve used Smarty on the command line to generate a dynamic e-mail for a mailing list, I’ve generated .smil files, and I’ve used it to generate HTML and it works great. Tracking down errors is very quick because you can look at one file for the logic, and the other file for the display. But the error you found is not in both, it’s in one or the other and you know by looking at the results where it is. Another nice thing is that it removes that mammoth 100k of HTML from your code and puts it somewhere else. Your logic becomes very short and scan-able. How nice that is.

What’s moronic is that people go to all this trouble building templating systems when all you really need is extract(). Sad, really.

I don’t think I want to get into that one.

The authors of the book ‘Web Application design with PHP 4’ had a good idea when they basically said, don’t try to roll your own templating system when a good one already exists. Bottom line, you didn’t think of everything, and you don’t have the team of designers they did, and in the case of the comment above, you didn’t think of security like they did. Rolling your own is ok if you are a student looking to experiment, learn, analyze and debug. Rolling your own is not ok when you have limited time, a boss or a client to answer to and an actual project to complete – you know, the reason you are using the template system in the first place.

Smarty is really a waste of time.

Quite contrary.

I have been working with Smarty for 7 years now, pretty much since it’s introduction and I can say, in every instance, Smarty has reduced the time it takes to develop an app. Even something as simple as a contact form interface, a single page with two possible outcomes.

Yes, Smarty takes time to learn, but didn’t PHP? If I was developing Smarty I would have done things a little different to reduce the time on the learning curve, and maybe that’s a possibility in the future. But indeed, a template system in general and Smarty in particularly is not a waste of time, and will speed up your development time thanks to the separation of logic and display. Once you consider Smarty’s incredible template and page caching mechanisms you’ll see it saves you resource time as well.

Why? I don’t really know, but I can tell you that when you don’t have the possibility of escaping PHP into HTML whenever you want you tend to think more in Data Structures. You create a real, formatted data structure and pass it along to the template via Smarty. When you do that, you tend to have better, more reusable code. More reusable code tends to mean less time programming.

The other thing I’ve found is that a shorter file size means less code, means easier scanning. There have been several studies done on formatting and productivity and it is easy to see why this works. If you can scan your code then you don’t have to spend time thinking ‘what is going on here?’ With no escapes in and out of PHP and HTML your brain doesn’t shift either. It stays in one mode the whole way and that, albeit subconsciously, seems to speed you up. In many of the scripts I do now, the logic takes up less than one screen on my editor, and that is sheer joy.

Summary

Overall I’ve been very happy with the Smarty Template Engine for PHP and I will continue using it on each project I do. In fact, on one project I was in such a rush that I chose to leave it off. Big mistake. It took me longer to complete. Less than a week after the project was finished and went live I was coming back to re-do it in Smarty. I’ve been using it for about 7 years and I keep choosing it again and again. If you enjoy shorter code, separate logic and display, caching and super fast applications with low overhead then Smarty is your choice for PHP.