If you are into Joomla! extensions development you are undoubtedly familiar with the rule of index.html, that is the necessity to put a “blank page” index.html file on any and all directories containing PHP files. This habit is so ingrained to the mentality of Joomla! developers that it’s now dubbed a “security feature” and made a prerequisite to publishing your extension in the Joomla! Extensions Directory. The thing is, is it really a security feature or are we trying to solve the wrong problem?

First, let’s examine the reason for putting index.html files all over the place. As you most likely know, on most web servers, if you don’t put an index.html file inside a directory, all of its contents are listed to the browser. For example, when I remove the index.html file from the cache directory and access it as http://www.example.com/cache/ I get a full listing of all of my cache directory’s contents.

This means that a potential attacker can click on any of those links and view the contents of my site’s cached pages. Even more sinister, let’s see what happens when there is a PHP file in there (like the example above), and click on it. Oops! The file got executed! 

This is a security risk, for three reasons. The first reason is that by directly accessing a PHP file in your site, a cracker may be able to retrieve sensitive information about your site (like your server’s path structure) to directly injecting SQL and files in your code. The other reason is that a cracker may be able to upload a hacking script on your site, through a vulnerable component, and access it over the web to compromise your site. The third reason is that an enterprising attacker could potentially use the names and size of the files to identify a vulnerable release of an extension installed on your site and launch an attack against it. Apparently, by placing the index.html files back in place, they don’t get file listings so we are safe and that’s why this is labelled as a security feature, right?

Bloody wrong, mate!

Whoever thinks like that, he’s thinking of security through obscurity which, as we all should have known by now, is no security at all. Let’s see why.

Why index.html files provide no security

First, we have to consider how a potential cracker works. There are two classes of crackers: script kiddies and serious crackers. The first class of crackers (albeit it’s a far cry calling them that) will simply try any vulnerability they know of on your site, no matter if it applies to it or not. You would laugh at how many times I see attacks in my logs that have nothing to do with my site: attacks against Joomla! 1.0.11, old versions of VirtueMart, even attacks against IIS. None of this is even installed on my site! They just “shoot at random” hoping that they hit a target. At any rate, the existence of the index.html file is completely irrelevant to their efforts.

The other class of crackers, the serious crackers, will not rely on directory listings. Let’s say that they want to figure out if your site is running Joomla!. As you’re aware, there is includes/js directory in your site, with an index.html file nonetheless, and a file named joomla.javascript.js. Have you guessed what I am at? Right, I’ll access it over the web: http://www.example.com/includes/js/joomla.javascript.js. If the file is there, the site’s running on Joomla!, if not, well, it’s not Joomla!. Using a similar process of analyzing static files (which index.html doesn’t protect) a cracker can figure out which version of the CMS you’re using and even which version of your extensions you have installed.

Finally, the presence of index.html does not block access to files uploaded to your site. If a cracker manages to exploit a badly written extension to upload an arbitrary PHP file (usually a hacking script) on your site, he’ll be able to access it no matter what.

Therefore the presence –or complete lack– of index.html files has no effect on your site’s security whatsoever.

Is there a problem with having index.html files?

Of course! Not only they provide no security, not only they provide a false sense of security but they end up diminishing the security of our sites and providing a disservice to our users. Not to mention how they can kill a software before it even gets the chance to be used by people... Sounds like a far cry? Read on and cry along...

Installation

During installation of a complex component, like Akeeba Subscriptions or Akeeba Backup, we end up having hundreds of useless files which make up 15% of the files included in the archive. Do remember that on shared hosts which require the FTP mode to install software we have to extract them to the tmp directory first (which takes much time on its own right), then FTP them to their final location. The amount of time required to install an extension is proportional to the number of files, not their total size, due to the inefficient way FTP works.

As a result, people start having blank page / 500 Internal Server Error pages when installing our components due to server timeouts. They never go to the JED to complain for the index.html rule. They always come back to us, the developers, screaming at us that we are incompetent idiots who should take up knitting instead of writing software. Here's the thing: even though I know what the problem is, I can't solve it because I am not allowed to by the JED rules. In other words, SNAFU.

Inode (file count) limits

Most shared servers tend to have an inode limit, i.e. they actively limit the number of files allowed to be stored on the server. By proliferating the inclusion of useless files we actively force Joomla! to artificially inflate the number of files "required" for its operation, counting towards the inode limit. Removing the index.html files can result to savings of about 20% on the number of files, providing an actual service to our users. Yet again, we know the problem, we know it's a borderline antifeature (artificial limitation which has no purpose of existence other than to bother legitimate users), yet we are not allowed to fix it!

Too revealing

The index.html files end up being too revealing. They do not serve a blank page. They serve an HTML page which has an empty body. That's a far cry from serving their intended, supposed, security purpose. Any potential attacker with half a brain will view the source and realise that there is something there. If it's something there, it means that the directory DOES exist (otherwise he'd get a 404 Not Found error). Therefore, the very existence of the index.html file serves as an admittance to the fact that the directory exists –a fact which would be known even without the index.html– therefore the extension exists and can be a potential target for exploitation.

In plain English: serving content, any content, when accessing a directory reveals that the directory is there. You have to return a 404 or 403 if you intend to do as much as confuse potential crackers.

Irreparable damage

Even this may sound like a long shot, index.html files can cause irreparable damage to an extension author's reputation and kill the software before it even reaches release 1.0. Here's my story about Akeeba Subscriptions and how following JED's rule on index.html sites made my efforts to create the only free subscriptions system for Joomla! go down the drain.

I had a pretty solid extension which was downloaded over 800 times before publishing it on the JED. However, in order for it to be accepted to the JED, it was mandatory to add index.html files throughout all the directories found in the package. After my direct appeal to abandon this rule (as a JED support ticket with a copy of 90% of the information found here) I was told off with something in the lines of "these are the rules, we're not changing them". So, I added index.html files throughout my package's files. One directory –aptly named "nooku"– is holding all the Nooku Framework files, which were to be copied to the site's root. An index.html file had to be placed there as well to satisfy the might forces of the JED. Inadvertently, this file was copied to the root of the site of any user installing the extension.

It all went downhill from there.

As you know, servers are configured to serve the default index files in case you don't specify a site. This is why visiting a Joomla! site at http://www.example.com shows your Joomla! site (it actually displays index.php) instead of a directory listing. Unfortunately, many servers are configured to show index.html before looking for an index.php file. As a result, a user on such a server installing my extension and trying to access the site would get a blank page. Despite me providing uninstallation instructions, I failed to notice the extra index.html file and people would report that I am a moron, idiot and other such nice things in the JED listing, downrating the shit out of it. This forced me to ask the JED to unpublish the listing until I found out what went on. It took a person who gave me access to his site to figure out what was going on.

The problem for me is now two-fold:

1. If I fix my extension to not have the index.html file, it won't break people's sites but it won't be accepted by the JED. If I do leave that index.html file in there, it will be accepted in the JED but break people sites.
2. The JED listing is forever "burned". It has been downrated to smithereens. If I ever ask for it to be republished, it will appear as my software is broken and kills people sites. Even though the problem was caused by the callous rules of the JED, they will not remove the unfavourable ratings. The only way to "reset" the negative ratings is to stop my extension being free and make it commercial instead. Or change its name, which is just plain stupid.

So, I am left with an irreparable damage. Many people consider me an arsehole who can't code and blame me for killing their sites (of course they are guilty for not looking for an index.html file, the first thing any half-knowledgeable webmaster would do, but I'll leave that to another rambling). I can't reinstate my JED listing unless I want to perpetuate this false perception. If I want to undo the damage, I have to provide a huge disservice to my users by making the software for-a-fee instead of free of charge. Therefore, I am forced to keep my software out of the JED, essentially flushing all the time I spent on it down the drain, as nobody will know about it and use it. And all that because of the ass-backwards JED rules which, still, add NOTHING to security, for crying out loud! Can you believe that? Wasting two months of your life and providing a disservice to the community because some non-developer who knows jack shit about security decided that index.html files are to be included in every directory for "security reasons"?

Really, is there a valid alternative to index.html files?

The only viable solution to fending off such attacks is the use of .htaccess files to limit access to arbitrary files and directories. I have written such a .htaccess (Master .htaccess) which I saw that it is now referenced in Joomla!'s documentation wiki. On IIS servers a similar solution can be devised, in the form of web.config files (the IIS equivalent of .htaccess). As a matter of fact, one can automatically convert the .htaccess file to a web.config file.

Of course, the other obvious solution is to use the defined(“_JEXEC”) or die() statement at the top of each PHP file. This simple trick ensures that the PHP files execute only when run from inside Joomla!, minimizing (or, should I say, eliminating) the surface area exposed to random web visitors, thereby obliterating the exploitation risk by directly accessing those files. It is a sad fact that even though perfectly secure extensions with a missing index.html are not published on JED, there are allowed modules with directly accessible PHP files, connecting to the Joomla!’s database without going Joomla!’s index.php pages, to be published. If you ask any security expert, he would conclude that allowing arbitrary entry points to your web application is much more insecure than allowing the listing of files in an arbitrary directory.

But, what do I know? I am just a developer who publishes his own security component and gave presentations about server security all over the world. The JED editors are, for the most part, non-developers who "know" what's best for us, even without having the faintest idea of what web site security is. Why should you trust me over JED editors? I can see no reason...

The rules should serve a purpose

The JED rule for the existence of index.html file throughout a component is the formalisation of a myth. About ten years ago, when Joomla! was Mambo and proprietary, this was indeed a security rule. By not allowing random visitors to poke through the contents of the directories Mambo was able to offer security through obscurity. In a system that only a handful of people know its inner workings, that’s an (almost) acceptable form of security.

Things have changed a lot since that era. Joomla! is now powering almost 3% of the web and its code is open source and available to anyone. Millions upon millions have downloaded it and hundreds –if not thousands– poked around its source code. A potential attacker needn’t go through the pain of accessing a directory listing to find out where each file is stored. It suffices to download Joomla! and the extension he’s interested in. As a result, the security through obscurity offered by index.html files is made obsolete by the very form of distribution and FOSS nature of Joomla! and its extensions (remember: the first requirement for being JED listed is to provide GPL'd software, therefore human-readable source code).

As a result, our insistence on clinging to an outdated, security-through-obscurity rule makes no sense. In fact, a quick look at other FOSS CMS would allow us to realize that we’re doing something which is obsolete and pointless. As a result, we are stuck with rules which serve no purpose other than their own existence, perpetuate a myth, enforce a false sense of security and can ruin the reputation of a developer.

And I will close this lengthy post with a Twitter message by security expert Jeff Channel, replying to another message by Joomla! co-founder Brian Teeman:

brianteeman: JFolder::create($path, 0777); has no place in ANY ext. instead of checking for blank index.html JED should check for real security issues

jeffchannel: @brianteeman How true - at least 2 of the top 5 "most favored" on the JED have SQL Injection bugs... ;)

That sums it up.