Written by: 
With the vote on the Joomla! restructuring coming to a conclusion pretty soon I would like to take a moment to reflect on what is the problem and how (or if) it's being fixed.
The problem from a contributor's perspective
If you ask ten people what's the problem with Joomla! you'll get ten different answers. A common theme in all the answers I have heard –and I did listen to a lot of clients, developers and acquaintances– is that Joomla! seems to be going nowhere. Or at least not fast enough. I believe that we all agree with that. The last major change in the CMS happened in Joomla! 3.0 three years ago. In computing this a century ago. This stall doesn't happen by lack of talent or want. We certainly have very talented developers in the Joomla! world, at least as many as other popular CMS. We most definitely do want to move forward, there's no doubt about it. What is the problem then?
The problem is in the processes of Joomla! the organisation, not the code of Joomla! the CMS per se.
Let's say that Alice has a great new idea she wants to implement, be it a new code feature, a marketing idea or a brand new service that would set Joomla! apart from its competition. Let's also say that Alice has the technical background, free time and will to carry out all the work necessary towards that goal. So, where does Alice start?
Alice would be a fool to make a ton of work only to get it turned down. So she must talk to someone. Who? Nobody knows. Right now there are too many cooks and no chef or recipe in the kitchen. Alice would have to start asking around –in forums, Facebook, wherever– to figure out who to contact. Usually there's no response and Alice leaves in frustration. This can be fixed by a directory of go-to persons published in the joomla.org menu, but I digress.
Assuming that Alice does find someone to talk to, that "someone" will at best be a small cog in the Joomla! machine with no authoritative power because there's nobody with authoritative power – not necessarily a bad thing.
I can't comment for things beside the PHP code, so let me pretend Alice has a PHP-based feature to contribute. At the very best she'd get in touch with a member of the PLT and ask them if she should contribute this feature. The typical response is "write it, make a PR and the PLT will decide if and when it should be included". You've just lost Alice. Alice only has so much time in her hands and the possibility of wasting it is off-putting. She'll let someone else go through this. There is a fix for that too, as long as we identify the actual problem.
The fundamental problem is one of vision
You'd be excused to think that the problem is just a matter of lack of communication. But this is just a symptom of the fundamental issue of Joomla!.
There is no vision.
And by vision I mean a roadmap. Not the joke of a roadmap currently published. If someone reads this roadmap the things that strike them as odd would be a. version 3 continues to minor version 11 and b. the changes between minor versions are miniscule. Coupled with the Version History and the backwards compatibility statement on the Roadmap page what they're told is that Joomla! is a dead project walking. The same codebase is scheduled to be maintained until the end of 2019. That would make it an eight year old code base by that time.
Let me remind you which code base was current eight years ago: Joomla! 1.5. Can you imagine someone using Joomla! 1.5 today? Yet this is what our current Roadmap promises. By itself this would be not entirely bad if Joomla! at least did it the WordPress way: add useful new features between releases. But Joomla!'s distorted view on backwards compatibility is that new versions can only be added in minor releases, 6 to 12 months apart.
Right now nobody has any idea what Joomla! should look like in 3, 6, 12 or 36 months.
There is no vision for the future. There's only a wish: people will come up with new features. But why would someone spend weeks or months building something if there is no authoritative way to know if it will be included?
Would you agree to complete a project that takes month without knowing if you would be paid for it?
This is what we're currently asking potential contributors. They should undertake a major project, without any help and without any guarantee that their work will be included. This even happened with the vast majority of Google Summer of Code projects. Instead of attracting talent we made damn sure that new talent will stay the hell away from Joomla!.
We are (still) not fixing what is broken
The current leadership restructuring proposal doesn't address the fundamental problem of lack of vision.
If anything, it adds even more cooks to the kitchen and still no recipe. Under the new structure any department has a say on anything else. I understand the reasoning behind it, I'm just not sure it's well thought out./p>
I'm afraid what the decision making process will be for important decisions, such as Joomla! 4, under the new structure. Ever heard of Joomla! 4? It's Joomla!'s unicorn. Joomla! 4 should have been already underway but it's not even in the planning stages yet. It's quite obvious to anyone who has a passing relevance to PHP code that Joomla! 4 would need to be bold and break backwards compatibility hard – which doesn't mean that you won't be able to do one click upgrades, don't be confused! If for no other reason, in the last 5 years the entire PHP community has shifted towards using common building blocks available through Composer instead of inventing their own solutions to the same, already solved problems.
Do you really think that a radical change of this magnitude would pass when everyone and their dog can object? Mind you, I've seen that exact process take place in real world organizations as a business consultant. It's design by committee and it's governed by Parkinson's law of triviality a.k.a. "bikeshedding". I don't have a readily available solution for this, at least not one palatable by most people in the Joomla! community.
Now what?
Regardless of the outcome of the vote on the leadership restructuring I firmly believe that Joomla! needs to fix its fundamental issue as soon as possible.
Joomla! needs to immediately work on a vision for Joomla! 4, in an express process which must be complete at the latest by December 2015. Joomla! needs to start amassing the developer, designer and UX expert talent necessary to build Joomla! 4 with a target beta date September 2016 at the latest. Any more delay would be catastrophic for the project and for all of us who make our living with Joomla!.
I can see that the most mistrustful among you may think that I'm conveniently passing along the hot potato. No. Even though I disagree with the proposed structure and the personalities of certain people involved I still do want to help. The big question is whether I'll be allowed to help or, once more, I'll be told to shut up. In the latter case all bets are off.
Up until 3-4 months ago Joomla 4 could only go as far as planning - it would be stupid to start writing code without knowing how PHP 7 was going to finish up. I think a lot of the framework v2 side of things will also help and that roadmap was published in February. Once that is finished we can look at porting that into Joomla.
There's been a lot of PLT talk (and this is something I personally want to get finalised at JAB in our meeting but tbh should have happened months ago) of having some kind of architectural code sprint to get a solid Joomla 4 architecture planned (which would, I expect have a roadmap at it's core) because even if other's are in denial we all know that there are few true architects sitting on the PLT. So the sprint would focus on what is essential? What features are going to hit extension dev's hard? How can we mitigate those things? (For example if we move to a Joomla Framework/Symfony style of writing plugins as listeners plugin structures change radically).
In terms of agreeing to features - it's always hard. When the Install from Web team made their written document it seemed good. However then when the code appeared things were obviously not good - the code is hard to maintain, poorly documented and as soon as JED upgraded we were left in a super hard position. I don't think we should ever be bound to accept features on paper if the code that accompanies them is rubbish sadly.
I agree that there's a lot of positive vision things we can do. But I also think that it's pointless to do these things without good structure, leadership and communication. I remember the sit down talk we had after JAB 13 where after fixing things in tags we sat down and discussed longer term things like web services, UCM etc. And yet look how fast that broke down!
Note these are all just personal observations. These are not official leadership views etc. etc. etc.
Kind Regards,
George
I disagree that planning Joomla! 4 required you to know where PHP 7 is going. Joomla! 4 will be anyway pinned to supporting at least 5.4 –or 5.5 at best, though unlikely– unless you want to lose everyone hosted on shared hosts. In the meantime I (re)wrote FOF 3 and Akeeba Engine 2 without knowing how PHP 7 will turn out.
Having a code sprint, hm. Planning doesn't necessarily require in person interaction but I see your point. It will expedite the process. The critical bit is who's going to be invited. If you don't include extension developers and designers you'll end up with Joomla! 3.20 or Joomla! Shipwreck.
Regarding the IFW example, this is a case of failed leadership and no clear vision. The bad code should've been caught long before it was written, let alone shipped. If you have a clear vision with the necessary provisions for the future (we already knew JED would be rewritten!) you can prevent bad code from being written.
I also want to remind you that vision drives daily operations, not vice versa. The structure's reason of existence is to efficiently support the vision of the organization. If something has to give it's the structure, not the vision. Your goal is to fulfill your vision, not make people happy with meaningless badges.
You are right that a code sprint isn't required (is it ever?) but it allows a group of people to get together and discuss things in person. I've always felt that face to face disagreements are generally less aggressive than ones done over the web. You are correct Joomla 4 is going to set the stage for another 5+ years so there needs to be representation from all coding perspectives. Core maintainers, Extension Developers (ideally both large and small - because things like Akeeba Extensions on FOF don't represent the majority of extensions using Core MVC). Either way we dedicated a increased amount of budget onto code sprints and so far have had 0 applications for that money pot....
Clearly in Joomla 3 bootstrap has failed the designers (also architecturally mistakes were made not allowing us to override markup in many places).
Obviously I don't know what the leadership discussion was at the time of IFW - but I do remember the detailed proposal google doc they gave on the Joomla CMS group (I'm sure it can be dug out from somewhere) which is much much more detailed than what most people give. But that never prepares you for things like https://github.com/joomla-extensions/install-from-web-server/blob/master/components/com_apps/models/dashboard.php#L113-L116 inside the code when it comes. Even now to an extent there is failed leadership on the rewrite - we're still leaving it down to the original team to do it at their pleasure instead of committing resources to help. In terms of the JED rewrite for sure we knew it was happening. But the IFW team did try but were never allowed to look at the API before new JED was released (and yes again that is a leadership communication problem again in part as well).
<blockquote>
Clearly in Joomla 3 bootstrap has failed the designers (also architecturally mistakes were made not allowing us to override markup in many places).
</blockquote>
You either need a Joomla! CSS framework or the balls to completely change the CSS framework every 2 years, i.e. whenever Bootstrap releases a new version. We do have the talent to get a decent CSS abstraction framework (Angie and Seth spring to mind). You just need to get them involved instead of sidelining them. Not you personally, "you" as in "the leadership".
<blockquote>
at the time of IFW - but I do remember the detailed proposal google doc they gave on the Joomla CMS group
</blockquote>
I am aware of it. I had read it and disagreed with it. But there's no point discussing this any more. It's hard to prove I'm not an after Christ prophet since I didn't make my objections public at the time. I did express my concerns to Beat in person (I think it was JD Netherlands – definitely while dining at an Italian restaurant).
BTW, if you ever need me to provide input on J! 4 I'll be glad to do so, on my own dime if you don't have the budget. I'd very much like to continue having a viable business the next 5 years at the very least :D
George - I appreciate your comments, I want to respond on one point that you wrote:
<cite>
Either way we dedicated a increased amount of budget onto code sprints and so far have had 0 applications for that money pot….
</cite>
If I understand your meaning, I don't think this is an area where we can expect some individuals to get things started with. I think this is a task that PLT has to make a priority and take the initiative to get started. Announce a J4 sprint, recruit the best and brightest from a variety of skill areas (also consider inviting some non-Joomla experts), ask for sponsors if you need to. Then come back and announce a date/time/general agenda, and maybe encourage/invite others to participate.
Thank you again for your interest to leave some comments here, and for everything you are doing to make Joomla better.
If I'm not mistaken the relevant documents can be found at the links in this mailing list post: https://groups.google.com/forum/#!msg/joomla-dev-cms/_f6iPHK7POk/mCHszqi8pZwJ
Anyone from a leadership team reading this, please let us know if there is a more current version.
I think there is a lack of vision for Joomla as a software product. There is vision, but it seems to be more around creating a "fun" social group for people who are interested in working on a thing called Joomla. Joomla really needs to choose which one it is going to be.
I think the biggest problem with moving forward is the lack of a plan for the software itself (some people call this vision, but I think we need to go to the next level of detail). Joomla 4 is virtually impossible to plan without first establishing, and agreeing on the "rules" that are the best compromise for everyone. Mostly that entails how much backward compatibility with 3.x do we want to maintain. More B/C is not necessarily better, and the longer 4.x waits, the more B/C will be required (not just desired).
My desire for Joomla, as a co-founder, is it picks the main thing it's supposed to focus on, and for me that should be the software. Further, I would say that a deliberate and ruthless pursuit of excellence is required.
To this end, I think the project should not be focusing on "structure" until it has the product itself nailed down. The product is coasting, neither innovating nor growing. The structure should serve what the product needs to be, not pick a structure and hopefully the best product magically pops out the other end (not withstanding that I think a lot of people ARE working hard at getting the structure right, I don't think they have a good handle on what the structure is actually for).
Something has to change. Unfortunately I've been saying this for a couple of years now but any discussion I've tried to start gets shot down.
Maybe we should play the "if I was King for a day" game? What would I do? Interesting question?
I'd have to think about it some more, but I'm pretty sure I'd be telling leadership that there is nothing more important than establishing the plan for the software for the next 1-2 years. That includes the source code (specifically the architecture), the user documentation, the developer documentation, and the support channels for using and developing for Joomla. Everything else is subservient to that.
I'd be suggesting to consolidate all the outgoing communication channels into as few as possible. There is no need for separate "news" on at least 3 different sites. It needs to be in one feed, and if you must, segment it with tags or categories.
One of my pet peeves is that the phpBB doesn't send the message body in the topic notification. It was ridiculous when the forum swapped from SMF, and it's still ridiculous. The way people collaborate needs to be fixed and frankly, the choices of software tools is holding people back.
I could go on but lastly I'll say there needs to be more synergy and collaboration in the extension marketplace. Quite frankly the quality of code is terrible more often than not (and I include my own older works in that comment), and developers are doing things that make it hard for other developers to do things the right way (or perhaps, the less wrong way). I recently bumped into a template framework that overrode the "Joomla Way" so much that it was impossible to make a simple fix in "the Joomla Way". This has got to stop and extension developers (and I include template developers in this - it's nuts that you guys are still not allowed on the JED) need to be working together to fix the basics in the core.
For that to happen, we need to learn how to debate better and that disagreement is healthy when done in a dignified way. Joomla doesn't do well at iron sharpening iron (that is, debating to ruthlessly improve the end result, not to ruthless tear down an individuals ideas). It used to.
I'm really at a loss to know how to help. I don't see a place where there can be a mature and frank dialog about what changes are required in the software product as a whole. I have lots of ideas, but I'm also willing to listen to other points of view providing everyone concerned is willing to compromise on what's best for the product (as opposed to what is only best for their best, or their blog, or their whatever).
Here's a final thought for giggle. Nic, how about you and me and a small group of top guns nut out the plan we think needs to happen for Joomla to get back to being the leading CMS. What's the worst that could happen LOL :)
I also do agree with your comments regarding Joomla! support and our forum software. The funny thing is that in a code project running a customizable forum software written in the same programming language as our main product we still fail to deliver a quality support system. Sigh...
And yeah, I do agree with you on 3PD code. All the crappy decisions in the products I'm currently shipping is workarounds to stupid implementations. Do I want to ship a private, namespaced copy of jQuery? Not at all. But when some plugins remove jQuery from the back-end (LOLWUT?!) or replace it with jQuery 1.2 (not a typo!) or some crazy dev adds their custom, oh-so-awesome jQuery plugin which screws up core functionality what choice do I have? At least the WordPress folks are better at this. If you report an extension doing this crap it gets immediately unlisted until the developer fixes the issue. Hint to our JEDis.
I do agree that you , me and the other mavericks should essentially get together and fork Joomla! inside the project structure. The code fork is for sure, the current product leaves a terrible lot to be desired. It's to everyone's best interests to do this fork inside the project's structure. Nobody wants to repeat the Mambo days 10 years ago. George Wilson, are you listening? :) You already have volunteers who want to get Joomla! 4 done. Seize the opportunity. It's not like you have an excess of developers to afford passing up on offers like that...
Thank you for this interesting read, although i'm not a developer but 'just a user', I understand what you mean by the lack of vision. And being part of the community, outside the developers zone, I also bump into a lack of vision, we're trying to do some stuff, but are missing the bigger picture...
We do not only need a vision for J!4 , we need a vision for the complete project... As the proposal got passed yesterday, i truly hope that vision will be created as soon as possible, and not by only a few people, but from within our active community... Now It's time to lead and not to influence...
I hope that people will continue giving their feedback and we can use my proposal as a first step towards building Joomla! 4 and beyond. I'm just trying to throw that first snowball which will cause an avalanche of progress.
Not sure you read it yet. But do read and comment. Hopefully with the new Structure and leaders, change will happen !
Just to put into perspective how stupid the so-called leadership is: it took me a week of working part-time to refactor Joomla! plugins to use the framework events package. Based on that, my experience and my most pessimistic predictions about people's ability to commit to deadlines it would have taken us 6 months to get the original Joomla! 4 plan to alpha-1, another 3-4 months to get it to beta and, assuming some light whip-cracking, another 2-3 months to get stable. In other words, we should be in Joomla! 4.1 now.
Then again what do I know? I'm just the guy who's publishing the top rated, most downloaded, most successful Joomla! extension of all times, has worked as a business consultant successfully and manages over 24 software projects in 5 programming languages and 3 frameworks with a development team of 2 people. It's not like I know how to make mass-market software and manage people...
Dude, you can't fix stupid. I tried. It's like wrestling a pig in a mud pit. You can't win, you'll get dirty and the pig enjoys it.
A lot of Joomla's problems step from we not having a central vision that can drive us towards our common goals. I've been speaking to a lot of people about this to a lot of people and everyone has a variety of perspectives towards what should Joomla be doing, for whom as well as how.. A lot of times they find it hard to understand each other though they are actually talking about similar things...
This is bound to happen in a community like Joomla - specially one without a 'benevolent' dictator or a company behind it. We need to ensure we all can subscribe to a common vision to give us the speed and agility to keep moving, retain our freedom and exercise it.. My blog post tries to explore that.
You have the top extensions in Joomla and bring great experience to the project. I know its like banging your head against a wall when you want to explain and change things. Can get extremely frustrating. Well I'm going to keep trying till I can !