Stop talking about scaling Drupal and make it easier to use

Once you have nodes/users/comments/taxonomy terms into five digits each, which is not hard for any moderately busy community site to get to, you will very quickly run into issues with things like taxonomy listings, complex views, tracker or forum module.

Scaling a site like the Emmys is relatively easy - looks like hardly any logged in traffic so nearly every request uses page caching - even with a vast amount of traffic it's fairly simple to scale Drupal to deal with that, this much is true as of the past couple of years.

However if you have logged in users and a lot of dynamic content, then you very quickly need relatively complex caching strategies and some way to denormalize queries and/or you'll be moving off shared hosting or cheap VPS accounts pretty quick.

Most sites Acquia are hosting (that I know of) are not dealing with large user communities, so "leaving [that kind of scaling] to Acquia" would be really idiotic - what they're doing is scaling the provision of lots of small and medium websites, and higher traffic sites with mainly anonymous traffic, at least at the moment I think that's the focus.

Most sites with large user communities cannot afford the hardware and double-figure development teams of sites like the Economist and Examiner.com, and don't get the free expert help of Drupal.org - so it's very important, to me personally as someone who both works on a large dynamic site as my day job, and has spent years running medium-sized community sites on a shoe string in my spare time, that it can handle those kind of sites better without having to use core patches (or Pressflow, which is largely packaged core hacks plus a few extras), or a massive reliance on custom code, or extremely beefy hardware, or all of these at once.

Working with Drupal 7 for the past few months it's going to be much, much better than this than D6 (not good on $5/month shared hosting, but much better on single VPS accounts), but there's a very long way to go before it's as easy to build a complex site that doesn't fall over once more than a few people use it at the same time, as it is to just build the complex site.

I'm also not clear if you want to focus on making Drupal sites easier to use, or Drupal as a site building platform easier to use - the two are not the same thing at all.

Apparently you missed an effort so grand that it actually got its own name: d7ux.

It seems odd that you would post this. Usability has been a giant focus of people working on core right now, with half a dozen people who've basically been working on it exclusively.

So let's leave the scalability question to Dries and Acquia

...

Perhaps you didn't mean to be, but I find this statement kind of insulting.

And what if I say that the backend is harder to replace than the user interface and also the UI is more often site specific so let's get Acquia to build usability? Drupal Gardens, hm?

Change starts with yourself, what are you going to do about it?

So, you've already felt the wrath of other devs, no need for me to apply ;-) Here's what I've been doing about usability. Why not go download ctools and context_admin and see if you can't make your site a little easier to maintain. Making drupal easier starts with you (if you're this fired up to make it more usable). Hopefully you can build on other people's work.

Eclipse

http://drupal.org/project/context_admin

I too am surprised about the "start focusing more on making Drupal sites that are intuitive, easy and a joy to use" part. It's up to designers and builders to create great ux for projects, regardless of the platform used.

If you mean better usability of Drupal itself, then did you really miss out on all the UX work of the last 18 or so months? Come to #drupal-usability on IRC or groups.drupal.org/usability and we'll get you up to speed :)

Just because someone can make a Drupal site scale to a million sites doesn't mean that same person can make a site easier for a novice to use. The two skill sets are completely different. Attempting to get developers to shift from performance to usability doesn't make any sense. Focussing on recruiting more UX specialists into the community would make a lot more sense.

People with the interest and the ability to address scalability should absolutely continue talking about scalability. People with the interest and ability to fix usability issues should do so. Sometimes they are the same people but often they are not.

Thanks for the info, I know very little about your company, and thankfully I'm glad that I'll now never have to bother finding out more. Ill informed, idiotic, comes to mind when reading your article. If you are the company, or employed by the company, then you or it deserves to fail with such an attitude.

Drupal does not scale well, even on medium sized sites, Drupal is not enterprise grade, it's not even medium business grade. And yes I'm well aware of all the big companies doing Drupal. Drupal takes a lot of work to get going for any project of worth.

Performance of a Web site, is also a feature of a website. If your website does not scale, it doesn't matter how pretty the front page looks, it's not going to be used.

While I get your point, you have made it in such a way that its meaning is hidden in a forest of objectionable statements.

At the very least Tag1, Chapter3 and 4Kitchens tend not to like leaving the scaling to Acquia.

Hi Joaquin, This post could have done a better job defining it's target audience. I think you are right most people shouldn't be worrying about the scalability of Drupal, it's been proven time and again that Drupal can be scaled. It's still skilled and hard work, but the best practices are shared effectively.

Of course, most people doesn't include contributors like Earl. If views, panels, ctools didn't scale well, most of the sites that rely on his modules would fall over. If catch and chx hadn't been doing so much detail performance work on drupal 7, while working on the d7 launch of examiner.com, performance hits we've taken to improve the D7UX would make D7 unusable for some. So for those core developers, please keep worrying about scalability and performance!

I am glad to see the CEO of a Drupal shop call for more attention to the details that impact every Drupal user, like the administration work flow, and for a better out of the box experience for common applications like blogging when using Drupal.

There are thousands of Drupal development shops and dozens of them are launching really big interactive Drupal sites. Collectively, we rely on the developers of those large sites to find performance and scalability issues in contributed modules and issues with Drupal core to keep the out of the box scalability of Drupal relatively high.

Products like project mercury and Acquia hosting are certainly helping to make scaling Drupal easier from an infrastructure level. Scaling anonymous traffic with pre-configured caching systems still requires significant effort and skill, particularly when scaling a high availability infrastructure with huge bursts, but it is a well worn path.

At Acquia we scale very large anonymous traffic sites as well as authenticated traffic sites, with lots of community features. That is often a combination our support and hosting operations team working with dozens of different customers and the many highly capable Drupal shops who design and architect sites to be scalable. At least one of those community sites is doing more than 20M pv a month, with a significant amount of authenticated traffic.

We are also developing a lot of very large Drupal commons sites, which in many cases has dozens of contributed modules and is exclusively authenticated traffic. Some of these sites are really going to push the envelop for Drupal community site sizes.

For people who are evaluating Drupal and how to scale the hosting infrastructure, see: http://acquia.com/products-services/acquia-hosting

To learn about the state of scaling and performance in Drupal join the high performance group, where dozens of skilled Drupal contributors share their knowledge. http://groups.drupal.org/high-performance

Don't forget to join the D7UX effort as well!

Cheers,
Kieran

2bits.com specializes in Drupal performance and scalability.

We presented on this back in April at DrupalCon San Francisco: see 2.4 million page views per day, 60 million per month, one server!. That site has actually hit higher records now, with 3.4 million page views per day and 92 million per month.

On our site you can find plenty of articles, case studies, and presentations on Drupal performance tuning and optimization for large web sites.