It reminded me about the following excerpt from Clay Shirky’s speech on cognitive surplus — he’s talking to a TV reporter about his new book, and she asks him about Wikipedia:

She shook her head and said, ‘Where do people find the time?’ That was her question. And I just kind of snapped. And I said, ‘No one who works in TV gets to ask that question. You know where the time comes from. It comes from the cognitive surplus you’ve been masking for 50 years.’ So how big is that surplus? So if you take Wikipedia as a kind of unit, all of Wikipedia, the whole project — every page, every edit, every talk page, every line of code, in every language that Wikipedia exists in — that represents something like the cumulation of 100 million hours of human thought. I worked this out with Martin Wattenberg at IBM; it’s a back-of-the-envelope calculation, but it’s the right order of magnitude, about 100 million hours of thought.

And television watching? Two hundred billion hours, in the U.S. alone, every year. Put another way, now that we have a unit, that’s 2,000 Wikipedia projects a year spent watching television. Or put still another way, in the U.S., we spend 100 million hours every weekend, just watching the ads. This is a pretty big surplus. People asking, ‘Where do they find the time?’ when they’re looking at things like Wikipedia don’t understand how tiny that entire project is, as a carve-out of this asset that’s finally being dragged into what Tim calls an architecture of participation.

Now, the interesting thing about a surplus like that is that society doesn’t know what to do with it at first — hence the gin, hence the sitcoms. Because if people knew what to do with a surplus with reference to the existing social institutions, then it wouldn’t be a surplus, would it? It’s precisely when no one has any idea how to deploy something that people have to start experimenting with it, in order for the surplus to get integrated, and the course of that integration can transform society.

The early phase for taking advantage of this cognitive surplus, the phase I think we’re still in, is all special cases. The physics of participation is much more like the physics of weather than it is like the physics of gravity. We know all the forces that combine to make these kinds of things work: there’s an interesting community over here, there’s an interesting sharing model over there, those people are collaborating on open source software. But despite knowing the inputs, we can’t predict the outputs yet because there’s so much complexity.

I’ve been television free since I was 18 years old and moved to my own apartment. That’s part of the reason why something like Plone could happen, a 250+ company worldwide phenomenon. Hanno got rid of his TV when he was 20. He's the most prolific contributor to Plone worldwide.

Get rid of your TV. Change the world, don’t just sit and watch.

…or, as Hanno said: “If you want to be the best — act accordingly.”

We have in one fell swoop eliminated the need for:

• Collections as a separate type,
• Images as a separate type,
• Composite page types,
• Proxy (represent one object in two locations) type,
• Default pages,
• The need for a separate "Contents" tab,
• Folder as a separate type.

We have made it substantially easier to handle:

• Rich (“opaque”) media like video, audio and Flash,
• Content reuse, i.e. generating new content by reusing other content (example: book with 3 different audiences),
• Application-specific functionality like forms/polls/maps inside a page.

We have made it possible to:

• Share “snippets of Plone” (widgets) to other systems — blogs, iGoogle, Facebook — you name it.

…all while making the UI massively simpler and easier to work with.

And if that wasn’t enough, the infrastructure improvements can be done gradually over a number of small releases in the 3.x series, and then culminate in a Plone 4.0 or 5.0 where we make the entire picture come together in a coherent new user experience. And we mostly don’t need to invent new infrastructure, instead we can build on work done in the Plone 3.x releases.

Next Steps

Where do we go from here? I want you all to help with fleshing out the plan, as well as contributing code and testing resources to make this happen. The next steps will be:

• Identify which individual components are needed,
• Figure out which of them can be built in parallel, and which ones can potentially be included in the 3.x releases,
• Identify champions for the different areas,
• Schedule sprints for big-picture integration sessions.

We hope to have the first of these sprints at the Plone Conference 2008 — and I hope we’ll see you there! Everyone is welcome this time around, and we’re trying to set up a range of sprint tasks, so that developers of all skill levels can participate in different ways.

I hope you will join us in taking Plone to the next level!

…and now for the pièce de résistance:

It’s time to reduce the number of concepts and approaches in Plone. Now that we have the “widget” concept and the new editor capabilities, we have the tools needed to:

• Eliminate redundant types,
• Eliminate the need for a separate “Composite Page” type,
• Eliminate the need for a “Display” menu,
• Eliminate the need for a separate “Content Proxy” type,
• Eliminate the “Default Page” concept,
• Eliminate the need for a specialized “Folder” type.

I’d also like to get rid of the “Contents” tab, but let’s keep this out of this particular proposal for the time being, since that’s a separate discussion. Ideally, there should be a separate “batch operations” interface for batch edit/rename/move/workflow changes. This proposal can be implemented independently, however — and indeed there is currently a Summer of Code project called plone.app.batch this year, working on some of this functionality.

Eliminating redundant types

• “Image” as a content type usually doesn’t make sense in itself. One exception here is the “photo album” use case, but that’s usually not what we refer to when talking about images in Plone. By making the images work the way content editors expect them to, we reduce the complexity of creating a page with Plone, and in one fell swoop get rid of the need for add-ons like RichDocument, PloneArticle and a lot of other generalized "keep the resources with the content" products.
• “Collection” (aka. Smart Folder) does really not make any sense as a content type by itself either. It’s a stored search, meant to be used in a context. By implementing it as a widget instead, we free up the content editor to use it in more flexible ways.

Eliminating the need for a separate “Composite Page” type

Another way of thinking about the widgets system is that every page is essentially turned into a page with composition abilities similar to the existing third-party “composite page” types.

Thus, every page is a composite page, and can pull in resources from other locations — folder listings, searches, images, movies, forms — you name it.

Eliminating the need for a dedicated “Display” menu

The reason why the Display menu exists in the first place is to make it possible to have a specialized view of a folder. But usually, the listing in itself only gives you half of what you want — you may want a small blurb talking about what the listing contains, an illustration or even a demo movie to go along with the listing.

What is currently thought of as “folder listings” are really just a special case of a “listing” widget — plain folder listing or a stored search.

Eliminate the need for a “Content Proxy” type

By introducing a “Content Proxy” widget — and yes, I’m sure we can find a better name for it — one can construct a page composed of any other number of pages.

From the simple case of mirroring one page in a different location to the more complex case of, say, constructing a “manual for Plone integrators” composed of 15 FAQs, 30 how-tos and 5 tutorials that all live in separate locations.

Another way to look at it is that to create a content proxy, you create a page with a single widget that pulls its content from somewhere else — and this content can even potentially be external. Hosted in Google Docs? No problem!

Eliminate the “Default Page” concept

After all this, you can probably guess how we can get rid of the “Default Page” concept. Yes, by using widgets and the new way of doing layout, on a folder — or a page, if we get rid of the “Folder” type as outlined below. The composite page abilities in the new approach will be able to turn a folder/page into any type of listing — even a mirror of an existing page.

Technically speaking, if you look at the folder from a file system view like WebDAV or FTP, its “body text” will look like a folder with an index document inside, similar to how index.html works for HTML files.

Eliminate the need for a dedicated “Folder” type

This a suggestion where I’m still not 100% sure we can make it happen without any performance implications or other problems, but ideally I’d like to get rid of the “Folder” concept altogether. Currently, folders are used to create listings of content added to a particular area — and that’s just as easily accomplished by adding a “dynamic listing” widget. It can list content based on containment, similar to what a “Folder” does today, or based on a set of criteria, like the “Collections” do.

To me, it makes sense that all objects are “folderish” — to use the Zope jargon for this — but I’m sure there are some constraints and considerations that need to be discussed. This is by no means a necessary part of this proposal, but it would make the experience a lot simpler for the end-user if we could make it work that way. Hopefully the new, unified plone.folder implementation scheduled for the next major release of Plone will make this easier too.

Let's summarize what we have outlined so far.

A little bit of history

The background for this approach comes from a concept Geir Bækholt, Martijn Pieters and others at Jarn came up with to help Trolltech support a seemingly simple use case: They had a centralized price list, and wanted to make sure that everything referencing price was always up to date when they were changed the numbers in a central location. They were using code templates do do this, which is not such a good idea, since it effectively removes things from the indexed content, as well as making things fragile by mixing programming logic and content. Predictably, none of the content editors in the site were able to keep track of this.

To solve the problem, they invented something they called “snippets” — small pieces of text that had a special class that made them replaceable by a transform-on-save. I’m not going to cover this implementation in detail, but the outcome is that they realized that this approach could support a wide range of dynamic insertion capabilities relevant to audio/video media and other “opaque” content formats in Plone.

Where this gets interesting is when we combine it with a couple of other concepts, namely arbitrary attachments and portlets.

Infrastructure: Arbitrary attachments and layout

Plone’s current notion of adding an Image object just to get an image into a page is a bit cumbersome. There are times when having a separate (i.e. end-user selectable) type for images makes sense, for instance in the “photo album” use case — but in the vast majority of cases, the user just wants to upload an image to use in a page.

In some cases, it may be desirable to use a centralized image repository, some of which may be pre-populated, but page editing is still the primary point at which the author considers images. What actually happens to the image when it is added isn’t really a concern to the user, and shouldn’t factor into how the UI works, which it currently does.

This problem is not specific to images. Adding a Flash animation or audio/video media files is no easier — quite contrary. From their experience with word processors and other desktop applications, the user expects to be able to embed files and resources directly into a document. Plone should make this as easy as possible.

The problem can be solved in a general, flexible way that makes it easy to create such attachments and lay them out on a standard page. The solution comes in three parts: widgets, editor integration and transformations.

#1: Widgets

A new type of “contextual portlet manager” is added that can keep track of portlet assignments annotated onto a content item. Unlike the existing portlet managers for the regular portlet columns, this portlet manager is never actually rendered directly, but it acts as a container for embedded portlets. Importantly, the new portlet manager has a particular marker interface so that specific portlet renderers can be used when a portlet is assigned to this particular type of portlet manager.

We call these “widgets”. New widgets are defined that support attachment behavior. These must have a view that can render it as a simple icon as well as a canonical rendering. This can be achieved by having different portlet renderers registered for different marker interfaces on the portlet manager and/or request.

Portlets of this type could include an image widget and a media object widget. Both of these would include UI to upload resources directly into the widget and way to traverse to the uploaded image or file.

Additionally, they can consult a global option that lets the user choose a folder to act a repository for all images or other assets, via a pluggable storage policy. This makes it possible to have a central image repository, even though (to the end user) it looks like he’s just adding images to his page.

#2: Visual editor integration

Kupu (or whatever visual editor we use as our core editor) gains the capability to manage such widgets directly. An add form that includes upload widget for the attachment widget may be shown directly inside Kupu. This can be made general enough so that other types of attachment widgets can show their own forms, although it is likely to require specific UI for images and media objects that use attachment widgets behind the scenes.

The important part: A widget is inserted into the body text by using an <img /> tag that renders the widget’s icon representation. For image attachments, this may be an actual full-size or thumbnail representation of the image, for Flash, it might be a more abstract representation like just an icon. It also includes the widget id stored in an HTML class or similar.

The reason we use an image for this representation is that it’s the easiest element to manipulate in an in-browser editor like Kupu. The user can drag the icon around the page however he wants, and on saving the page, the transformation layer takes care of converting the placeholders into the real widgets. If you try to use other HTML tag constructs for this, you quickly run into visual editor issues specific to the various browsers.

#3: Transformation

When the page is saved, a text transform looks for these special images and replaces them with the rendered representation of the relevant widget.

It should also be possible to optionally include attached items in the SearchableText of a content item, so it will return as a result if you search for a term included in the attachment.

Turning portlets into widgets, a note on terminology

The idea of “widgets” needs some more explanation, as well as some implementation-specific notes:

To make portlets more understandable and generic, we rename them to “widgets”. The implementation still refers to portlets, this is a UI change only. There are some behavioral changes that would make widgets more generally useful.

The main reason for the renaming is that “portlet” is very “jargon”-y — your mom certainly does not know what a portlet is. To add insult to injury, the Plone portlets implementation isn’t really what the rest of the world is talking about when they are referring to portlets — they are usually talking about the elephantine JSR-168 Java specification¹, so let’s eliminate that source of confusion once and for all. ¹To be fair to Plone, the JSR spec was still in its infancy when this particular part of Plone was named.

“Widget” is the term used by several web-based and desktop-based implementations of this concept, and people understand what they are at this point. “Gadgets” is another term that might be an alternative, but for the remainder of this proposal, I will use the term “widgets”.

On a related note, the intent is also to get rid of the notion of left and right columns (as well as the general separation that forces you into a certain layout at the moment), but I’ll save that layout discussion for a later point. It’s covered superficially in the technical explanation at the end of this part, if you’re interested. The short version is that everything becomes a widget (the search box, the navigation, breadcrumbs, logo, etc).

Demonstration & Notes

Movie showing the new way to handle rich media
(no sound, 5.5MB)

• Insert a Movie widget
• It allows you to Upload, Search/Browse an existing content object, or Embed from an external web site, potentially with some settings related to height/width, etc.
• For movies and audio, an add-on product suite like a stripped-down version of Plone4Artists could supply transcoding and metadata extraction for these types. Simple metadata extractors like EXIF (photo metadata) and ID3 (mp3 metadata) could be in the core, but we don’t really want to ship a full suite of video and audio converters by default.
• Right-clicking a widget and/or possibly clicking an information icon brings up the widget settings.

As you can see, this solves a lot of interesting use cases, and the potential new capabilities here can not be understated, it’s an incredibly flexible and powerful way of handling “opaque” content.

If you’re interested in the technical implementation details, continue reading — if not, you can jump directly to Part 3: Composite Pages, Listings & Content Proxies.

Implementation notes

Some implementation specifics from Martin:

• Some widgets are global. They are shown on all pages. This is not the same as assigning a widget at the portal root that is acquired everywhere. One use case here is things like promotions or site-wide information, but this mechanism can also be used to manipulate site layout (see below).
• There is a “Layout and widgets” control panel were all global widget assignments are managed. This includes site-global, group-assigned and type-assigned widgets.
• Global categories of widgets (assigned by group, content type or globally) can be blocked by location as they are now. This should show a visual representation of what gets blocked. It should be possible to block all widgets in the given category, or only some.
• When a contextual widget is added (i.e. one that is attached to a content item), it is for that exact content item only by default.
• A particular widget assigned to a folder may optionally be set to apply to items in the folder only, or to items in the folder and its subfolders. However, acquisition is not the default.
• Widget assigned to default pages behave as if they’re assigned to the relevant folder.
• A portlet manager can be associated with a layout grid. A widget’s position in the grid is annotated onto the widget’s assignment instance. This alleviates the need for multiple portlet managers that only serve to represent different areas of the screen.
• For this to work, there must be a more granular way of asking a portlet manager to render itself than the current approach which uses the structure: expression type in TAL. Most likely, this will just be a view with a function to render a particular grid cell of a particular portlet manager.
• Some widgets can only be added as site globals, never to groups, content types or content items. These are layout elements that fit into a global grid that includes the entire page.
• A convenience wrapper widget type makes it possible to wrap any supported viewlet into a widget. “Supported” here means that the viewlet provides a particular marker interface.
• By moving viewlet widgets around the site-global grid, it is possible to move the search box, say, from the top right hand corner to anywhere else on the screen.

With that out of the way, it’s time to move on to Part 3: Composite Pages, Listings & Content Proxies.

Over the past few weeks, I have done some experiments in simplifying the UI to make Plone less intimidating for newbies. A lot of these experiences are based on deploying it in a limited fashion internally here at Google. Generally, Plone is in a state now where most of the major areas of functionality is in place, so it’s time to look at it with fresh eyes — it’s not exactly Plone 1.0 when it comes to number of things you can click anymore.

I have built a clickable prototype of the new UI, since the best way to show you what the new editing experience looks like is to show it in action.

Note that while I’m using the NuPlone theme in this demo, the approach works equally well with any theme — this is about functionality, not looks.

Demonstration & Notes

Movie showing the new approach (no sound, 6.6MB)

• First we show the existing UI, and highlight the various features.
• Then we switch to the new proposed UI. The major difference is that interface usually only has one button — “Edit” — instead of the entire tabs/menu/button structure that was there in the earlier version.
• Adding content: Showing the simplified new add menu, notice how there’s fewer types. We’ll get to the rationale between these going away in the next parts.
• When we click the edit button, it does an async request and brings back the edit form inline. It turns out that making everything editable inline all the time is confusing users, and contributes significant amounts of visual noise.
• A number of interface elements and functionality has been moved to the edit screen. Sharing and History are now on the Edit screen, since they are usually part of the editing flow.
• Another common trait of these actions is that they are used only occasionally compared to the adding and editing of content.
• The “Advanced” menu handles everything that isn’t in the “Default” fieldset, i.e things like Effective/Expiry dates, ownership, copyrights, local workflows, etc. It also handles add-on screens that would sometimes show up as additional tabs in the old-style interface.
• Share, History and Advanced show up layered on top of the main content, lightbox-style when clicked — a modal window in UI terms. This reinforces that the metadata and functionality is a layer around the content. It’s not shown in the screencast, as it takes more time to fake than I have right now. Just imagine the existing fieldset implementation in a lightbox-style overlay, and you’ll get the idea.
• Moving on to the visual editor, we have collapsed down some less-commonly used buttons to a pull-down menu to make the interface as minimalist as possible. Not to worry, though: if you have certain things that deserve a dedicated button like — let’s say, heavy use of the “highlight” style — these can appear on the main toolbar too. Don’t let the super-minimalist button setup distract you at this point, it’s there to illustrate the concept.
• Most significant of the two menus is the “Insert” menu, which can insert a number of familiar and unfamiliar content components called “widgets”, we’ll get to these in part 2:
  • Tables
  • Widgets
  • Listings — both folders and dynamic searches (Collections)
  • Application functionality like Polls and Forms
• This is another one of the ideas that will Change Everything™, since there is no need for a dedicated Collection type anymore. If you want to mimic the current Collection behaviour, it would be a page with only the Collection widget in it. This is an example of how the widget idea starts simplifying everything in unexpected ways. I’ll cover this in more detail in part 2.
• Some things would be moved to the main edit screen, like tags and other elements deemed important enough to live there. Don’t pay too much attention to the UI on the tag field, it’s just an input box in this mock-up — of course it would have autocomplete in its final form.
• Workflow state has moved to the bottom of the form, as the natural action is to do some edits, and then submit the document as part of the editing flow. For people with the Reviewer role/permission, the idea is to add an additional pulldown on the view of the content too, so they don’t have to go to the edit screen to make state changes — in other words the same way as things work in the old-style UI, but limited to the power users that are reviewers.
• We show how you would submit the document for review, and Save it. The current thinking is that changing the state would show a review comment string field next to the pulldown, so you could enter an optional comment to go along with it.

Frequently asked questions and additional notes

What happens to in-line editing?

If there’s something that we have heard loud and clear from our integrators, in-line editing — aka. “click to edit” — doesn’t work that well with end-users, so we’ll disable it for general content authoring. Yes, I just admitted we were wrong the first time around.

Instead of trying to work around the problem of editing efficiency with “UI hacks” like these, we should attack the root of the problem, which can be done in much the same way, but with fewer moving parts. There are use cases — mostly specialized applications — where inline editing may be the right approach. This proposal just states that for the basic content authoring use case, it doesn’t add much — except for confusion, annoyance and visual noise.

Where is the current state shown on the view, since the menu bar is gone?

The current plan is to fold it into the by-line along with the publishing date and author info.

What happens to products that have defined custom tabs and/or menus?

Ideally, there are very few legitimate use cases for doing this, but I agree that some exist. I won’t call out which are gratuitous and which make more sense, but I’ll use LinguaPlone — Plone’s multilingual support — as an example of where it might make sense to add a new menu.

• Additional tabs end up as sheets in the “Advanced” edit page.
• Menus end up as pulldowns in the view UI (but are discouraged unless there’s a really good reason for them to live there).
• Single-item menus become buttons.
• A “Translator” role can minimize the UI noise, similar to how we could show the state-change menu for people with the Reviewer role to ease their day-to-day dealings with the system.

In sum, these changes to the user interface make it less cluttered, more efficient, and easier to integrate with custom layouts, as the only thing you have to find space for in a custom design is a button and a pulldown.

Next, let's look at better rich media handling in Plone.

But what about the people that will use those Plone deployments as their intranet, web site or collaboration space in the years to come? Don’t they deserve just as great an experience the first time they are introduced to Plone? Shouldn’t they be able to master the basics of Plone in about half an hour of tinkering, and be able to self-graduate to higher understanding of the system when they see the need for it?

Let’s skip the all rhetorical questions for which the answer is obviously a resounding “yes” — and let’s not forget that Plone was a revolution at the time, and enabled a lot of non-technical people to publish and work with content through their web browsers when few such solutions existed.

This was before Google Docs, Basecamp, mainstream wiki usage, and even before the word “blog” was something most people knew what was. Netscape 4 was still a viable browser, Windows 2000 was the new hotness, and Napster was teaching people the about the power of networks of people and media. Mac OS X and the iPod was just a glimmer in Steve Jobs’ eye, and Clippy from Word was the most widespread alternative UI for content authoring. People thought we were crazy to use CSS everywhere for layout, and a new project called Wikipedia asked if they could use our style sheet and layout, and we said yes.

In order to to understand this proposal, an appreciation of what was done in Plone from 2001 and up until today — versions 1.0, 2.0, 2.1, 2.5 and 3.0 — is useful. Painted with a broad brush, all those releases were incremental — in a UI sense, that is. The infrastructure went through great improvements, but the user interface just adapted to fit around the new concepts that were introduced, and the core usability approach didn’t change much. Sure, Ajax functionality, a visual content editor and other nice improvements were added, but the basic way of doing content authoring remained.

There are good things to be said for interface stability, and I’m really not proposing major changes in how you interact with Plone on a day-to-day basis, since it generally works well. Changing a few key elements can have profound effects on the overall approachability for the average Plone end-user, however. We can also make it easier for the power users who create and manage complex content layouts and rely on rich media like movies and audio to make their sites more compelling and rewarding.

This new UI approach is gradual in its implementation — “progressive disclosure” for you UI designers out there — but revolutionary in the way it makes things both simpler and more efficient and powerful at the same time. Rather than trying to separate the users into explicit "simple user" and "power user" roles, we make it easier to reach the things you use all the time, and push some of the functionality to locations where people can find them without interfering with the general content authoring. So it’s similar to — but more implicit and gradual than — the idea of a dedicated Power User role.

Whether the elements of this proposal end up in Plone 4.0, Plone 5.0 or Plone 3000, I don’t know. What’s important is that it’s possible to implement pretty much all of these things as incremental improvements in the coming releases in the 3.x series, as they make sense as smaller improvements by themselves — or don’t need to be visible until we flick the switch in a later version.

What can be improved?

Plone mostly works great, but there’s no reason to stop innovating! Some of the major areas I think should be improved — and are covered in this proposal — are:

• Rich media handling and dealing with anything that is not a simple page of text is harder than we’d like it to be. We want it to be equally easy to write content that consists of images, movies, audio and content embedded from other web sites.
• Composite pages — pages that pull content from a number of locations — is a hard concept to get right, usability-wise. None of the existing add-on products have supplied a satisfactory solution to this. No offense intended, of course — it’s a very hard problem, and people have done a great job exploring various different approaches to this.
• The content authoring process is currently very artificial — you generally need to make sure your images, audio or movies are uploaded to the system before you start writing. This is the opposite of how people usually think and work.

What are we proposing to do?

We suggest a set of changes that will make Plone easier to use, as well as reduce the complexity of (or getting rid of entirely!) the concepts of:

• Collections
• Images
• Composite page types
• Content Proxies (represent one object in two locations)
• Default pages
• “Contents” tab
• Rich media
• Content reuse

A tall order, for sure. Suspend your disbelief for a moment, and let me show you how! I know this can seem a bit overwhelming when taken in one sitting, but I want you to see the progression and thinking along the way — as well as the ideal end state.

Preamble and credits

This proposal is comprehensive and wide-reaching, but it’s not a result of me putting on my thinking hat for three months, retreating to my secret mountain hideaway with an unlimited supply of my favorite whisky and music. Although that does sound tempting, in reality it’s a result of great conceptual ideas from people like Geir Bækholt, Martin Aspeli, Benjamin Saller, Helge Tesdal, Danny Bloemendaal, Cornelis Kolbach, Duncan Booth as well as my fellow UI designers at Google — you know who you are! As usual, my role is mostly putting all these ideas together into a coherent whole, and hopefully I give their ideas enough credit and appreciation by proposing something that is more valuable than the sum of its parts, clichés notwithstanding.

So to anyone who has been involved in this discussion over the past couple of years — too many to mention, so I won’t even try to name you all — thank you for your great input and suggestions! Also a very special thank you goes out to Martin, who wrote the initial summary of the infrastructural description on his way back to London based on the description Geir and myself gave him a few hours earlier at the Planning Summit.

How to read this proposal

There are several ways to explain what this proposal covers, and Martin and Geir have helped me with the implementation specifications. If you’re mainly interested in how this will end up from the end-user point of view, I suggest that you read through the technical parts anyway — and ignore anything you don’t immediately understand. It will give you a fuller understanding of the screencast demonstrations and screenshots that follow.

I started out with two separate articles on how this would work, but realized that for our developers it’s all connected. In the interest of efficiency — and actually getting this published — you’ll have to live with the somewhat interleaved explanation for now.

It’s tempting to write a proposal that was five times as long trying to cover all the details that are touched on in this proposal, but then nobody would read it. So let me know what needs clarification either by contacting me directly or by posting to the Plone Core Developer list, and I will explain in more depth — and if necessary update the proposal to clarify when things are confusing or non-obvious. Most of the edge cases — believe it or not — have been thought through. At least we think so, but you are of course welcome to show us new ones.

It’s time for a revolution, it’s time to make Plone’s user experience more powerful, as well as making it simpler to get started! But enough pillow talk, let’s get down and dirty with Part 1: Simplify the Editing Experience.

Today, he announced that he landed a book deal, and in that context he wrote something that sounds very familiar:

The best thing about being a blogger is the people you get to meet. I have found this to be true and self-evident. When I was younger, the people who inspired me the most professionally were famous, dead, or both. Since I become a blogger the people who inspired me the most became good friends of mine. We hung out. We drunk beer. We ate pizza. It wasn’t a big deal, it was just… lovely. Back in 2004, my blogging buddies and I knew we were onto a good thing. Something powerful and creative and earth-changing. But that’s not the main reason we liked it. We liked it because we enjoyed it, because it was interesting, because of the smart, passionate, fun people we were starting to hang out with.

Sound familiar? It perfectly captures my feelings about the outstanding Plone community, which has been an amazing journey for me and hundreds of other developers throughout the years.

It’s been a crazy ride — starting from being unemployed in a small city in Norway, I got to build a world-class company with some of the smartest people I know. I now work at Google with another group of exceptionally talented people, call San Francisco my home, spend time with the most wonderful girlfriend in the world, and drive the only car I ever wanted. Plone has enabled several hundred companies all over the world to run their own businesses, and do what they love to do. It’s mind-boggling, and sometimes the massive reality of it all comes crashing down on me — in a good way.

If you’re interested in my personal perspective on how Plone happened, it was covered in the foreword to Martin’s book.

So thank you, Plone community. You’re fantastic, and my life is exciting and interesting because of you. I couldn’t have asked for a better group of people to change the world with. I can’t wait to see what the next eight years will be like.

• This is my personal list, and doesn’t necessarily reflect what the Plone Community wants to do.
• Even though I work at Google, there’s nothing here that constitutes knowledge of any secrets about future products. Any mention of Google here is as an outsider looking in, as I don’t work in the below mentioned focus areas — and have no insights into what they are up to next.
• Some of these are confrontational, some are not — they are all here to stimulate discussion, and not to imply failure or dissatisfaction in any of the areas. The Plone community is a fantastic community exactly because we can have these sorts of discussions out in the open without people taking it personally.
• Make no mistake — we are also a leader in a lot of the areas mentioned — I’m just making sure we keep pushing the curve and continue delivering the best damn content management system on the planet.

This list is merely a brain dump meant to inspire and provoke discussion in the days leading up to the Plone Strategic Planning Summit at the Googleplex in February 2008. The discussions there will certainly be a bit more high-level than what I talk about below, but this is a good chance for you to read about some of the baggage I will bring to the summit — as well as add your own views to the discussion so we can make note of them.

With that out of the way, get a cup of your favorite beverage, and let’s move on to the list! These are in no particular order, but I numbered them to make it easier to keep track of how far along you are in the document.

#1: Simple development should be possible entirely through the web interface

For those of you familiar with Zope, this point can be summarized as “Deliver on the promise of ZClasses”. Yes, I said ZClasses. For those of you unfamiliar with Zope, ZClasses is an old — and now frowned-upon — part of Zope that allows you to construct content types entirely through a web interface. It is what I usually refer to as the “gateway drug” of Zope. Most people that got into Zope in the beginning did so especially based on what ZClasses were supposed to deliver.

Of course, history shows that this didn’t really work out, as there was no transition path from clicking around and making your own simple type definitions, and then move to more advanced programming later on. With Zope 3 and the tools we now have at our disposal, it’s time to revisit this notion, since we can pull it off without any serious drawbacks this time around.

The path to get started with Plone should be:

1. Install Plone, change options and settings to your liking, add simple types for the main business processes you want to mimic — all using the web interface.
2. When you have the need for more advanced types — or just want to make sure the setup is repeatable and possible to check in to a code repository or a product: click a button → get a filesystem-based product.
3. Once you are comfortable with the architecture and general approach, you can continue doing the rest using a filesystem-based approach to development.

I know Martin Aspeli has done a lot of thinking around this, and has a proposal based on Zope 3 technologies ready, so I think this is already in good hands — but worth exploring and giving feedback on. My major goals to ease the approachability of Plone is to be able to solve the following common tasks using this solution:

1. Create simple types — then serialize (and diff) to an FS product
2. Change setup and configuration — then serialize (and diff) to an FS product
3. Create user properties — then serialize (and diff) to an FS product

Zope is one of the few systems that can do this, and we should exploit it for what it’s worth. The ultimate step is always filesystem-based development, but the goal is to be able to do all the simple things without programming — and then be able to dump to file system when you are ready to take it to the next level.

#2: Themes should not require Plone-specific knowledge

There are several reasons why there aren’t a lot of high-quality themes available for Plone. Most of it is a reflection of the fact that Plone is a more sophisticated solution that tends to have paying customers and businesses as a core audience. Obviously, this isn’t conducive to a large theme library in the same way as blogging software like WordPress — simply because companies and organizations pay a lot of money for their designs, and don’t want other web sites looking like their site.

The other reason is that it currently requires a lot of CSS knowledge to work with Plone to produce a theme that will be maintainable as well as being future-proof. Separating the theme layer from the main templates in a way that makes it possible to hand a template to a web designer independently of Plone is what we have always wanted to do.

This reality is getting closer with the arrival of Deliverance, an approach that makes a separate “theme layer” that is independent of Plone, and can be worked on without having any knowledge about how Plone works. Deliverance is currently being used by several sites, and is maturing rapidly due to the great work of Paul Everitt and the OpenPlans Team. I believe having Deliverance — or something similar — as part of the Plone core would be a sensible and game-changing move, provided it can deliver on its promises.

Again, this is something that I believe is in good hands, but needs shepherding, cheer-leading and feedback to make sure we make it as simple as possible to put your own look on Plone.

#3: Fantastic searching/grouping/batching operations for content should be accessible to everyone

The best way to sum this up for those of you who are familiar with Plone already would be Collections on steroids. There is a real need to organize and group content in Plone in various ways, and the current Collection and search implementations in Plone barely scrapes the surface of what we’re capable of in this area.

• The first step would be to mimic a slice/dice interface that all Plone developers are intimately familiar with: the Trac search. For those of you unfamiliar with Trac, it looks a lot like the Smart Folders in Mac OS X. The goal would be to offer similar query — and especially grouping — abilities in the core Collection type and also the advanced search form. This alone would make Plone a fantastic tool for content management.
• Assigning metadata to items in a collection of content — be it permanent or temporary — should be possible for all content. Construct your collection of content using the search/grouping interface and then apply tags, expiry dates and contributor metadata to it is a necessity. I cannot stress this enough.
• Another related quick fix would be to treat CSV as a first-order citizen for export/import of data — especially in Collections and users/groups — as well as wherever else data comes in a tabular format. CSV is the lingua franca of structured data and data manipulation/exchange, and we’re not exposing this in Plone at all right now. Since Python has great libraries for handling CSV data, it’s time we start using them.
• In the same vein, faceted browsing of the search results to make it easy to narrow down the result set. This is essentially the same back-end with a slightly different UI on top.

#4: Rich media should be a seamless part of the Plone Experience

One of the major growth areas we see in the Ploniverse is the need to handle rich media in a graceful and elegant way. Audio and video data in particular, but also interactive media like Flash.

This presents a number of challenges on both the front-end and back-end:

• On the back-end: Seamless and efficient storage of large files (aka. “BLOBs”), and performance in serving up these without using too much memory or CPU on the server side
• On the front-end: Easy upload of a large number of files, as well as playback and dedicated view modes for audio, video and picture collections.

The back-end part of this equation has received a lot of attention lately, with Zope 2.11 shipping with native BLOB support that massively improves the memory and storage requirements as well as the performance of serving up large files. On the Plone side, Andreas Zeidler has done an amazing job and created an implementation that makes this available on the Plone side of things, and this work is scheduled to land in Plone 4.0.

On the front-end, projects like Plone4Artists have explored a number of different approaches to handling these types of data in Plone, and there are several important lessons to learn there.

#5: Streamlined, unified install experience on all platforms

Plone already has the best install experience of any content management system — it’s really the Gold Standard in how easy you can make it to get started with such a sophisticated framework across all the major platforms. So why do we need to change it? The reason is simple: recent progress towards integration and unification with the Python world — especially eggs (similar to JAR files for those of you from the Java world) — necessitates some improvements here.

The current situaton is also that we have three separate cores of the installers, one for each of Windows, Mac OS X and Unix/Linux. The more we can share between these implementations, the easier it becomes to release software as well as troubleshoot problems regardless of what platform people are on.

Steve McMahon and Kamal Gill have done great work done on updating the Unified Installer to be egg-based recently, which lays most of the groundwork in getting the Windows and Mac OS X installers to use the same infrastructure. I want to make sure Windows is not left behind, as this is a very common evaluation platform for Plone — even if people end up installing it on Linux/Unix-based platforms when it goes into production.

However, I want to take this further, and think about the next steps in both initial install experience, upgrade experience, and how to install add-on products. Going into great detail on this would necessitate an entire document in itself, but I’ll list my current brainstorming notes in this area. If you have any feedback on this, please help out, these are loosely joined ideas that may or may not make sense — or even be realistic:

1. Install Plone/Zope in a consistent way that is 100% insulated from other changes to the system Python and other Plone/Zope instances on the system.
2. Install add-on products without having to do the download/unpack/locate directory dance. One way to do this would be to let “known good” projects be installed using the web interface, others might require server access. “Yes, I have a backup” button when installing.
3. Ability to ask questions during install — before the add-on product gets installed. In a lot of cases, this will influence how the add-on is installed and set up.
4. Ability to ask questions during upgrades: “We changed the default policy for X to do Y, but unless you want us to change your default setting, we will leave it the way it was.”
5. Show an indicator on whether we consider it safe to upgrade to the next available version X of Plone, based on the dependency information listed in the currently installed eggs — ie. if all add-on products say they work with the next version of Plone, tell the user this. If not, let him know which ones don’t have support yet.
6. Is it possible tocreate a roll-back mechanism to downgrade installs in case add-ons screw something up? Yes, you should always have backups, but it would still be a great addition that would alleviate a lot of nervousness on behalf of the site admin.
7. An easy way to clone your production instance to a development server.
8. Upgrade Plone/Zope using the web interface with just selecting to do so — ask for backup locations, etc.
9. Make the installer more like a "network installer" — ie. comes with the core, and then gets the relevant eggs online. Could even check for newer versions when you run it, and ask if you want the newer version instead. Minimizes the initial download, and makes it easier to handle release files. The role of the installer: set up a sane, insulated environment with Python + easy_install that works!
10. Andy McKay had a great idea of super-simple, standalone ZPT-based (not Zope!) application to replace the Plone Controller — making it cross-platform. Upgrade options, test running and so on could also be added here.

#6: Increased focus on a culture of systematic benchmarking and performance tuning

Like in Alcoholics Anonymous, the first step to healing is to realize that you have a problem. Unoptimized Plone sites tend to be quite slow, and while there are numerous ways to make them go faster, they all require additional knowledge from the integrator. We can do a lot with a little investment here, as earlier optimization efforts have shown. We need to get rid of the attitude that “there’s not much we can do about performance, Plone is a bit slow since it does so much for you”. Every single time we have examined this closer, there are huge wins to be had on a number of levels.

The complimentary side of this is to establish a nightly testing framework, in the same manner we already run automated, comprehensive code tests every night. Knowing when something started being slower is just as important as speeding up things — especially since we rely on the combined stacks of CMF, Zope and Python. A small screw-up can have massive consequences for us, and it can be hard to track down what changed. If you have a day-to-day log of some central performance statistics, you can narrow this down to specific changes — both upstream and in Plone itself.

Some of the things we can do:

1. Nightly speed benchmarks (we want to know when code was added that made things slower)
2. More analysis on where the CPU time goes — will need Zope gurus here, as it’s very opaque to analyze at the moment, we tried this with Mike Solomon (Python performance guru at YouTube), and it’s really hard to know what the components are for an outsider — even for our seasoned, battle-scarred Plone veterans.
3. Graph dependencies in an attempt to clean up code and make things simpler. The current inheritance trees in Plone + CMF + Zope are quite convoluted at times, and there’s probably a lot of simplification potential here once we see the bigger picture.
4. I want a random content population mechanism! To do proper automated benchmarks, you need a realistic body of content to test it with. Luckily, Tarek Ziade and others at the Snow Sprint 2008 already started work on this.

#7: Improved tabular data story

Right now, Plone is a great choice for content-centric applications, and it also has the possibility to integrate with more structured, relational data via the SQL database adapters. However, most of these solutions are old — and while they work, Python has acquired a number of great SQL integration tools in the meantime, SQLAlchemy being the weapon of choice for most relational-thinking Pythonistas.

I believe this is an important focus area, but since I’m not an SQL guru, I’ll leave it to others to comment on how this can best fit into the Plone experience. I have a related suggestion, though — one that we have been kicking around for quite a while at various sprints and workshops, but never had time to look into:

I want us to have a simple “Grid” type as one of the core Plone types. Looking at the average user, they usually reach for a spreadsheet not when they need calculation abilities — but when they need something that looks tabular. Week plans, project planning and overviews — all very structural data. Of course you can do this in a table in a normal HTML page, but that’s not how people think.

The Grid type wouldn’t even have to have any spreadsheet-like properties included — as long as it could deal with text input in tables and do some rudimentary operations — like import/export CSV, and possibly feed other parts of the system with the data. This would be the perfect back-end for things like polls, questionnaires and other similar applications. You could then export the data as CSV for specialised handling and analysis/graphing, if needed. Of course, using the Chart API from Google would be an easy way to provide an optional graphing capability if your data is public. But I assume I’m a bit biased in this area.

Whether this would be backed by something like SQLite with the ability to scale up to another SQL database as the storage later — or simply be a basic type — I’ll leave for the people with that kind of experience to decide. In any case, a Grid type would fill a real need in the current Plone line-up.

#8: Unify similar concepts

Programmers often have a tendency to abstract away things until they are almost unrecognizable, to the great frustration of UI designers everywhere, of course. Making things too generic makes it hard for people to recognize patterns and makes it harder for them to let the tool guide them in what they want to do. However, the opposite tendency is also very much prevalent, and leads to a lot of redundant implementations, and confusion in what add-on products to use.

Building a unified back-end for some central application structures would make it easier to maintain an infrastructure that is well-tested, performance-audited and maintained. I won’t go too far into this, since it’s way outside my league, but some suggestions for common applications that could share APIs and implementations — but have different user interfaces to fulfill the specific needs — are:

• Ratings, comments, forums, mailing lists
• Blogs, feeds, mail-in support, mail-out support, newsletters

These could even ship with the core, but be deactivated by default. But I know I’m pushing it here — and I’m also tackling another element of this in a later point — so I’ll leave it as an exercise for the reader to help identify clusters of similar functionality.

#9: Provide a fantastic page compositing story

At the moment, there are several page compositing add-ons available for Plone. None of them make for a fantastic experience in creating pages that are composed of other pages, listings and resources — but it’s one of the most important focus areas for Plone 4.0. If I had my way, the user would never have to think about abstract concepts like viewlets, portlets, listings and content — the approach would be the same for everything.

Like the unified installation area, this is highly complex and could fill a document by itself — but I wanted to have a placeholder here, so we can discuss the current approaches and where to go in Plone 4.0. Hopefully this will be where I get to spend most of my development time for the 4.0 release.

#10: Content re-use is overrated — people like folderish

This is a personal pet peeve, but since I said I would include some confrontational issues for discussion… ;-)

There are absolutely user stories where content re-use is important, but those tend to be highly specialized cases, not the common case. Example: why isn’t an Event folderish? How about a Page? You can always find them later, and Plone’s built-in Link Integrity will stop you from doing stupid things like deleting content that is referenced elsewhere. There might be technical reasons why we’re not doing this — and if there is, I’d like to hear them.

Note: I’m not advocating using File field/widgets, which I agree are an abomination in most cases. I want to be able to add resources inside a content item, though. The simple reason is: that’s how people think, and we’ll have to build around that notion, at least for the basic functionality. Combined with our powerful search capabilities, this is a problem I’m confident we can provide an elegant solution for.

#11: Take project workspaces seriously

Plone is an ideal platform for highly collaborative work, and we have been improving this story over the years. Organizations like OpenPlans have also done a lot of great exploratory work in this area. It’s time to bring some of that hard-earned knowledge into Plone proper. Using an approach similar to what Martin Aspeli has created with his b-org add-on, it should be possible to offer simple, compelling and efficient project workspaces without adding a lot of complexity or code.

#12: Increase efforts on plone.org

The situation of plone.org is familiar to anyone that runs a successful company: Who has time to update and improve the company web site when business is doing so well? The same goes for the plone.org web site — most of our developers and integrators are busy and happy working on client projects. However, without a major facelift for our communal property, we aren’t doing Plone real justice.

Some things I believe we need to tackle are:

• Performance tuning — currently, nobody has time to look into tuning the setup so it scales better. Currently, it’s a bit painful to use as a logged-in user because of the delays. This is to be expected with our growth, but that doesn’t mean we shouldn’t fix it.
• More functionality for community input — we need ratings on everything, but especially add-on products.
• Identify documentation leaders — at the documentation sprint, we put in place the framework and social notion of people having responsibility and oversight of certain areas of the documentation (e.g. LDAP integration). This needs to be put into action, and a consolidation effort for the documentation needs to happen as soon as possible.
• OpenID support everywhere — yes, especially in things like the issue tracker. Make it easy for casual observers to comment and interact without creating new accounts. Since Yahoo, Google and others are currently adding OpenID support to various services, everyone will soon be able to use this.
• Sell Plone better — The web site needs to sell Plone better, both in prose and design. The marketing committee and the Strategic Planning Summit are gearing up to address these issues.
• Make it easy to translate web site content — This is on our more long-term radar, LinguaPlone could be used for this already, but it’s important to have a good workflow and process around this, so translations aren’t outdated when content changes. Expect it to take a while, but make sure you help out in finding the right balance of convenience and structure. Plone is successful on an international scale, our web site should be too!

#13: Lower the Migration barriers from other systems

If we can make it easier to upgrade from other systems to Plone, our adoption will increase. Migration is a complex thing, and can never be 100% automated, but there are a number of things we can do to make it possible for people to receive the Plone goodness even if they started out on a different platform:

• A robust XML export/import strategy — GSXML seems to be the best option I have seen so far, but figuring out what to standardize on, and have one solution that we recommend will make things much easier.
• Share migration scripts — even if what you did was a horrible hack to get that Sharepoint content into Plone, share it! Over time, people can improve and contribute back to these scripts, and people who are considering a migration have some hard-won knowledge to start off with.
• Support common markup formats from other systems — The classic case here is people who install MediaWiki, and then discover “you know, having a bit more granular security would be nice”. Having optional markup modules available so people can stick with what they know will increase the chances of them migrating upwards on the food chain.

#14: Realize that web publishing isn’t our main arena

Another slightly controversial idea. I’m not saying that we should ignore simple web publishing — simply that it’s not an area we will ever be a dominant player in — and we’re not trying to be. A great static deployment story will help tremendously here, but there are several factors that will always make us more of a fringe player in the brochureware corporate web site / blog arena:

• PHP systems will always win in pure numbers from pure widespread hosting availability.
• Plone is massive overkill for a simple web site.
• Software-as-a-Service providers like Google and others will also take on this area — and win.
• It’s where the volume is, but not where money and talent is. We do money and talent better than volume. ;-)

• Intranet deployments for companies and organizations.
• Highly collaborative workspaces with sophisticated security requirements.
• Document management.
• …and numerous other specialized fields.

#15: Let Favorites come back — in a big way

Early versions of Plone had the notion of “Favorites” — a way to bookmark content that was useful to you in a particular Plone site. This implementation unfortunately was very basic, so it didn’t get much use among the power users, and it never got much maintenance or improvement. Later on, it was dropped, since nobody was willing to maintain it, and it really didn’t do much anyway.

I’m bringing sexy back — uhm, I mean…Favorites back. There are a number of great uses for this feature outside of a simple list of documents, as witnessed by many an inflated web startup over the last few years. I’d like us to bring back new, improved favorite support in Plone. Some ideas for cool things we can do with these:

• Use Gmail’s “Star” concept to make an efficient, simple UI to mark content for later retrieval.
• Dashboard portlets with your favorites.
• Let the number of "stars" on a document imply things about its ranking in search results. As del.icio.us shows, this is a great indicator of popularity.
• Introduce the "follow" concept from Twitter to Plone — “star” users like content — and be able to follow their updates and additions in the site. Dashboard portlet to list these, with RSS support.
• The favorite marker can mark content to be available offline via plugins like Google Gears
• …and I’m sure you have other great examples.

#16: Improve desktop integration

Plone is already far ahead of its competitors in this area, thanks to the tireless efforts of Enfold Systems on the Windows side of things with Enfold Desktop. There’s still a lot of challenges here, and some specifics I’d like to discuss are:

• Can we extend this kind of functionality to the OS X and Linux desktops?
• Can we publish guidelines that show how to write your add-ons so they are friendly to WebDAV and desktop integration?
• Can we make the Content Type Registries capable of having local configuration mappings, so they do the right thing depending on context — and make it easy to set up new mappings?
• Can we make it easy to batch-apply metadata on upload using some sort of URL mapping?
• Can we build a comprehensive test suite to make sure that our assumptions don’t break when we do changes in the various underlying implementations? Can we make it a given that a shipping Plone release has working WebDAV integration on all the major implementations — every time?

#17: "Recommended apps" tier

This is more a pie-in-the-sky discussion item. Looking at other successful open source communities, it strikes me that Plone has a lot in common with how the KDE project manages their platform. They have a core that is useful in itself, but also a lot of add-ons that are more or less recommended — and the best-of-breed even ship with KDE itself. Working out how we can achieve a similar model with cues taken from the earlier point on installation capabilities and upgrade/dependency management would be very interesting.

There are a number of add-ons that deserve tighter integration (mostly in a political sense) with Plone — having recommended solutions for certain common add-ons helps the platform in general. We already have some projects that in some ways behave like this — LinguaPlone comes to mind — but having an accepted process and some guidelines around this is something we’ll need sooner or later.

#18: Stay hungry/foolish

Plone has always been great at encouraging experimentation and wild ideas. I want us to continue doing this, and make it clear that we value it — but at the same time let people know that experiments aren’t necessarily going to end up in the core. Do more experimentation and stuff that has “fun” PR value — things like iPhone support and social network features are relevant contemporary examples.

What’s on your list? What are the things you’d like to see Plone focus on? Do you have comments or suggestions on any of the above points? Leave a comment on the mailing list, or blog about your own list!

Thank you for your attention. It’s truly an exciting time in the Plone world.

Fortunately, Apple did make this a hidden preference in OS X Leopard.

To change the menu bar to be the old-style white, issue this command using the Terminal — paste in the whole thing, I split it into several lines for readability:

 sudo defaults write \
 /System/Library/LaunchDaemons/com.apple.WindowServer \
 'EnvironmentVariables' -dict 'CI_NO_BACKGROUND_IMAGE' 1

You will have to restart your computer for the change to take effect, but after that the menu bar will stay a solid white.

Enjoy your white, readable menu bar!

If you should ever want the translucent version back, here's the command to revert the change to the default setting:

 sudo defaults delete \
 /System/Library/LaunchDaemons/com.apple.WindowServer \
 'EnvironmentVariables' -dict 'CI_NO_BACKGROUND_IMAGE'

Credit: UsingMac.com

Plone Conference 2007 Keynote (~7MB, PDF format)

Enjoy!

I'm using the now free MyBrand service from FeedBurner. It allows you to host the RSS with them with no future migration pains. Should you ever want to move away from using their service, you have control of the domain your feed is hosted on, since it's just a CNAME under your own domain.

The old feed will continue to work for the foreseeable future, but if you really want to make sure; update your feed reader URL for this site.

PS: The reason Planet Plone suddenly started listing all my posts all over again is that it recognizes them by modification date instead of publication date. We should maybe consider pushing out effective date as our date for feeds. It had nothing to do with the feed address change. I apologize for the noise.

I’ll talk about some of the smaller enhancements that few of the reviews touched on, but are worth a mention — since they make me happy. Happy is important. Hopefully there are some gems in here that you haven’t discovered yet:

• On-the-fly repartitioning and resizing of disks without erasing content.
• Terminal is massively improved, and supports tabs.
• Preview can edit and merge PDFs! You can rearrange, combine, annotate and delete PDF pages with simple drag and drop, and save the result. As a result, most people won’t have to buy Adobe Acrobat anymore, as this is 99% of what they use it for. My prediction is that they will still buy it out of habit, but at least you know that it‘s not necessary anymore.
• Keyboard modifier keys can now be set on a per-keyboard basis, so you can remap that Microsoft keyboard you use at work to have the Apple keys in the right location without screwing up your keyboard at home.
• Simple settings to share a particular folder with people on your network without sharing the entire drive.
• Breadcrumbs in the Finder: Select View → Show Path Bar.
• Scripting and Cocoa bindings, as well as full Interface Builder and XCode support for everyone’s favorite language — Python. I’m excited about what this will let us do.
• “Repeat what I do” mode for Automator — finally this application starts being useful to automate repetitive tasks. Piecing together pre-built actions in the previous versions never worked out quite the way you wanted it.
• Experimental ZFS read/write support for developers. Need to test whether it makes a difference for Plone startup time soon, I know that HFS is not the most efficient file system when it has to deal with to lots of small files.
• Safari is the fastest browser on the planet right now — although Opera 9.5 (currently in beta) will be faster, Safari is the smoothest browsing experience out there at the moment. Also worth mentioning is that the “Inspect Element” part has been improved significantly, and is very valuable when debugging HTML and CSS in Safari.
• “Merge all windows” function in Safari. Don’t you hate it when websites open new windows without your consent? Now you can merge them back easily.
• Best virtual desktop implementation on any platform — try hitting F8 to activate spaces and press C to collect all the windows, or move the windows between the spaces using drag and drop — even Exposé works in the Spaces view!
• Parental Controls + Guest account that is wiped when user logs out → perfect kiosk mode to use for iTunes/Safari when you’re using your computer as a jukebox at a party, or just handing it over to others so they can use the web for a while. Your Plone instances will stay safe, and your party people won’t read your email.
• Finder is now multithreaded, so it pretty much never locks up, even when dealing with disappearing network devices. AutoFS to the rescue!
• Network settings panel is now sane and well-organized. Internet Connect application is gone, all the VPN and 3G/modem settings are now where they should have been in the first place.
• Related, Leopard supported my HSPDA 3G (mobile broadband) card out of the box, no need for ugly third-party software or drivers. It didn’t even need any setup, just plug it in and connect. Awesome.
• Spotlight is much faster, to the point where it is equivalent to Quicksilver when it comes to pure speed. It also ditched the “magic, invisible results window” that made it so annoying in OS X 10.4. It also supports AND/OR/NOT searches now¹.¹ Oh, and try typing in an equation in the Spotlight search, and it will feed it to the Calculator as you type, and give you the answer directly in the menu. Nice. While I still prefer QS for most operations, this makes it possible to survive on other people’s Macs.
• If you search the help menu — which has an inline search bar now — for something that matches a menu item, it will show you in which submenu it resides. Very elegant.
• All my installed applications from OS X 10.4 worked! This is very impressive. The Plone 3 Unified Installer has also been updated to work on Leopard.
• Faster! People have mentioned this, but seriously — it’s like getting a new computer. Less disk grinding, more efficient use of multi-core CPUs. After logging in, my startup apps are up and running in about half the time they took on OS X 10.4.

…the only downside is that Leopard is ugly as sin, and the new additions have pretty serious UI issues — Stacks and the Dock in particular. I’m hoping that UNO will be upgraded to support Leopard eventually. I never have — and never will — use the Dock, so I’ll leave those UI rants for others².² For an extremely detailed review of Leopard, including its UI warts — I recommend John Siracusa’s review of Leopard at Ars Technica.

The same goes for the translucent menu bar, WTF were they thinking? Visually and functionally, it’s the worst crime Apple has inflicted on its users in quite a while. While there are utilities that claim to put back the old-style menu bar, they fake it by putting a white area behind the menubar, which screws up the font rendering and doesn’t work with multiple monitors. Hopefully there’s a hidden setting to return to the old look somewhere.

Enough with the UI rants — is Leopard a worthy upgrade?

In my opinion, a resounding yes. It’s an extremely polished and solid release, and has served me well for quite a while with no problems. You can tell that this has been a major cleanup release, things are more consistent, faster and just plain well done. It’s like Plone 3 for your desktop. ;-)

Many small annoyances are fixed. Random example: renaming a file with an extension selects only the text before the dot, so you don’t end up removing the extension by mistake. There are hundreds of these tiny improvements.

Apple continues their great work on refining their operating system and the Mac experience — and although I would have liked to see something earth-shattering and revolutionary on the UI front in this release of OS X, I'm happy with a solid, incremental upgrade like Leopard is. There's always OS X 10.6 for the revolutionary, fun stuff.

Upgrade tip: I tried moving over my home folder manually instead of letting the Migration Assistant handle it for me — that didn’t work so well, and I had several permission issues. Trust the Migration Assistant, it is your friend. I did a clean install, and then used the Migration Assistant to move my backed up OS X 10.4 home folder and settings to the new OS. It even migrated my weird network setup perfectly. Slick.

It takes a single click to enable it for your searches, so head over to the Google Experimental Search page.

PS: I had nothing to do with these features, I'm just a big fan, and think people here would appreciate the keyboard shortcuts — I know I can't live without them anymore.

I had the honor to write a foreword for the book, reproduced below.

It’s always fascinating how life throws you a loop now and then that changes your future in a profound way—and you don’t realize it at the time. As I sit here almost six years after the Plone project started, it seems like a good time to reflect on how the last years changed everything, and some of the background of why you are holding this book in your hands—because the story about the Plone community is at least as remarkable as the software itself.

It all started out in a very classic way—I had just discovered Zope and Python, and wanted to build a simple web application to teach myself how they worked. This was back in 1999, when Zope was still a new, unproven technology, and had more than a few rough spots. I have never been a programmer, but Python made it all seem so simple that I couldn’t resist trying to build a simple web application with it.

After reading what I could find of documentation at the time, I couldn’t quite figure it out—so I ended up in the online Zope chat rooms to see if I could get any help with building my web application.

Little did I know that what happened that evening would change my life in a significant way. I met Alan Runyan online, and after trying to assist me, we ended up talking about music instead. We also reached the conclusion that I should focus on what I was passionate about—instead of coding, I wanted to build great user interfaces and make things easy to use. Alan wanted to provide the plumbing to make the system work.

For some reason, it just clicked at that point, and we collaborated online and obsessed over the details of the system for months. External factors were probably decisive here too: I was without a job, and my girlfriend had left me a few months prior; Alan had just given up his job as a Java programmer at a failed dot-com company and decided to start his own company doing Python instead—so we both ended up pouring every living hour into the project, and moving at a break-neck pace towards getting the initial version out.

We ended up getting a release ready just before the EuroPython Conference in 2002, and this was actually the first time I met Alan in person. We had been working on Plone for the past year just using email and IRC chat—two technologies that are still cornerstones of Plone project communication. I still remember the delight in discovering that we had excellent communication in person as well.

What happened next was somewhat surreal for people new to this whole thing: we were sitting in the audience in the “State of Zope” talk held by Paul Everitt. He got to the part of his talk where he called attention to people and projects that he was especially impressed with.

When he called out our names and talked about how much he liked Plone—which at this point was still mostly the effort of a handful of people—it made us feel like we were really onto something. This was our defining moment.

For those of you who don’t know Paul, he is one of the founders of Zope Corporation, and would go on to become our most tireless and hard-working supporter. He got involved in all the important steps that would follow—he put a solid legal and marketing story in place and helped create the Plone Foundation— and did some great storytelling along the way.

There is no way to properly express how much Paul has meant to us personally—and to Plone—five years later. His role was crucial in the story of Plone’s success, and the project would not be where it is now without him.

Looking back, it sounds a bit like the classic romanticized start-up stories of Silicon Valley, except that we didn’t start a company together. We chose to start two separate companies—in hindsight a very good decision. It never ceases to amaze me how much of an impact the project has had since.

We are now an open-source community of hundreds of companies doing Plone development, training, and support. In just the past month, large companies like Novell and Akamai—as well as government agencies like the CIA, and NGOs like Oxfam—have revealed that they are using Plone for their web content management, and more will follow. The Plone Network site, plone.net, lists over 150 companies that offer Plone services, and the entire ecosystem is estimated to have revenues in the hundreds of millions of US dollars annually. This year’s Plone Conference in Naples, Italy is expected to draw over 300 developers and users from around the world.

Not bad for a system that was conceived and created by a handful of people standing on the shoulders of the giants of the Zope and Python communities.

But the real story here is about an amazing community of people—individuals and organizations, large and small—all coming together to create the best content management system on the planet. We meet in the most unlikely locations—from ancient castles and mountain-tops in Austria, to the archipelagos and fjords of Norway, the sandy beaches of Brazil, and the busy corporate offices of Google in Silicon Valley. These events are at the core of the Plone experience, and developers nurture deep friendships within the community. I can say without a doubt that these are the smartest, kindest, most amazing people I have ever had the pleasure to work with.

One of those people is Martin Aspeli, whose book you are reading right now.

Even though we’re originally from the same country, we didn’t meet that way. Martin was at the time—and still is—living in London. He had contributed some code to one of our community projects a few months prior, and suggested that we should meet up when he was visiting his parents in Oslo, Norway. It was a cold and dark winter evening when we met at the train station—and ended up talking about how to improve Plone and the community process at a nearby café. I knew there and then that Martin would become an important part of the Plone project.

Fast-forward a few years, and Martin has risen to become one of Plone’s most important and respected—not to mention prolific—developers. He has architected and built several core components of the Plone 3 release; he has been one of the leaders on the documentation team, as well as an active guide in Plone’s help forums. He also manages to fit in a day job at one of the “big four” consulting companies in the world.

On top of all this, he was secretly working on a book to coincide with the Plone 3.0 release—which you are now the lucky owner of.

This brings me to why this book is so unique, and why we are lucky to have Martin as part of our community. In the fast-paced world of open-source development— and Plone in particular—we have never had the chance to have a book that was entirely up-to-date on all subjects. There have been several great books in the past, but Martin has raised the bar further—by using the writing of a book to inform the development of Plone. If something didn’t make sense, or was deemed too complex for the problem it was trying to solve—he would update that part of Plone so that it could be explained in simpler terms. It made the book better, and it has certainly made Plone better.

Another thing that sets Martin’s book apart is his unparalleled ability to explain advanced and powerful concepts in a very accessible way. He has years of experience developing with Plone and answering questions on the support forums, and is one of the most patient and eloquent writers around. He doesn’t give up until you know exactly what’s going on.

But maybe more than anything, this book is unique in its scope. Martin takes you through every step from installing Plone, through professional development practices, unit tests, how to think about your application, and even through some common, non-trivial tasks like setting up external caching proxies like Varnish and authentication mechanisms like LDAP. In sum, this book teaches you how to be an independent and skillful Plone developer, capable of running your own company—if that is your goal—or provide scalable, maintainable services for your existing organization.

Five years ago, I certainly wouldn’t have imagined sitting here, jet-lagged and happy in Barcelona this Sunday morning after wrapping up a workshop to improve the multilingual components in Plone. Nor would I have expected to live halfway across the world in San Francisco and work for Google, and still have time to lead Plone into the future.

Speaking of which, how does the future of Plone look like in 2007? Web development is now in a state we could only have dreamt about five years ago—and the rise of numerous great Python web frameworks, and even non-Python solutions like Ruby on Rails has made it possible for the Plone community to focus on what it excels at: content and document management, multilingual content, and solving real problems for real companies—and having fun in the process. Before these frameworks existed, people would often try to do things with Plone that it was not built or designed to do—and we are very happy that solutions now exist that cater to these audiences, so we can focus on our core expertise. Choice is good, and you should use the right tool for the job at hand.

We are lucky to have Martin, and so are you. Enjoy the book, and I look forward to seeing you in our help forums, chat rooms, or at one of the many Plone conferences and workshops around the world.

— Alexander Limi, Barcelona, July 2007

You can buy the book Professional Plone Development here.

Click the “Oracle Content DB” link, and you get:

I have to admit, their tagline “Oracle is the Information Company” seems somewhat… ironic at this point?

Sounds like they need a Content Management System that avoids broken links.

(Thanks to Brian Johnson for pointing this out)