In search of online publishing middleware

From Jonathan Morgan on March 18, 2007 at 7:37 AM

Here are a few more detials on my quest for decent, free, open-source publishing middleware. The initial things I'm looking for:
- Software that has a framework for content flowing into the system that allows easy plugging in of modules that support different types of data (RSS, NewsML, NITF, flat files that are comma-delimeted, etc.) and that supports multiple sources.
- A system with an internal data format that allows for standard types of content (like video, audio, interactives, etc.) and that allows you to produce them, categorize them and tie them together.
- Output from the system should be the same kind of modular as the input, so that it also supports different output types (RSS, NITF, NewsML, SaxoTech and other proprietary export formats, flat HTML, dynamically rendered pages, etc.) that can be added or removed as needed and used to publish things to different destinations.
- It needs to have grouping and hierachies at the core of its data model, so that you can group anything and assign properties to the group, then override them at the individual item level as needed.

I am at a site where our pagination software and CMS don't really lend themselves to working together without middleware, and our home-grown system has about reached the limits of its expandability. So what are most people using now for this kind of middleware? Does anyone know of a good open-source project that is trying to make something like this?

From Eric Wignall on March 19, 2007 at 11:46 AM

What you're asking for is what might be considered a 'grail' (I'll leave it to you to consider whether 'holy' should be attached.) in learning management systems as well. Content management systems like Ektron, PHPCow and ViaArt can pull together a variety of content types governed by various user levels/permissions but the input is not automated. With .Net, RSS, and search tag coding you may be able to create a content community with media being pulled in and posted in new ways. Is that what you're suggesting?
-- There's also a set of interactive tools being made available by Adobe: http://labs.adobe.com/wiki/index.php/Apollo

From Jonathan Morgan on March 20, 2007 at 6:15 AM

Yes, that is pretty much it, though I think you would need to be able to specify how automated the path from import to publication would be. You would probably want to allow some types of content to flow directly into publication, with "publication" being anything from an export as XML to the content being added to lists that appear on a web site, but also to allow for some content to be either peer-reviewed or edited if part of the value an organization added was news judgment.

If designed well, this kind of system would be flexible enough to be able to underly a whole lot of different, interesting projects (sort of like the reporting management system I helped design that underlies the NewAssignment.net project).

I don't think it is a grail, so much, either. I actually have some experience in learning management systems - I implemented a simpler version of a framework like this for Knowledgeplanet, to allow them to import content from different third-party partners, and we ended up being able to make something modular so we had one API that we re-used for vastly different integrations - one company used XML for its APIs, for instance, and one had us use the same HTML forms and scrape the same HTML response pages that their users used when they interacted with their system.

I ended up refining that framework a lot over the 4 or so years I did integrations, and from what I learned in that process, I think this project is doable (an integration tool that imports and exports modularly but doesn't integrate with a CMS is already available, from a company I used to work for, but I want to make one that is open-source so that it can be used by anyone, be they large or small).