Publishing with Silicon

Anybody who works with automating Adobe Illustrator long enough will find some level of flakiness in the behavior of this application. It is very old, and doesn’t enjoy the same level of support for automation as does InDesign, a much more modern application. I got this email this morning, and it seems we are one step closer to taming the Illustrator beast…

”
Colleagues,

To place a graphic in Illustrator, you create a new placedItem using placedItems.add(), then you set the file property of the placedItem to a file reference for the graphic you want to import. That’s okay. But when you’re creating/exporting/closing lots of files, Illustrator sometimes loses track of what’s going on. In times like these, Illustrator will create a placedItem that doesn’t really exist. When that happens, trying to set the file property will fail, but it won’t throw an error until later—because the placedItem.file property is null. In fact, the whole placedItem is garbage—but Illustrator doesn’t *know* that it’s junk until it tries to do something with it. If you set the placedItem.file property to an existing file and the property remains null, you know you’ve got a bad object.

So…be encouraging. I find that asking Illustrator to do something a few times usually produces the result I want. Something like this (I can probably go through and remove a few of the extra steps here, but, however inelegant, this does work):

//Place the EPS file by setting the file property of the placedItem to the EPS.
try{
      SuiteTalk.log("Setting file property for placedItem to EPS file for layer " + lName + ".", "DEBUG");
      placedEPS.file = epsFile;
      SuiteTalk.log("Setting file succeeded.", "DEBUG");
}
catch(e:Error){
      SuiteTalk.log("Error creating placed EPS for layer " + lName + ".", "DEBUG");
      SuiteTalk.log("File exists? " + epsFile.exists + ".", "DEBUG");
      SuiteTalk.log("File URL: " + epsFile.url + ".", "DEBUG");
      //Try again.
      if(epsFile.exists == true){
            var epsFilePlaced:Boolean = false;
            var desperationCounter:int = 0;
            while(epsFilePlaced == false){
                  try{
                        //Try again from the beginning.
                        newDoc.activate();
                        placedEPS = newDoc.placedItems.add();
                        placedEPS.file = epsFile;
                  }
                  catch(e:Error){
                        //"SuiteTalk" is for writing to a log file
                        SuiteTalk.log("Retry failed for setting file of placedItem for layer " + lName + ".", "DEBUG");
                        if(desperationCounter >= 100){
                              se = new SuiteTalkError( "Error creating placedItem for EPS: " + e.message, "103");
                              se.other = lName;
                              outErrors.push(se);
                              SuiteTalk.log("Could not create placedItem for EPS for layer " + lName + ".", "DEBUG");
                              epsFilePlaced = true;
                        }
                        desperationCounter ++;
                  }
                  if(placedEPS.file != null){
                        epsFilePlaced = true;
                  }
            }
      }
}

So far, it’s never run through the desperationCounter, and it’s always–eventually—imported the file.

And now, a round of applause for Jonathan Brown’s “isValid” property on InDesign objects.

Thanks,

Ole
”

LOL, we can’t always be elegant, especially with older applications.

For 7 years my friend Gary Preuss has been telling me I need to work with Esko technology, and I liked what pieces I saw of it but never found a situation where we worked with it extensively. I haven’t had the time to explore the technology or reach out to Esko until recently: now that I have been able to see what they have to offer, I realize that Gary was right.

Esko seems to complement Silicon Publishing almost perfectly: they focus on Illustrator automation, probably because Illustrator was the hot new thing in the time they started out… we on the other hand have focused much more on InDesign, yet we are the world leaders at translating between the two formats. While Esko has many, many products, we are really focused primarily on one, Silicon Designer. This is about the only product that Esko does not have in their arsenal.

We also have things that are overlapping and in common: we are both in much of our work extending Adobe technology, in other parts of our work building tangential software that Adobe does not build. And, importantly, we both work with some depth at connecting data sources to rendition. Silicon Publishing started out primarily working with XML for companies like Boeing, and document-centric XML/SGML is in our blood. Esko has one tool in particular, their Dynamic Content Engine, which is very similar to work we’ve done over the years making semantic XML work with high-quality, Adobe-generated, rendition.

XML: Abstraction and Rendition

There are two distinct Esko products in particular that excite me, in the context of connecting Silicon Designer to Esko technology: WebCenter and the Dynamic Content Engine.

Esko WebCenter is a workflow portal around document production – this facilitates review, approval, proofing, annotation. Notably everything but editing. How perfect for us, as we do none of the workflow. We are completely agnostic as to what contains Silicon Designer – it may be a shopping cart in a B2C app, a workflow engine like WebCenter, or any form of container application: the point is that we specialize on the editing and leave the challenges solved by WebCenter to others.

Esko WebCenter

Esko Dynamic Content Engine is a set of tools that facilitate something that we do offer: working with high-quality rendition and editing of semantic XML. I authored the chapter of the XML Handbook on WYSIWYG XML editing and formatting,  so this sort of thing is dear to my heart.

XML Handbook

The Dynamic Content Engine lets users associate XML with text and art in an Illustrator document. It is flexible, but generally assumes a specific Schema, that of the Intelligent Packaging Consortium (and I believe in the future these specifications will come from GS1). As we have found in this space, limits are your friends. By assuming this schema, or this and extensibility within some limits, the tooling becomes easier than building tools around completely arbitrary structures.

Esko Dynamic Content Engine

All in all I am very impressed with Esko and look forward to working with their technology and Silicon Designer.

I have been working on several web development projects recently, even doing some coding myself which has been very rare the past few years. As I compare what we are doing now to what we did 15 years ago, it really strikes me that the trends in computing and communications have led us from a state of a very limited tool set where there were a few difficult ways to accomplish something, to a world in which there are too many possible ways to quickly and easily do the very same things that were so difficult years ago.

Take dynamic web sites, for example. In 1997, creating a dynamic web site typically required a programmer, and the programmer would typically use Perl or C to work with CGI directly at a low level. PHP, Microsoft’s Active Server Pages, and Java Server Pages soon offered higher-level constructs that lowered the bar and made it easy for beginning programmers to make quite effective dynamic sites. By the late 1990s there were tools such as HAHTSite and FrontPage that even showed the vision of letting non-programmers create such sites, yet these were quite ridiculous at the time and it took quite some time before that sort of approach bore fruit.

When it became apparent that the Web would be central to human communications, technology supporting it very quickly evolved. With the steady exponential improvement of processing power, bandwidth, storage, etc., the underlying hardware has evolved steadily, while software has moved more in fits and starts, with web and information standards held hostage by proprietary self-interest. Still overall the trend has been continuously upward, such that these days we actually do have software (free software, even) that makes it completely easy for non-programmers to create dynamic web sites.

I am writing this with WordPress, which I think shines as a high-level technology that enables average people to easily accomplish what was once a daunting challenge for advanced programmers. WordPress followed a decade of  attempts (commercial and open source) at the same thing… software has gotten really good in some domains.

Probably the most frustratingly slow technology, for those of us developing web sites, has been the web browser itself. While technologies such as WordPress have come of age, the combination of committee-driven web standards with limited efforts from the very small number of browser creators has made browser evolution an extremely slow-moving process.

HTML5, SVG, CSS3

I tech edited a book, SVG Unleashed, in 2003. With hindsight we should have titled it SVG Leashed. SVG was a wonderful standard, yet it threatened browser makers and software companies more than it inspired them: it had some resonance in the Open Source community, yet it moved slowly even there. Ironically it recently rose from the ashes when Apple, Microsoft, and others saw it as a way to prevent Adobe’s Flash from attaining ubiquity. Funny for the most monopolistic, proprietary companies in the universe to proclaim a love of standards, but that is what they had to do when they found their operating systems threatened by an uppity plugin turned VM.

So now even SVG has finally won the victory we thought would have been much quicker, and there is fantastic technology available supporting the same things we wanted to do years ago. Yet while SVG has evolved, other overlapping technology has also evolved. We now have HTML5 and its Canvas, and CSS3, coupled with the wonderful capability to combine these with SVG, so there are more ways than ever to accomplish things that used to be impossible or could only be done a single (and often cumbersome) way.

There is still programming required to use SVG and HTML5. Much of the human insight required is in fact due to the redundancy of SVG, Canvas, and CSS: you can in many cases produce exactly the same result from an entirely different approach, and the art of picking the right tool for the task at hand is currently best done by humans, yet certainly tools will evolve that make this more automatic. I do not see coding going away completely, yet I love the trend that things get easier over time.

So while fifteen years ago the question was “how on earth can we do this?” now we face more commonly the question “which of the many ways we could do this makes sense in this specific situation?” There are times when it truly doesn’t matter, and times when it matters quite a bit. In the specific domain I work in, online editing, the choices between HTML5 technologies have extreme repercussions, yet we’ve seen in some of the output-only publishing cases where it is truly a tossup. Overall it is wonderful to see things where they are today and I wish I had been born many years later.

We first announced our CS Extension for Fotolia at the Adobe MAX conference in October: as I posted at the time, this gives users of Photoshop, Illustrator, and InDesign a very easy way to connect to stock imagery, directly from within the Creative Suite applications.

We have just released a major update, still considered “beta” yet quite usable and stable. The biggest change has been localizing it for many languages:

• English US
• English GB
• French
• German
• Italian
• Japanese
• Korean
• Polish
• Portuguese BR
• Portuguese PT
• Russian
• Spanish
• Turkish

We also made many improvements, including making sure it works with the latest Lion OS, which does require that you have the latest update to Extension Manager.

The Fotolia CS Extension, Localized for 13 Languages

It was interesting that the proofing process for languages also led to feedback and resulting tweaks that had nothing to do with the text of the user interfaces: different cultures seem to have different ideas of user interface, and it is interesting that different graphic formats have different levels of priority in different parts of the world. SVG, in particular, seems to be more popular in Europe than in the US. Fotolia offers vectors in AI, EPS and SVG formats.

It is unfortunate that vector art is not yet handled in a perfect way, and this is due to a combination of factors:

• Preview images are typically raster, so in the first place we don’t have a sample image to play with that will map to the final, purchased, image
• Adobe software has poor support for SVG at the moment (although the pendulum is swinging back towards standards so this may change): InDesign can’t import SVG, Illustrator can import SVG but not link (or re-link a comp image) to an SVG image
• For EPS and AI formats, Fotolia lets you download vector images in a .zip package, with purchased filenames that are inconsistent with comp images

The end result is that you can currently purchase vector images through the extension, but it is not the joyous “buy and swap comps with real images” that you can experience with raster images. Hopefully as we go forward this will improve. Perhaps Fotolia could use transparent pngs for the comp images? That coupled with SVG support from Adobe (one would hope SVG is coming back in CS Next thanks to HTML5) and consistent naming from Fotolia could enable exactly the same powerful workflow for vectors, some day.

While the concept was not original, this is very similar conceptually to what Adobe envisioned with “Adobe Stock Photos” years back, it is implemented better than was ever before technically possible, and we remain thoroughly impressed with the extensibility group at Adobe. The fact that we can maintain a single application that installs on 3 products across 2 platforms (and multiple versions of the products, platforms, and languages) is testament to how far Adobe has come with their core offering, the Creative Suite. We look forward to connecting CS to other systems – assets, workflows, collaboration, print/web round trip, this is a great start but is just scratching the surface.

You can try the CS Extension here.

We have been automating Adobe InDesign for 12 years, InDesign Server for 7 (since it came out), and it seems that just now we are finally being proven right in our bet long ago that going with the slowest, yet far and away most robust, composition engine would in the long run be the best path. After all, hardware just gets faster and faster. Desktop tools and capabilities do tend to end up on servers.

Now, in 2012, we can finally say that advances in Hardware, RIP software, and InDesign Server itself, have given us many tastes of sufficiently fast performance, and we can see a vision when InDesign Server is the tool of choice for even the most throughput-intensive documents, such as transactional statements.

In the year 2000, it was quite a long-term vision. Having just started Silicon Publishing, we wanted to do exactly what we had done previously at Bertelsmann: compose data-generated documents like catalogs and directories from data sources, flowed through document templates. At Bertelsmann we had primarily used Xyvision, which was an industrial-strength composition tool with powerful automation, robust page composition, and stunning throughput. Yet the cost was prohibitive to our small, new company and we weren’t eager to hire a “xy operator” who knew the esoteric ins and outs of making the thing work. There was something liberating about controlling the complete job from one piece of code, and seeing the output in real time in a desktop application like FrameMaker or InDesign.

Those were the two tools we tried: FrameMaker 6.5 and InDesign 1.0. Frame had MIF, a wonderful language for defining documents, and we got quite adept at generating MIF from code, even from XSLT in some cases. with InDesign it was a matter of using a scripting language: AppleScript or VBScript at the time. We later moved to ExtendScript as our primary language for automation, which we now augment with C++ plugins or Flex-built CS Extensions as appropriate.

Frame was much more robust, especially back then when InDesign didn’t even have tables. But Frame was old, while InDesign was exciting and new: the exposure to automation was unparalleled, and as we saw the momentum of the product we could feel that this was going to be the place Adobe put their best typography and layout capability, and we knew Adobe was uniquely qualified to see this through.

We got work automating catalogs and directories, and when we provided data-generated typesetting as a service, nobody knew that we weren’t using Xyvision or something comparable. We found we could automate InDesign to do pretty much anything we could do in Xyvision. It was sometimes a pain to automate things that really belonged in the product (tables didn’t show up until InDesign 2.0) but the momentum of the product with each release felt wonderful, and the fundamental full exposure to automation was a joy: anything we could imagine, we could do. This could not be said of any previous layout application we had seen, nor any we have seen since.

Output from Early InDesign Automation

It was a bit ridiculous, however, when we would deploy automation solutions to our clients. At Bertelsmann we had mainly been a service bureau or ASP-model solution provider (what they call SaaS today… in the 90s we did have SaaS, Cloud, and AJAX, by the way, they had merely not yet been named and hyped by the late-adopters). With a hosted solution we could turn 300-page complex directories in seconds or a few minutes, depending on complexity. Tolerable speed in any case, from a true server.

As a small company, we could have provided hosted services, but (a) at the time we weren’t even perceived as a company, we were seen as consultants because there were so few of us, and (b) the tools we were using weren’t yet licensed for server usage. FrameMaker Server did come out soon after we started, yet InDesign was not licensed to be used as a server nor would it be practical, though we did hear of companies setting up “desktop farms” some of which we know persist to this day.

Yet our client base grew, and many wanted to run automated publishing themselves, in house. In such cases, we had to explain to them that they would need to set up a desktop workstation, and in some cases (i.e. producing 150 100-page directories driven by a business rules table) let a process run overnight: they would literally see the documents compose on the screen. They scratched their heads, but they didn’t complain. Many of these solutions have run for years and we still have clients that run old desktop code.

Still, it was surprising how solid the applications were and how much throughput moved through them. We found InDesign kept getting better and better, and the precision was stunning: for example we automated generation of patient encounter forms, and we could analyze proximity of black ink to the bubbles that users would mark, and we could define proximity rules and copy fitting logic in composition that allowed maximal usage of the limited space on these crowded forms.

Adobe Altercast Diagram, circa 2003

It was just a bit silly that this was not on a server. We had been doing work for Adobe since 2000, yet we weren’t deeply connected to the InDesign group. We asked for an InDesign Server from all sorts of groups at Adobe, which seemed to come and go at random. There were server efforts such as “Altercast” and “Document Server” and there was at one point a “Server Tools Group.” We begged anyone who would give us the time of day for an InDesign Server, and it gradually dawned on us that Adobe was a collection of largely un-connected product silos, so whatever emotion we brought to bear on an Acrobat Sales Rep was unlikely to have great impact.

It was probably completely unrelated to our impassioned pleas, but in 2004 we finally got wind of an effort to create an InDesign Server. Based on InDesign CS1, there was an internal project that split out the model from the UI and allowed for server-like usage of the core InDesign engine – notably, this also had XML round trip that was prototyped but never showed up in the product. In 2005 when I got a chance to talk to the people in charge of the InDesign Server project for CS2, I said that we should definitely be in the beta as I was probably the only person in the world who literally dreamed of an InDesign Server. We were included in the first group of solution providers/resellers.

I actually got a bit of work for Adobe writing about the release of InDesign Server, and I wrote an article for InDesign Magazine about our love of this new thing. Yet when I went up to meet the great Whitney McCleary, one of InDesign’s spiritual leaders, I was given the bad news about performance… it was still slow. “We decided to release this technology to developers,” she said, “and we decided not to support it.” There was extremely little server about InDesign server, they had essentially hacked out the parts that didn’t depend on UI and wrapped them in a little bit of a server-esque container so you could send a SOAP message that would tell it what script to run. You could spawn multiple instances, but the rewards were not nearly linear, even a fairly decent server would not stun us with throughput.

So here was the real bet I’m talking about – 5 years had gone by, we weren’t impressed with speed, and were disappointed that the software itself wouldn’t help us much with making it go faster. Yet the quality… by now, there was pretty much every bell and whistle you could imagine: Transparency, tables, text on a path, referenced objects from Illustrator and Photoshop, really high quality PDF output with fine-grained control, I think by CS3 the feature set was at the point you could say it had a far-and-away advantage over any other page layout engine. Yet it was… slow.

At least we had a server, we had a mechanism to add servers, but for transactional documents and even some classes of promotional documents it was prohibitively slow. We used XMPie and built our own plug-ins to do what XMPie did, but even that was slow. And the documents created could often take forever to RIP. We liked XMPie’s export to VIPP, PPML, VPS, etc. yet there was still no comparison to a native VIPP job or old-school AFP.

Why was it so slow? Mainly because it was doing substantial processing: no pain, no gain. Managing layers of transparency has obvious processing cost.  When you use the Paragraph Composer, it is not the dumb, brute-force, “slap text on a page” process of most transactional engines, but the artificial intelligence that makes InDesign output look like it was crafted by a seasoned typesetter requires that the content of the entire paragraph and all contained styles are analyzed to determine how characters are placed on the page.

We dabbled in faster, less robust tools but decided fundamentally to stick with InDesign, as by then it had completely differentiated itself from any other composition tool both in terms of quality of output and exposure to automation. Our logic in sticking with InDesign Server was that some day, hardware would reach performance levels where throughput would be acceptable for more and more document types, and that PDF-based RIP software would one day work.

Having been heads down the past 5 years deploying InDesign Server solutions, I look around today and I’m happily surprised that our dream is starting to come true. We are not benefitting from a throughput improvement on quite the scale of Moore’s Law, as processor speed is just one factor in IDS throughput, yet we are finding new multi-core servers are exponentially faster than what we used to see and more and more document types are becoming viable with IDS. And PDF-based RIP software is truly stunning. HP in particular has been fantastic at building out RIP software and Adobe has made some steps towards optimizing InDesign Server for high throughput applications. The beauty of this to us is that the page layout capability is not sacrificed at all; there is less and less need over time for compromise between throughput and output quality.

It has been over a year since I wrote a comparison of InDesign Server and Scene7 Web to Print technologies. This entire past year I’ve been asked about this every few weeks, if not every day (which was the case last week), and I continue to point to that post. I just re-read that blog post again, and I think it is still largely correct in outlining core differences between these two technologies provided by the world leader in print software for server-base document rendition to print.

I am thankful to work with both InDesign Server and Scene7 Web to Print day in and day out, both on an increasing scale. Our Silicon Designer product is rapidly becoming the tool of choice for editing documents online that render in these technologies, and we continue to watch these evolve, help them evolve, and pray that Adobe corporate doesn’t oppress Adobe engineering any further than they have in the past.

Adobe is a pendulum-swinging sort of company, turning back and forth in slow motion on directions such as acceptance/rejection of Web Standards, and I remain optimistic that the engineering-centric nature that characterized the first 20 years of Adobe will return some day and bring the company back to full health and economic stability.  Meanwhile, we watch our favorite tools advance or stagnate in different dimensions depending on political climate in a business sphere far-removed from the technical.

Overall, I am optimistic. I have calculated that Adobe has 143 times the engineering staff required to make the world safe for web-to-print. Yet the game isn’t over, at a business level Adobe is at a crossroads, and the core technologies that they have succeeded with in the past are political footballs. Even if InDesign is the default application for making serious documents for print, yet we have no guarantee it won’t be destroyed by ineptitude and pure corporate politics. Same with Scene7: it is obvious that the values and criticality of these technologies are under-estimated by corporate management in this season, but equally obvious that their value will be understood over time.

If you asked me what changed with both over the past year, I would say “not much, yet both are more stable” – I think next year things may be quite different. The InDesign Server momentum is around scalability, cloud-based rendition, and DPS export, while Scene7 Web to Print momentum is around CQ integration and finishing out the feature set. Both have exciting HTML5 stories, yet it remains to be seen how these relate to the other HTML5 developments at Adobe.

http://blogs.adobe.com/flex/2011/11/your-questions-about-flex.html

So, no more Adobe Flex after version 4.6, etc. I sympathize with the comments here:

http://blogs.adobe.com/flashplatform/2011/11/flash-to-focus-on-pc-browsing-and-mobile-apps-adobe-to-more-aggressively-contribute-to-html5.html

Interesting. This is hardly surprising, the Flex framework was already “open source” to some extent technically; yet Adobe has acknowledged there were few if any external contributors of substance. It will be quite interesting to see what happens with Flex… the sad thing is that for all the HTML5 hype, all the work of Google, Apple, Microsoft have not moved it forward quickly enough, and Adobe dropped the ball on standards quite a while back when they chose Flash over SVG, so they will take time to ramp up and the tools for HTML5 are likely to remain sub-standard by compare to Flash Builder for some time to come.

Conversion of Flex applications to HTML5 is not obvious or trivial, as there are just too many ways to do the same thing with HTML5: canvas, SVG, CSS3 can all do text, for example, all with unique features and limitations the others don’t have. iOS is a far easier target. Flex has been a wonderful development framework, partly because it and Flash were not open source, and could be engineered in a way that was more controlled and integrated, more guided.

Adobe had finally gotten Flex to work in so many ways: the text layout framework, the spark component model, 3-D, etc… the irony. It will be very nice when HTML5 development tools rival Flash Builder and nicer still when WebKit rivals Flash player.

Now we have the bizarre situation where most business owners are convinced HTML5 can do everything everywhere today, so there will be even more work converting “legacy” Flash apps to HTML5. When results are apparent against the real-world browser environment, many applications will want Flash fallbacks. While Flash developers today are thrilled to get HTML5 output (limited) from Flash Professional or iOS AIR output from Flash builder, in the future it will probably go the other way, with Flash being a possible output from HTML5 authoring tools, as it will remain a meaningful technology for legacy browsers and specific environments for some time to come.

I was finishing up a very busy week this past Friday, when I got an email from Ed Kotnik – “Congrats on Best of the Best at XMPie Conference.”

This was for an application we built some time ago, but one that is very dear to my heart: our Silicon Paginator implementation for Royal Caribbean Cruise Lines. We worked very hard on this, and it represented a stage in the evolution of Silicon Paginator that was important to me at the time, and is still relevant to our future.

Silicon Paginator wins Best of the Best Award

There are many cool aspects of this application, but what I think most significant is the pure InDesign Server automation that renders the very high-quality output from very complex data, following a structure-to-style mapping that lets InDesign templates format content from (mainly) XHTML structures into output that looks hand-crafted. Also the way it was built was a significant step up from the ways we had done this before, as we took on this project right after hiring one of the first in a string of top-notch software engineers that has joined us the past few years, and he took my crude hacks of years earlier on the processing and built it out the right way with more modern technology.

The Royal Caribbean implementation of Silicon Paginator represents the culmination of earlier work with XHTML as a data source for InDesign Server applications. One particular project that had a great deal of similarity was the content management system we built for ACTS Seminaries, and attempted to productize as “Instacatalog” in 2003. This was prior to InDesign Server, but at the time we automated InDesign desktop for catalog output. The application worked by ingesting XHTML, parsing it with a stream parser (then SAX-based) and analyzing each node of content to determine its structural context to render it in InDesign in the correct style, given style mappings between the CSS for the web and the InDesign object styles for print. That application and others like it worked quite well, but I had hacked them in fairly precarious ways. In particular, the processing wasn’t elegant at all.

Instacatalog output

Despite the lack of elegance back then, it was impressive how well it worked. The one limitation (still a limitation) is that it didn’t handle absolutely arbitrary structures: you could put a list inside a list, but only so many levels deep, for example. Yet it was able even then to validate and reject cases where users would try to go outside of such limitations. And it seemed whatever structures anyone needed, it was easy enough to extend.

With Royal, it had to scale. The volumes were huge, there was certainly not going to be any manual inspection of things. We were fortunate to have more product-level developers working with us, and the whole transform piece was refactored to work very efficiently. Template setup was fairly manual, though we had some utility scripts that were helpful and we developed some straightforward setup guidelines. These days we are much better at tooling the document setup aspect of this sort of application.

The text-intensive, tables flowing across pages form of output with sources like this is still in the minority in Silicon Paginator implementations. More commonly we have relational data sources, with only crude markup from within database fields, producing directories and catalogs more than heavy stories of rich text, but here in applications like this are the fun part. How far are we going? Do we merge cells? Lists within lists within tables? Tables within lists with tables? It is fun to gain control over the transformation between different forms of rendition, with HTML posing some unique challenges and opportunities. While Royal may enjoy the tangible benefits of automated publishing, I enjoy the technical details in seeing what I had personally cobbled together 10 years ago get transformed by real developers into a truly reusable set of tools.

Today I am at SVG Open in Cambridge, MA where we are announcing the HTML5 Version of Silicon Designer. It is hard to convince some of our clients that Flash has any place at all in an applications, yet we are still very happy that we mastered Flash as it makes our HTML5 efforts focused on WebKit rather than pretending that it magically makes IE8 into an RIA (Rich Internet Application) powerhouse.

Deploying the Flash player and Webkit with parallel forms of the same application lets us reach the entire browser base, from legacy PC browsers to the full spectrum of mobile devices, without degrading user experience. We write once, and don’t have to test everywhere. Our approach is: ‘Flash where Flash goes, HTML5 where Flash doesn’t go.’ And we don’t see Flash going away anytime soon in the online editing space; it still provides the richest experience in many contexts. Targeting the most powerful media available on each distinct client lets us ensure the optimal user experience across all devices.

That is my philosophy for Silicon Designer. Not for all web apps. But with Silicon Designer, here we have an extremely rich client experience using Flash, which currently has no parallel in the HTML5 world, quite, though Safari Webkit and Chrome Webkit are getting close, and the Adobe extensions to Webkit bring them closer.

At SVG Open we saw various approaches to legacy browsers, when apps start out in standards-based mode vs. in Flash:

1. Sometimes old browsers are simply ignored
2. Sometimes, as with Raphael.js, elaborate schemes to produce VML for the older Internet Explorer are used
3. Sometimes, as with SVG Web, Flash is derived from HTML5/SVG

These all can be fine, depending on user base, application characteristics, etc., but with an online editor for high quality print documents, none are currently ideal. Flash still has the ultimate power for robust vector graphics… Webkit and other implementations of HTML5/Canvas/SVG are just catching up. SVG Web is not an option for us, as our HTML5 version uses HTML, Canvas, and SVG, not pure SVG (I doubt there is currently an “SVG Web” equivalent for the full gamut of HTML5).

Deriving one format from another generally leads to poor quality in the format that gets derived. It is more powerful, if you can afford it, to have both forms of something derive from a model that is not tightly coupled with either one, using a higher level of abstraction.

In our case, as we already have a robust Flash app, and we specialize in defining the document model agnostic to rendition engine (as this model has to work in Flash, InDesign, and Scene7) it’s not the biggest deal to have two versions, one Flash and one HTML5. With the HTML5 version, do we really want to target IE9? Firefox? It would be nice, yet it is not critical.

Essential to us is hitting the iPad, especially, secondarily providing good mobile experience (as designer tends to be most relevant to larger screens, phones are not often used for high-end forms of design) and providing on each device/OS the best possible experience. Targeting just Flash and HTML5 that works with Webkit, we get all we need.

I am really enjoying our work on the CS Extension for Fotolia, which we announced at MAX and will be refining and re-announcing throughout the rest of the month, as we localize it for 14 languages and spread the word among users of Photoshop, Illustrator, and InDesign.

I was not really aware of the “Microstock” industry until quite recently. A few months ago, I was brainstorming with my colleagues about possible uses of our Silicon Connector technology, which brings true HTTP links to Adobe InDesign. One developer suggested integration with Fotolia, which he said was a stock photo company with good APIs, and a good company to work with.

It worked out really well. I think we are in the right place at the right time, because we have managed to exploit state-of-the-art extensibility technology from Adobe to connect designer desktops to stock photos, enabling purchases without leaving the design app. This is not exactly an original idea, yet I don’t know of it being done recently in InDesign, Photoshop, and Illustrator. We also have many ideas how to take this further, and don’t see many technical limitations.

We have been extending Creative Suite applications for 10 years (since before they were considered a “suite”) and the current state of extensibility is absolutely great compared to what we used to have to go through to create something at all like this.

In the early days, one had to build plugins in C++ targeting specific CS Apps and Operating Systems: one program for Photoshop/Mac, one for Illustrator/Win, etc. Different applications provided more or less support for cross-platform starting points, but still each app was different, the platforms differed as well, and installation/testing was multiplied out by version/platform/app.

This was the context in which Adobe launched the ill-fated “Adobe Stock Photo” which they discontinued for mysterious reasons. While the engineering challenges were far greater back then, it would appear that Adobe would have even greater challenges on the front of alliance/partnership, as they could very easily alienate their users/partners with any direction chosen for the program.

To sell expensive stock photos might seem to make the high-end photographer/photoshop customer king, yet for each of the photographers enjoying high-end stock photo revenue at the time, there were many aspiring photographers itching to get into the game, and a growing “microstock” movement supporting them. Many of these had bought Photoshop. Whose side was Adobe on?

“Low end vs. high end?” was one decision, another was “specific companies to partner with?” – here was another minefield, as traditionally Adobe would want their software purchased by all; showing any bias towards a specific company might jeopardize the  balanced, equal opportunity “partner story” they had strived to attain.

I remember when Adobe had the bright idea of adding a “Print at FedEx Kinkos” button to Acrobat. Within moments of the news, our phones began to ring with calls from irate printers (I don’t mind people thinking Silicon Publishing is Adobe, except at the times we are held accountable for mistakes like this). They had to take that button out quickly.

Adobe Stock Photo went away, but Adobe invested at the same time in fundamental technologies to support 3rd parties doing exactly the same sort of thing. At the time they discontinued the program, John Nack wrote:

“we’re working hard to make Photoshop, Bridge, and the other Creative Suite apps much more easily extensible so that they can support whatever services customers find useful–whether from Adobe or from third parties.”

It makes sense… something like a “print at Printer X” button or “get assets from Source Y” panel can make the Creative Suite applications much more powerful, yet Adobe may not want to be in the printing/stock photo business: it makes much more sense for third parties like us to provide points of integration. I am very glad they did what John Nack said they were planning to do.

Rather than limiting us to building plugins in C++, Adobe began prototyping ways to use Flash panels inside the creative suite applications that let us extend the applications with Adobe Flex. We were excited by “PatchPanel” – an early version of this put out on Adobe Labs, yet that prototype technology suffered from a disconnect between the events of the Flash application and the container application; it was just not ready for prime time.

With CS5, Adobe made the Creative Suite Flash panels much more robust and introduced a form of “CS Extensions” that really worked. This technology is now solid enough to compete quite favorably with C++ and scripting alternatives. You can find out more about CS Extension Builder on Adobe’s Developer Connection site. The main benefits:

• Build one extension in Flex that can run in multiple apps/platforms
• Compile to a single (cross-platform) .zxp file that installs the application with CS Extension Manager into multiple applications (if you have targeted more than one)
• Connect to Web Services or other external data/content sources from the CS Extension
• Fully interact with the event model of the container application

Our one big complaint is that we have to use Flex 3 to build these, currently… we would like to leverage the greater power of Flex 4. At the Creative Developer Tour recently, Adobe indicated that future versions will support up-to-date Flex versions. Still, the power of the  current form of CS Extensions is already a huge step forward.

You can find our free CS Extension at www.fotolia.com/adobeplugin