The Insider’s Guide to InDesign Server
Adobe InDesign Server is perhaps the most obscure product ever released by Adobe, yet it is in our humble opinion the greatest piece of software ever to grace this Earth. What follows is everything you ever wanted to know about InDesign Server, but were too terrified to ask, as presented in our recent webinar:
What is Adobe InDesign Server?
InDesign Server is best understood in relation to a tool that should already be familiar; desktop Adobe InDesign. More than 90% of the InDesign Server code base is in fact literally the InDesign desktop codebase.
The common code is the core composition engine. Conveniently, the primary automation techniques (C++ plugins and ExtendScript coding) are also the same in both desktop and server flavors of InDesign.
The difference is that InDesign has a user interface, which is ideal for one highly-skilled designer creating documents and document templates. InDesign Server has no default interface. You can run it by a command from a command line, or you can call it from your own user interface that you can create: this can be a web page, an application, or even InDesign itself. Thus desktop and server deployments are really different from each other. Adobe adds a bit of server-specific functionality, such as a SOAP interface for calling scripts.
Critically, InDesign Server lets you run multiple, parallel instances of InDesign concurrently, on a single server. This can help immensely, for example, if you have multiple users on a web site requesting dynamically-created InDesign documents simultaneously, or if you need to produce a million documents in an hour.
With the desktop form of the product, users tend to have to be highly skilled. InDesign is extremely powerful, so the very advanced interface allows completely fine-grained control over anything you’d ever want to do with page layout and document editing. However, the learning curve is steep: I have worked with InDesign professionally for 20 years, and I still feel like a novice-level user of the product. Power users like David Blatner, Chris Converse, and Bob Levine are rare on this earth.
InDesign Server, on the other hand, can cover a vast range of use cases, most of which do not require InDesign expertise: this allows mere mortals to produce fantastic print output. For example, you can have a web page from which you select a data source and a document template, to generate a parts catalog. Or, a salesperson can edit just the product price on a spec sheet, calling InDesign Server to update the template (originally created with desktop InDesign) with the latest pricing information.
This is not to say that InDesign skill isn’t required in an InDesign Server workflow: there usually is an InDesign expert involved, building the template that is used by the application. But there is an appropriate division of labor: without learning a thing about InDesign, a salesperson can create a gorgeous document using updated information, while adhering to brand guidelines she knows nothing about. Designers, meanwhile, can focus on their art, rather than receiving an email for every price change or re-worded slogan.
Why Adobe created InDesign Server
InDesign Server lets you create InDesign documents programmatically. This is required for large-scale database publishing: if you have 300 variants of a 100-page catalog, for instance, automation makes sense. If you want to send a million personalized letters, InDesign Server can render print-ready files with a lights-out process.
Beyond database publishing, online editing solutions that use the InDesign application require InDesign Server. The end-user license agreement for desktop InDesign prohibits direct usage in a web-based application, and in any event, InDesign desktop is not practical for multi-user applications. InDesign Server, on the other hand, allows for numerous concurrent instances of the InDesign engine, and can scale to arbitrary throughput and performance requirements, even for the largest sites on earth.
Finally, InDesign Server can also offload time-consuming processes for InDesign workgroups. Say your design team needs to create three different types of PDF for a given InDesign document. By simply saving the InDesign document to a network drive, InDesign Server can render such output automatically, freeing up the desktop for creative work.
InDesign Server uses templates built in standard desktop Adobe InDesign. It lives and breathes InDesign and Adobe graphics formats, just like InDesign itself, so it provides the optimal capability for high-quality output. It renders PDF, PSD, AI, and other graphic formats perfectly, and uses the same PDF job options as Acrobat and other Adobe tools, so it has quite an advantage over other engines when it comes to document automation. The core InDesign product is such a phenomenal success, that it is the tool of choice for the creation of any document that prints, as well as graphics that require deep typography and exquisite graphic perfection. Why would anyone use another engine for server-based composition?
How InDesign Server came to exist
We at Silicon Publishing started to automate desktop InDesign in the year 2000, and within a few hours we realized two things:
- InDesign is the ultimate tool for automated page composition.
- The world needed an InDesign Server.
Of course, at the time we weren’t so well connected with Adobe, and Adobe back then was a profoundly desktop-focused company. Our passionate requests of Acrobat Sales reps, Partner Program representatives, random acquaintances who worked at Adobe (we knew a parking attendant who had seen Jon Warnock once!), and “new feature” forms on Adobe’s site, probably never managed to reach the InDesign team.
“Yes, Mister Dunn, an InDesign Server is a wonderful idea, we’ll get right on it.”
— Acrobat sales rep, being polite, circa 2001
Thankfully, though, we were not alone in deducing the obvious; others did a better job of scaling the walls of the Adobe Seattle office. It turns out that some brave souls actually used InDesign like a server and asked Adobe the naive question, “is this legal?” Olav Martin Kvern, who worked for Adobe back then and subsequently joined Silicon Publishing, explains that it all started with beer.
As Ole explains, a brewery in upstate New York was using InDesign (desktop!) to power a web-to-print application for custom labels on their beer bottles. This challenge to Adobe’s legal and product teams led to the creation of this inevitable product.
Several of the amazing team of people at Adobe who were evolving InDesign back then, such as Ole and Whitney McCleary, actually got quite enthusiastic about this new product, envisioning sites that would say “Powered by InDesign” on them, letting the world leverage the InDesign engine to do things that had never before been possible.
We at Silicon Publishing were ecstatic, as by October 2005, when Whitney announced it along with InDesign CS2 at the IFRA conference in Europe, we had already built quite a business around automating InDesign itself. We ported all of our code to the server in less than a week.
How InDesign Server is used today
At a very high level we see the three use cases mentioned above:
- Database publishing
- Online Editing (often known as “Web to Print”)
- Workflow automation — offloading tasks from power desktop InDesign users.
Each of these has numerous variations based on different vertical markets, different types of business model (B2C vs. B2B applications, for example), different document types, and various integrations (with DAMs, data sources, web portals, etc.). Considering all of the integration use cases, and its ability to sit as an embedded layer in a huge number of solutions or applications, InDesign Server has an even broader applicability than does InDesign desktop. I consider it inevitable that one day Adobe will make more money with the server flavor of the product.
Database Publishing
We first considered InDesign in light of database publishing, as we had been focused on the automation of documents such as catalogs, directories, statements, etc. We had, in the 1990s, mastered “electronic publishing” — automated page composition of data-driven output — using older technologies such as Xyvision. We found that InDesign Server beat Xyvision in every way, with the exception of throughput. When InDesign Server first came out, servers were slow, and there was not much optimization of the product for server usage. Still, if you threw enough hardware at it, you could attain far better performance. We quickly immersed ourselves in load balancing and job queuing of multiple parallel instances of the server. That fine art continues to this day, and I’m happy to say that Xyvision no longer has a throughput advantage: we can pretty much scale IDS to arbitrary throughput given modern technologies.
Database publishing is used across a wide variety of document types: below is a grouping of some of these in terms of the complexity and throughput requirements.
The Royal Caribbean cruise booklets represent a classic use of InDesign Server for database publishing. Each customer that purchases a cruise (and there are thousands of these per day), is given an automated cruise booklet with their itinerary, ports of calls, and whatever other content is deemed appropriate for this single recipient. It is built in a ready-to-print format using InDesign Server, and the logic for output includes very subtle rules.
For example, sections may grow or shrink based on the amount of data that needs to be rendered. Never is there excessive whitespace, as filler images, copy fitting, conditional information, and other techniques automatically ensure a hand-crafted look and feel, regardless of the shape of the data that needs to be flowed. In our experience, this level of perfect automation can only be attained with InDesign Server, in our experience, as it enables a greater deal of precision and nuance in exposing the composition engine to automation than any other tool in existence. Of course it doesn’t hurt that it has all of the typographical and graphic capability of InDesign, along with its harmony with other Adobe technologies and formats.
Such automated publishing allows for communications that are personalized and powerful. They also aren’t necessarily rendered to print, either: InDesign is increasingly used for non-print output. In the case of Royal Caribbean, most of the content is distributed electronically, and can be navigated through web- or app-based document readers. Yet high quality print is always an option, and can be essential for the few remaining “physically printed” items such as bag tags.
There was an era when InDesign Server did not go fast enough for the world of “Variable Data Publishing” or VDP: situations in which large company needs to produce, for example, 40 million bills for Internet, TV, and phone. Today, however, advances with servers, cloud scaling, InDesign, and printer RIP technology make InDesign infinitely scalable: it is now an ideal tool for this niche of database publishing, with significant advantages over the older tools out there. This is because its quality of rendition and control of automation are second to none, and it works with design constructs, templates, and object styles that are already familiar to the majority of designers in the world.
When we invested in automating InDesign in 2000, we had complete faith in the inevitability of the server form of the product, and the likelihood that Moore’s Law would continue as the norm for computing. As 2020 approaches, we think that we were right. It did take a bit longer than anticipated, but as of now InDesign Server is the optimal tool for nearly every database publishing use case.”
Online Editing
The world of online editing or “ Web-to-Print” is the second major use case for InDesign Server. This technology is extremely dependent on something that is way out of Adobe’s control: web technology itself. For some reason, we see the same dichotomy among developers that one finds among designers: humans tend to think in more of a web-oriented way, or more of a print-centric way.
There are fantastic print designers who think of “pixel perfect” layout and the perfect geometry of fixed-layout composition, like a book whose pages are set at 8.5 x 11. There are also wonderful web-centric designers who think more of responsive design, mastering the nuance of content that will reflow and shape itself across aspect ratios, devices, graphic resolutions, etc. Rarely do you find designers who can do both really well.
In similar fashion, developers tend to be more web-centric or more print-centric. While a database publishing application can often be built by a single person (usually someone who knows InDesign and can figure out “just enough” data), it is hard for us at Silicon Publishing to imagine the complex form of web to print being developed by any single person really well. We certainly have two teams, sometimes three when we get ambitious or silly enough to consider the middle tier as its own domain. So building robust web to print solutions that are complex is best done by groups of developers, with the operative word being “complex.”
Is there a simple form of online editing? Yes, there sure is. In our many years of building online editing solutions, we see two absolutely opposite approaches, and the very first step in creating a web to print application is determining which approach you want to take. These two approaches are:
- Simple: server-centric, form-based solutions.
- Complex: client-centric, WYSIWYG, on-canvas solutions.
While both can be considered “online editing” or “web to print” and both require an InDesign Server back end of the highest quality, the level of development between the two is night and day. Form-based, server-centric solutions can be built quickly by almost anybody, while WYSIWYG, on-canvas editing is quite nearly impossible. A few companies, such as ours, have had the tenacity to make this work at scale with perfection, but it is not a great starter project.
Yet as InDesign Server resellers, we have seen even very small companies, often with just a single designer or developer, build the simple case rapidly, and with great success. We didn’t even offer a product along these lines for over 10 years: we simply said “purchase IDS and read the scripting guide.” We recommended, however, that if they wanted the complex form: editing on the canvas and rendering precisely with InDesign Server, that they consider our Silicon Designer product, which was developed over more than 10 years by more than 10 expert developers.
The simple case involves sending a request to InDesign Server from a web page, which includes only the pieces of content that update, such as the name, photo, and address on a business card, and getting back an image (InDesign Server serves up fantastic PNG or JPG) to show those updates applied to an InDesign template.
Now, while this is constrained, (for example, not letting users re-arrange objects on the page), constraints are often your friend. This allows for an absolute division between content and design intent. You can be assured, that if content is updated through a form where only the text can be changed, brand guidelines will absolutely be maintained. Sometimes simple is best!
As much as we love to do the impossible, we now support and embrace the simple form-based approach whether you want to build it yourself, or if you want us to give you a turnkey solution. The advantage of using Silicon Designer for a form-based solution is the incredible scalability and speed that we can offer. This is based on 14 years of tuning the load balancing and job queuing, as well as the prescribed forms of setup and bullet proof APIs, but if you don’t have tons of users/massive scale, or robust application demands, this may not matter.
The general functionality of merging data with templates is something that we love to teach designers and are happy to explain in nuanced detail. If it works, it works, and many InDesign Server-based online editing applications have generated huge revenue and massive efficiency gains with minimal development by taking the simple approach.
Yet, complexity does have its place. What if you actually do want to make design changes through a browser-based interface? Maybe you aren’t a salesperson changing a price but you’re designing a greeting card for your mother, and you want to layer 16 of your own images and arrange them precisely, cross-fading among them using InDesign’s phenomenal blend modes, while changing each letter of “Mom” to a different font with a different drop shadow? WYSIWYG, on-canvas editing solutions can offer complete creative freedom, or just the right balance between freedom and constraint, making them quite valuable for such use cases. School yearbooks, for example, could barely be created through a form-based interface: freedom and creativity are essential elements when editing that sort of content.
Ever since the 1990s, Silicon Publishing’s founders have been building solutions allowing users to edit in WYSIWYG ways, from PC applications to the very first web pages that rendered dynamic PDF. The very first thing we did with InDesign, when we got it, was to render HTML, SVG, and XSL-FO with it. As the web technologies advanced — CSS, JavaScript, SVG, Flash, HTML5, Canvas, web components, etc. — we mapped these forms of rendition to InDesign rendition as closely as we could. After building WYSIWYG editors based on InDesign Server from 2006 through 2008, we announced the Silicon Designer product at Print 09.
The rest, as they say, is history. We actually managed to round trip HTML5 with InDesign, using a basic rendition model rooted in our work from the 1990s, which has been maintained by a growing team since 2006.
With true WYSIWYG editing, you save out from Adobe InDesign (using ExtendScript) an XML or JSON file that serves as a rendition-agnostic expression of the Document Object Model (DOM). This lets the document render as an editable HTML5 document in a web browser. Edits made to the HTML5 are expressed back in that same format, which is rendered by InDesign Server as desired, typically for previews, thumbnail images, and final output PDF.
In the case of our Silicon Designer product, this intermediate format enabling HTML5/InDesign “round trip” is called SDXML: this XML namespace is somewhat similar to the IDML export format of InDesign, yet it has a whole set of special characteristics allowing for fluidity and interoperation with web-centric DOMs. It includes metadata informing the editing experience: when you set up the document template, you can indicate which page objects are editable or not editable, and exactly how far edits can go.
Not only have we been deploying Silicon Designer for over a decade, we have also consulted on numerous development efforts around the world where editing directly on the canvas of the document was required, thus something of an “InDesign on the web” functionality was the goal. We are happy to help others, even our competitors, solve this challenge. But this complex flavor of online editing can be a sinkhole of development time: the sad reality is that the text engines of browsers and devices are not naturally very similar to InDesign’s text engine at all, so there is some serious software engineering required to even approach the level of a Silicon Designer.
While Silicon Designer and solutions like it provide some form of “InDesign-like” capability (you might start from a blank canvas and create a whole document from scratch, for example), our goal has never been “to replace InDesign with the same thing on the web.” No, in every serious deployment of Designer, desktop InDesign is an important component of the solutions. End users don’t want an InDesign interface unless they happen to be InDesign experts, while the other 99.9% of humans want the opposite, as ease of use is essential. An optimal balance between constraint and freedom is ideal. Perhaps the “advanced features” are only available to some people in the organization, or users may be allowed to crop and scale an image but not allowed to re-position it.
The beauty of abstracting out the document model is that the experience on the web can have precisely the characteristics desired. Just as InDesign is beautifully extensible we have always maintained the goal of extensibility when developing WYSIWYG on-canvas editing solutions such as Silicon Designer. The intention is not to replace InDesign, but to deliver “ InDesign for the rest of us” — allowing those of us not skilled in InDesign to make gorgeous InDesign documents based on templates created by real designers, using interfaces appropriate for our level of skill and the desired degree of creative freedom.
Workflow Automation
The third and final high-level use case for InDesign server is perhaps the most boring one, but it can also be profoundly pragmatic and useful. This is workflow automation in support of InDesign authoring groups.
Say you have a recurring task that you and your design/authoring team must repeat throughout the day: making 3 variants of PDF output, making a thumbnail image for the cover of a document, making various resolutions of thumbnails in different format for different web workflows, etc. Such tasks can generally be automated with InDesign desktop, thanks to the power of ExtendScript. However, maybe that will tie up your computer for a significant time. That is when InDesign Server comes in, to automate tasks server-side, letting you work continuously on the desktop, rather than going to get coffee while the 14 flavors of output are generated.
InDesign Server is available with several DAM and Content Management products/solutions providing exactly this sort of functionality. Asset management systems/solutions, for example, often require multiple forms of rendition to be automatically generated from a high-resolution “source” assets upon ingestion. InDesign Server is ideal for such a task, due to its native InDesign capability, as well as its friendliness towards other Adobe formats such as PSD, AI, and PDF. You may be using InDesign Server but not even know it. The Adobe Creative Cloud is a great example of an asset management system that uses InDesign Server.
Automating InDesign Server
If you’ve gotten this far, you may be wondering “how is InDesign Server automated?” There is plenty of documentation from Adobe on this subject, but at a high level:
- Scripting, mainly using Adobe’s flavor of JavaScript called ExtendScript, is the primary way most InDesign Server solutions are coded.
- C++ plugins are a second option for automating InDesign. As these are really hard to develop and maintain, we avoid them unless they provide substantial performance gain (not always the case!) or in those rare cases where scripting can not produce the same results.
- The desktop and server forms of InDesign automation are nearly entirely identical. This is fantastic, as the process for development is generally a matter of creating a single script that runs in both flavors, with desktop used for initial testing and debugging. In desktop InDesign, you can see quite easily in real time how your document is being constructed, which is invaluable. Once the script works in desktop, you simply deploy the same script to InDesign Server.
Link to this section of the video
Scripting is not hard to learn: the ExtendScript documentation and scripting examples from Adobe can give you quite a good start. Designers can become scripters, and coders from other languages (or JavaScript in other contexts) can automate InDesign.
We have, however, noted that being a great developer doesn’t necessarily make you a great InDesign developer. Not only do you have to know something of the InDesign object model to automate it, there are little gotchas and known issues with the InDesign engine that must be learned over time. One thing to avoid is giving InDesign raw programming tasks that it isn’t optimized for: you wouldn’t want to, for example, build an XML parser inside of InDesign, or maintain a large database of information you are filtering and sorting: rather, you want to feed InDesign the most “ready to go” and efficient, data (just what’s getting printed, in the order it is rendered in), and script the InDesign engine to do what it does best (flow that data onto the page). The place to have a database is in a database application, which should be upstream from the InDesign server.
Deploying and scaling InDesign Server
InDesign Server is typically deployed in Windows: despite the availability of a Mac form of the product, it is extremely rare to find the Mac server deployed (the one exception seems to be shops that are still dependent on old Mac-specific fonts). It can be deployed to your own server, to a virtual machine in your data center or a dedicated hosted server, or in any cloud that supports Windows, such as AWS and Azure. We are currently seeing AWS as the primary deployment of choice, comprising something like 50% of the large deployments we encounter.
Load-balancing and job queuing the multiple parallel instances of IDS can be done in a relatively simple way, or you can go full out, like monitoring job health, re-runnnig failed jobs, optimizing the instance usage based on detailed monitoring, or autoscaling to spawn additional instances (Server/AWS instances, not just IDS instances) based on load. We have built a series of 4 robust solutions for scaling IDS over the past 14 years, and we have scaled very large solutions with enormous throughput successfully.
Yet we’ve also seen solutions run for years with simple round-robin job queuing implemented by an average developer in a day or two. How far you go with load balancing, and whether you’d want to buy a product such as Silicon Designer that includes such functionality depends on a number of factors, including: how many users do you have? how stable are your scripts? how reliable are your templates?
When we first automated InDesign Server, we found it necessary to frequently re-run jobs and adjust to crashes. After we had hired expert scripters, such as Ole himself, we hit long periods free of crashes, probably due to a combination of perfect scripts and advances in the product. Yet with larger solutions, random content sources, and/or frequent code/template/asset updates, robust load-balancing and job queuing can become a requirement for success.
How to try InDesign Server
InDesign Server is available as a fully-functional 90-day trial. We explain how to access and try InDesign Server, as well as other core Adobe resources, in our blog post entitled “ How to Try Adobe InDesign Server.”
How to buy InDesign Server
As an authorized Adobe reseller since product launch, and with a number of the original InDesign developers on our staff who share a passion for delivering the best InDesign Server solutions around the globe, Silicon Publishing is the best choice. Although our pricing is the same as others, we offer a level of support that is truly second to none. Thanks to our 20-year partnership with Adobe, we have quick access to their assistance, but the product is generally so stable and well-known by our team that we very rarely need Adobe’s help.
InDesign Server has two main flavors, depending on what you do with it:
- The Premium license, priced at $13,500/year, lets you use InDesign in the context of a web-based application, or in an application where a real-time request returns a document. This is required for web-to-print solutions whether on the Internet or a corporate intranet.
- The Limited license, at $5,000/year, lets you use InDesign for batch composition solutions within your local network. This is typically the license used, for example, if you are rendering 100,000 financial statements in a single batch process.
Q&A
At the webinar, we were asked the following questions, and had these responses.
- What version of JavaScript does InDesign Server support?
- InDesign ExtendScript is basically the same as ECMAScript version 5. (Not very modern).
- That only has to do with communicating with the InDesign model itself. In your web front end or database publishing back end, you can use the latest and greatest JavaScript.
- Also, it’s worth noting that InDesign can also be automated with any scripting or programming system that can connect with the platform’s Inter-application communication standard (i.e., COM on Windows).
2. Is there a way to dynamically add/remove table columns based on the data retrieved with a component in a template?
- Yes!
- This is a typical challenge in many database automation projects. You can use the variability of incoming data by either dynamically manipulating the table itself, or selecting an appropriate template for that table (typically expressed as an InDesign snippet).
- A common use-case is parts catalogs, where it’s perfectly possible for each type of part may have its own data schema, thus requiring a very different table structure.
3. Can InDesign Server be used for making CNC files?
- Ole just happens to use InDesign as a CAD tool: he creates g code files that drive control computer-controlled routing tools and 3D printers, as a hobby. So, yes…
- It is all geometry: the geometry of anything you create in InDesign can be conveyed to 3D applications with automation.
4. Is InDesign Server considered just an engine to process with, or can it house assets itself?
- It is usually a very bad idea to use InDesign for things that it does not do best.
- While IDS does have a generic scripting language inside of it, you would not want to persist or manipulate data, build data-centric algorithms, or perform complex calculations unrelated to page layout.
- The best practice is to feed InDesign ready-to-render data and use the InDesign engine for what it does best: flow content onto pages.
5. What about animation in InDesign?
- There are some limited animation capabilities built into InDesign.
- One third party application, In5 from Ajar Productions, is a powerful tool for creating HTML5 applications with InDesign:
ajarproductions.com - It is a myth that InDesign is merely a “print production” product. It is a design tool, and it is seeing wider adoption than ever, in many cases growing through demand for highest-precision authoring of documents/graphics that never print. Infographics, for example.
We are thankful to be part of the network of third-party InDesign Server solution providers, and we want to continue to see InDesign Server expand in usage and find its way into more and more solutions. It has steadily grown since its inception and we see some really exciting frontiers ahead for this fantastic product.
