My Tools for Writing and Self-Publishing “The Yii Book”

My first book, PHP for the World Wide Web: Visual QuickStart Guide, came out about 12 years ago now (it covered both PHP 3 and 4!). In the dozen years since then, I’ve written 22 more books (including revisions), with three different publishers. About 2-3 years ago, I first started thinking about self-publishing a book, and as of Fall 2012, am finally doing so with The Yii Book. I occasionally get asked, probably by people that also want to self-publish, about the tools I’m using for writing and self-publishing this book: technically speaking, how am I doing it? The introduction to the book does discuss this, but as not everyone has purchased the book, I thought I’d write up my process.

To best appreciate the system I’m using, let me explain the traditional alternative. With a traditional publisher, you’re normally writing in Microsoft Word (yes, Word). Even in the technical writing field, publishers that don’t use Word are the exception more than the rule, in my experience. With a programming book, I would provide to the publisher:

• A chapter as a Word document
• All of the images (as TIFs)
• All of the scripts (i.e., code examples) as separate Word documents

After I submit a chapter to the publisher, it’d be passed through three editors:

• Project/managing
• Copy/line
• Technical

Then the chapter would be returned to me for revisions. After revising the material, it would be examined by the project/managing editor again before going to the compositor. The compositor takes those materials, tosses them into a desktop publishingapplication such as InDesign, and creates a PDF. I’m sure the workflow is optimized as much as the compositor is able to, but with a traditional book (print or electronic), an actual human person lays out the columns, the text, the images, the page breaks, etc.

Then the managing editor, a proofreader, a composition supervisor, and myself would all look at the PDFs and make suggestions, corrections, and so forth. The compositor would go back into InDesign, make those changes, and output another PDF. By hand.

To put a dollar sign on what this is worth, Brett Kelly self-published a guide to Evernote. In an interview, Kelly said that just having revisions to the existing work done would cost him hundreds of, if not a thousand, dollars. I have no idea how much the original layout would have cost, but I’m sure it wasn’t cheap.

I was not planning on paying a compositor, so I assumed I’d be learning InDesign myself. I used PageMaker a million years ago, so learning InDesign isn’t impossible for me, but I certainly wasn’t looking forward to it. And I wasn’t looking forward to having to make changes and updates with each minor correction. There had to be a better way…

Technically speaking, I had two goals when it came to self-publishing. First, as a developer, I wanted to optimize the process so that it was as efficient as possible. Second, as a reader, I wanted to be able to output the work in as many formats as needed: PDF, epub, mobi (Kindle), HTML, even print. I’m not sure about the second goal, but as for the first, using MS Word and InDesign would not cut it. Fortunately, I found Markdown and Scrivener.

Markdown was created by John Gruber as an easy way to create HTML content for blog posts. Markdown allows you to write in plain text, with a minimum of markup, and output good HTML. I began using an extension of Markdown, MultiMarkdown, in 2010 to write my newsletters. About that time, I also discoveredScrivener. Scrivener is far and away the best writing application Ive ever come across. If youre thinking about doing any serious amount of writing, download it today! Scrivener makes it extremely easy to focus on writing, while also providing a top-notch resource for managing your materials. Scrivener also supports MultiMarkdown, and the application itself is capable of outputting in every format you’ll want:

• PDF
• epub
• mobi (Kindle)
• iBooks
• Rich Text Format (RTF)
• Microsoft Word
• OpenOffice
• HTML
• LaTeX

Unfortunately, when writing in Markdown, Scrivener is more limited in its output formats. This is where Pandoccame in.

Pandoc was created by Dr. John MacFarlane, a philosophy professor at the University of California at Berkeley. Pandoc is a command-line conversion tool that can take many different types of input and create many different types of output. For example, Pandoc can work with (take as input):

• Markdown
• HTML
• DocBook
• LaTeX
• And more

With that source, Pandoc can output:

• HTML
• Word
• OpenOffice
• epub
• DocBook
• LaTeX
• PDF
• Markdown
• And more

Pandoc also extends Markdown slightly.The only thing Pandoc can’t do (for my needs) is output mobi (Kindle) files. For that, I turn to Calibre.

Calibreis yet another excellent open source application. Calibre can convert and export a book into multiple formats, including mobi. And although Amazon provides its own tools for creating mobi files, my research suggested that Calibre is the better tool.

This might all sound like a lot of work, but once I figured out the right tools, and how to use them, it’s quite elegant and supremely flexible. Here’s what the process looks like, in sequence:

1. In Scrivener, I write in plain text formatted using Markdown.
2. Images are taken separately, using Snapz Pro X.
3. From Scrivener, I output Markdown. With the current version of the book (at the time of this writing), 14 chapters, plus an introduction, results in 10,000 lines of Markdown source.
4. Then I turn to the command line and I use Pandoc to convert the Markdown to PDF, using my own LaTeX template. LaTeX is amazing, but not for the faint of heart. With 10,000 lines of Markdown, over 100 images, a self-generated table of contents, and some layout formatting, it now probably takes Pandoc four minutes to create the PDF.
5. Then I use Pandoc to convert the same Markdown source to epub.
6. Then I open the epub in Calibre and output a mobi (Kindle) file.
7. When I need to, I use Pandoc to convert parts of the Markdown to HTML, for posting excerpts online.

Thanks to Markdown, Scrivener, Pandoc, and Calibre, I’m able to meet both of my goals. I’m outputting in all the formats I want and the process is fairly optimized. There’s still some work I can do towards that end, but I’m able to write in plain text, which is fantastic. I dont have to take my hands off the keyboard to highlight text or click buttons, and I dont need to memorize keyboard commands. I can write, or edit my writing, using any application. The images are kept separate, so they are easy to use and maintain. While Im writing, I dont have to worry about layout or formatting or document size or anything but the words. As a writer, this is paramount.

If I want to change the features or formatting of any output typePDF, ePub, HTML, I just need to change a template or my Pandoc command and I get new results without touching my original source. Write once, publish everywhere!

Its really a supremely capable and flexible solution, vastly superior to MS Word and InDesign in almost every way. It just requires that I learn some new things, thats all. By far, the biggest hurdle has been learning enough LaTeX to get the PDF results I wanted. And that’s still an ongoing process.But Id rather learn how to use the best tools for the job than just accept the tools I already know how to use (e.g., MS Word). And, I haven’t spent a dime! (Well, I did donate $100 to Calibre, but I didn’t have to.)

I hope this explanation is helpful to those that might also want to self-publish or, if not, then is at least interesting to learn about.

As always, thanks for reading and let me know if you have any questions or comments.