The English book printed project: production report 1

After the release of the eShop (thank you for all the purchases!). I started this week another huge item on my TO-DO list for Pepper&Carrot: self-publishing it as a comic book. I do it because I find it exciting to make a big high quality comic book using only FLOSS from scratch! I'm not expecting much from the sales. I just want my own Pepper&Carrot book, 100% FLOSS to crown the last five years of production. Focusing on that target even drives me away from the Inktober challenge I usually very like. But it's a project with a lot of difficulties and I want to make it perfect. So I took the necessary time to do research and experiments before going into production. And that's where I am at start of October 2019. Here is the results of one week of intense research and tests:

The book

Deciding the format was hard: I bought samples done by my printer, reviewed all book at home. After a lot of thought on it, my project will be a large 8,5"x11" (21,59x27,94cm) and thick book with a hard-cover. It will include 193 pages of comics -from episode 1 to episode 29- printed in full resolution on a high-quality almost mate paper inside and glossy for the cover. ( I have a sample of this quality on my desk, this is really what I want). A gallery of additional artworks will be added on full page to the comic, full credits and the list of patrons to shape a 200 pages large book. My target dream price: in between 30$ and 40$ (without shipping). I'll see if I can do that, if I can't I'll explain why in a future update. So far, it looks like I can make it from my estimation.

A double page in Scribus

A dedicated cover

I want the book to have its own cover, so I plan to paint a specific illustration for it. It will take me days but I don't want to reuse the illustrations already in usage for the Glénat book project, the TokyoPop publishing, the Breton version, etc... I'll start this artwork next week and I'll show the progress on social medias.

100% made with FLOSS

"[...] there's an entire industry out there that spent the last ~25 years making sure it's a pain in the rear to do anything useful with PDFs without their expensive tools. Now you want to undermine their collective efforts? Think of the children! :)"

~ Satō Katsura (from here).

This quote summarizes perfectly the situation. As usual, I'll use only Free/Libre and open-Source software: Krita, Inkscape, Scribus on my Kubuntu 18.04.2 LTS. Yet, my printer provides a tutorial for Scribus, the *.icc and templates. (A FLOSS-friendly printer? Only when I'll receive perfect prints I'll be able to tell).

Testing various CMYK convertion methods (comparing a CMYK red from Krita and CmykTools)

Multilingual, collaborative on Git

English only? What about Spanish? Russian? or more? That's one of the most ambitious part of my project: I started to design a multilingual system for the maximum automation around a GIT repo with scripting. I plan to use the data already translated in order to publish a lot of languages; but I'll also need help to translate specific texts. This will come after, I'll start with the English version.

An early brainstorming/mind-map of the translation system for the book publishing repository.

Printer specifications

My printer works only with the CGATS21_CRPC1.icc CMYK color profile and the PDF/X-1a or PDF/X-3 file format. I can't expect in this process a man in the middle reviewing my file on a machine with proprietary software like the one of Adobe: everything goes directly to the print machine: it must be compliant and perfect. If anything fails, I'll have to pay for the mistake and the time of the operator. Also, PDF/X-1a or PDF/X-3 mean no transparency for designing the page.

Without transparency, I'll have to overlay page with translation and without to clean credits.
(Example gif animation of the process)

The test and bugs:

During the process of exporting dozens of test PDFs with various settings and trying everything the web wrote on the topic with trial and error, I made discoveries. I report them here if you are curious enough to read them. But also for them to get referenced later by search engine; I'm also posting my workarounds: they might help.

Rendering comparison, click to enlarge

Comparison details of picture on top (from left to right):

• Scribus (Softproofed with CGATS21_CRPC6.icc but output with CGATS21_CRPC1.icc)
• Krita displaying the extracted image from PDF.
• Okular displaying the PDF output (not color managed).
• Nomacs displaying the original sRGB.

The fog of CGATS_CPR1.icc

Haaaa, color management on Linux... All a poetry! The color space of the profile looks really small at a first glance, but according to the sample I bought; it is very capable of wonderful printed comics colors. Unfortunately, the Soft Proofing of this CMYK profile renders an horrible bleached rendering, feeling like being lost in the fog of a haunted wood. This rendering is consistent in Gimp, Krita and Scribus and it took me hours of test to exclude I was doing a mistake with my color workflow. But once rendered the issue disappear, and colors looks as good as a CMYK image can be (so after a real convert, not a soft-proof preview). This is also consistent in Krita and Scribus (not in Gimp because well.. still no CMYK as far as I know). So, no possible direct WYSIWYG workflow with Soft Proofing (What You See Is What You Get) but a WYSHITFOG workflow (What You See Happens In The Fog Of Gamma). Just kidding. I found a workaround: Switch to another CMYK profile during prod ; after hours of research I found that the CGATS_CPR6.icc while being softproofed looks similar CGATS_CPR1.icc rendered (not 1:1, but visually Ok). Note for later: don't forget to switch back for export.

Bug Report URL (Krita):

• [CMYK] CGATS21_CRPC1.icc has a different rendering between softproofed and converted : https://bugs.kde.org/show_bug.cgi?id=412604_

Soft Proofed preview VS real convertion with CGATS21_CRPC1.icc.

The 72ppi TIFF of Scribus

When I started to get a somehow functional color managed workflow, It was obvious I'll have to bring modifications to the comic pages and adress issues I could spot with the CMYK profile (too dark pages, out of gamut effect that translate badly to CMYK,etc). That's something no other publisher did with Pepper&Carrot (certainly to 'respect the art') and they often just went at printing the RGB as it is with more or less success (and sometime, disasters for certain crowdfunding). Well, I'll want to adapt the art and I can. So, I ran a series of test to evaluate how I could save my modifications directly in CMYK and get them in Scribus. I decided to pre-convert pages to CMYK in Krita and export them as Tiff then import them on Scribus: this was fixing the "preview in the fog" workflow but I discovered Scribus was skipping the resolution ppi of Tiff and import them as 72ppi by default. I then tried to update my version. I tested also the last version in development. All of them had this issue, this is too hard to work like that for 200 pages, too many manual tweaks and I wasn't feeling confident for the TIFF format in general. So better to drop this option.

Bug Report URL (Scribus) :

• 0015837: Import of Tiff files: missing resolution/dpi/ppi. Note: It turns out Krita is guilty to not write the unit "inch or centimeters" while writing the Tiff.

Comparing the same page with various convertion method in Scribus: original on right.

Remastering the artworks on pages for print

On newer tests, it has appeared the best cross-format for CMYK pictures between Krita and Scribus was 'CMYK JPGs'. That's really curious; this format was a taboo for my teachers when I learned Pre-Press with QuarkXPress and Indesign around 2000 (in my youth). But it works and with a compression as near as possible to lossless for this format (100% quality) it is even a good solution for a transition flat format. Things I learned. But I don't feel good about using this format: thumbnails on the file browser looks terrible and it still feels like a hack in a way.

Anyway, I discovered during the process that -with the right settings- the export to PDF of Scribus was doing the conversion to CMYK on the fly whatever input was entered on Scribus; including sRGB format. Splendid technology. I then tested the quality of the exported PDF CMYK of Scribus in competition with Krita and Cmyktools. I obtained very similar results with Scribus and Krita. Cmyktools had other very interesting optimization, but the software being not maintained since 2011, the GUI was spanning accross my two monitors and the resulting CMYK was color inverted after extraction from the PDF. I had to exclude it. The quality of auto-converted CMYK via Scribus had similar (if not exact) quality of the one converted by Krita. I guess it is the same LittleCMS magic under the hood for both of them.

I then decided to go with only the 8Bit RGB workspace for my modification: unify my artworks for web and for print in a single format and remaster the sRGB with Soft Proofing activated for CGATS_CPR1.icc (in fact, 'workarounded' with CGATS_CPR6.icc to avoid the fog). Tweaking slightly the RGB will mean modifying Pepper&Carrot online pages too or how the episodes were designed at release. Remastering them for the book project is the right effort to do at the source to avoid CMYK mistake for the future converting while keeping always the sources artwork in a single place.

Left: Original, Right: the type of RGB corrections that might happens, (work in progress, not happy with the contrast yet).

Validation of the PDF for print

On testing the output PDF of Scribus, the harder is to know when I produce a valid PDF export. How to inspect it? I have not found a perfect solution... I still blindly have to trust something on the output of Scribus. Knowing the buggy nature of FLOSS and how the project degenerate quickly this is not making me confident. But I found a couple of options, mostly via command line interface, on a GNU/Linux system:

Imagemagick:

Imagemagick can list a lot of information about the PDF with the command here under, including the CMYK nature of the PDF. Unfortunately, no color space. Does it mean ImageMagick doesn't do it? Or Scribus skip including it? I'll never know.

$ identify -verbose output.pdf

Imagemagick can also convert a page of the PDF to a PSD. This command under will return a wrong resolution PSD (a thumbnail) and Krita can open it. CMYK values are well set on the PSD, but the color profile is lost: Krita fallback to Chemical Proof profile: showing something quite similar to Poppler.

$ convert output.pdf output.psd

Poppler:

Poppler via Poppler-utils can list all the image inside a PDF and report if they are CMYK. Same, no color profile, so it can be CMYK-pizza or CMYK-chocolate, this is not Poppler-utils business. I would really like to know if they are all profiled with CGATS_CPR1.icc or if the PDF as a global file is tagged with CGATS_CPR1.icc...

$ pdfimages -list output.pdf

A useful output!

It is also possible to extract the content of the PDF to a folder using the Poppler-utils this way:

$ mkdir extracted

$ pdfimages -all output.pdf extracted/img

The Tiff, PPM or JPG obtained this way are CMYK files but unfortunately have no profile so Krita will display them with the default generic free/libre CMYK color profile: ChemicalProof. You'll need to convert them manually to CGATS_CPR1.icc with Imagemagick (apply simply the profile; no convert/scaling):

$ cd extracted

$ convert img-000.tif -profile /path/to/your/CGATS_CPR1.icc img-profiled.tif

Then you can open in Krita and control the quality of the Scribus rendered files with the color picker to see how it manages the black, how out of gamut colors are scaled, etc.... That's my best solution so far!

Okular:

Okular, the Plasma desktop PDF reader can read the PDF thanks to Poppler under the hood. But same error here: the CMYK colors are not profiled and fallback to a default profile like Chemical Proof. It is still a blast to have the possibility to read a version of a CMYK PDF on a PDF reader, even if the colors are not faithful: the picture are still readable.

Krita:

Krita can open a page of the PDF; but will do a bit the same than a Poppler conversion: the picture has same generic CMYK rendering and the result is even not CMYK anymore but all converted in the default RGB used by Krita (sRGB-elle-V2-srgbtrc.icc)...

Conclusion and more links:

I know a lot of you will skip reading this long log and that's fine. I wrote this long results of my research for a future person who will get same trouble as I do and will search Internet to see if other tempted this quest. That's a bottle in a ocean, and a way to log my efforts on the way.

Here are my sources and best Internet links I could find on the topic. I triaged them from reading hundreds of pages.

COLOR MANAGEMENT:

• Krita documentation: Soft Proofing: https://docs.krita.org/en/user_manual/soft_proofing.html
• Krita documentation: all sub-chapters of "Colors" category, a must read: https://docs.krita.org/en/general_concepts/colors.html
• Elle Stone website: Nine Degrees Below, a goldmine for every articles: https://ninedegreesbelow.com/
• FLOSS Manual : Scribus. One of the easiest guide into Scribus: http://write.flossmanuals.net/scribus/introduction/
• Scribus Wiki: Color Management setup, especially good to read after reading Krita documentation on the same topic:https://wiki.scribus.net/canvas/Color_Management_setup
• Fedora Project wiki: How to set CMYK color on a design for printing. A practical guide, a bit outdated but still nice because practical: https://fedoraproject.org/wiki/How_to_set_CMYK_color_on_a_design_for_printing
• Preparing Your Book For Print with Scribus goldmine documentation from my printer: https://onebookshelfpublisherservice.zendesk.com/hc/en-us/articles/360022742353-Preparing-Your-Book-For-Print-with-Scribus

PDF:

• The PDF/X-1a file format, to know more about the compliance: https://www.prepressure.com/pdf/basics/pdfx-1a
• Unix.stackexchange.com : What are GNU/Linux tools for checking PDF documents before publishing? To know more about the tools available: https://unix.stackexchange.com/questions/316760/what-are-gnu-linux-tools-for-checking-pdf-documents-before-publishing
• Adobe Support Community - Transparency Compliance with PDF/x-1a ; to understand why no transparency in this format: https://community.adobe.com/t5/InDesign/Transparency-Compliance-with-PDF-x-1a/td-p/9277582
• Thatch Tran: Improving PDF export in Scribus, to understand a bit more the situation: https://wiki.scribus.net/canvas/Thatch_Tran:_Improving_PDF_export_in_Scribus

DOWNLOADS:

• INTERNATIONAL COLOR CONSORTIUM, CGATS21-2-CRPC1: http://www.color.org/registry/CGATS21_CRPC1.xalter
• INTERNATIONAL COLOR CONSORTIUM, CGATS21-2-CRPC6: http://www.color.org/registry/CGATS21_CRPC6.xalter
• The repository of CMYKTools by BlackFiveImaging: https://github.com/blackfiveimaging/cmyktool