The difficulties of doing an open comicbook project for print
I spent the first week of June here to finish a side project: an open comicbook project for Pepper&Carrot to be sold and printed. I finished the beta version of book 1 last night in Scribus and the book is now ready to be proofread, exported and printed. The Gitlab (Framagit) repository is here. Why did it take me a full week? ... Here is my report if you want technical details:
Screenshot: two instances of Scribus 1.5.4dev running side-by-side on my Linux Mint desktop:
Inner pages on right, cover on left.
The Pepper&Carrot comic mainly uses Krita (for artworks, Krita kra files exported to flat JPG at 95% quality) and Inkscape (SVGs for speech bubbles and text, exported as PNG). For the book project, I used Scribus. Scribus is the only free/libre open-source software to provide a multipages CMYK workflow with advanced text and layout tools. Scribus is a very solid project, and I planned to overlay my artworks and my speech bubbles in it. But Scribus also has a lot of traps: many format (*.pdf, *.eps, *.tiff, *.jpg, *.png) do not really react as expected in the first place and you cannot know it until you try it. One must do the mistakes, do trials-and-errors, export after export, to learn how to prepare in the best way the media before being able to export a good and solid CMYK hi-resolution PDF output ready for the printer. If your project has a small amount of images, it's easy to switch formats and encodings for each of them; but if your project needs a renderfarm to fusion hundreds of Krita files with thousands of SVG files to output the correct files, it can quickly become a tedious exercice.
Dual screen screenshot while scripting and re-rendering translations.
So, it was a long and difficult task to find the best recipe and adapt all the other parts of Pepper&Carrot to be dynamically linked to the book project. The book project is designed to be fully dynamic with the other parts of Pepper&Carrot: if a user submits a correction on one SVG for the webcomic in the git of an episode, the change will be propagated to the book's files. The book project can also switch translation, to ease the work of many micro and local self-publishing effort. For the book, I used Scribus 1.5.4dev daily build: it was the only way to work around a Dpi problem in the released version making the UI too big or too little. The software is stable and has good ergonomy. It's really easy to use if you have an experience with this type of software (eg. QuarkXpress/Indesign, I was a prepress layout graphic artist in a past life) and has many features, especially for text, paragraph, advanced layout and color management. One specific issue I met with Scribus was related to the PNG hi-resolution I generate with the renderfarm. This PNG are generated from the SVG sources by Inkscape and then handled by Imagemagick and couldn't be read by Inkscape without getting dark edge artifacts around speech bubbles. All pages were affected, no exception. A solution was found in the way PNGs can be encoded, but this change required me to re-render all 5,800 SVG translations of Pepper&Carrot to get the new PNGs...
_Issue I met with PNG transparency: speech bubble is a PNG inside an Image Frame laid over the artwork. Full, bug report and solution here.
Since the creation of Pepper&Carrot, I never ran a full re-render of all the sources; I just keep a good render besides the source. I only regenerate this render in case of a modification of the source. Langage by langage, with review of the translator after the render, this system was working. But I started to get more and more export problems with newer versions of Inkscape.
A common mix of two rendering SVG issues: buggy border radius (bold effect) applying to only a part of the text. Episode 6 is translated in 34 langages, and sound effects like this appear 4 times in this episode. 136 file to manually fix. If you can fix a file in 2min, it's 4h30 of non-stop work. It was done for all 22 episodes.
Let's be clear: It's not the fault of Inkscape's developers: I have a very wide variety of SVGs like no other project around. I'm totally out-of-the-scope of many common uses of Inkscape because yes, Pepper&Carrot is now super-complex and has the most challenging base of SVGs you could find on the globe (except maybe for Openclipart and Wikimedia Commons). I'm using special font effects, special text boxes and rare combinations of filters to workaround specific speech bubble shapes and typography on the sounds effect. I also have in the repository source SVGs saved from Inkscape 0.48 to 0.92, from MacOS, Windows or Linux versions, from many different computers with different local settings, with left-to-right langages, right-to-left, using experimental open fonts. It's not a surprise then if this large SVG database is sensitive to breaks: I'm simply pushing the limits of the Inkscape SVG file-format too far!
Original VS Re-rendered translation, with a combo of many new bugs I met:
Right-to-left support punctuation issue + bold radiusx2 + offset in position + missing sound effect.
I first tried to find a way to re-render all within the same Inkscape version; the latest stable is 0.92.1 and converts every previous version while keeping the look of the original. But I had too many failures even with the help of an Inkscape developer directly on the Pepper&Carrot IRC channel. So I decided to build a script to compare before and after versions; cycling over all the SVGs and with a smart automatic detection process to guess when the render was similar and acceptable. I received precious help from the community to build this type of complex script. The script tried to auto-fix everything, taking around 7 second by SVGs analyzed and about 15 hours to cycle over all the database with all my processors screaming their 100% usage on my desktop computer. But many renderings were too different, writing in my console "Irregular rendering issue found" in red. Out of 5,800 SVGs, I had more than 3,000 bad renderings (script sources available here).
The script used to identify problematic rendering in action. Console on left, Image viewer in the middle showing a detected issue, and on the right a little GUI to interact. "Is this OK? .... huh ... No!" 3,000 files had to be reviewed/fixed this way.
Another computer script was able to quickly display the "before and after" in a quick gif animations at the center of my screen and propose a Yes/No interface for the 3,000 irregular SVG renders detected. The idea was simple: the gif pops up on the screen, blinks the before/after versions with a pop-up: Is this OK? Yes or No. If yes, the file commits itself, if no the software opens Inkscape for manual fixes. After days and nights of a brainwashing repetitive work across hundreds of file comparisons, I could only fix a bit less than 300 irregular SVGs and started to get a massive headache (and also loosing any passion for Pepper&Carrot!). 300 files solved for days of work is a tiny dust if I compare to the 3,000 problematic SVGs detected. My first week of June was already finishing, still not a page of the book was done and I was clearly wasting a lot of time to workaround Scribus being picky with PNG encoded background color in transparency. I was also screwing all the database, my health, and in general... my passion for the project. It was emergency time and I decided to stop: for a single man, fixing this alone would be equal to a full month brainwashing death. I reverted all the converted files even validated, because I was really affecting the quality. I then decided to take another cheap workaround: I installed on my machine multiple versions of Inkscape and made the renderfarm detect the version of the SVG input and redirect the render to the right version of Inkscape. This makes my renderfarm script now super hard to install because you need this setup of special Inkscape versions and having multiple versions of the same software on GNU/Linux is hard. I simply now render 0.48 and 0.91 SVGs in Inkscape 0.91 and 0.92 SVGs in 0.92. A simple way but not a elegant or a long term solution. Anyway, I wasn't loosing the original project: doing my Scribus book 1, and I could continue with this patch. I was able to finally re-render all the speechbubbles in all langages to workaround my original PNG problem in Scribus and I could get the good PNGs white-backgrounded transparency. Something only Imagemagick can do with special flags.
The inner pages of the book project in Scribus. Totally in sync with the source of the webcomic
Finally, what a story and difficult time! I had the feeling to meet cascading issue a bit like in the famous scene of "Malcolm in the middle" where the father attempt to "Replace a lightbulb" (Imgur gifv link). I hate when this happen and I feel this type of big things will only attack more and more the project. But let's see the bright side: the book project was an opportunity to review the state of all the SVG database, find many bugs in software and SVGs and it will probably make the future brighter for other artists taking the open-source way. Thanks again to the developers of Inkscape and Scribus, always giving a lot of their time, giving me detailed information and being around when I have to solve this type of big problems. Now the situation is stable again, maintained but at the price of running multiple Inkscape versions on the renderfarm. This situation will not be possible to keep forever and I already feel the subtle long term pressure of this Damocles's sword. Every SVG translations (especially from episode 01 to episode 12) will need at one point a manual review, file by file, and I'll need help from contributors. I'll publish a little how-to about it in a near future. Now let's turn the page of this big chapter and let's start the next step with the book project. On my side, I'll start the production of tutorials and Episode 23, I can't wait to draw Pepper&Carrot instead of debugging Pepper&Carrot ! Thank you if you could read it all, If you have questions, please comment: I'll be around.
