Figure 2.15: Rendering of6 dierent images, all with5 dierent runs with dierent stroke scales (labeled by roman numbers), plotting compression ratios (CR) against PSNR (see equation 2.1).
Points show the positions where the rendering processes converged to. Sample images in Figure 2.3.
painting only with the smallest stroke (i.e. 10), knowing that it will produce the largest output. If a somewhat slower convergence can be allowed, painting with three layers can produce the same quality but half the size. Also, starting with a large stroke, then later rening with a smaller one, can signicantly reduce the size of the stroke-stream - for example see the lowest two curves on the lower graph of Figure 2.16: the dierence is that the lowest curve is rendered only with the smallest stroke, the one above uses the largest stroke as a rst step.
2.5 Scalability, coding, compression, storage
We have also investigated ecient lossless coding and compression possibilities of stroke based rendering techniques. In stroke based rendering, the emphasis has been on the generated eects, and little care has been about how the painted images should be stored and handled. Images generated with SBR are usually highly susceptible to encoder noise and artifacts, thus in order to retain the painterly quality, one has to store them in a lossless raster image format. But there are usable alternatives to this approach, and this is what we deal with in this section, and later also in Chapters 5.4 and 5.5.
In this section the main idea is, that if we store the painted images not as raster images but as a stroke series that it is built of, we will obtain a lossless representation, which is at the same time more compact than losslessly encoding these images with known rater image coders. Later, in
2.5. Scalability, coding, compression, storage 26
Figure 2.16: Detailed runs for the buttery image from Figure 2.15, plotting render times and compression ratios (CR) against PSNR (see equation 2.1). With a selected goal e.g. given quality in the shortest time one can select the appropriate rendering mode and vice versa.
the above mentioned chapters, we will also introduce vector graphical representation and encoding of painted images, which another step towards lossless, easy to handle and portable representations of these painterly rendered images.
Stroke-series representation makes a painted image scale-independent to an extent, in the sense that dierently scaled versions of the painted image can be displayed without re-painting: only the strokes are scaled and the positions adjusted. This is also the case when images are converted and stored as vector graphics (in SVG). The image versions in Figure 2.17 were generated this way, i.e. the smallest one was a painterly generated image, the larger versions are just larger version displays.
The scaling of images produced by stroke-based NPR methods has always been a question
2.5. Scalability, coding, compression, storage 27
Figure 2.17: Sample showing scalability. The images are all displayed from the same painted stroke-stream.
with no denitive answer (e.g. see Chapter 5.3 in [100]). Raster-based interpolation methods are denitely not suitable for creating dierently scaled versions of stroke-generated images, since the easily introduced unpleasant artifacts (blockines, aliasing, etc.). In the case of 3D NPR, when surface points and local textures are available, such scaling is an easier task. But in 2D SBR, scaling is not a trivial task. As we just mentioned above, when we store the painted images as a series of stroke parameters which they are built of, it is easy to scale and shift these stroke parameters before displaying them, thus producing dierently scaled views of the same rendered images. That, of course, does not mean an increase in detail in the painted image. It just means that a larger version can be easily displayed, which will give the same painterly look and feel, when - and this is important - viewed from a greater distance, just as in the case of the painterly rendered images in their original scale.
What we will discuss rst in the following is a way to store the stroke series of the generated paintings in a way that makes displaying dierent scale views of the rendered images a transparent
2.5. Scalability, coding, compression, storage 28
and easy task. And that is storage in scalable vector graphics representation.
Scalable Vector Graphics (SVG) is a standard language for describing 2D vector graphics.
The such described graphics les are platform independent and can be manipulated and viewed by a variety of applications and tools, from browser plugins to free and non-free applications. One of the main advantages of storing images as SVG is that these provide scale-independent handling of the included graphics.
Our painterly rendered images are represented and stored as a series of strokes. As described earlier, our strokes usually are geometrical shapes which can be easily described as SVG elements.
For example a rectangular shape with color at a given position and orientation has the form
<rect
width="60" height="15"
transform="translate(21,128) rotate(22.5,30,7)"
fill="#897F0C" />
Many dierent shapes are supported including ellipses, polygons, lines or polygonal lines.
For a sample image which has been painterly rendered into SVG and displayed, see Figure 2.18.
See more on the SVG representation in Chapters 5.4 and 5.5.
Figure 2.18: Sample for a painterly rendered image displayed from SVG format.
Regardless of painting with 10layers and always covering all the canvas, or painting with less layers with edge following orientations and layer lling after a certain coverage percent, the output always is a sequence of stroke parameters. We code these outputs in a lossless way with
2.5. Scalability, coding, compression, storage 29
Human coding to achieve even smaller sizes. In the compression scheme the entire parameter set of all the placed strokes are treated as dierent symbols. That is because at least the position data of the strokes is always dierent, there are no placed strokes with identical parameter sets.
Therefore if one would use the entire parameter set of a stroke as a symbol for encoding, there would be as many dierent symbols as the number of placed strokes and ecient coding could not be used. Figure 2.19 shows compression ratios of stroke-series encoding related to lossless image coders for sample images, 2.19(a) also contains data for SVG storage. What can be seen is that coding the painted image by compressing the resulting stroke stream is more ecient than coding the rendered images with usual raster image coders in lossless mode. All sizes are relative to the size of the painted bitmap.
Figure 2.20 presents data showing that if we compress the painted output (the stroke-series representation of the painted frames) in our lossless way, than we can achieve at around3-4times better compression ratios compared to what we can achieve when compressing the painted output bitmaps with lossless image coders (e.q. Huyuv and JPEG2000 in lossless mode).
(a) (b)
Figure 2.19: Comparing lossless compression rates of two example rendered image bitmaps by dierent codecs and by stroke-stream compressions. Subgure a) also contains stored in SVG format (last 2 columns).
The painting process has a few more properties that make an interesting option. For example the transformation has the ability to decrease errors produced by standard DCT-based techniques.
In Figure 2.21 there is an example of an oil painting (JPEG and painted versions), showing that the painting technique smooths the surfaces, preserving contours and main edges, producing a visually
2.5. Scalability, coding, compression, storage 30
Figure 2.20: Lossless coding ratios of output stroke-series relative to lossless coded bitmaps of painted images.
pleasant output. One should also notice that this "denoising" (more a smoothing) eect which comes with the use of the Painting technique is not a pre-dened goal or strategy we wanted to achieve, it's more a side-eect of the stochastic painting (SPT) method (ergo we don't just paint to lter the noise, the painting itself lters).
Figure 2.21: JPEG coded oil painting segment (left) and the JPEG recoded with Paintbrush rendering (right) - showing the ability to mimic stroke styles and noise ltering eects.