LukasT.mediablog.sk

I renamed my very first brush engine, sumi-e, I wrote a long time ago. It was Google Summer Of Code 2008, the year I joined the Krita team. This brush engine started my joy over  the brush engines. I have improved it lately.

In hairy brush you don’t define the brush mask, but you define the bristles. Every bristle has a position and lenght. For setting up the bristles I used to use Guassian. I like this function for it’s naturalness, that’s why I used it. The value of the function defined the lenght of the bristle. But in turned out that it is not very effective. It produces masks you never had enough control over its dimension. You had to understand the sigma in Gaussian and not every artists care about math.

So I decided to reuse the brush dialog that is shared among the other paintops and use pixel masks to define the bristles. Every pixel define the bristle. The intensity and alpha of the pixel define the lenght of the bristle. The position is computed according the center of the brush. What it gives? Many new possibilities for shapes of the hairy brush. You could have only round shapes before, now you can have any shape. If the pixel is transparent, the bristle is not in the brush – that is the trick to have any shape. Other bonus is that you can have colourful shape also, that was not possible before.

Confeti brush can define position and the length of the bristle

You can control the hairs spacing in the brush shape by parameter scale. This parameter scales the position of the bristle from the center of the brush. Next parameter is random offset. The path of the bristle is randomly deformed according this parameter. You can setup the limit of the offset of the bristle from the initial position. So far the scale has been always controlled by the pressure. I made this optionally now by adding the size sensor and implement that support in hairy brush.  So you can turn off the change of the size according pressure or you can control the size by different sensor like time or distance or tablet’s tilt. And now you can paint also grass more easily as enkithan reported to me.

Lukáš Tvrdý – Developer tries to paint grass

There were some performance problems also with sumi-e, now hairy brush. I benchmarked and fixed the related bug and I introduced also some feature that helps with performance. Usually the more bristles, the more slowdown. I added possibility to control the amount of the bristles. You can specify the percentage of the density of the brush. Quite useful when you need big brush.

Related feature is also threshold according the pressure. Now when you paint, only the bristles which are able to touch the canvas, will be painted. Before I used opacity to control the used/unused bristles. Now it is more fun. You can turn off the size sensor and use threshold to control the size of the brush stroke. Or you can combine as these features complement.

Next feature is called soak ink in Ink options. Bristles can be colorified according the first contact with the canvas. So you can use photo as the source of the color. Or you can process some photo like I did to demonstrate the feature

Hairy brush used for post-processing image to give hairy look

I have some other ideas how to improve Hairy brush. Rotation has to be also controllable according the sensors.Ink depletion has to be reconsidered, more color effects has to be added. The bristles should be more anti-aliased. We will see.  Feel free to contact me if you have anything related to hairy brush features.

Here is the crop of the sketch that Boud, the Krita maintainer, did with hairy brush lately.I like especially the hairs and the eyes.

Boudewijn Rempt – Wounded Girl

Last week we finished the part of the action plan regarding the performance. I started to work on usability improvements. The first usability improvement is canvas mirroring.

Canvas mirroring is useful in your painting work flow and Boudewijn explained that it is useful for artists to check errors in their paintings. In the classical painting you need some hardware for that — you need to find a real mirror and see if your painting looks creepy or not. In Krita you don’t have to buy a new mirror, you can use the canvas mirroring feature.

It is also useful when you paint something that supposed to be “almost” symmetrical. You paint ears of your hero or you paint head outlines. You can paint the first half, mirror the canvas and paint the second half and check back quickly. Other possibility is to use symmetrical feature I wrote for Softbrush lately.

In Krita you could already do this by transforming the image and it’s data but that can be slow if you are painting on big canvases. Krita aims to be professional tool, we want to support big canvases. So we need this also as projection feature. We will not transform the image data, we transform the projection of the image data.

I started to work on this and I had a feeling that this is not going to be about a lot of code but this task is going to be about orientate yourself in some complex part of the Krita called projection and about orientate in coordinate systems we have. You also have to find out and understand how the scrolling and zooming works.

I contacted mypaint maintainer maxy with some short mail asking how mirroring is done within mypaint. The answer was what I basically expected and it proved me that I might go in right direction.So first I started with mirroring the coordinates of the tools. We have few coordinate systems in canvas. First coordinate you get is the widget coordinate. That one is transformed according the scrolling offset. Then document origin, the top left coordinate of the image in canvas, also has to be taken into account. Next the zooming transformation follows.After this you are on document level coordinate system. On this level I mirror the coordinate. That means that the document is changed in mirrored manner. This works so far very nicely. For vertical mirroring you need to do transformation like scale(-1,1) and then translate(width,0) where width is the width of the document in my case. We mirror just vertically.

The next step is to mirror the projection of the image. I started to work in QPainter canvas as that one is the more complicated one. First I mirrored extra buffer QImage we have in canvas to be able to use RasterOp operations with Arthur. QImage::mirror()  was just quick&dirty hack to see what will be broken. So tools outlines were broken as they don’t know about the mirroring and some features worked in mirrored way. I removed the dirty hack and I used QTransform for transforming the QPainter. I left the QPainter transformed this time also for tools so the outlines were fixed.

The OpenGL canvas was similar to QPainter canvas, I needed to do just the transformation and again setup the QPainter which is used even in OpenGL based canvas for painting the tool outlines.

Then I started to work on fixing some related, quite complicated, bugs. I spent a lot of time with them without solution. The issue does not seem to be complicated. You just need to mirror some coordinates. But it is hard to know which ones, if you did not write that projection code. The issues are that when you scroll and zoom-in  in the part of the image, it became transparent. I suppose that projection thinks that part of the image is not visible so it does not update it or something like that. But I don’t know really if that is true and I can’t fix it either. Similar problem occurs in OpenGL canvas. I was not able to fix it either. I was trying but without lack. Try&error method did not bring any good results.

Then I spent time integrate the feature into Krita. You can mirror the canvas by View->Mirror Image (CTRL+I).To sum it up, the feature is in koffice-ko feature branch as trunk is frozen. It works with bugs we know about. We just need to fix them. The week I spent on it was nice start for this feature. Let’s hope the feature will be bug-free for Krita 2.3.

In week 8 I started working on a part of the Krita I had never touched before. It is actually a library inside KOffice called Pigment. Pigment is responsible for the color management. It contains some colorspaces we actually don’t use in Krita because they are too simple for Krita’s needs. They are intended for use by other applications in KOffice. So far the applicationss do not have color management, but they can if the developers want it. Pigment contains many useful classes which operates on the color in some colorspace in many ways like doing the math, compute the histogram. It also contains the implementation of the composite operations. And here was my interest as we wanted to try to vectorize the composite operations by using SSE instructions and the vectorization feature in GCC4.x.

So first I started to write benchmarks for the various composite operations. And then I started to work with GCC feature. Vectorization of the composite operation is already implemented in GEGL by Øyvind Kolås. Also GIMP 2.6.x is using MMX, SSE, SSE2 so I had inspiration and I was trying to map it to our implementation of composite operations. GEGL is using nice code, but GIMP is using assembler directly. I don’t have much experience with assembler. I wish I had write assembler lessons previous year at the university. It is not hard
to code something, but it is hard to do it correctly. I read somewhere in Inkscape mailing list that they had some assembler code to speedup some work but it ended up to be slower then code optimized by compiler.

I had quite a hard time and I did not manage to implement the vectorization even with help I get regularly from Cyrille and boud and other Krita hackers and
GEGL hacker. We stopped it on Wednesday because we discovered that the issue is more complicated then we thought and it would require much more than two days to finish. Maybe another week or even weeks. And the result could be not faster. One of the problems we discovered was that the RGBA 8-bit colorspace uses the unsigned char datatype (quint8 in Qt) for the memory storage but when you do a composite operation, you have to retype it to the bigger data type like a int32. Why? If you have a pixel in quint8 with value 255 and other pixel with value 200 and you add or multiply them, you overflow the data type. And the result is bad. If you retype, you have solved that issue. I studied the GIMP code and how it is
implemented there. It is solved using MMX instuctions for this case. The MMX technology supports both saturating and wraparound modes. You can read about that more in details here. Another issue was that GIMP does not compute every composite operation with vectorization but only some composite operations are implemented this way.

So we decided to start to work on the other item in the action plan which is mirroring of the canvas and possibly rotation. On Thursday and Friday I was back in the canvas code. So far I have working code of the mirroring of the events from input devices. Now the hardest part will be to implement  mirroring in the projection. Projection is the code responsible for correct displaying of the zoomed image and it computes the image you see in the canvas when you scroll or move or some tool paint it’s outline or some part of the
image is changed by some tool. The task will continue also for OpenGL canvas as we have two canvases in Krita.

You see, the vectorization week does not bring any speedup, but I don’t want you to be sad so I decided to write a blog post about other Krita work I do in my spare time. Read it here.

This week of the sponsored Krita developer was a little shorter. On Monday I traveled back from Holland to Slovakia. I was at the Krita Hackfest 2010. We worked there quite hard so we decided to take a break for me for 2 days. So I started to work on Thursday and finished on Friday.

As I wrote in my previous blog post, I started to work on the smudge brush, because according the bug report it was horribly slow. Although I wrote many brush engines, this one is not mine. But I have many experiences with brush engines so I hoped to use them.

Thanks to the consultations with Cyrille Berger,author of this brush engine, I managed to speed up the smudge. I removed some unnecessary memory device and then I had to write famous custom bitBlt function which takes selections saved in different memory class into account. The performance bottleneck was transformation of our paint device into selection.

We introduced fixed paint device longer time ago which should speed up the composition of the brush masks because it is lightweight memory device, but in the smudge it introduced some workarounds which slowed down the performance. I removed that workarounds but it was complicated, that’s why nobody did it before. I ported some of my paintops to use the fixed paint device because according our benchmarks it is really faster.

Here is the table with the performance times of smudge in our KisStrokeBenchmark, where the brush engine draws the big stroke:

1. 7,519 msec per iteration (total: 7519, iterations: 1)
2. 5,275 msec per iteration (total: 5275, iterations: 1)
3. 1,202 msec per iteration (total: 1202, iterations: 1)
4. 653 msec per iteration (total: 653, iterations: 1)

You can see, that the speedup factor is almost twelve. The table consists of the iterations when optimizing. From initial time to the final optimization.

On Friday I was done so I started to work on a vectorization of the compositing operations. Sounds cool, nah? First I wrote benchmarks for our composite operations and now I will continue to write example code which will use the vectorization in gcc4.x. Here is what I’m going to study and use. The point is to speedup the composition of course.

I’m writing this blogpost from Boud’s kitchen. After 10 days of the Krita hackfest I feel quite dizzy. What I was doing?

According the action plan, this two weeks could be spent on some unfinished business. So we decided to work futher on the Adobe Photoshop brush integration. I fixed the type of the brush internally. Photoshop brushes for Krita consist of the alpha-map and the presets of the Photoshop’s brush engine. I ported the GIMP plug-in which loads the bitmaps in the abr brush. But the presets were not parsed.

Frob wrote the python script which I mentioned previously in my blogpost. Frob and Alexandre Prokoudine are working on reverse engineering of the Photoshop’s resources like brushes and gradients. And their work includes also parsing of the brush engines presets. So I ported the script from python to Qt/C++ for Krita. The last step is to integrate the script. That was not done yet.

Then I arrived to Deventer, the Netherlands. Dmitry Kazakov was waiting for me at the airport and we went together to Deventer from Amsterdam. The weekend was gone so fast. We discussed a lot of issues together with Peter Sikking, interaction designer. That was very very interesting for me. I also met other Krita hackers I haven’t met yet like Adam or Vera. It is nice to be together, because you can be very productive.Together with Dmitry we fixed a bug in few minutes. It would take much more time if we would do it like a Report Bug -> Assign -> Fix.

Week 6 was the hackfest for me, Sven, Cyrille and Boud here in Deventer. We decided to give a try to 1/4 computation of the brush mask to have faster painting. This time it was implemented correctly. Before I tried to copy the 1/4 of the mask to the other parts of the mask. This time I decided to compute the brush mask with bi-linear interpolation. I spent 2 days on it fixing some artefacts that started to occur. Final measurements showed that the 1/4 computation is more expensive then the function itself as this time the arctan2 was not involved. Also other benchmark with arctan2 involved showed that the code is equally fast as the version without the interpolation. Maybe some bigger brushes will be faster but I need to investigate. The code is in trunk, we just need to find it’s usage.

Another optimization was for flood fill tool. Write benchmark, valgrind, identify the bottlenecks, try to fix them, do it few times and I was done. Cyrille helped me a lot, he is very good developer and he had always some good advices how to get the speedup. Finally I managed to make flood fill run 3.5 times faster.
Here is table and you can see how time dropped:

Start of the optimization
1. 5,921 msec
2. 5,752 msec
3. 5,574 msec
4. 3,581 msec
5. 1,711 msec
End of the optimization

The flood fill is still quite slow in the Krita. I tested on 4096×4096 pixel image. The performance problem now is that there is difference function used in the flood fill algorithm and the function convert the pixels to LAB. I suppose that in GIMP it is probably done on RGB 8-bit colorspace. But in Krita there are many colorspaces and the best way to do the difference is on L channel in LAB as Cyrille told me.

Then we were in Amsterdam in the Blender studios. We met Ton Roosendaal and the Durian project team. It was exciting for us. I managed to talk with Angela, the Durian artist from Canada. I had nice chat with her, lovely person. I also managed to talk with Ben, the artist who tried Krita and he also produced some art with it. I was in Amsterdam for the first time. I managed to see arrogant Amsterdam bikers, nice architecture and the cannal. We had long but nice walk around the town.

At the hackfest we discovered more performance issues. So I spent some time on smudge brush. I started with simple fixes and I started to work on some complicated issues. I’m writing bitBlt function for our fixed device. Fixed device is used for brush masks and it is faster then using paint device with tiles as the brush masks are usually small and they does not require tiles and undo. We need more function for bitBlt when the selection is stored in fixed device. So far I  the smudge is 1.42x faster.

Krita guys also started to work on various stuff that Peter Sikking proposed, e.g. we have a new widget and scratch box so I had to fix paintops also. Also I rewrote the deform brush. I’m trying to unify the brush engines in Krita and I wanted to add some new features to old brush engines, so rewrite was needed. The code was almost one and a half year old. I did a lot of work on brush engines also. Usually many fixes etc, but that is for other blogpost maybe later.

I would like to thank Boudewijn and Irina for taking care of us. The home-made food was “best thing ever” as Vera would say. I had nice time with the Krita team at Rempt’s house. Thank you for inviting me!

Busy week, I’m really tired and I’m looking forward for some rest and my home.