Flex 4: A multi threading solution

Alex van Beek

Syndication

The sample Flex application

The sample application in this blog post is a Flex application which turns an image into greyscale. A better option for this would be to use a custom filter built with PixelBender, but this example demonstrates the problem quite well and with PixelBender you lose the ability to report progress. Take a look at the screenshot below:

It has a nice photograph of a parrot, two buttons and a progressbar showing progress when the image is being converted to greyscale. Next up, the code behind:

<?xml version="1.0" encoding="utf-8"?>

<s:Application xmlns:fx="http://ns.adobe.com/mxml/2009" 

               xmlns:s="library://ns.adobe.com/flex/spark" 

               xmlns:mx="library://ns.adobe.com/flex/halo" creationComplete="saveOriginalImageData(event)"

              >

    

    

    <fx:Script>

        <![CDATA[

            

            private var _originalImageData : BitmapData;

  

            private function greyWithPauseHandler(event:MouseEvent):void

            {

                var clone : BitmapData = _originalImageData.clone();

                _parrot.source = new Bitmap(clone);

                setProgress(0,0);

                convertToGreyScaleWithPauses(clone);

            }

        

        

                    

            private function convertToGreyScale(toGreyScale: BitmapData) : void {

                var totalPixels: int = toGreyScale.width * toGreyScale.height;

                var amountForProgressUpdate : int = totalPixels / 16;

                setProgress(0, totalPixels);

                

                for(var i : int = 0; i < totalPixels; i++) {

                    var y: int = i / toGreyScale.width;

                    var x :int = i - y * toGreyScale.width;

                    var pixel:uint = toGreyScale.getPixel(x,y);

                    var red:uint = (pixel >> 16) & 255;

                    var green:uint = (pixel >> 8) & 255;

                    var blue:uint = pixel & 255;

                    var grey:uint = (red * 0.3) + (green * 0.59) + (blue * 0.11);

                    toGreyScale.setPixel(x,y,(grey<<16) | (grey<<8) | grey);

                    

                    if(i % amountForProgressUpdate == 0 && i != 0) {

                        setProgress(i + 1, totalPixels);    

                    }

                }

                setProgress(totalPixels, totalPixels);

            }

            private function convertToGreyScaleWithPauses(toGreyScale: BitmapData) : void {

                var totalPixels: int = toGreyScale.width * toGreyScale.height;

                setProgress(0, totalPixels);

                

                    UIUtilities.pausingFor(0,totalPixels, function(i:int) : void {

                        var y: int = i / toGreyScale.width;

                        var x :int = i - y * toGreyScale.width;

                        var pixel:uint = toGreyScale.getPixel(x,y);

                        var red:uint = (pixel >> 16) & 255;

                        var green:uint = (pixel >> 8) & 255;

                        var blue:uint = pixel & 255;

                        var grey:uint = (red * 0.3) + (green * 0.59) + (blue * 0.11);

                        toGreyScale.setPixel(x,y,(grey<<16) | (grey<<8) | grey);

                    },setProgress ,this);

                    

                    _parrot.source = new Bitmap(toGreyScale);

            }

            

            private function setProgress(processed : int, amountThatNeedsToBeProcessed : int) : void {

                _progress.setProgress(processed, amountThatNeedsToBeProcessed);                

            }

  

  

            private function greyHandler(event:MouseEvent):void

            {

                var clone : BitmapData = _originalImageData.clone();

                _parrot.source = new Bitmap(clone);

                setProgress(0,0);

                // Need an artificial delay (Timer) to let the Image update itself and show the original image before 

                // the greying begins...

                convertToGreyScale(clone);

            }

  

            private function saveOriginalImageData(event:Event):void

            {

                _originalImageData = Bitmap(_parrot.content).bitmapData;

            }

  

        ]]>

    </fx:Script>

    

    <mx:ProgressBar mode="manual" minimum="0" maximum="1000000000000" id="_progress" horizontalCenter="0" bottom="6" label="Greying: %3%%"/>

    <mx:Image id="_parrot" source="@Embed('./assets/parrot_heada.jpg')" left="10" right="10" top="10" bottom="77" scaleContent="true"/>

    <s:Button label="Grey it with pauses" horizontalCenter="-107" bottom="48" click="greyWithPauseHandler(event)"

              />

    <s:Button label="Grey it" bottom="48" horizontalCenter="65" click="greyHandler(event)"/>

    

</s:Application>

At this point I encourage you to download the sample project here and run it. If you don't have Flash Builder installed, I've exported a release build here which you can run by double clicking the .html file. First click on the “Grey it” button, you’ll see the ui freeze for a while, when the application has finished converting the image to greyscale, the ui unfreezes and the progressbar jumps to 100%. Next, click on the “Grey it with pauses” button. You’ll see the image being converted to greyscale gradually and you’ll see the progressbar showing progress, you’ll also notice that the “Grey it with pauses” takes more time to complete than the “Grey it” button. Let’s explain what happens:

Take a look at line 67, this method get’s called when the “Grey it” button is clicked. It clones the original BitmapData (which gets saved on the creationComplete event), resets the progress method using the “setProgress()” method on line 62 and calls the "convertToGreyScale()" method which does all the work. You can see a problem indicated in the comment. The problem is that when supplying the “_parrot” Image component with the original colored image data, the user interface doesn’t update until the “convertToGreyScale()” method has completed. The only way to work around this is to build in an artificial delay using a Timer for example and experiment to find the correct delay in which the Image get’s updated. That’s why I previously told you to first click on the “Grey it” button and then on the “Grey it with pauses” button. The other way around you don’t see the image going from colored to greyscale.
In the “convertToGreyScale()” method on line 23, the BitmapData data is processed and converted to grayscale using a standard algorithm, maybe it’s not the most optimized algorithm, but it suffices. Most people would use nested for loops, I’ve converted it to a single loop in order to explain the next part more easily. This method updates the progressbar once in a while, in total 16 times, this is determined on line 25. If I updated the progressbar every iteration, I would get a script timeout, even if this timeout was set to 60 seconds, which is the maximum. As you’ve seen, the user only sees the progressbar at 100% when this method is finished, because only then can Flash update the ui. In fact, all the calls to "setProgress()" could be removed from this function, because they don’t have any visual effect.
Next take a look at the “convertToGreyScaleUsingPauses()” method on line 44. This method get’s called when the user clicks the “Grey it with pauses” button. The heart of this method is on line 48. The “UIUtilities.pausingFor()” method is called, in almost the same way as I configured the for loop in the “convertToGreyScale()” method. The loop body has become an anonymous function that must accept an int (the current iteration) and is called multiple times depending on the first two arguments. Next to the first anonymous function I supply another function which is responsible for setting the progress. The “pausingFor” function makes sure that pauses occur, in which the ui can update itself, hence the name “pausingFor”. Take a look at the code below:

package

{

    import mx.core.UIComponent;

  

    public final class UIUtilities

    {

        

        /**

         * Executes a for loop that pauses once in a while to let the UI update itself. The parameters of this method

         * are derived from a normal 

         * for(var i :int =0; i <10;i++) {

         * }

         * loop.

         * @param fromInclusive The 0 in the loop above.

         * @param toExclusive The 10 in the loop above.

         * @param loopBodyFunction The loop body, everything between {} in the loop above. This function must accept an int,

         * which represents the current iteration.

         * @param updateProgressFunction The method that needs to be called to update the UI, for example a progressbar.

         * this method must accept two ints, The first is the number of iterations processed, the other is the total number of

         * of iterations that need to be processed.

         * @param componentInDisplayList Any component that is connected to the displaylist. This method makes use

         * of the callLater() method which is available on any UIComponent. The root Application is an easy choice.

         * @param numberOfPauses The number of times this method pauses to let the UI update itself.

         * The correct amount is hardware dependent, 8 pauses doesn't mean you'll see 8 UI updates. Experiment

         * to find the number that suits you best. A higher number means less performance, but more ui updates and

         * visual feedback.

         **/

        public static function pausingFor(fromInclusive:int, toExclusive :int,loopBodyFunction : Function,updateProgressFunction : Function,componentInDisplayList:UIComponent,

                                   numberOfPauses : int = 8) : void {

            executeLoop(fromInclusive,toExclusive, toExclusive / numberOfPauses, loopBodyFunction,updateProgressFunction, componentInDisplayList)

        }

        

    

        private static function executeLoop(fromInclusive:int, toExclusive :int,numberOfIterationsBeforePause : int, loopBodyFunction : Function,

                                             updateProgressFunction : Function,componentInDisplayList : UIComponent) : void {

            var i : int = fromInclusive;

            for(i; i < toExclusive;i++) {

                //determine the rest of the number of iterations processed and the numberOfIterationsBeforePause

                //This is needed to determine whether a pause should occur. 

                var rest : Number = i % numberOfIterationsBeforePause;

                

                //If the rest is 0 and i not is 0, a pause must occur to let the ui update itself

                if(rest == 0 && i != 0) {

                    

                    //use callLater to pause and let the UI update.....

                    componentInDisplayList.callLater(

                        //Supply anonymous function to the callLater method, which can be called after the pause...

                        function(index:int) : void {

                            //after pausing, resume work...

                            loopBodyFunction(index);

                            //We need to continue with the executeLoop() method. The current index has already

                            //been processed so continue this method with the next index

                            executeLoop(index + 1,toExclusive,numberOfIterationsBeforePause,loopBodyFunction,updateProgressFunction,componentInDisplayList);

                        },);

                    //When using callLater to let the UI update, my own code must be finished. So break out of the loop 

                    break;

                } else {

                    //No time for a pause

                    loopBodyFunction(i);

                    //Just before a pause occurs, report progress so that a user can set progress values

                    if(rest == numberOfIterationsBeforePause - 1) {

                        updateProgressFunction(i + 1, toExclusive);

                    }

                }

            }

            //Final progress update

            updateProgressFunction(i + 1, toExclusive);

        } 

        

    }

}

This is the source code of the UIUtilities class. The only public static function is the “pausingFor” function. Take your time reading the ASDoc, it explains what this function does and what the different parameters are for. This method calls another private static function “executeLoop()” which has almost the same parameters as the “pausingFor” method, except for the “numberOfIterationsBeforePause” parameter. This parameter is the result of the “toExlusive” parameter divided by the “numberOfPauses” parameter of the “pausingFor” method.

Let's take a look the "executeLoop()" method on line 34. I’ve written inline comments that explain most of what’s going on. The most important part of this function is on line 46. This is the “callLater()” method that any UIComponent has and this is one of the most overlooked methods in Flex. It accepts a function as the first parameter and an array with arguments to supply to that function as the second parameter. When you call this method, Flex schedules the supplied function to be called in the next frame. This way, the ui can update itself for the rest of the duration of the current frame. This means that there must be enough time left in the current frame and there mustn't be any of your own code that needs to be executed after the call to “callLater()” , or else you still won’t see any ui updates. In the example above I supply an anonymous function that needs to be called in the next frame. In this anonymous function I do two things:

Let the "loopBodyFunction" parameter execute.
Afterwards call the "executeLoop()" function again. This way I’m sure that after the ui has had time to update itself, the "executeLoop()" function resumes with the correct iteration after being suspended by the "callLater()" method.

Another interesting part is on line 61. In the iteration just before the iteration in which the "callLater()" function is called, I give the user the chance to set any progress values that the ui needs for updating itself, by calling the "updateProgressFunction" parameter and supplying the number of iterations processed and the number of iterations that need to be processed. This way, I minimize the calls to the ui to when they matter the most: just before a call to "callLater()".

Conclusion

With my "UIUtilities.pausingFor()" function, I’ve shown how you can let the ui still be responsive and let it update itself, in a way that’s quite similar to using a normal "for" loop in a background thread. My "UIUtilities" class can also be easily expanded with a "pausingForEach()" function, building on the "pausingFor" function. While the api of the "UIUtilities" class is quite simple, be aware that using this api has a performance cost, letting the task finish without giving the ui the time to update itself, is always faster, but not necessarily better for the user of the application. Also, this class is nowhere near a replacement for having the ability to do true multi threading, which would be better for the performance, certainly on multi core processors. But until we have that possibility in Flash, we have to work around the lack of multi threading using solutions similar to the solution provided above. While my post was titled with "Flex 4", the solution provided above should also work on Flex 3. You can find the working solution with the source code, here.

Posted 01-03-2010 8:43 by Alex van Beek

Filed under: Flex, Flex 4, Responsive, Multi threading, Progress

Comments

uberVU - social comments wrote Social comments and analytics for this post

on 01-03-2010 9:54

This post was mentioned on Twitter by Beekje: New blog post providing a Flex multi threading solution:

http://tinyurl.com/yk2h6kb

James Ward wrote re: Flex 4: A multi threading solution

on 01-03-2010 13:10

This is a nice example. Another alternative would be to write a Pixel Bender filter to do this on a separate thread.

Alex van Beek wrote re: Flex 4: A multi threading solution

on 01-03-2010 13:29

Hi James,

Thanks for your comment. I mentioned Pixel Bender as another option in my post, but if you use pixel bender, I think you lose the ability to report progress.

Grayscaling was just an example, in practise you could use above example for processing incoming business objects, or parsing XML as well.

David R wrote re: Flex 4: A multi threading solution

on 01-03-2010 17:53

One improvement you can make is to base number of iterations off of time, instead of a simple count. That way people who have faster PCs will be able to do more loop iterations before pausing. It would be annoying to have to wait for some process to finish on your super fast computer because the programmer artificially limited the speed.

You can let the developer then specify how many pauses per second (or effectively, what FPS to update at). Around 5-10 FPS will give maximum CPU utilization, while still keeping UI updated often enough.

Erno de Weerd wrote re: Flex 4: A multi threading solution

on 01-03-2010 18:48

I do think that 5-10 FPS can be too low in a lot of scenarios. Yes, when you 'block' the interface with a progress bar 5 to 10 might be enough but when you are allowing the user to perform other actions by partially blocking the UI you might need more depending on the animations you need.

In the end you need multiple threads to really gain performance

Elena wrote re: Flex 4: A multi threading solution

on 01-03-2010 18:56

It's very interesting article. Thank you for information.

Brett W wrote re: Flex 4: A multi threading solution

on 01-03-2010 19:49

The execution loop approach is pretty tried and true (Flash developers have been dealing with the monolithic thread for a long time), though I'd agree with the above poster that it's better to go with a Timer than a loopcount.

But who is "developing applications in Silverlight A LOT"?? Are a lot of companies using it for internal tools? Because it's presence on the web is pretty limited to sites that Microsoft has paid.

Alex van Beek wrote re: Flex 4: A multi threading solution

on 03-03-2010 10:14

You do benefit with a faster computer, simply because the iterations are processed faster. For example if a pause occures every 30 iterations, on faster computer obviously the pause will occur sooner, since it processed the 30 iterations faster.

About the timer, a great example with a timer can be found here:

blogs.adobe.com/.../threads_in_actionscript_3.html

It's true that using a timer allows faster computers to perform more iterations, but a timer also introduces some overhead.

I tested the solution from the URL above with the greyscaling example and on my computer the solution I presented above performed better, partly because I can tweak how many ui updates I would like, but with about the same amount of ui updates, because of the timer overhead. Of course, this doesn't mean that this is the case on every system.

Alex Bevilacqua wrote re: Flex 4: A multi threading solution

on 03-03-2010 13:04

I wrote something similar for and AIR project (code.google.com/.../Chunker.as) a while back.

This limitation in Flex is going to really cause issues in the future if we want to roll out more robust applications that process a lot of information locally :(

Alex van Beek wrote re: Flex 4: A multi threading solution

on 03-03-2010 15:18

Yeah, I also think it will become a limitation. That's why I hope Adobe introduces threading in Flash in the future. It doesn't have to be a full multi threading model, but something that allows us to do processing in the background. You can vote for Flash threading here:

bugs.adobe.com/.../ASL-23

You need an Adobe account to vote...

Add a Comment

Name: (required)

Website: (optional)

Comments (required)

Remember Me?

Enter code (required)

Syndication

Recent Posts

Tags

Archives

The sample Flex application

Conclusion

Comments

Add a Comment