Asynchronous Programming

[bchip]

Just a general question here on async programming as I dont have much experience on this.

I wrote an windows forms desktop app that imports data. In a child window I run some calculations on the data.
So far its been sync programmed so if I click "do calcs" then the whole system would freeze for hours
doing the calculations. They are pretty hectic so this is expected.

I see that I can run the same calculations through a background worker async.
So the system doesnt freeze anymore but it seems that the calcs are a bit slower.
My question is - what dont I know that I dont know about this?
For example is async running the calcs more slowly, is it the same? Is this bad practise? etc

Any advise on this is much appreciated

 

[cguy]

Just a general question here on async programming as I dont have much experience on this.

I wrote an windows forms desktop app that imports data. In a child window I run some calculations on the data.
So far its been sync programmed so if I click "do calcs" then the whole system would freeze for hours
doing the calculations. They are pretty hectic so this is expected.

I see that I can run the same calculations through a background worker async.
So the system doesnt freeze anymore but it seems that the calcs are a bit slower.
My question is - what dont I know that I dont know about this?
For example is async running the calcs more slowly, is it the same? Is this bad practise? etc

Any advise on this is much appreciated

The design makes sense. It shouldn’t be significantly slower unless perhaps your worker thread is touching the GUI, causing updates that would otherwise have been deferred or accessing a concurrent data structure of some sort.

 

[Kosmik]

Have you considered running calcs on a database level vs application? Or deferred calculations with periods of results etc. Just seems a bit silly to have it pull down to a client and crunch on that.

 

[bchip]

Have you considered running calcs on a database level vs application? Or deferred calculations with periods of results etc. Just seems a bit silly to have it pull down to a client and crunch on that.

Unfortunately its not possible, the calcs are a bit too interwoven with the application
to make this possible.

 

[bchip]

The design makes sense. It shouldn’t be significantly slower unless perhaps your worker thread is touching the GUI, causing updates that would otherwise have been deferred or accessing a concurrent data structure of some sort.

Thanks.
Whats weird is that if I run the calculation sync then it takes less than 2 seconds (in my test).
But if I run them async ...Ive been waiting over 3 minutes. So not sure if Im doing something wrong.

It does a calc then adds a row to a datagridviewer, repeatedly with a different variable each time.
Other than that I try and update a progressbar and a label for estimated time left.
At most it updates about 3 labels, which seems redundant as its unresponsive.

 

[cguy]

Thanks.
Whats weird is that if I run the calculation sync then it takes less than 2 seconds (in my test).
But if I run them async ...Ive been waiting over 3 minutes. So not sure if Im doing something wrong.

It does a calc then adds a row to a datagridviewer, repeatedly with a different variable each time.
Other than that I try and update a progressbar and a label for estimated time left.
At most it updates about 3 labels, which seems redundant as its unresponsive.

I would try comment out the progress bar update and viewer widget and see if that helps. Each update could be triggering a redraw. Then put them back in if this solves it, but try to update less frequently.

 

[retromodcoza]

I also have a windows forms app that runs intensive calcs in the background and updates the GUI with progress.

Async is not slower by default. Something is slowing it up. As above , it could be the GUI , but I would break out the profiler and see where the slowdowns are occuring.

 

[bchip]

I would try comment out the progress bar update and viewer widget and see if that helps. Each update could be triggering a redraw. Then put them back in if this solves it, but try to update less frequently.

Thanks, it seems its getting stuck at some points.
Like changing the tabpage on the tabcontrol made it go into an infinite loop.
Taking it step-by-step now, making sure those display changes are done at the start while still in sync.

Thanks for the help.

 

[Batista]

Thanks.
Whats weird is that if I run the calculation sync then it takes less than 2 seconds (in my test).
But if I run them async ...Ive been waiting over 3 minutes. So not sure if Im doing something wrong.

It does a calc then adds a row to a datagridviewer, repeatedly with a different variable each time.
Other than that I try and update a progressbar and a label for estimated time left.
At most it updates about 3 labels, which seems redundant as its unresponsive.

You should update the datasource for the datgridview once you are done,not on each iteration.
proc dowork()
{
alldone=false;
dgv.datasource = null
//do work code set alldone after
If (alldone)
{
dgv.datasource = dtFinal
}
}

 

[cguy]

Thanks, it seems its getting stuck at some points.
Like changing the tabpage on the tabcontrol made it go into an infinite loop.
Taking it step-by-step now, making sure those display changes are done at the start while still in sync.

Thanks for the help.

What type of number crunching are you doing, BTW?

 

[[)roi(]]

Thanks.
Whats weird is that if I run the calculation sync then it takes less than 2 seconds (in my test).
But if I run them async ...Ive been waiting over 3 minutes. So not sure if Im doing something wrong.

It does a calc then adds a row to a datagridviewer, repeatedly with a different variable each time.
Other than that I try and update a progressbar and a label for estimated time left.
At most it updates about 3 labels, which seems redundant as its unresponsive.

Your described design most certainly performs IO on the UI thread as cguy mentioned; is this something that's really necessary on such a frequent interval?

Also have you assessed your processing ito of its potential for parallel computes? Simply running the same interactive process on background thread is always going to be slower because of the increased inter thread IO. To take advantage of this your process must really be redesigned for concurrency.

 

[bchip]

You should update the datasource for the datgridview once you are done,not on each iteration.
proc dowork()
{
alldone=false;
dgv.datasource = null
//do work code set alldone after
If (alldone)
{
dgv.datasource = dtFinal
}
}

Thanks, I see that it updates at the end but just wanted to know
how do I know I'm back in the Sync part again?
Or asked otherwise, the alldone part is that in the Worker_ProgressChange portion
where e.ProgressPercentage = 100

Just want to know where I can start changing the UI parts again.
It seems anytime I touched any control the system just stalled, so had to rewrite but luckily not a lot.

 

[bchip]

What type of number crunching are you doing, BTW?

Im sure its stuff your quite familiar with
Its building my own backtester/optimizer on stock market data.
I found that most of these were lacking quite a bit, so just developed my own.

What shocks me most with this is how when I compare my results to certain
brokerage firms and their provided apps I find so many mistakes with theirs...

 

[bchip]

Your described design most certainly performs IO on the UI thread as cguy mentioned; is this something that's really necessary on such a frequent interval?

Also have you assessed your processing ito of its potential for parallel computes? Simply running the same interactive process on background thread is always going to be slower because of the increased inter thread IO. To take advantage of this your process must really be redesigned for concurrency.

Have redesigned this to be better thanks.
If I knew how to redesign for concurrency though I would, but think thats way over my head

 

[[)roi(]]

Have redesigned this to be better thanks.
If I knew how to redesign for concurrency though I would, but think thats way over my head

There are a number of different approaches to this, some that involve a lot of complex choreography with thread management; but there are alternatives that simplify everything by designing interdepency between processes, which not only makes it simpler to reason about, but also greatly enables the potential for massive parallel and distributed processing.

Linq FYI also caters for lazy parallel computing of transforms so if your process uses Linq, its relatively simple to enable concurrency at a basic level; you also have Task Parallel Library (TPL) in C# which allows you to run for loops in a parallel context; data & process design is key to making this work though. For those working in Java, it too has added this functionality in Java streams; naturally the entire topic is a lengthy / complex discussion so it would be best to flesh this out in a thread dedicated to this.

@cguy also does a lot of this stuff, and would most certainly have a different approach to mine (which is a functional approach).

 

[[)roi(]]

Im sure its stuff your quite familiar with
Its building my own backtester/optimizer on stock market data.

This btw sounds very much like something that can be computed in parallel. So I suggest you experiment with Parallel Linq and/or TPL for/foreach loops.

 

[John_Phoenix]

Some great advice here, thank you all!

 

[Spacerat]

Tpl is what you want to look at. You can create a Task for each computation or group of computations and fire them off. Update ui as each completes. Depends though on the nature of the computations and whether they are dependent on each others’ results.

 

[[)roi(]]

Tpl is what you want to look at. You can create a Task for each computation or group of computations and fire them off. Update ui as each completes. Depends though on the nature of the computations and whether they are dependent on each others’ results.

It really depends on the data and process; parallel Linq is preferred if you need to take a single dataset in; compute based on its elements in parallel and end up with a single output dataset; whereas TPL for loops is typically more suited to independent processing of objects, e.g. file processing.

Parallel Linq utilizes a FP technique (laziness & applicative functor) and TPL is a more of basic imperative approach, where you split the input parameter sets, and then spin a thread for each.

 

[Thor]

God i missed reading your posts.