Mappredictor 5.20

Message boards : Malaria Control : Mappredictor 5.20

Author Message
Profile maire
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: Nov 7 05
Posts: 438
Credit: 118,258
RAC: 0

We\'re getting ready to run another batch of mappredictor workunits. The first workunits will be sent out starting tomorrow Friday 14th. The new workunits are very short (around 1 min on an avarage PC), mainly because the prediction focus on a much smaller geographical range (see below). Otherwise the same as usual applies for the mappredictor: The application makes use of BOINC\'s wrapper approach for legacy applications, therefrore there is no checkpointing and no feedback from the application about progress. As usual you can opt out by changing your project preferences.
Here\'s what the scientists say about the new version:

The purpose of this application is to produce a smooth map of malaria risk in Mali taking into account two important factors:
1) Malaria risk data are correlated in space since common environmental risk factors may influence the malaria risk similarly in locations within the same geographical area. In this application we modeled the spatial correlation as a function of the distance between locations and dependent on locations (non-stationarity).
2) The relation between malaria risk and environmental factors may not be linear; therefore we use non-parametric regression models to quantify this relation.

This application is similar to the previous one (West Africa) and it is used mainly to validate the previous results (compare the two maps of Mali).

Nick

____________
Nicolas Maire
Swiss Tropical and Public Health Institute
http://www.swisstph.ch

Profile The Gas Giant
Avatar
Send message
Joined: Mar 7 06
Posts: 1213
Credit: 3,503,340
RAC: 1,667

Has anyone seen any of these yet?

Marky-UK
Send message
Joined: Mar 7 06
Posts: 4
Credit: 78,672
RAC: 0

Has anyone seen any of these yet?

They seem to be available now. Run time on an Athlon 1500 is ~25 seconds (and there\'s a daily limit of 500, which that PC will reach in under 4 hours).

Profile maire
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: Nov 7 05
Posts: 438
Credit: 118,258
RAC: 0

We\'ve started sending them out this morning, phasing in slowly as usual. The daily quota has been increased, this should not be limiting any longer.
There are a total of 95590 workunits. Visit the server status page and have a look at the batch progress monitor! The image is updated every 5 minutes.
Nick

____________
Nicolas Maire
Swiss Tropical and Public Health Institute
http://www.swisstph.ch

Ageless
Avatar
Send message
Joined: Jun 29 06
Posts: 261
Credit: 146,907
RAC: 217

I had one. Quick little buggers. :)
____________
Jord.

BOINC FAQ Service

Profile The Gas Giant
Avatar
Send message
Joined: Mar 7 06
Posts: 1213
Credit: 3,503,340
RAC: 1,667

YAY. I got some! And they are very quick little buggers! ;)

Redbull
Send message
Joined: Mar 8 06
Posts: 28
Credit: 307,058
RAC: 0

Im getting quite a few.

I have taken to aborting workunits which are normal ones for now to give you guys some tests.

Also I aborted some of the map predictor work units and it seemed to abort fine.
____________

Profile The Gas Giant
Avatar
Send message
Joined: Mar 7 06
Posts: 1213
Credit: 3,503,340
RAC: 1,667

YAY. I got some! And they are very quick little buggers! ;)

So quick in fact that it clogs up the client_state.xml file as it needs to download so many to keep the wu cache at the desired level. Let\'s see
- 30 sec per wu
- 0.5 day cache = 21600 seconds
- 2 cores
- required work = 43200 seconds
= 1440 wu\'s!

I\'ve reduced my cache setting to 0.25 days to reduce the load on boinc.exe having to write 4MB of data (ie client_state.xml) every 15 seconds or so. In the last 14hrs boinc.exe has consumed 2hrs of cpu time on my 3.2GHz P4. Hmmmm, surely if the wu\'s only take 30sec then they could be run internally within reasonable time limits.

Live long and BOINC!
____________
Paul
(S@H1 8888)

Profile The Gas Giant
Avatar
Send message
Joined: Mar 7 06
Posts: 1213
Credit: 3,503,340
RAC: 1,667

Gotta hate it when the power goes out in the middle of an edit....

0.5 day cache = 43200 secs
2 cores
required work = 86400 secs
= 2880 wu\'s

Which makes for a very large client_state.xml

Ageless
Avatar
Send message
Joined: Jun 29 06
Posts: 261
Credit: 146,907
RAC: 217

- 30 sec per wu

Throttle your CPU. ;-)

I\'ve got mine throttled at 65% and then they run in 1m 13s (average).
But run they do. Good thing there\'s no graphics included.
____________
Jord.

BOINC FAQ Service

Tom Philippart
Send message
Joined: Jun 25 06
Posts: 29
Credit: 220,888
RAC: 0

These small WUs are getting a big problem for me, Boinc.exe takes up 100% CPU Time, because Boinc downloads and uploads so many WUs to get enough work for the 2 cores of my CPU. It takes around 11 seconds to crunch a single unit!!
____________

Profile maire
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: Nov 7 05
Posts: 438
Credit: 118,258
RAC: 0

These small WUs are getting a big problem for me, Boinc.exe takes up 100% CPU Time, because Boinc downloads and uploads so many WUs to get enough work for the 2 cores of my CPU. It takes around 11 seconds to crunch a single unit!!


We still have about 18\'000 waiting to be sent for the first time, and expect the last one to go out later today.
If the short map workunits are a problem, please opt out using the project preferences.
Thanks
Nick
____________
Nicolas Maire
Swiss Tropical and Public Health Institute
http://www.swisstph.ch

Ageless
Avatar
Send message
Joined: Jun 29 06
Posts: 261
Credit: 146,907
RAC: 217

Good thing I only work half days, as this morning, during the download of a lot of the small ones, my computer hung. Nothing spectacular about that, but worrisome in some sense anyway. No results that hung, though, so it was probably \'about time it hung again\'.

I\'ve reduced my connect to + additional days to 0.02 plus 0.02 and will continue testing.
____________
Jord.

BOINC FAQ Service

Ageless
Avatar
Send message
Joined: Jun 29 06
Posts: 261
Credit: 146,907
RAC: 217

Hmmm, let\'s not run this mappredictor anymore. It rebooted my system while I was away and since BOINC runs as a service, it continued to run on the login screen, then hung my system again.

I\'ll try to coerce/run the cache dry and have already changed my preferences to not run predictor again. Sorry, perhaps that it works better on a more stable system.

Edit: I managed to post the above, then my system hung again. If you don\'t mind, I\'ll be aborting all map predictors.
____________
Jord.

BOINC FAQ Service

Profile maire
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: Nov 7 05
Posts: 438
Credit: 118,258
RAC: 0

We received the last result for this batch last night. I\'ve passed all the data on to the scientists of this project to analyze.
Many thanks for your help!
Nick
____________
Nicolas Maire
Swiss Tropical and Public Health Institute
http://www.swisstph.ch

Profile The Gas Giant
Avatar
Send message
Joined: Mar 7 06
Posts: 1213
Credit: 3,503,340
RAC: 1,667

We received the last result for this batch last night. I\'ve passed all the data on to the scientists of this project to analyze.
Many thanks for your help!
Nick

Wow, that took a long time! I haven\'t seen one of those wu\'s for many days. To speed up the response in the future is it possible to target specific hosts known to be reliable to run any wu\'s that are stragglers?
____________
Paul
(S@H1 8888)

Profile maire
Volunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: Nov 7 05
Posts: 438
Credit: 118,258
RAC: 0


Wow, that took a long time! I haven\'t seen one of those wu\'s for many days. To speed up the response in the future is it possible to target specific hosts known to be reliable to run any wu\'s that are stragglers?

Yes, turnaround time is an issue in volunteer computing. We are now using a scheduler that resends error or no_reply results to reliable hosts only. This seems to make a big difference, but there is still the rare case where more than one resend is necessary.
The last result to come back, for example, was for a workunit that was sent out at the very beginning (Sept 17th), and then resent on the 21th and 24th.
Nick
____________
Nicolas Maire
Swiss Tropical and Public Health Institute
http://www.swisstph.ch

Post to thread

Message boards : Malaria Control : Mappredictor 5.20


Return to malariacontrol.net main page


Copyright © 2013 africa@home