ThinkingParallel...

Additional operating system/hardware discussion (Windows, Linux, *BSD and others)
Forum rules
Any posts concerning pirated software or offering to buy/sell/trade commercial software are subject to removal.
User avatar
Arie van Schutterhoef
Posts: 149
Joined: Sat May 07, 2005 9:58 am
Location: Over the Rainbow
Contact:

ThinkingParallel...

Unread postby Arie van Schutterhoef » Mon Oct 22, 2007 2:12 pm

Seeing this link:
http://www.hpcwire.com/hpc/1847953.html
subsequently reading & downloading the mentioned PDF,
I was pondering, together with my 'schreck-programming-compatriots' (info on request), how MPI, PVM and OpenMP compare to one another.
I saw that SGI is offering it for their Altix 450. Also Apple nowadays uses Intel cpu's we've been staring at these options for audio applications in realtime during concerts. For some considerable time we work with SGI/Apple set-ups and these are getting older, though still working quite nicely. But speed improvements on the new generation cpu's are making it more interesting to make a transition to these architectures.
So I wonder how people's experiences have been with porting and working with these different protocols.

AvS

User avatar
R-ten-K
Posts: 1889
Joined: Mon Nov 15, 2004 10:36 pm
Location: Nor Cal

Re: ThinkingParallel...

Unread postby R-ten-K » Mon Oct 22, 2007 4:29 pm

I have used a mixture of OpenMP, MPI, and SIMD programming modesl. PVM for all intents and purposes is dead and has way too much overhead.

OpenMP has been stagnated for a while, and at the end you only end up getting good performance by hand crafting all the pragmas in your code. Which means that you have to do all the work by hand anyways, so most compilers rely on the hints by the programmer to figure out which loops (for the most part, OpenMP is fairly loop oriented) to parallelize. MPI is a PITA, as a simple scatter gather code will require a shitload of overhead just to send the threads on their merry way. I have also worked a lot with pthreads, as long as you know what you are doing that ends up being the best approach (in my experience) to get most of the juice from a few extra symmetric cores.

Parallel programming is still a bit of a dark art, and it is close to impossible to hire people who know what the heck they are doing. In fact, I am experiencing that hiring programmers with a lot of serial experience is almost a liability nowadays, few of them seem to be able to wrap their head around efficient data partitioning and parallel programming in general. The debugging clusterf*cks that these guys generate are legendary :-(, anyhow... in my team we also deal a big deal with asynchronous data parallel programming models, so that adds to the fun.

There is for the most part a lack of good parallel programming tools, tracing and debugging parallel stuff is a fantastically under "attacked" problem. If anything, I am glad everything is going parallel though as there is a clear opportunity and there will be a higher barrier of entry that will keep some of the fantastically bad programmers populating our field... for a while at least.
"Was it a dream where you see yourself standing in sort of sun-god robes on a
pyramid with thousand naked women screaming and throwing little pickles at you?"

User avatar
porter
Posts: 2917
Joined: Wed Nov 01, 2006 10:37 pm
Location: NZ

Re: ThinkingParallel...

Unread postby porter » Mon Oct 22, 2007 5:01 pm

R-ten-K wrote:I have also worked a lot with pthreads


My belief is that the two most important thread "APIs" are pthread_cleanup_[push|pop]. :)
Land of the Long White Cloud and no Software Patents.

User avatar
squeen
Moderator
Moderator
Posts: 2933
Joined: Fri May 09, 2003 6:10 am
Location: Maryland, USA

Re: ThinkingParallel...

Unread postby squeen » Tue Oct 23, 2007 2:42 am

I have worked with pthreads (and SGI's REACT using pthreads). It's a nice API, but in writing fast parallel code I've still had issues with barrier latency. Word to the wise--pthread_barrier_wait() is a SLOW, non-spinning barrier. Better to write your own.

User avatar
R-ten-K
Posts: 1889
Joined: Mon Nov 15, 2004 10:36 pm
Location: Nor Cal

Re: ThinkingParallel...

Unread postby R-ten-K » Tue Oct 23, 2007 11:40 am

Barriers are costly, as they usually imply strong consistency. Some of the openMP barriers were not that bad, However I dunno if you were experiencing the same problem, but in our case once the number of threads/processors gets large enough, you have to implement some sort of smart tree of "gathers" so your barrier only has a few elements. Having a barrier on lots of threads will bring any communication network to a halt :-(
"Was it a dream where you see yourself standing in sort of sun-god robes on a
pyramid with thousand naked women screaming and throwing little pickles at you?"


Return to “Miscellaneous Operating Systems/Hardware”

Who is online

Users browsing this forum: No registered users and 2 guests