Nekochan Net

Official Chat Channel: #nekochan // irc.nekochan.net
It is currently Wed Sep 17, 2014 7:31 am

All times are UTC - 8 hours


Forum rules


Any posts concerning pirated software or offering to buy/sell/trade commercial software are subject to removal.



Post new topic Reply to topic  [ 110 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6, 7, 8  Next
Author Message
Unread postPosted: Tue Jun 17, 2008 5:41 am 
Offline

Joined: Sun Oct 05, 2003 7:42 am
Posts: 786
Location: Frankfurt (Rhein-Main Area) / Germany
Quote:
You've probably heard the saying if it looks like computer graphics, then it's bad computer graphics. Well those shots definitely don't look like computer graphics. In particular, I like the top of the car in the black & white airport image---how the glare from the window shield and roof merge into the single reflected color. (I'm curious, do you ever find yourself looking at real objects now with the same critical eye you examine a rendering and think they sometimes look fake? Maybe its just me...:))


No we are doing the same. Especially since we are dealing with all kind of people like designer, engineers, marketing people, photographers. Everyday I learn something new about cars and this makes it a real challenge. What is missing here is real tonemapping or a real 16bit display like the one from brightside, the eye does very funny things when it gets to lighting. A

Quote:
Now I understand you are a commercial outfit, so can't freely give away your secrets, but I'd like to turn your own question back at you. "What's your idea on dealing with the noise?" I don't see any noise in the images you posted, so I'm curious as to why you asked. How many rays/pixel? What kind of render times? Just wondering I'm wondering on how far away I am from professional grade applications.


You are right, there are of course several things I cannot disclose because of commercial reasons but there are also some things I cannot for technical reasons (sampling is really not my field of expertise). The render time is 3 minutes for the image in 2560x1600pixel including antialiasing on an 8core machine.

Quote:
Also, are you running Windows on the 32p machine?


Yes, Windows 2003 Server, everything out of the box.

Quote:
Do you get 6 fsp with the same image quality you showed? If so, that absolutely incredible. How much is hardware (GL) and to what degree is it ray traced.


see comments about speed above. rendering is done 95% CPU and 5% GPU. But this may be subject of change.

Quote:
With all the buzz, it seems as if real-time ray tracing is the new programmable shaders. Your comment about the Telsa is true, however I've heard its been difficult to get the K-d Tree etc. accelerators to run on the GPU because of the SIMD and data base access. Programming CUDA is supposed to have a small cache size (32K) or something like that, and that currently GPU based ray tracers are no faster than multicore CPU. However, NVIDIA did just hire Peter Shirley...

Your (or anyone's) thoughts?


GPU programming is of course much harder than CPU programming for two reasons. First one is that your hardware faces some real limitations like bad sequential memory access as well as shader clusters that are bigger than useable. Second reason is that the development tools or the lack of them makes it really hard to do.

What nvidia has bought from Peter Shirley I don't know, judging from the photos on the web page I see some basic raytracing functionality like reflection and refraction and that's all. But we will see in future what kind of product will appear on the market. Their goal is of course to have hybrid rendering, something they also want to achieve with the acquisition of mental images. I am very curious how all this will turn out. In the meantime we will continue the development of our software in a small team and not concentrate on all this marketing hype :)

Matthias

_________________
Life is what happens while we are making other plans


Last edited by Brombear on Tue Jun 17, 2008 9:53 am, edited 1 time in total.

Top
 Profile  
 
Unread postPosted: Tue Jun 17, 2008 9:44 am 
Offline
Moderator
Moderator
User avatar

Joined: Fri May 09, 2003 5:10 am
Posts: 2931
Location: Maryland, USA
Brombear wrote:

You are right, there are of course several things I cannot disclose because of commercial reasons but there are also some things I cannot for technical reasons (sampling is really not my field of expertise). The render time is 3 minutes for the image in 2560x1600pixel including antialiasing.


That's truly impressive to me. I've still got my work cut out climbing that learning curve. Thanks.:)


Top
 Profile  
 
Unread postPosted: Wed Jun 18, 2008 10:24 am 
Offline
Moderator
Moderator
User avatar

Joined: Fri May 09, 2003 5:10 am
Posts: 2931
Location: Maryland, USA
Sorry, one more in the series. I tought it might be called "Eeek!" but my 6 year-old named it...
Attachment:
File comment: "Yum. I Love Spiders."
eyeflop.gif
eyeflop.gif [ 417.03 KiB | Viewed 200 times ]

go figure... :)


Top
 Profile  
 
Unread postPosted: Wed Jun 18, 2008 12:16 pm 
Offline

Joined: Sun Oct 05, 2003 7:42 am
Posts: 786
Location: Frankfurt (Rhein-Main Area) / Germany
:lol:

Matthias

_________________
Life is what happens while we are making other plans


Top
 Profile  
 
Unread postPosted: Wed Jun 18, 2008 1:14 pm 
Offline
User avatar

Joined: Wed May 28, 2008 5:00 pm
Posts: 222
Location: San Diego, CA, USA
Love it!

Being Bender it should be "Bite my shiny metal ass Spider!" :)

_________________
:540: :Octane: :Octane2: :O2: NeXTStation x 2, A4, A7000, RISC PC


Top
 Profile  
 
Unread postPosted: Fri Jul 11, 2008 5:51 pm 
Offline
Moderator
Moderator
User avatar

Joined: Fri May 09, 2003 5:10 am
Posts: 2931
Location: Maryland, USA
Here's one I liked, still a work in progress (materials!).

Attachment:
bay_test.jpg
bay_test.jpg [ 114.25 KiB | Viewed 223 times ]


I'm kinda excited because I may be able to run our little ray tracer on a 2500 core cluster next week.


Top
 Profile  
 
Unread postPosted: Sat Jul 12, 2008 12:27 am 
Offline

Joined: Mon Jan 28, 2008 11:50 am
Posts: 305
Location: Bristol. UK
squeen wrote:
Here's one I liked, still a work in progress (materials!).

I'm kinda excited because I may be able to run our little ray tracer on a 2500 core cluster next week.


Stunning Model!!! Lot's o' detail...!

Looking forward to "the monster crunch" test.

Jacques

_________________
No SGI box currently...Snif!


Top
 Profile  
 
Unread postPosted: Sat Jul 12, 2008 1:06 am 
Offline

Joined: Sun Oct 05, 2003 7:42 am
Posts: 786
Location: Frankfurt (Rhein-Main Area) / Germany
squeen wrote:
I'm kinda excited because I may be able to run our little ray tracer on a 2500 core cluster next week.


Hope you have enough time to work on the efficiency. We are currently implementing computation clustering as well and there are all kind of strange things to be aware of (bandwidth issues, latency, compression, multicast, upload to gpu, etc. etc.)

Don't forget to put some efficiency table here, I am really looking forward to your results.

Matthias

_________________
Life is what happens while we are making other plans


Top
 Profile  
 
Unread postPosted: Sun Jul 27, 2008 12:10 am 
Offline

Joined: Sun Oct 05, 2003 7:42 am
Posts: 786
Location: Frankfurt (Rhein-Main Area) / Germany
@Squeen

Are you still alive or been sucked away by the fan arrays of your cluster :lol:

Matthias

_________________
Life is what happens while we are making other plans


Top
 Profile  
 
Unread postPosted: Sun Jul 27, 2008 12:48 pm 
Offline
Moderator
Moderator
User avatar

Joined: Fri May 09, 2003 5:10 am
Posts: 2931
Location: Maryland, USA
Flight code delivery in 2 weeks---down deep in the Belly of the Whale.


Top
 Profile  
 
Unread postPosted: Fri Aug 22, 2008 2:06 am 
Offline
Moderator
Moderator
User avatar

Joined: Fri May 09, 2003 5:10 am
Posts: 2931
Location: Maryland, USA
This image completed with 20K rays/pixel on 512 cores in 5 minutes. Lighting (that you see) is mainly Earth albedo. The non-visible side of the arrays are pointed nearly at the sun with some beta offset. There are also 8 shuttle bay lights (not seen) that contribute very little.
Attachment:
hst1.jpg
hst1.jpg [ 93.07 KiB | Viewed 196 times ]


Top
 Profile  
 
Unread postPosted: Fri Aug 22, 2008 2:18 am 
Offline

Joined: Sun Oct 05, 2003 7:42 am
Posts: 786
Location: Frankfurt (Rhein-Main Area) / Germany
The image looks real great.

What do you think is your biggest bottleneck ? What kind of network topology are you using ?

We have found out that standard gigabit ethernet on the master does not make much sense anymore when you are passing a certain number of slaves (thus reaching a certain speed). You either have to bundle them or go to 10gbit or use infiband/myrinet based networks reducing the latency even more.

We will be testing a cluster with approx ~90 cores this week and I am really looking forward to it.

Matthias

_________________
Life is what happens while we are making other plans


Top
 Profile  
 
Unread postPosted: Fri Aug 22, 2008 2:41 am 
Offline
Moderator
Moderator
User avatar

Joined: Fri May 09, 2003 5:10 am
Posts: 2931
Location: Maryland, USA
The cluster here uses IP over Infiniband @ ~ 20Gb/sec and implements a "fast" shared file system.
We had to break the scanlines down further (row subdivisions) in order to get enough parallelism granularity. Also, while previously we had sent out all the model files via socket (distributed render farm), the shared file system makes this unneccessary.

Right now the bottleneck is Kd-Tree build (depth=30). I'd like to get some degree of parallelism on that (cutting the time in a 3rd or better would help a lot---it can take up to 1.5 minutes). Pipelining the tree build would effectively eliminate any wait time, but that means more code restructuring to keep two trees in memory instead of just one.

Photon mapping or bi-directional path tracing (Metropolis Light Transport) would also mean a whole lot fewer rays to get the same image quality. That's my next most important task when I can get some time.

I'd also like to get in one of the labs here that measures BRDFs and implement look-up tables as an alternative to the A-S AnsioPhong.

Displacement maps (as opposed to just bump maps) is also on my wish list, as is real (physical) atmospheric shattering.

No time...whew! :(


Top
 Profile  
 
Unread postPosted: Fri Aug 22, 2008 5:17 am 
Offline

Joined: Sun Oct 05, 2003 7:42 am
Posts: 786
Location: Frankfurt (Rhein-Main Area) / Germany
Very interesting. Can you give me the raw render time (without loading file and building the kd-tree) ?

Typically it should be better to build blocks instead of scanlines, because neighbouring rays touch nearly equal segments of memory. Of course this is only valid for primary rays ... remember primaries cache while secondaries trash :-)

Did you profile IP over Infiniband ? I remember some statistics that do not look that good. Of course its more than ok to get a small speed increase by doing nothing. We will have to look into this sometime in future and I presume no picknick.

Photon mapping will also be on our list next. We have no BRDFs and I am not a big fan of it. In german we say ... wer misst misst Mist ... who measures ... measures shit :)

Seems you still have a long way to go ... as we do too ...

I am really looking forward to your next results :)

Matthias

_________________
Life is what happens while we are making other plans


Top
 Profile  
 
Unread postPosted: Fri Sep 05, 2008 11:15 am 
Offline
Moderator
Moderator
User avatar

Joined: Fri May 09, 2003 5:10 am
Posts: 2931
Location: Maryland, USA
They just upgraded the cluster to over 4500 cores and since the existing workload hasn't expanded to fill the need I'm using 1536 cores with our little path-tracer-that-could and rendering 1000x1000 pixel images with 100,000 rays/pixel, (5 bounce max depth) in about 5 minutes/frame (70 seconds of which is single-threaded kd-Tree build time).

I am certain I've got mindless communication bottlenecks all over the place, but whoo-hoo!!

This is just nuts. :P


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 110 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6, 7, 8  Next

All times are UTC - 8 hours


Who is online

Users browsing this forum: No registered users and 0 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group