Through the Looking Glass

3D/2D CGI and the tools used in their creation (Maya, Photoshop, Blender, GIMP, etc.).
Forum rules
Any posts concerning pirated software or offering to buy/sell/trade commercial software are subject to removal.
Brombear
Posts: 786
Joined: Sun Oct 05, 2003 8:42 am
Location: Frankfurt (Rhein-Main Area) / Germany
Contact:

Re: Through the Looking Glass

Unread postby Brombear » Tue Jun 17, 2008 6:41 am

You've probably heard the saying if it looks like computer graphics, then it's bad computer graphics. Well those shots definitely don't look like computer graphics. In particular, I like the top of the car in the black & white airport image---how the glare from the window shield and roof merge into the single reflected color. (I'm curious, do you ever find yourself looking at real objects now with the same critical eye you examine a rendering and think they sometimes look fake? Maybe its just me...:))


No we are doing the same. Especially since we are dealing with all kind of people like designer, engineers, marketing people, photographers. Everyday I learn something new about cars and this makes it a real challenge. What is missing here is real tonemapping or a real 16bit display like the one from brightside, the eye does very funny things when it gets to lighting. A

Now I understand you are a commercial outfit, so can't freely give away your secrets, but I'd like to turn your own question back at you. "What's your idea on dealing with the noise?" I don't see any noise in the images you posted, so I'm curious as to why you asked. How many rays/pixel? What kind of render times? Just wondering I'm wondering on how far away I am from professional grade applications.


You are right, there are of course several things I cannot disclose because of commercial reasons but there are also some things I cannot for technical reasons (sampling is really not my field of expertise). The render time is 3 minutes for the image in 2560x1600pixel including antialiasing on an 8core machine.

Also, are you running Windows on the 32p machine?


Yes, Windows 2003 Server, everything out of the box.

Do you get 6 fsp with the same image quality you showed? If so, that absolutely incredible. How much is hardware (GL) and to what degree is it ray traced.


see comments about speed above. rendering is done 95% CPU and 5% GPU. But this may be subject of change.

With all the buzz, it seems as if real-time ray tracing is the new programmable shaders. Your comment about the Telsa is true, however I've heard its been difficult to get the K-d Tree etc. accelerators to run on the GPU because of the SIMD and data base access. Programming CUDA is supposed to have a small cache size (32K) or something like that, and that currently GPU based ray tracers are no faster than multicore CPU. However, NVIDIA did just hire Peter Shirley...

Your (or anyone's) thoughts?


GPU programming is of course much harder than CPU programming for two reasons. First one is that your hardware faces some real limitations like bad sequential memory access as well as shader clusters that are bigger than useable. Second reason is that the development tools or the lack of them makes it really hard to do.

What nvidia has bought from Peter Shirley I don't know, judging from the photos on the web page I see some basic raytracing functionality like reflection and refraction and that's all. But we will see in future what kind of product will appear on the market. Their goal is of course to have hybrid rendering, something they also want to achieve with the acquisition of mental images. I am very curious how all this will turn out. In the meantime we will continue the development of our software in a small team and not concentrate on all this marketing hype :)

Matthias
Last edited by Brombear on Tue Jun 17, 2008 10:53 am, edited 1 time in total.
Life is what happens while we are making other plans

User avatar
squeen
Moderator
Moderator
Posts: 2931
Joined: Fri May 09, 2003 6:10 am
Location: Maryland, USA

Re: Through the Looking Glass

Unread postby squeen » Tue Jun 17, 2008 10:44 am

Brombear wrote:
You are right, there are of course several things I cannot disclose because of commercial reasons but there are also some things I cannot for technical reasons (sampling is really not my field of expertise). The render time is 3 minutes for the image in 2560x1600pixel including antialiasing.


That's truly impressive to me. I've still got my work cut out climbing that learning curve. Thanks.:)

User avatar
squeen
Moderator
Moderator
Posts: 2931
Joined: Fri May 09, 2003 6:10 am
Location: Maryland, USA

Re: Through the Looking Glass

Unread postby squeen » Wed Jun 18, 2008 11:24 am

Sorry, one more in the series. I tought it might be called "Eeek!" but my 6 year-old named it...
eyeflop.gif
"Yum. I Love Spiders."

go figure... :)

Brombear
Posts: 786
Joined: Sun Oct 05, 2003 8:42 am
Location: Frankfurt (Rhein-Main Area) / Germany
Contact:

Re: Through the Looking Glass

Unread postby Brombear » Wed Jun 18, 2008 1:16 pm

:lol:

Matthias
Life is what happens while we are making other plans

User avatar
un_
Posts: 222
Joined: Wed May 28, 2008 6:00 pm
Location: San Diego, CA, USA

Re: Through the Looking Glass

Unread postby un_ » Wed Jun 18, 2008 2:14 pm

Love it!

Being Bender it should be "Bite my shiny metal ass Spider!" :)
:540: :Octane: :Octane2: :O2: NeXTStation x 2, A4, A7000, RISC PC

User avatar
squeen
Moderator
Moderator
Posts: 2931
Joined: Fri May 09, 2003 6:10 am
Location: Maryland, USA

Re: Through the Looking Glass

Unread postby squeen » Fri Jul 11, 2008 6:51 pm

Here's one I liked, still a work in progress (materials!).

bay_test.jpg


I'm kinda excited because I may be able to run our little ray tracer on a 2500 core cluster next week.

JacquesT
Posts: 305
Joined: Mon Jan 28, 2008 12:50 pm
Location: Bristol. UK

Re: Through the Looking Glass

Unread postby JacquesT » Sat Jul 12, 2008 1:27 am

squeen wrote:Here's one I liked, still a work in progress (materials!).

I'm kinda excited because I may be able to run our little ray tracer on a 2500 core cluster next week.


Stunning Model!!! Lot's o' detail...!

Looking forward to "the monster crunch" test.

Jacques
No SGI box currently...Snif!

Brombear
Posts: 786
Joined: Sun Oct 05, 2003 8:42 am
Location: Frankfurt (Rhein-Main Area) / Germany
Contact:

Re: Through the Looking Glass

Unread postby Brombear » Sat Jul 12, 2008 2:06 am

squeen wrote:I'm kinda excited because I may be able to run our little ray tracer on a 2500 core cluster next week.


Hope you have enough time to work on the efficiency. We are currently implementing computation clustering as well and there are all kind of strange things to be aware of (bandwidth issues, latency, compression, multicast, upload to gpu, etc. etc.)

Don't forget to put some efficiency table here, I am really looking forward to your results.

Matthias
Life is what happens while we are making other plans

Brombear
Posts: 786
Joined: Sun Oct 05, 2003 8:42 am
Location: Frankfurt (Rhein-Main Area) / Germany
Contact:

Re: Through the Looking Glass

Unread postby Brombear » Sun Jul 27, 2008 1:10 am

@Squeen

Are you still alive or been sucked away by the fan arrays of your cluster :lol:

Matthias
Life is what happens while we are making other plans

User avatar
squeen
Moderator
Moderator
Posts: 2931
Joined: Fri May 09, 2003 6:10 am
Location: Maryland, USA

Re: Through the Looking Glass

Unread postby squeen » Sun Jul 27, 2008 1:48 pm

Flight code delivery in 2 weeks---down deep in the Belly of the Whale.

User avatar
squeen
Moderator
Moderator
Posts: 2931
Joined: Fri May 09, 2003 6:10 am
Location: Maryland, USA

Re: Through the Looking Glass

Unread postby squeen » Fri Aug 22, 2008 3:06 am

This image completed with 20K rays/pixel on 512 cores in 5 minutes. Lighting (that you see) is mainly Earth albedo. The non-visible side of the arrays are pointed nearly at the sun with some beta offset. There are also 8 shuttle bay lights (not seen) that contribute very little.
hst1.jpg

Brombear
Posts: 786
Joined: Sun Oct 05, 2003 8:42 am
Location: Frankfurt (Rhein-Main Area) / Germany
Contact:

Re: Through the Looking Glass

Unread postby Brombear » Fri Aug 22, 2008 3:18 am

The image looks real great.

What do you think is your biggest bottleneck ? What kind of network topology are you using ?

We have found out that standard gigabit ethernet on the master does not make much sense anymore when you are passing a certain number of slaves (thus reaching a certain speed). You either have to bundle them or go to 10gbit or use infiband/myrinet based networks reducing the latency even more.

We will be testing a cluster with approx ~90 cores this week and I am really looking forward to it.

Matthias
Life is what happens while we are making other plans

User avatar
squeen
Moderator
Moderator
Posts: 2931
Joined: Fri May 09, 2003 6:10 am
Location: Maryland, USA

Re: Through the Looking Glass

Unread postby squeen » Fri Aug 22, 2008 3:41 am

The cluster here uses IP over Infiniband @ ~ 20Gb/sec and implements a "fast" shared file system.
We had to break the scanlines down further (row subdivisions) in order to get enough parallelism granularity. Also, while previously we had sent out all the model files via socket (distributed render farm), the shared file system makes this unneccessary.

Right now the bottleneck is Kd-Tree build (depth=30). I'd like to get some degree of parallelism on that (cutting the time in a 3rd or better would help a lot---it can take up to 1.5 minutes). Pipelining the tree build would effectively eliminate any wait time, but that means more code restructuring to keep two trees in memory instead of just one.

Photon mapping or bi-directional path tracing (Metropolis Light Transport) would also mean a whole lot fewer rays to get the same image quality. That's my next most important task when I can get some time.

I'd also like to get in one of the labs here that measures BRDFs and implement look-up tables as an alternative to the A-S AnsioPhong.

Displacement maps (as opposed to just bump maps) is also on my wish list, as is real (physical) atmospheric shattering.

No time...whew! :(

Brombear
Posts: 786
Joined: Sun Oct 05, 2003 8:42 am
Location: Frankfurt (Rhein-Main Area) / Germany
Contact:

Re: Through the Looking Glass

Unread postby Brombear » Fri Aug 22, 2008 6:17 am

Very interesting. Can you give me the raw render time (without loading file and building the kd-tree) ?

Typically it should be better to build blocks instead of scanlines, because neighbouring rays touch nearly equal segments of memory. Of course this is only valid for primary rays ... remember primaries cache while secondaries trash :-)

Did you profile IP over Infiniband ? I remember some statistics that do not look that good. Of course its more than ok to get a small speed increase by doing nothing. We will have to look into this sometime in future and I presume no picknick.

Photon mapping will also be on our list next. We have no BRDFs and I am not a big fan of it. In german we say ... wer misst misst Mist ... who measures ... measures shit :)

Seems you still have a long way to go ... as we do too ...

I am really looking forward to your next results :)

Matthias
Life is what happens while we are making other plans

User avatar
squeen
Moderator
Moderator
Posts: 2931
Joined: Fri May 09, 2003 6:10 am
Location: Maryland, USA

Re: Through the Looking Glass

Unread postby squeen » Fri Sep 05, 2008 12:15 pm

They just upgraded the cluster to over 4500 cores and since the existing workload hasn't expanded to fill the need I'm using 1536 cores with our little path-tracer-that-could and rendering 1000x1000 pixel images with 100,000 rays/pixel, (5 bounce max depth) in about 5 minutes/frame (70 seconds of which is single-threaded kd-Tree build time).

I am certain I've got mindless communication bottlenecks all over the place, but whoo-hoo!!

This is just nuts. :P


Return to “SGI: Computer Graphics”

Who is online

Users browsing this forum: No registered users and 1 guest