I forward from an R4600 Indy to an IBM ThinkPad with a lowly 8MB graphics card. It works fine with most apps under both BeOS (on which I have a very rudimentary X server) and FreeBSD (6.2-RELEASE, ie. X.org 6.9). The MediaRecorder uses GLX (OpenGL extension to X11) which BeOS doesn't have, but I think it's only for the actual video stream, not the interface as such. System preferences, fm, nedit, and other apps all run as good if not better than they do on the Indy. MediaRecorder works fine with FreeBSD with GLX enabled, certainly not as slow as you describe.
So, err. It works for me, and out of the box too on FreeBSD. Hope this helps!
You do of course know that the interface, when forwarded over X11, is running on the machine you're sitting at, so if it's sloooow, then the apps you try to run will respond sloooooowly?