Finally go it to work... it seems the "-x" option is not supported on Irix 5.3 or 6.2 on Indy (might work on others - not sure?)
So tried "setmon -w 60HZ" and simply "setmon 60HZ" but they did not make the 60HZ stick either (kept defaulting to 76Hz are re-boots). Changing PROM monitor to L, did not help much either.
Next tried the suggestions on the link below; sub-directory is NG1/vof for Indy (quoted the text below;). At first they did not work also, but after setting monitor to "L" in prom with these changes below, somehow the new resolution of 60Hz finally stuck. After this, I reversed the two changes below and it still remains at 60HZ. Not sure if anyone has any thoughts on this...http://www.tek-tips.com/viewthread.cfm?qid=632551
The best way to come out of this situation is, go to prom monitor by pressing esc key when the system shows the initial screen. Type single at >>> prompt.
Once you enter into single user mode, goto /usr/gfx/ucode/CRM/vof or /usr/gfx/ucode/CRIME/vof.
Copy the file 1024x768_60 to 1280x1024_96.
go to /etc/rc2.d
/usr/gfx/setmon -n 1024x768_60.
write and close.