On a somewhat related note the mine planning software I use (which used to run on SGI machines ~ 12 years ago) chokes when I try and triangulate more than 1.5 million data points. The points are are actually stored as topography (x,y, elevation) contours (ie. strings) and are obtained from an aerial survey company. Many of these flight companies have switched over to using LIDAR and generate billions of points (point clouds) and provide much higher resolution (detail).
The project I'm working on is approximately 24 km x 12 km and consists of about 26 million points residing in 9 rectangular "tiles". Just displaying these 26 million points on my computer screen (dual quad core Dell T5400 with 4gb of ram) is painful as the disk I/O is slow. The vendor is still a year or more away from having a 64bit software version.
To display a large topography area like this the steps recommended by our mining software vendor is to build a 2D grid of the area and then interpolate the Z elevation into each grid cell thereby avoiding the whole DTM (triangles) model. The datasets I routinely work with are large and always break the software. It took my workstation 9 hours just to build my (small grid size) gridded surface and then the software refused to display the grid.
My question is: Does anyone know of any software which can handle between 1.5 and 3.5 million points (in a single "tile") and create a DTM without crashing?
Next question: I can "thin" the points on each topography line to reduce the number of points however, the algorithm is stupid and thinning can cause adjacent lines to cross. Elevation lines can't cross (they only do this in the real world when there are cliffs and overhangs) as this "breaks" the DTM engine (in every mining or AutoCAD related program I've ever seen or used). Does anyone know of a "smart" line thinning algorithm and looks at the uphill and downhill lines to make sure deleting a point doesn't cause a problem?
I've asked 3 of the big mining software vendors and they all give me the deer in the headslight look and say that might be a useful tool. I've thought about the problem but I'm too busy doing design work to tackle this problem. I keep thinking there must be an algorithm out there somewhere. Most of the code (or ideas) that these mining software vendors use was "borrowed" from other industries and academics and they don't really create much us we as a company fund them.