kuxv wrote:I have done some temporary modifications to the code which are not supposed to stay as they are so I don't want to commit them now. Although I can do that if you are interested and just use my repository as a backup and clean it all at the end.
Please commit early & often - it's your fork so it doesn't matter if it has some temporary hacks in it - just mark them with a comment in the proximity so people don't start telling you it's hacky when you already know
Solution would be to reorganize those arrays so all vertices from lowest LOD are stored on the beginning of the array and highest at the end. So they can be allocated/deallocated as needed. For compatibility there could be just a function like getHeightDataPoint(x,y) which would translate coordinates from modified array organization to classic one.
The two issues you mentioned (editing and height queries) are valid ones. For editing, you can justify having to force a load of the full precision data, because you can't edit down-sampled data, but for the height query, if this just during runtime most applications would probably be happy to accept the lower quality results based on the currently loaded data, because that's what they're rendering anyway. Therefore you probably want to have some way to force the loading to full detail, which would be called for editing, but when this hasn't been called, just sample the currently loaded data as accurately as is possible, and interpolating the rest.
One thing to notice: To allow separate read of height data stream seeking is needed and currently prepare() method takes StreamSerializer which can't do seek by bytes(at least I didn't find out how to) thus I can't skip reading of height/delta data. So I added another prepare() which takes just DataStreamPtr. There is also load() method which takes StreamSerializer as argument. So those functions have to load all height/delta data at the beginning. But their equivalents using Filename could use separate LOD load.
StreamSerializer is just a wrapper around DataStream in order to simplify the reading / writing of a chunk-based file. Feel free to add a seek in bytes method to it, the only reason it's not there is that no-one has needed it before. Please note: the whole concept of a chunked file is that there are blocks of data preceded by a header, which tells you how big the following data is. This is very robust and is generally much better than just laying out the file arbitrarily. You could just use a single chunk to contain all of the sections of data for the terrain, and just have a data header section which describes how that data is internally partitioned, which would provide the information needed to calculate the seeks. The nice thing about this is that it's self-describing, and therefore easier to debug / test.