I have just finished a computer graphics project course at my university, which focused on using shaders to render trees using the model described in "Creation and Rendering of Realistic trees" by Jason Weber and Joseph Penn. This is the same model that MeshTree, Arbaro and other similar tree generating projects have used. All of them, however, have generated a static mesh, and none have considered level-of-detail-schemes in detail. (no pun intended)
My approach has been somewhat different to theirs. I initially envisioned my project for the Geforce8, as a geometry shader would in theory let you render a single tree at any detail with just its position (and possibly orientation), tree type and a seed value (for the random number sequence to use) as the only parameters. But lack of such a card made me reconsider my ideas for a Geforce6 card.
A tree is rendered att full detail by drawing each stem as a segmented cylinder of specified detail and its vertices are modified in a vertex shader, to make it curve and taper its radius. I first chose to do away with random variations in the curvature, which allowed me to explicitly compute a vertex's position in the vertex shader linearly in time, but was not satisfied with the result, as curved stems without any random variation looked somewhat synthetic
My current solution is to use a big heavily quantized vertex texture as lookup table, which allows a fixed number of stems with random variations in curvature. The curvature is linearly interpolated when a stem sways in the wind, allowing the vertex texture generating code to quantize the maximum allowed curvature to just a handful of possible indices.
Leaves are drawn as a cluster for each stem, consisting of quads with identical positions, but differing with an index. This index is used to place them randomly around the stem. (which implies reading the same vertex texture as the stem vertex shader uses)
Here are pictures of the test tree with and without leaves:


The trunk, main branches and sub-branches all sway individually in the wind.
I have also implemented a level-of-detail-scheme. For this I first looked at SpeedTree's method, but deemed it unsatisfactory. While SpeedTree looks outstanding from stills, I personally find the heavy aliasing resulting from only using alpha-rejection disturbing. Also, SpeedTree's simplistic LOD scheme means that the illusion is shattered once you get too close to a tree so the camera-aligned billboards become obvious.
Instead, I chose a LOD-scheme taken from a thesis by Emil Jansson, where branches are increasingly replaced by textured billboards axis-aligned-billboards. (not camera aligned) I.e., once distance to the tree increases, the subbranches are replaced with billboards, then the main branches and finally the tree itself. (becoming the tree billboard we all know and love from any tree rendering software)
To be able to re-use the same stem textures somewhat more and also make them sway in the wind the same way the geometry it replaces does, I implemented the same formulae I used for bending stems in the pixel shader to bend the pre-rendered textures viewed from the side. (without the random variation though) These two pictures should give you an idea of what I mean:


And this is how it looks when the main branches are rendered as billboards:


Less symmetric results should be possible by rendering several billboards for each branch level.
The same pixel-bending could be applied to a tree which just consists of a single billboard. At that distance, you won't take notice of individual branches' movements, but still be impressed with seeing trees blow in the wind at a distance.
The billboard stem vertex shader is very similar to the leaves shader I described above, but each stem consists of four orthogonal quads, who are faded away (and then discarded completely) with increasing angle between the normal and the viewspace Z-axis.
Real alpha-blending is used to get anti-aliasing for free via mip-mapping, which does require some sorting. However, it isn't very noticable if the billboard branches are drawn in a random order since you're at a distance from the tree. Also, I think the vertex shader could be rewritten in a smarter way to take the viewing angle into account when placing the billboard stems so that the ones at the back of the tree are drawn first. (remember the quads have identical vertex data except for the index) So for the LOD where the trunk is geometry and all main branches are billboards, it would be enough to sort trees only.
For batches of billboard trees at a distance, I think you could get away with the same reasoning as for the branches and draw them unsorted if you eventually cross-fade them with a sorted billboard batch. But I haven't tested this in practice.
An initial goal of mine was to provide a simple "TreeManager" class which would let you add different tree types to it and set the quantization and limits of the curvature vertex texture parameters. You could then "compile" this data to make it generate the vertex texture and build texture atlases of the tree barks and billboard stems. This class would also handle the drawing order, which needs to look like this:
* First draw all fully opaque geometry stems in the scene
* Then draw all batches of billboard trees back-to-front
* Then draw stem billboard clusters for billboard main branches for each tree, and draw the fading geometry stems for this tree
* Then draw stem billboard clusters for sub-branches
I have started work on this class but didn't finish it in time. I also haven't had time to optimize the shaders. There's a lot of work left to be done, and I was thinking it might make a decent candidate for the SoC. I envision an external library for OGRE where the game programmer can easily edit parameters of a generated tree like in meshtree studio, only in real-time this time. Then throw all the designed trees to the TreeManager and get the scene populated with lots of tree species and even more individual instances of each species where no instance is identical to the other.
Of course, there are drawbacks to the method I've described. The big question is if the lack of aliasing makes up for the performance loss when using alpha-blending and (approximate) sorting. And of course, the shader requirements are a drawback in itself. You might also question how much heavy use of instancing really bothers a player, and whether unique trees are really worth the effort. More testing and evaluation is needed for this, and a mid-term evaluation would likely consist of deciding on which features features are really worth the effort, or if the project ought to turn its focus to a simpler more SpeedTree-like scheme after all.
Phew, even less time left now. Tell me what you think about the proposal.