vSprites in Depth

Why go to all this trouble? (Editors note in 2023: Remember this was written in 2003...)

When large numbers of highly detailed objects (many thousands) are in a scene, pre-rendering can offer very efficient render times. An equivalent scene with as many objects in it - Paint Effects or geometry - with shadows - could require more than 10 times as long to render.

Other solutions exist, and of course have their own trade-offs. Below are a few of them with discussion of the issues that come with them.

Maya Paint Effects:
Works great for a few hundred in a single frame. But with shadows (ray traced or depth mapped), over 3,000 bushes, for 3000+ frames, I think the render time was not even worth bothering to address. Plus, you'd have all this computation going into all these Paint Effects strokes, most of which would affect at most a pixel or two in the image. Also, even with Paint Effect's efficiency of display, having 100,000 paint effect bushes in a scene would still make the scene very bulky. The final nail in the coffin, as usual with Paint Effects, is the need for motion blur, which is pretty much out unless you're willing to "fix it in post" and comp it.

Advantage of vSprites:
Since all the objects are a single plane with a texture map, 2d motion blur works perfectly.

Particle Sprites:
Simple and efficient, but liable to the "twisting" artifact - sprites rotate around their center. With all these bushes spinning it would look like a low grade arcade game.

Advantage of vSprites:
Since all vSprites are offset from the virtual origin of each imaginary object, the object appears stationary even when the camera rotates around the object.

Crosshatched Planes:
This would work if the camera wasn't moving much, and shadows could be simulated with a piece of geometry serving as the floor. But with the animation of the camera, the structure of the few surfaces would be viewed edge-on for at least some bushes all the time...again more low low-grade arcade game feel.

Advantage of vSprites:
Since each object is pre-rendered in full, there is no lack of detail on the object from any angle.

All above approaches: No adjusted Level of Detail
All the techniques above fail to optimize geometry or texturing as a function of distance from the camera - an object 10,000 units away has just as much detail as one of 10 units distance, and takes just as much render time.

Advantage of vSprites:
Sprites can reuse texture maps as a function of distance from the camera. This way objects very far from the camera don't need to be re-rendered each frame.

All above approaches: Geometry out of view consuming memory and compute resources
All the techniques above fail to optimize the geometry in the scene to be only that which is within the camera field of view. The memory footprint for a valley filled with bushes would be overwhelming, and most of it unused at any one frame render.

Advantage of vSprites:
The implementation of vSprites is a runtime MEL command per frame. The scene stored to disk has no bushes in it, and opens in about 4 seconds. The bushes are created and removed at render time so that only the vSprites that are in camera view exist - they are added and removed as needed as the camera's view changes. Runtime memory footprint is minimized, and there's no need for big bulky files saved to disk which take forever to load.

My over-ambitious, kill-the-fly-with-a-nuclear-weapon solution:
Pre-Rendered Objects, or "vSprites"

There are other working solutions out there - some film and game studios have some good ones to be sure. I still wanted to come up with my own.

Ideally, the camera would never know the plant (or any object) was never really there.

My initial approach attempted to render every object every 10 degrees of change, and then snap or interpolate as the camera view swept in between these ranges. As technically interesting as it was, the visual artifact wasn't acceptable.

So, like most people writing scripts, the I redesigned a second time, making vSprites render each object from the relative camera position fresh each frame. This works, but results in a bit of overkill for each frame being rendered - the relative change in angle for the majority of objects was too small to be noticed. So I added a distance requirement to the updating of any vSprite - only objects close enough to the camera to exhibit parallax would be re-rendered. This results in a one-time up front slow first frame render (as all the vSprites are rendered), but each incremental frame gets completed very quickly as only the few that are close to the camera need to updated.

Benefits:

1. Fun coding exercise. Yes, I'm a programmer, looking to make things that work.

2. Motion blur works fine within the Renderer...unlike Paint Effects for example.

3. Implementing more visual detail for very little geometry is a rendering optimization I can't pass by.

4. FAST render times. Yes, the MEL script has to execute once before each frame to adjust the planes and corresponding textures, but since this only involves rotating existing geometry and breaking/making connections with shader nodes, even this is fast. The scene itself has a lot of 2 triangle planes in it...not very taxing for the software renderer. The details already exist in the shading texture. I've rendered frames with thousands of bushes (complete with shadows..they're pre-rendered too) in 1 minute. Try that with thousands of paint effects brushes.

5. Geometry in the scene at any one time is limited to geometry that is in the camera view. vSprites are added and deleted as the camera view moves, greatly reducing the load time and optimizing total memory footprint for rendering.

Detriments:

Initial frame of a sequence takes over an hour to create and pre-render the initial set of textures. But that's 1 hour, one time startup cost for all subsequent (1000's) of frames.

Implementation:

A quick overview of the vSprite architecture below.

- A mel script vSprites is added to Maya's "Pre-Render MEL" render globals. "vSprites nameOfCameraToBeRendered"

- This script verifies the camera to be rendered, and ensures all scene prerequisites exist (if not it makes its own default on the fly). This includes Closest point on surface nodes for camera field of view determination. Average time: less than 1 second.

- vSprite transforms are captured via mel an put in an array. The array is saved to a file - so for scenes where the vSprites don't move, the transform capture occurs only once. Subsequent frames rendered simply read in the file if it already exists. Average time: 2 seconds if file exists, 4 seconds for the first time transform data is created.

- vSprite array data are parsed to determine their field of view in the camera, size, render resolution, creation, update, or delete status.
Average time: 1-10 seconds.

- vSprite array data are processed - adding vSprites to the scene, updating them, or deleting them as necessary. When vSprites are added, their file-based texture shader networks are created, but the source files for them don't exist.... yet. Average time: 2 minutes for the first frame (as 1000's are created, usually a few seconds for all subsequent frames as only a few 100 objects get added or subtracted each frame.)

- vSprite array data are saved to a new hand off file. This is just a binary file written out from Maya via mel containing relative camera positions and bush item attributes. Average time: less than 1 second.

- vSprites then launches another instance of Maya in prompt mode. This instance of Maya reads in the hand off file, creates a camera, imports in all the object geometry types to the origin (in this case a bunch of different paint effect based bushes), and then keyframes the camera to all the relative positions from the objects as exists in the parent scene. This Maya file is then saved, and this instance of Maya in prompt mode quits.

- vSprites then launches batch renders of the file created above. If multiple machines are available, it will divide up the jobs amongst many machines to greatly reduce render times. Average time: up to 2 hours for first frame (again as 1000's of texture files are created the first time, usually less than 4 minutes for all subsequent frames)

- vSprites then renames all the rendered images from the batch render above to the appropriate texture file names for the textures to be matched up in the parent Maya scene. Average Time: 1-5 seconds.

So vSprites basically takes a few hours to create the textures for the first frame rendered (since it has to generate 5000 texture files or so) , but then subsequent render times are less than 4 minutes for vSprite processing. For most of the frames in this animation, total render times (including vSprite processing) were ~10 minutes a frame, even with an average of 4000 bushes in camera view at any given time.

Here is the current list of mel for vSprites. It looks scary, but it all works by simply adding the one line to the render globals mentioned above.

vSprites.mel
vSpritesCaptureTransforms.mel
vSpritesDeletevSprite.mel
vSpritesGetRelativeVectorAndDistance.mel
vSpritesGlobals.mel
vSpritesInitArray.mel
vSpritesIsInFrustum.mel
vSpritesLightLinkExclusive.mel
vSpritesMakePathForWindows.mel
vSpritesMakevSprite.mel
vSpritesParseTransforms.mel
vSpritesProcessTransforms.mel
vSpritesRefreshTextures.mel
vSpritesRendervSprites.mel
vSpritesUpdateSpriteInfo.mel
vSpritesUpdateSpriteTexture.mel