3.6 Render Loop Overrides

 
 
 

A render loop override is represented in the API as an MRenderOverride.

As the name implies a plug-in of this type overrides the entire rendering for a complete frame. Unlike older interfaces there are no fixed entry points to insert or remove specific render logic in the render loop. For example there is no such thing as a “pre” and “post” “pass” callback. Instead the intention is to provide the appropriate mix of building blocks and pipeline exposure to allow the plug-in to define arbitrarily complex pipeline logic.

An override is broken down into a set of operations. Predefined operations, which execute parts of the internal pipeline, as well as custom operations represent the base set of building blocks. Operations are designed to fully describe data inputs and outputs, and by default execute independently of each other. The main connection between operations is thus data. All operations render into a render target and data is explicitly passed between operations by sharing render targets.

The following is a set of basic operations available:

As these operations are embedded into the rendering pipeline, a rendering state is provided to operations as necessary. This information is generally determined at the beginning of a frame and is implicitly set as default values for each operation. Overrides which are set per operation do not persist between operations.

Depending on the amount of customization, the interfaces are mostly draw API agnostic.

Unlike previous API implementations, sequence is specified by plug-in code and is not defined by the interface. The plug-in is therefore required to break down the desired render loop into a linear list of operations. A basic look at the API interface shows an example of the type of render loop that can be specified (left). The plug-in determines the appropriate ordering of operations. On the right is the basic multi-pass looping logic for old interfaces. Here, sequence is predefined based on the interface.

Figure 40: On the left, render targets serve as a means to pass context between operations as outputs and/or inputs. On the right, a series of scene render ‘passes’ get called one after the other to update a particular viewport. Callbacks in the old interface are provided between renders to access and update external data or state, which can be used to affect the current render or subsequent renders.

3.6.1 Render Override Registration and Activation

As overrides are integrated into the rendering framework, they work for all exposed interfaces (3d viewport, Playblast, RenderView and batch rendering), while avoiding dependencies on any specific interface such as 3d viewports (M3dView).

Any number of overrides can be registered via the MRenderer class but only one can be actively set per exposed interface. Each 3d viewport (which Playblast uses) can have a different override, while command line rendering (RenderView and batch) can use yet another different override.

The main interfaces for querying the available set of overrides as well as setting the active override are: M3dView and the modelEditor for interactive rendering and render options or MRenderer for non interactive (RenderView and batch) rendering.

Using an MRenderOverride does not always preclude usage of old interfaces but their usage is strongly discouraged for a number of reasons including the fact that old interfaces:

3.6.2 Base Render Operation And Utility Overrides

All render operations which can be instantiated are derived from the base class: MRenderOperation. These classes can be used as is, or they can be further customized through further class derivation. Derivation is the approach used to provide override parameters or required methods per operation.

MRenderOperation provides interfaces which are generic to all operations. This includes:

  1. A name: All operations require a unique identifier.
  2. A possible override for the output render targets (MRenderTarget).
  3. A possible override for the 2d viewport rectangle (of the draw context).

Some operations such as scene renders and custom user operations can override camera inputs. The utility structure MCameraOverride is the interface for providing such an override.

The override of the camera can be specified at two levels:

The camera override parameters can also be used to hide the 3D UI for a list of cameras.

Another utility structure which is coupled with scene operations and quad render operations is a render target clear. This structure is MClearOperation. The utilities parameters can be set to selectively indicate the following options:

User operations can perform their own clear as required.

Note that clear operation overrides which are embedded in the render loop disallow any internal 2D post effects from rendering as the clear operation can cause required data for a post effect to be removed.

All operations and support classes are only descriptions of behavior and are not resources themselves. As such, they can be persistent beyond the lifetime of a frame render.

3.6.3 Scene Render

The construct to represent a scene render is MSceneRender.

By default a scene render operation renders all or part of the current visible scene into the current render target. Visibility takes into account the current state of objects, any display filters, and camera frustum culling.

When querying pass context information if a scene render is active then its name will be the current pass identifier and the pass will be marked with a “color pass” semantic.

There are a fair number of per operation overrides. These overrides reflect what can be set either in a 3d viewport or from command line rendering. The overrides are set for the duration that the operation is being executed.

The possible overrides include:

For per scene render preparation, pre and/or post render callbacks can also be specified.

There is no limit on the number of scene operations that can be specified, though there is the inherent cost of rendering multiple types. Each invocation runs the rendering pipeline and thus it is possible to have multiple update and draw phases invoked. The main difference between the new and the old systems is that multiple scene traversals are not invoked, as this is no longer an inherent part of the pipeline logic.

An example of a shaded versus non-shaded pipeline split is show in the following diagram. Each variation in an override can be used to create a different pipeline per scene operation.

Figure 43: The top-most pipeline is an example of what roughly occurs when there is no separation of “shaded” from “non-shaded”. The split into two pipeline configurations is shown below: a “shaded” only pipeline (middle) and a “non-shaded” pipeline (bottom).

3.6.4 Quad Render

This is a pre-packaged render of a 2D screen aligned geometric quadrilateral. The construct to represent a quad render is MQuadRender.

Depending on the shader and the viewport region specified, the operation can render to fill the entire render target or just part of it. A clear operation (MClearOperation) can be returned from this operation for cases where sub-region rendering is required.

This operation can be used to render into any of the render targets (MRenderTarget) specified as output by writing the appropriate shader logic for the shader instance (MShaderInstance) associated with the quad operation. As the shader instance can also take, as input, a set of MRenderTargets, a quad operation can form connections between operations via the use of shared targets.

The following is the possible flow of render targets as source data and output data.

Figure 44: The shader instance can take 0 or more targets as input. The shader can write out into 1 or more output targets.

It is possible to render using a render target as both an input and as an output. Depending on device resource management a temporary target may be required. The inherent cost of this functionality should be weighed against any simplification or clarity gained by describing the render loop logic in this way.

“Ping-ponging” of render targets (alternating targets) in general does not create temporary targets.

For quad rendering, a few basic semantics are provided to allow for automatic binding of parameters which can be used for quad rendering. This includes:

In general, as parameter binding is the responsibility of the operation, it can query the available parameters on the shader instance and bind as appropriate. For example render target binding is performed manually.

Example of an invert operation (CgFx)

// World-view-projection transformation.
float4x4 gWVPXf : WorldViewProjection < string UIWidget = "None"; >;

// The single filter input, i.e. the image to be manipulated.
texture gInputTex : InputTexture
<
    string UIName = "Input Texture";
>;

// Filter input sampler.
sampler2D gInputSampler = sampler_state
{
    Texture = <gInputTex>;
    MinFilter = Point;
    MagFilter = Point;
    MipFilter = Point;
};

// Vertex shader input structure.
struct VS_INPUT
{
    float4 Pos : POSITION;
    float3 UV : TEXCOORD0;
};

// Vertex shader output structure.
struct VS_TO_PS
{
    float4 HPos : POSITION;
    float3 UV : TEXCOORD0;
};

// Vertex shader.
VS_TO_PS VS_Invert(VS_INPUT In)
{
    VS_TO_PS Out;
    
    // Transform the position from object space to clip space for output.
    Out.HPos = mul(gWVPXf, In.Pos);
    
    // Pass the texture coordinates unchanged.
    Out.UV = In.UV;
    
    return Out;
}

// Pixel shader.
float4 PS_Invert (VS_TO_PS In) : COLOR0
{
    float4 output = tex2D(gInputSampler, In.UV);
    return 1.0f - output;
}

// The main technique.
technique Main
{
    pass p0
    {
        VertexProgram = compile glslv VS_Invert();
        FragmentProgram  = compile glslf PS_Invert();
    }
}

3.6.5 User Operation

Instead of having a fixed structure for pre and post pass callbacks, a user operation is exposed in the API. The API construct for this is MUserRenderOperation.

The operation is most similar to MViewportRenderer in the old system. The main difference is the level of integration. An MViewportRenderer is an unstructured render override while a MUserRenderOperation is an action which is integrated into the render loop support structure.

A MUserRenderOperation allows for a camera override to be specified, in addition to the viewport and render target overrides inherited from the parent MRenderOperation class.

When querying pass context information if a user operation is active then it's name will be the current pass identifier and the pass will be marked with a "user pass" semantic.

There is one entry point which is exposed to "execute" the operation. In order to have some frame of reference a draw context (MDrawContext) is provided.

Figure 45: The camera can be overridden for a user operation. At execution time, an MDrawContext is provided. The output can be directed to one or more render targets.

As with all plug-ins which affect draw state via direct API calls, the operation should always restore to the entry state. This is especially important in relation to the current output target(s). State such as the active OpenGL context, FBO, PBO should not be set within the operation, and if they ever are, then the previous values must be restored on exit from the operation.

For example, calls such as M3dView::beginGL()/endGL() should never be called since they set active OpenGL context. Also, they never run under DirectX or in batch mode.

3.6.6 Present Operation

To present a render target for interactive viewing a present operation is provided. In the API this is represented by MPresentTarget.

This is roughly equivalent to a swap buffers call in OpenGL or a present of a SwapChain in DirectX.

When batch rendering, this operation has no meaning and is automatically ignored.

The operation can specify a color render target to be displayed. If none is specified, the active internally set color target is used. It is possible to optionally specify depth to be presented. If a depth target override is specified then the depth values from that target are copied to an onscreen depth target.

When dealing with active stereo rendering the output can be directed to the left or right eye. The way to achieve this in the render loop is to present the contents of a render target to the left buffer", and the contents of a render target to the right buffer. Mono viewing, which is the default, is achieved by specifying rendering to the center buffer.

If active stereo rendering is supported by the video card, and the driver has been appropriately set up, then the MTargetBackBuffer input can be set to route targets for display as appropriate.

Figure 46: The back-buffer option can be set to override the output buffer. The “present” operation routes the appropriate render target to the correct buffer on the “on screen target”.

3.6.7 HUD Render

This operation allows for the display of the 2D HUD for a 3D viewport. In the API, this is represented by MHUDRender.

3.6.8 Structure Summary

Figure 47

The above diagram shows the overall structure of all of the various operations and how various overrides and resources (shader instances, render targets) can be shared and reused.

The connection from the MShaderInstance to an MRenderItem, though not explicitly part of the render loop structures, shows the possible flow of data (render targets or textures) from the output of a render loop (pipeline) to the input for a render item flowing down another pipeline. An example of this is to provide shadow maps as an input to shaders which render an item with shadowing.

3.6.9 Override Setup And execution

With a basic understanding of the available operations, this section shows how a plug-in should set up the override to ensure proper configuration and execution of these operations.

The term phase introduced when discussing the rendering pipeline is reused here to describe the key functionalities of a plug-in. The basic phases are: setup, execution and cleanup.

During the setup phase:

During the execution phase:

During the cleanup phase:

Instances of render overrides and operations have no implicit resources themselves, but instead only keep references to external resources. It is the responsibility of the plug-in to determine the life-time for these resources. It is best to use the available resource managers if possible. As long as the plug-in holds on to a reference to a resource, it remains allocated. Resources which are passed back to an operation (for example through an MDrawContext) should never be held on to as it could be a resource which only exists during the lifetime that the operation is being executed.

Figure 48: A high level breakdown of the override “phases”: up-front data evaluation and update, execution time logic definition and operation invocation, and the final cleanup. One or more pipelines may be setup and updates to operations can represent changes to operation data dependencies. For example, the render target used by a shader instance parameter can change during execution time.

3.6.10 Render Override Examples

In this section, a few example operation configurations are described.

Post Scene Render Color Effect

Figure 49

This example shows, a basic scene render into depth and color output targets. The second operation applies a color effect (for example, “posterize”) to the color target from the first operation and writes back into the same target. This is similar to allowing reading and writing to the same color target. The implementation can follow the example above; or, a second color target can be acquired and used as the destination for the color effect, or a color target can be not specified at all. In the later case, the active internal color target is used.

A more complex configuration which is used to support a “glow” post effect is part of the viewRenderOverride SDK sample.

Stereo Rendering

Figure 50

This is an example from the plug-in which is used to support stereo display. On the left are two scene renders which are the first two operations. Each scene render uses different camera overrides to render from the left and right eye (camera). The output is passed to two color targets in order to perform a final composite based on two color inputs. The depth target is reused as the final contents are not required. At the top right a quad render operation is specified as the next operation in order to support Anaglyph passive stereo display. Different shaders can be returned dynamically for the quad operation, depending on the required final passive stereo display mode. Active stereo is shown at the bottom right. It would be an alternative operation configuration that passes the left and right color targets to the left and right onscreen buffers respectively.

The stereoCameraOverride plug-in code in the SDK can be referenced to show the implementation.

Beauty Pass Color Composite

Figure 51

This example shows how an external renderer, which only generates color images as output, can be composited with the rest of the scene rendered using the internal framework.

There are three basic operations performed:

  1. The first (left) MSceneRender prepares the depth buffer with depth values based on shaded objects in the scene. The display mode is set to be shaded and the scene filter is set to only draw shaded items. There are a few choices for the shader used. Either a custom depth shader can be applied; or, as shown in this case, a solid color shader using an ambient light can be used. The color mask is set to only write to a depth target.
  2. The next operation (MUserRenderOperation) is used to blit the external renderers color image only to the color target.
  3. The last operation (right) MSceneRender is responsible for overlaying the non-shaded scene elements. Here, only the active Maya objects are rendered. No targets are cleared but depth testing is still enabled to support proper shaded versus non-shaded scene compositing (for example, “wireframe-on-shaded”).

User Operation With Device Access

This is a simple configuration from a render loop logic perspective. Its logic is closest to the free-form MViewportRenderer implementation from the old system. There is only one MUserRenderOperation. The SDK example viewDX11DeviceAccess shows how various resources can be allocated from the framework and how GPU handles to the render targets and texture (shader resource views), state blocks, sampler states can be extracted. The older MDrawTraversal interface is used to scan the entire scene to draw some bounding boxes.

Figure 52: In the setup phase the GPU device handle is extracted to create some stock DX geometry. A texture and a sampler state are allocated via the appropriate manager in the framework. An output target is also allocated. During the execution phase, a series of steps is performed to get the GPU handles of resources and rendering is all done using native DX calls.