SIMD RenderMan Shading Language Plugins 
May, 2006

 
Introduction

A Simple Example

Writing Plugin Functions

A Useful Example

RSL Plugin API Reference

Old-style plugins (deprecated)

1. Introduction to RSL Plugins

RenderMan Shading Language (RSL) has always had a rich library of built-in functions, already known to the RSL compiler and implemented as part of the runtime shader interpreter in the renderer. This built-in function library includes math operations (sin, sqrt, etc.), vector and matrix operations, coordinate transformations, and higher level functions like noise and texture. New and useful built-in functions have been steadily added with each new PRMan release.

It has also been possible to write arbitrary functions in RSL itself. However, defining functions in RSL has limitations. The RSL compiler inlines a function every time the function is called and thus its code is not shared among its invocations, let alone by separate shaders that might call the same function.

This AppNote shows how PRMan allows one to write new built-in RSL functions in C++. Such functions overcome some of the limitations of RSL-defined functions.

Writing new built-in functions in C++ and linking them as plugins has advantages over writing functions in RSL, including:

Plugins do have some limitations that one should be aware of:

 

2. A Simple Example

We start out with a simple example for implementing a varying value reduction function, pxGridMax(). It is polymorphic; in other words, you can pass it a float, a point-like type, or a color and it will compute the component wise maximum in a like type. Since the shader compiler prevents a function that takes a varying argument from returning a uniform result, the second argument to the function is an output argument containing the uniform result.

pxGridMax.cpp:

gridmax.sl:

 

3. Writing Shading Language Plugin Functions

3.1 The C++ File

Plugins must include the provided header, RslPlugin.h, which contains class definitions and macros that are used for writing your own plugin functions. You must compile your plugins with a C++ compiler, but you must use C linkage. To accomplish this, use extern "C" around the tables and functions to guarantee C-style linkage for the plugin loading system.

3.2 The RslPublicFunctions Table

The C++ file contains a table describing the functions contained in the plugin — essentially mapping the RSL functions (with arguments) to the C++ implementations. The functions may be polymorphic; in other words, one may have separate functions of the same name that are distinguished by the types of the arguments passed to them or by the type of the value returned by the function.

To give a more concrete example, here is an example table that describes implementations of a squaring function for several types of arguments:

Note the following properties of the table declaration:

  1. A static array of RslFunction structs must be used to construct the table, which must be called RslPublicFunctions.
  2. The end of the table is denoted by a NULL entry.
  3. The table contains one entry for each polymorphic version of a plugin function. In this example, there are versions that square floats, points, vectors, normals, and colors.
  4. The file can contain any number of functions, as long as they are all defined in the RslFunctionTable.
  5. The first string in each RslFunction struct is the declaration of the function as it will be called from a shader.
  6. The second argument is the name of the C++ function that will execute the plugin function.
  7. The third and fourth arguments are the names of the C++ functions that will run when the plugin function first executes after being loaded and at the end of a rendering pass.

3.3 Plugin Function Definitions

Plugin functions are declared as follows:

The function takes three arguments: rslContext is a pointer to the thread-specific state in which the plugin function is running, argc is the number of arguments to the function (including one for the return value), and argv is an array of argument pointers (each of which is a const RslArg*). The argument array starts with the return value, if any (argv[0] should not be used if the RSL return type is void). The plugin function returns an integer return value: 0 indicates no error and a nonzero return value indicates an error. The RSLEXPORT macro is an OS-specific declaration used to ensure the function is visible to the program loading the plugin. Be sure to enclose all the plugin functions in an extern "C" { ... } block, as shown in Section 2.

The RslArgs have a number of useful methods used to determine attributes of a given argument. IsFloat(), IsString(), IsPoint(), et cetera can be used to determine if an argument is a given type. IsArray() can be used to determine if an argument is an array or not. If the argument is an array, GetArrayLength() can be used to determine the number of elements of an array argument. See the RSL Plugin API Reference for a description of other advanced methods.

3.4 Iterators

In PRMan a plugin function will operate over a collection of points at the same time. However, not all of the points in the collection will be active at any given time (due to the nature of loops or conditional statements in the shader). In order to perform operations on only those points that are active the plugin interface provides iterators. The data for each active point of a given argument is accessed through an RslIter object of a given type. The example from Section 2 demonstrates how iterators are used: 

    // Get iterators for result and argument.  The result is always uniform.
    RslFloatIter result(argv[2]);
    RslFloatIter    arg(argv[1]);

    // Copy the first value (the initial maximum) from arg to result.
    *result = *arg;

    // Query for the number of iteration values from the argument.
    // It will be one if the argument is uniform.
    int numVals = argv[0]->NumValues();

    // Copy the maximum argument value to the result.
    for (int i = 0; i < numVals; ++i) {
        if (*arg > *result)
            *result = *arg;

	 // Increment the argument iterator.  Result is uniform, 
	 // so it does not require incrementing (but doing so wouldn't hurt).
         ++arg;
    }
An iterator is constructed from a given argument:

The iterator can then be treated like a pointer to the given type, so one can access the data for a given argument at a given point by simply dereferencing it:

returns the value of the first active float. The iterator is advanced by simply incrementing it:

The pre-increment operator should be preferred, since it is more efficient than the post-increment operator. Assignment to arguments is equally simple:

Note that any argument (not just argv[0]) can be used as an output argument. It is up to the programmer to ensure that plugins only modify assignable values that have been specified with the output modifier on the arguments.

In this new interface, it is crucial to know how many iterations must be performed by the plugin function. The IsVarying() method of the RslIter class can be used to determine if an argument is varying or uniform. For varying arguments, the number of iterations will be equivalent to the number of active points on the grid; for uniform arguments, only one iteration should be performed. The number of iterations to perform can be obtained from the NumValues() method of the RslArg. Functions that return a value in the result argument should use argv[0]->NumValues() as the number of iterations, since the shader compiler guarantees that the detail of the result argument matches the detail of all the other arguments. Functions that return void or return any results in output arguments must examine all their arguments to determine the correct mix of varying and uniform argument iterations. There is a static convenience function in RslArg to simplify this: int numVals = RslArg::NumValues(argc, argv). It should be passed the argc and argv that were passed into the plugin function. As an example, a plugin function that adds its arguments would iterate as follows:

If the result is varying, the loop will run once for each active point. If the result is uniform, the loop will run just once. If the result argument is varying and either or both arguments are uniform, the loop will still run over all the active points for the result. If the result argument is uniform and either argument is varying, the shader compiler will flag the detail mismatch as an error. An example that uses output arguments, pxGridMad.cpp is discussed in Section 2.

3.5 Strings and Arrays

String and array arguments can present interesting issues when iterating and assigning. We will start with an example that appends '.tx' to a string argument and returns the result.

Dereferencing a string iterator gives a string pointer (char*). Strings are assigned by pointer assignment (e.g. *outString = tempString). However, the contents of an RSL string should never be modified. Also, the renderer does not free strings that are allocated in plugins, so the above example would cause memory to remain in use for the result until the end of render. One should never free memory associated with a plugin argument; doing so will result in a crash. Storage management is discussed in Section 3.6.

Arrays are accessed through a separate class of iterators, RslArrayIter. Array iterators for all of the types in RSL are provided. Here is an example that searches an array of vectors for a negative z component and returns an array of zeroes and ones based on the results.

3.6 Threading and Data Lifetime

As mentioned in the string example above, memory allocated in a plugin method is not freed by the renderer; the plugin developer must perform their own memory management. There are five different ways this can be done: per frame plugin initialization, per frame function intialization, global store allocation, thread-local allocation, and shader-local allocation. The first three are not thread safe and the second two are. For memory management that is not thread safe, the developer must place synchronization primitives around memory accesses to ensure correctness.

Per frame initialization is an easy way to allocate memory for data tables that need to be created exactly once for the life of a plugin; the memory allocated by per frame initialization will persist for the duration of a frame. At the end of the frame, a cleanup routine can be called to free any allocated resources. An example that uses per frame initialization is shown below and also in Section 4.

This version of the plugin does not leak memory on each execution, but, as noted in the comments, it has a major problem: it is not thread safe. The next example shows the use of per function initialization and uses RixInterfaces to create lock objects that ensure thread safety. However, in the example above, the string from one thread of execution will most likely be copied to strings from other threads, potentially causing an error that would be hard to debug. In addition, this version is not ideal because it assumes the strings coming into the function will never exceed 1024 characters in length.

The example below uses per function initialization to initialize global memory and locks for the given appendTx plugin function. Note however, there are two caveats with per function initialization, the first is that an initialization function will be run for every plugin function it is associated with. So if the same init function is associated with multiple plugin functions, it will be executed multiple times. The second is that cleanup functions are only run on plugin functions that have defined init functions, NULL init function pointers will prevent cleanup functions from executing.

The initializers in this example use RixInterfaces to obtain a factory object called RixThreadUtils. Once we have the RixThreadUtils factory object, we then create an RixMutex object that can be used to lock and unlock access to the shared global data in the plugin. Ideally one would never use locks in a plugin, since they reduce the paralellism that can be achieved by the multi-threaded renderer.

The final example describes the use of RixInterface objects that can be used to create memory stores that can be shared across plugin functions as well as across plugin files and across plugin types. We now revisit the global appendTx example one last time:

This example is a little more complex, because it attempts to maintain thread safety. The GetGlobalStorage method of the RslContext allows one to retrieve a global storage object. Since this is a global memory store, one has to lock it before performing operations on it with the Lock method. Then memory associated with an arbitrary key can be obtained with the Get method. If no data has been stored for this key, a NULL value is returned. If one detects that no data has been stored with this key, it can be created and put back into the global memory store with the Set method. A full description of the interfaces available via the GetRixInterface method can be found here: RixInterface Reference

An much simpler and more efficient, thread safe way to write this function involves using the RslContext methods GetThreadData and SetThreadData to manage a memory buffer for each thread of execution. The code to do this is shown here:

This version of our string appending function works well — it is thread safe and efficient. However the code still expects the strings will never exceed 1024 characters in length. A final revision to this function involves the use of SetLocalData and GetLocalData. These are very similar to the SetThreadData and GetThreadData methods, but the allocated memory persists only for the time that it takes to shade the current collection of points being operated on by the shader. This makes it more efficient in terms of peak memory utilization, but it can be less efficient in terms of the time utilized to call memory allocation and deallocation routines for every collection of points, rather than once per thread. The code for this version of the function is shown below:

3.8 Compilation Issues

Compiling and using a new plugin requires three steps:

  1. Compiling the C++ file that contains your plugin functions.
  2. Compiling the shader that uses your functions.
  3. Rendering a frame.

Compiling your C++ file is straightforward: just use the standard C++ compiler to generate an object (.o/.obj) file, then generate a shared object (.so/.dll) file from the object file. Remember that, though using C++, you must use C style linkage. You also must ensure that your C++ compiler and libraries are compatible with the compiler and runtime libraries used by PRMan (gcc for Linux and OS-X and Microsoft Visual C for Windows). Here are example commands for building a plugin on several architectures:

The resulting file myfunc.so or myfunc.dll is the plugin that implements your new function. It is not important that the filename matches the name of the function.

When compiling your shader, if the RSL compiler comes across a function reference that is neither a known built-in nor a function defined in RSL, it will search all plugins defined with the plugin directive until it finds one within the exported RslPublicFunctions table. Note that the plugin file must be specified with the plugin directive. (The .so or .dll extension is not required.) The path to plugins can be relative and can be modified with the -I command line option of the RSL compiler. The RSL compiler will typecheck your function call and issue a warning if you pass arguments that do not match any of the entries in the RslPublicFunctions table of your plugin.

Once your shader is successfully compiled, you are ready to render. The renderer requires the plugin to be in one of the directories that it searches for shaders. In other words, the searchpath specified with the option Option "searchpath" "shader" also specifies the searchpath for the RSL function plugins.

 

4. A Useful Example

In 2002, Ken Perlin published a concise paper in Computer Graphics; Vol. 35 No. 3. This paper addressed two issues in the original Perlin Noise algorithm. Below is a plugin that provides a C++ implementation for the improved noise function, based on his java code.

 

Pixar Animation Studios
(510) 752-3000 (voice)  (510) 752-3151 (fax)
Copyright © 1996- Pixar. All rights reserved.
RenderMan® is a registered trademark of Pixar.