Render Spooling - Job Setup

Display ... Camera ... Reyes ... Rays ... Acceleration ... Spooling

Frames... Alfred... Job Setup... Custom

The Job Setup portion of the RenderMan Globals Spool panel enables you to tune the performance of your rendering job on your network. This subpanel is probably the most powerful and potentially confusing set of controls you've seen in a spell.

Job Setup Overview

Before getting into the specifics of Job Setup, it behooves us to take a look at the big picture. Rendering 3D imagery is a time consuming process requiring fast processors, lots of RAM and big disks. We need lots of expensive machines to compute a sequence of frames in a modicum of time, so the first step in building a studio is to buy enough machines to obtain the desired frame throughput. For some, a single machine is sufficient and for others, 300 machines isn't enough.

As we increase the number of machines in our studio, we also increase the amount of data (images, textures, shaders, models) that must move from artists' machines to the remote rendering servers. Depending on the style and complexity of your imagery, the Asset Distribution part of the process can become the primary bottleneck. In this scenario, you might have lots of fast processors but they spend most of their time sitting around waiting on the network to deliver the pixels or textures. NFS is the most common mechanism for distributing assets. This may not be the best solution because NFS cache sizes are small compared to rendering assets. This can result in the same texture being sent over the network multiple times to the same machine. The recent CacheFS system was designed to alleviate the problem and, used in conjunction with NFS, can be a very effective solution for network rendering.

Another hidden hot spot in the rendering pipeline is the process of model evaluation and RIB Generation. Whether animating complex characters with skeletal deformations or running complex dynamic simulations or even making images of complex models, the process of transferring the data from the modeler to the rendering can take as much or more time that the rendering itself. Intimately connected with this issue is the Asset Distribution issue because it's often the case that the RIB Generation phase involves data amplification. This is the case where a 1 MB model file can turn into hundreds of MBs of RIB files. To minimize network traffic, it's often desireable to perform RIB Generation on the same machine that will render the image.

Once your image is rendered you often want to process it, retrieve it from servers or send it to a recorder. This phase is often intimately connected with the 3d rendering phase because the renderer knows more about the image than just the rgb values at every pixel. Again, network traffic is an important consideration.

Now your frame is complete, it's time to clean up after ourselves. We've just spewed megabytes of data across the network onto remote servers; if we don't clean up after ourselves, we'll quickly fill those tiny 9MB disks on the servers. Sometimes, however, we want to re-use assets to speedup the iteration loop. So we don't alway want to clean up completely.

Add into this tasty stew the case where your system administrators are, well, downright opinionated (you know who you are). You may find these crusty individuals unwilling to add a new NFS mount points or give you logins on remote servers or set up a permanent daemon running on remote machines.

All of these issues turn the apparently simple problem of getting your frames rendered efficiently into a potentially complex one. This panel will help you solve nearly all of them (we can't help you with your sysadmins).

Now to the specifics

Job Settings - A site-configurable collection of job-wide settings for tcl and RenderMan settings. MTOR comes with several default modes, but these can be expanded simply by adding new entries to the tor.ini file. New entries will be populated automatically in the RenderMan Globals. The following is and example of the "Bake" custom spool mode. The "Bake" mode attaches several options and attributes globally, throughout the scene . . . in this case "Bake" changes culling behavior so that surfaces hidden from camera view will still be rendered, which is useful when baking data. "Bake" also sets a couple MTOR and RSL variables. (You'll notice that the "none" option does nothing.)   

SetPref RMCCustomSpoolModes {
	none {}
	bake {
		-tcl {set ::TOR::theGlobalBakeMode write}
		-riopts {
			Option "user" "string bakemode" ["write"]
		}
		-riattrs {
			Attribute "cull" "hidden" 0 "backfacing" 0
			Attribute "dice" "rasterorient" 0
		}
	}

Job Chunk Size - This allows you to control how many RIB files are generated, which in turn determines the granularity of your rendering tasks as seen by Alfred.

• subframe - Subframe will cause separate RIB files to be generated for every computed map and every main pass, for every frame. For example, if you are rendering a 10 frame animation, where each main pass requires two shadowmaps, you will get 30 individual RIB files. You would use subframe if you want to get the quickest turnaround time for a single frame, as this creates a very high granularity job description where Alfred can schedule each computed map to be rendered on a separate processor.
• frame - Frame will cause one RIB file to be generated for each frame. Each frame will contain, in order, all computed maps within.
• superframe - Superframe is the most efficient mode for maximum throughput when using Distributed RIB generation. When doing this form of RIB generation, RIB files for each frame are generated on separate remote machines by starting and running the mtor standalone executable on the remote machine. In frame mode, this executable needs to be run once per frame. This becomes very expensive because the cost of loading the scene as well as doing any necessary particle dynamics run up needs to be incurred once per frame.

  Superframe reduces this cost by having remote machines generate sequenced "chunks" of frames, instead of a single RIB file for a frame. In this way the cost of a scene load and dynamics runup is incurred only once per "chunk", and not once per frame.

Superframe Count - This setting controls the number of frames in each "chunk" when using superframe mode. Larger superframe counts will be more efficient since you incur the cost of scene load and dynamics runup less often. However, your Alfred job granularity will also decrease as a result, which means your CPUs may be tied up for more often.

RIB Generation - This setting controls when and how RIB generation takes place.

• Immediate - All RIB files are generated immediately when you select Render, via the MTOR plugin. This is usually faster than Deferred mode, however this may affect your currently running Maya session since MTOR may need to change the frame counter to properly account for dynamics.
• Deferred - MTOR will submit an Alfred job which will in turn invoke a completely separate standalone mtor process on your machine. This separate process will use a temporarily saved copy of your Maya scene to generate the RIB. Something to keep in mind is that since this is an entirely separate process with its own copy of your scene, you will need twice the memory as you would in Immediate mode; also, unless your machine has multiple CPUs it may adversely impact the performance of your machine. However, your current Maya session will not be otherwise affected (i.e. no changes to the current frame.)
• Distributed - RIB generation is distributed via an Alfred job to remote servers which have the pixarMTOR service key. In order for this to succed, the remote server must have a full RAT installation, and you must be able to run remote jobs via Alfred. In other words, the RemoteCmd Alfred command must be capable of running, which may require either set up of the rsh mechanism, or that Alfserver be installed on the remote server machine. If you are using this mode, you should also consider the use of superframe Job Chunk Size.
• Distrib+Render - Like Distributed, this will cause RIB generation to occur on remote servers; however, it also forces the rendering of the created RIB to occur on the same machine which generated the RIB file. In this case, the renderer used cannot be netrender, it must be render or custom.

RIB Format - RIB files can be in one of four formats. ASCII RIB is in plain text, while binary RIB creates an unreadable (binary) file. Both ASCII and binary RIB can also be gzipped, which is a compression mechanism to reduce the size of the output file into an also unreadable form. The most compact form (binary & gzipped) is recommended unless you wish to read your RIB files; however, note that the catrib utility can be used to convert existing RIB files of any type to their ascii representation.

RIB Style - Changes the way MTOR writes out RIB files. With this setting MTOR can be configured to write out RIB effeciently, archiving as much geometry as possible, by using "Archive Object" mode. This can dramatically accelerate RIB generation times, especially for animated sequences.  

• Flat - RIB files are written in the traditional MTOR fashion with each object specified in the RIB file.
• Archive Objects - All objects in the scene are written out to individual archive RIB files. All references to these objects in the generated RIB files are done through DelayedReadArchive calls. This mode supports lazy acceleration of geometry.

Compute Location - This setting determines whether Alfred will schedule tasks to run on your local machine or on remote processors. In order for the remote setting to work correctly, the RemoteCmd Alfred command must be capable of running, which may require either set up of the rsh mechanism, or that Alfserver be installed on the remote server machine. Furthermore, you will need a strategy for having any necessary resources such as shaders and textures available on the remote machine. This typically occurs via a networked file system such as NFS.

Note that this does not apply if you are using the netrender Renderer setting, nor does it apply if you are using Distrib+Render RIB Generation.

Renderer - This setting gives you control over the choice of renderer used.

• netrender - This is Pixar's standard network capable frontend to PRMan. You should use netrender if you want to have maximum throughput on a single image (since multiple processors can cooperate on that image), or if you want to guarantee that your shaders and textures will be found on remote servers (since shaders and textures will be sent across the network from your local machine as needed).

  Note that netrender ignores the Compute Location, since the netrender client itself must run on your local machine (it does however bind to the NetRenderMan network server on remote hosts). Also, you can separately configure how many processors will be assigned to the netrender task via the Alfred subpanel.

  Remote hosts which are capable of serving your netrender client must have the pixarNRM Alfred service key.

• render - This is Pixar's standard non-network frontend to PRMan. You should use render if you are rendering locally.

  If you are rendering remotely, and if you are interested in maximum throughput on an animation, you should also use render: it is more efficient than netrender over multiple frames since there it does not incur network overhead in sending shaders and textures over the network. By the same token however, you will need to ensure that these resources are available (i.e. via NFS). Also, any remote hosts which you want to render on must have the pixarRender Alfred service key.

• custom - You can use this setting if you want to use another program to handle your rib files. Typically, you would use this to filter rib files through your own program before rendering them, or to send them to a non-Pixar RenderMan compliant renderer. Make sure that you also enter the appropiate information in the Custom subpanel

Imager - Setting this to custom allows you specify an command which will be invoked on each image after rendering. The command you specify will be passed a frame number and the name of the image file as arguments, and can then perform some operation on the image. Typical usages include copying the final image to a disk farm or renaming the image. Make sure that you also enter the appropiate information in the Custom subpanel.

Cleanup - Enabling each option causes that type of temporary file to be removed after the frames has completed rendering. You can use this option in conjunction with the Lazy Compute Acceleration setting to, most commonly, prevent shadows from being recomputed during interactive operation.

The types of temporary files include:

• rib - RIB files
• map - computed maps (generated once per frame)
• tex - converted Pixar format textures (generated once per frame)
• jmap - computed maps (generated once per job)
• jtex - converted Pixar format textures (generated once per job)

Common Settings - This popup menu allows you select from common premade configurations which affect portions of the job setup subpanel.