Writing mental ray shaders: Compiling

Environment Setup

Here are whats needed to compile and load the shaders. If in doubt, check the Autodesk MentalRay Technical Documentation . Everything needed is there, its a convoluted process to get it working so read it through carefully. Compiling on linux with gcc seems a more straightforward process, but it might just be because I did not have to install and setup the packages. Here are my notes on setting up the windows environment.

Loading Custom Shaders

I'll start with this, as there might have existing shaders that needs to be loaded. There are two ways. You can put .mi and .dll into the default shader directory

C:\Program Files\Autodesk\Maya2012\mentalray\include
C:\Program Files\Autodesk\Maya2012\mentalray\lib

or

you can specify a path in maya.env found here

C:\Users\equinoxin\Documents\maya\(maya version)

and add this line,

MI_CUSTOM_SHADER_PATH = add a path here.

Put both mi and dll at that location and it will load at startup.

Compiling Shaders

Using visual studio express 2013, free iso download from Microsoft here, Open the command line compiler called "VS2013 x64 Cross Tools Command Prompt" found under Visual Studio Tools. The command line uses DOS commands, so navigate to your source directory. Also make sure you set the correct compiler version with vcvarsall.bat in the visual studio directory. Description and options are here. Depending on your machine and output target, run the correct cl.

Compile

cl /c /O2 /MD /I "C:\Program Files\Autodesk\Maya2012\devkit\mentalray\include" /W3 -DWIN_NT -DBIT64 jc_color_gain.c

cl compiler options can be found here. Here's an explanation of the options above,

1. /c compiles without linking. Note: lower case c.

2. /O2 creates fast code

3. /MD Compiles to create a multithreaded DLL, by using MSVCRT.lib.

4. /I specifies an include library, point this to the directory containing shader.h

5. /W3 sets warning level to 3.

6. -DWIN_NT -DBIT64 sets the compiled target to windows 64 bit.

This will output an obj file. 

Link

link /nodefaultlib:LIBC.LIB /OPT:NOREF /DLL /OUT:jc_color_gain.dll jc_color_gain.obj shader.lib

make sure to copy shader.lib from devkit/mentalray/lib to your visual studio/vc/lib directory.

This will output the dll. 

Declare Shader

Create a .mi file to declare the shader, make sure there's a version and node id statements. 

 version 1

#: nodeid 3002;

C Source File

Make sure there are DLLEXPORT preceding the shader,  version, init and exit shaders. Otherwise the shaders wouldn't work. Under linux, DLLEXPORT evaluated to an empty word. Explanation here.

Resources

MentalRay Technical Documentation

http://docs.autodesk.com/MENTALRAY/2014/ENU/mental-ray-help/files/manual/index.html