Wednesday, April 11, 2012

How I write a MEL script - Example Script

We are ready to write the actual script.  Using the template, we add inputs, error check and do the work.  You can download the MEL script and view it while I break it down section by section.

Reminder: We are making a script that will create a matte render suitable for use in Photoshop.  As an extra, I've modified the code so we can also make RGB mattes.

That's a suggestion, not an order.  I use Maya's scriptEditor as my text editor. I type in code and test it by hand by highlighting a line or a group of lines and hitting enter on the number pad.

I know, I know, "How quaint."

This has its pros and cons:
  • Instant results from Maya.
  • Maya 2011 onward ( or was it 2010? ) has syntax highlighting that is updated with each release.
  • The history window is right above the text input, so copy and pasting Maya's commands is convenient.
  • Text editing features are workable enough for me.
  • If Maya crashes and you haven't saved your work, tough luck for you.  ( I have a script to save the MEL tabs and content in a single button push.  Maybe I'll post it someday. ).
  • If you are not careful and have an infinite loop, tough luck for you.  No help from me on this one.
  • Lots of text editors have Maya syntax highlighting.
  • The text editing features in a fuller featured, text editing app can be a time saver.
Really it comes down to idiosyncratic workflow.  Mine is to use the scriptEditor, a co-worker swears by nedit, and only uses the scriptEditor to copy and paste his working code. I suggest to try it all and see which fits you best.

If you open the MEL script in your favorite text editor, you will notice it is much larger than the scriptlet.  When I talked about how error checking should take up at least 50% of your code, the additional code in the Matte Script is mostly error checking.

Game developers used to have an imaginary gamer in mind as they developed a game.  This kid purchased the game right away, beat all the levels and found every Easter Egg.  This helped the developers create ever increasing challenges in games by thinking about how this gamer would break their game.  As a programmer, you need to keep in mind a user who will break your code.  You need to find your own Jessel.

One particular animator I worked with had a habit of finding every possible bug or shortcoming in code that I would write, whereas 100 other users would have no problems.  He did this by attempting to use the code in ways I never intended, inappropriate inputs and all manner of user error.  Over time I would ask myself, "What would Jessel do to break this?"  At first, admittedly, I dreaded the extra coding, but now I realize that it helped me write robust code.

// make random matte colors for selected geo
global proc spaMatteCreate( string $input ) {

Pretty simple stuff, just a quick line to say what it is and the global proc line.  No returns are needed and again only one input.

    // this is where I put in test inputs
    // so I can run examples through the parser below
    string $input = "";
    string $input = "-geo pSphere1";
    string $input = "-user_r 1.0";

    //////// init variables - the defaults
    // anything that the user is intended to input
    // gets a default at the top
    string $geo_ls[];clear($geo_ls);
    string $switch_node = "";
    float $min_val      = 0.0; // lowest value in case user wants 

                               // to define darkest color
    float $max_val      = 1.0; // highest value in case user wants 

                               // to define brightest color
    float $gamma        = 1.0/2.2;  // inverse of 2.2 for linear color workflow
    float $user_r       = 0.0; // user red input
    float $user_g       = 0.0; // user green input
    float $user_b       = 0.0; // user blue input
    int $user_rgb       = 0; // flag for user rgb input
    int $g=0; // geo counter

The very top is part of my debugging scheme.  I make example inputs and run them through the parser ( see below ).  It helps me catch errors in the parser if I'm trying to do anything tricky.  That way I'm not chasing an error from the input down in the work code.  I try to give myself little sanity checks like this throughout the code.  If I have multi line comments or debug only code, I'll use the "/*" "*/" markers rather than a bunch of lines that start with "//".  This way I can highlight the code block in-between the markers rather than each individual line.  Also I can double-click on the line and it will highlight the whole line, ready to be run.  If I had "//" I'd have to select around it.

Any variable that will be used in the entire proc, needs to be returned and/or will receive a user input gets a default.  When I start filling in the variables I keep it small.  As I code and think about the many uses of the code, I'll add an input.  The "user_rgb" and "min" "max" variables were a late addition.  In the process of testing the code, I noticed the colors could get close to black and white, so at first I hard-coded an upper and lower limit, then I decided to make the limit be user defined.  After that, I figured if I'm letting the user define a limit, why not let them decide the actual color.  And suddenly I have 6 more proc-wide variables and 6 more defaults.

The defaults allow me to set a base level.  The user can run the proc with no flags and get what I consider to be an acceptable result.  And because I define them at the top of the code, the user can override the defaults with the input flags which we will parse next.

    //////// parse the input
    string $buffer[]; // holds the input array
    // break apart the input into an array
    int $tok = tokenize( $input, " ", $buffer );
    // iterate through all the pieces of the input
    for( $t=0;$t<$tok;$t++ ) {
        // use switch to look at each piece with case statements
        switch( $buffer[$t] ) {
            // the flag names match the variable names
            case "-geo":
                $t++; // we found a case, so increment the loop counter
                // we need to put the geo into a list
                // we check for a match for anything starting with a "-", 

                // which means it's a flag
                // and we check to see if we've run out of pieces of the input
                while( !`gmatch $buffer[$t] "-*"` && $t<$tok ) {
                    $geo_ls[$g] = $buffer[$t]; // put it into the list
                    $t++; // increment the loop counter
                    $g++; // increment the geo counter
                $t--; // decrement the loop counter 

                      // because we found a fail case above
                $g--; // decrement the geo counter 

                      // because we found a fail case above
                break; // go to the top of the loop
            case "-switch_node":
                $t++; // we found a case, so increment the counter
                $switch_node = $buffer[$t]; // put it into a variable
                break; // go to the top of the loop
            case "-user_r":
                $t++; // we found a case, so increment the counter
                $user_rgb = 1; // turn on the user rgb flag
                $user_r = $buffer[$t]; // put it into a variable
                break; // go to the top of the loop
            case "-user_g":
                $t++; // we found a case, so increment the counter
                $user_rgb = 1; // turn on the user rgb flag
                $user_g = $buffer[$t]; // put it into a variable
                break; // go to the top of the loop
            case "-user_b":
                $t++; // we found a case, so increment the counter
                $user_rgb = 1; // turn on the user rgb flag
                $user_b = $buffer[$t]; // put it into a variable
                break; // go to the top of the loop
            case "-gamma":
                $t++; // we found a case, so increment the counter
                $gamma = $buffer[$t]; // put it into a variable
                break; // go to the top of the loop
            case "-min_val":
                $t++; // we found a case, so increment the counter
                $min_val = $buffer[$t]; // put it into a variable
                break; // go to the top of the loop
            case "-max_val":
                $t++; // we found a case, so increment the counter
                $max_val = $buffer[$t]; // put it into a variable
                break; // go to the top of the loop
            // case default: // I never do this
        } // end input switch
    } // end input parse
We break apart the input with "tokenize" ( note that we could use "stringToStringArray" ).  Then we loop through each piece of the input with "switch".  Each case statement is the flag the user should be passing to us.  I try to use the same flag name as the variable because I am easily confused and this keeps it simple.  Besides, we will make a UI so the user won't have to memorize of our flag names.

Most of the case statements are straight-forward, the exception is the geo list which uses a "while" loop.  "While" and "do-while" are notorious for infinite loops ( for me that is ), so make sure you double check your logic and copy your working code to a file before you run a test.

This line in particular is the strangest:
while( !`gmatch $buffer[$t] "-*"` && $t<$tok ) {
While means that I'm going to loop as long as the stuff in the ( ) is true. The stuff in the ( ) says the the loop can continue if a line DOES NOT match anything with a "-" AND the loop counter is less than the total number of loops.  So in other words, I'm going to be looping as long as we don't have another flag ( they start with a "-" ) and we don't run out of pieces of the input.  Inside the loop I dump whatever the user gives me into the geo list, and increment the loop and geo counters. After the loop completes I need to decrement ( subtract one from ) the loop counter and the geo counter because the "while" exits on a failure aka one too many on the counter.


     /* debug
    print( "// geo list:\n" );
    print $geo_ls;
    print( "// switch node:\n"+$switch_node+"\n" );
    print( "// user_rgb: "+$user_rgb+"\n" );
    print( "// user_r: "+$user_r+"\n" );
    print( "// user_g: "+$user_g+"\n" );
    print( "// user_b: "+$user_b+"\n" );
Debug printouts help in the sanity check process.  I place them in lots of places to spot check the main variables to make sure the data is changing as I expect.  While the code is in WIP mode, I'll remove the top and bottom "/*" "*/".  Then when it's time to release the code, I'll add them.


    //////// error check the input

    // geo
    if( ! size($geo_ls) ) {

        // list is empty so check to see if anything is selected
        $geo_ls = `ls -sl`;

        // remove any switch nodes
        string $temp_ls[] = `ls -type tripleShadingSwitch -sl`;
        $geo_ls = stringArrayRemove( $temp_ls, $geo_ls );

    // switch node
    if( ! size($switch_node) ) {

        // switch node is empty so check for any selected nodes
        $switch_node = stringArrayToString( `ls -type tripleShadingSwitch -sl -head 1`, "" );

        // the proc will function with or without a switch node,
        // so we can continue

    // min/max values - make sure min is lower than max
    if( $min_val>$max_val ) {
        float $temp_min = $min_val;
        float $temp_max = $max_val;
        $min_val = $temp_max;
        $max_val = $temp_min;

As far as error checking goes, this is the simplest.  For the geo list, I'm making sure that if the user didn't send me any geometry by way of the input, that I look for anything currently selected in the Maya scene. I do the same for the switch node, but I limit the scope of the selection to the specific node type.  Last is a check to make sure the minimum is higher than the maximum.

The same code block as Debug Block 1.  This lets me know if the error check is performing as expected.

Now we finally get to the actual working code section.  Up until now we've set some defaults, processed the user input and done preliminary error checking.  This is where the scriptlet grows.  In our example, the scriptlet was 19 lines, while the working code section is 125.  As we go through it piece-by-piece, you'll see how to protect against the end user breaking your code and allow for user-defined variables.

    // iterate through the geo list
    for( $geo in $geo_ls ) {
        // I usually grab the first item in the list 
        // so I can test the code by hand, line-by-line
        // $geo = $geo_ls[0];

Many of my procs tend to iterate over some list through the working code section.  I find that if I will be performing an operation on one object, I'll need to do it on many.

The last three lines that are commented out are another debug/test convenience that I do anytime I have a loop.  I grab the first item in the loop to test the code in the loop by hand.  Since I work in the scriptEditor, once you run the "$geo = $geo_ls[0];" line of code, all occurrences of $geo will have a usable node to test, that is if you ran the geo error check above and had some nodes selected.

      // check to see if the listed geo is a transform or shape
        if( size(`ls -type transform $geo`) ) {
            // grab the shape
            string $temp_ls[] = `listRelatives -ni -s $geo`;
            if( size($temp_ls) ) {
                $geo = $temp_ls[0]; // grab the first shape

We're making sure we have what we want, and that is a shape node.  I'm not requiring the user to hand me a shape node, most likely I will get a transform.  So if it's a transform, I get the first shape.  We're slowly narrowing the scope of acceptable items given by the user.

        // continue only if the geo is a valid shape
        if( size(`ls -type mesh -type nurbsSurface -type subdiv $geo`) ) {

Now that we possibly have a shape node, we don't know for sure because we only looked for one if the top node was a transform, we will only continue if indeed $geo is a poly, nurbs or subd shape.  Error checking to this level ensures we have exactly what we want but will fail gracefully.  Let's say that the user gave us a node that does not exist.  The "ls" will return nothing but will not produce an error, thereby allowing the proc to loop to any other nodes in the list.  That's why I use "size" in this case.  It just is looking to see if anything at all was returned.

            // make a switch node if one is not supplied, selected or connected
            if( ! size($switch_node) ) {
                // no user supplied node, so check for a connected node
                if( size(`listConnections -type tripleShadingSwitch $geo`) ) {
                    // found one
                    string $temp_ls[] = `listConnections -type tripleShadingSwitch $geo`;
                    $switch_node = $temp_ls[0];
                } else {
                    // no node listed or connected, so make one
                    $switch_node = `shadingNode -asUtility tripleShadingSwitch`;

We are getting to the part where we actually make something rather than deal exclusively with error checking.  There of course is error checking built in.
The tripleShadingSwitch is a Maya node that we can connect shapes and output 3 values for each shape.  Rather than having to make a bunch of shaders with different colors, we can make just one shader and have each shape with its own unique color.

I like to think big to small.  The biggest question to ask is if the user has supplied a tripleShadingSwitch node to us.  No?  Then next question is if there is a node already connected to the geometry?  No? Then we are out of questions so we'll make one.

            // check to see if the swtich node is connected to a shader
            string $shader_ls[] = `listConnections -type surfaceShader $switch_node`;
            string $shd = $shader_ls[0];
            string $sg = "";
            if( ! size($shader_ls) ) {
                // make a surface shader
                $shd = `shadingNode -n "matte_shd" -asShader surfaceShader`;
                $sg  = `sets -renderable 1 -noSurfaceShader 1 -empty -name "matte_sg"`;
                // connect the shader to the shading engine
                connectAttr -f ($shd+".outColor") ($sg+".surfaceShader");
                // connect the switch node to the shader
                connectAttr -f ($switch_node+".output") ($shd+".outColor");
            string $temp_ls[] = `listConnections -type shadingEngine $shd`;
            $sg = $temp_ls[0];
            // assign the geo to the shader
            string $par = stringArrayToString( `listRelatives -p $geo`, "" );
            sets -e -forceElement $sg $par;


Now that we have identified the tripleShadingSwitch node, we look for a shader, then connect the shader to the switch node.

Almost the same process as above, but this time because we are not asking the user to supply us with a shader, we skip to checking for a connection.  The reason I do not ask the user for a shader is that I don't want the user to have to jump through a bunch of hoops just to give me information I can find.  If you want your code to be used for a long time, then you have to trim down the information you want the user to supply to you.  Ask for the absolute bare minimum to get the task accomplished.  By keeping the procs focused, then the likelihood of asking for too many inputs is low. If your proc is single purposed, but you're asking for tons of inputs, then you need to rethink your approach.

I like to initialize variables that will only be used inside a loop ( aka. local variables ) right before they will be used for the first time.  A long time ago I would initialize all variables at the very top of the proc, but I found myself scrolling up and down the code to recall what I named each variable, so it wasn't very practical.  But be careful where you initialize a variable, it's only visible to the loop in which it was made, so if I try to use the variable outside of the loop, MEL will return an error.

We check to see if there the switch node is connected to a surface shader, and we toss the first item in the list to the shader variable, no matter what is returned.  We check it for errors below.

Now we take a look at what was returned, if nothing at all was returned, then we can make a shader.  If we have to make a shader, we need to make the shader itself, a shading group, connect them and then connect the tripleShadingSwitch. 

Because we specifically asked MEL to return a list of connected surface shaders, if the size of the variable was not zero we know we have the right type of node, so we don't have to error check the shader type. 

Once we have the shader, we need to find the shading group, or as MEL calls it the shading engine.  Then we get the parent of the geometry and assign the geometry to the shader.

No comments:

Post a Comment