## Thursday, April 14, 2011

### Writing mental ray shaders: Quantization Part I

Introduction
At this point in the book, Writing mental ray Shaders, it starts with the UV shader, which uses the quantization function. I think the function deserves its own little write up, so we'll divert a little bit and play with the color quantization function.

Quantization
From Wikipedia
Quantization, in mathematics and digital signal processing, is the process of mapping input values that are members of some relatively large set of admissible input values to output values that are members of a smaller countable set of output values. The set of possible input values may be infinitely large, and may possibly be continuous and therefore uncountable (such as the set of all real numbers, or all real numbers within some limited range). The set of possible output values may be finite or countably infinite. A device or algorithmic function that performs quantization is called a quantizer.

The most common type of quantization is known as scalar quantization. Scalar quantization, typically denoted as y = Q(x), is the process of using a quantization function Q( ) to map a scalar (one-dimensional) input value x to a scalar output value y. Scalar quantization can be as simple and intuitive as rounding high-precision numbers to the nearest integer, or to the nearest multiple of some other unit of precision.

Rounding
From Wikipedia
Rounding a number x to a multiple of some specified increment m entails the following steps:
1. Divide x by m, let the result be y;
2. Round y to an integer value, call it q;
3. Multiply q by m to obtain the rounded value z.
$z = \mathrm{round}(x, m) = \mathrm{round}(x / m) \cdot m\,$
Regardless, it is recommended to read through the wikipedia entry for details on rounding numbers.

Type Conversion
From Wikipedia
As we can see, we can use explicit type conversion as a rounding function. Getting all these definitions out of the way, we can start working on our UV as color with quantization shader.