Pixel Bender Language 10
Pixel Bender Language 10
Adobe, the Adobe logo, After Effects, Flash, and Pixel Bender are either registered trademarks or
trademarks of Adobe Systems Inc. in the United States and/or other countries. All other
trademarks are the property of their respective owners.
The information in this document is furnished for informational use only, is subject to change
without notice, and should not be construed as a commitment by Adobe Systems Inc. Adobe
Systems Inc. assumes no responsibility or liability for any errors or inaccuracies that may appear in
this document. The software described in this document is furnished under license and may only
be used or copied in accordance with the terms of such license.
Adobe Systems Inc., 345 Park Avenue, San Jose, California 95110, USA.
2
Adobe Pixel Bender Language 1.0
Tutorial and Reference
Introduction
Pixel Bender is a programming language designed for hardware-independent description of
image-processing algorithms. It is designed to compile efficiently for both GPU and CPU back ends,
including multi-core and multiprocessor systems. Since efficient execution on modern high-
performance hardware requires parallel processing, the Pixel Bender programming model is
explicitly parallel.
This document is a tutorial and complete reference for Pixel Bender. The language will evolve; the
features in this document compose a minimal subset that Adobe will support in future versions.
The basic syntax of Pixel Bender should be familiar to any C programmer. This section briefly
introduces the unique features of Pixel Bender programming. It is followed by a complete
language reference.
Kernels
The basic unit of image processing in Pixel Bender is the kernel. Each Pixel Bender program defines
one kernel. A complex image-processing algorithm may require multiple passes using different
kernels; this is described in the Pixel Bender Graph Language Reference.
A kernel is an object that defines the result of one output pixel as a function of an arbitrary number
of input pixels, possibly from different input images. It is executed in parallel over all desired
output pixels, to generate an output image. This parallel model means there are no interactions
between the individual invocations of a kernel function for each output pixel; state cannot be
shared between pixels. This is known as a strict gather model: a kernel gathers multiple input pixel
values to produce a single pixel output.
The simplest Pixel Bender program consists of a kernel that returns a solid color everywhere:
3
<languageVersion : 1.0;>
kernel FillWithBlack
{
output pixel4 result;
void evaluatePixel()
{
result = pixel4(0,0,0,0);
}
}
A kernel is defined like a class in C++, with member variables and functions. Every kernel must
provide at least the evaluatePixel function and at least one output parameter of type pixel. The
kernel above produces one output image with four channels (e.g., red, green, blue, alpha), denoted
by the declaration output pixel4 result. Because a kernel is executed for all pixels of the output
image, each pixel output parameter defines an entire image.
void evaluatePixel()
{
result = color;
}
}
A kernel can take any number of parameters of arbitrary types. The parameters are passed to the
Pixel Bender runtime system, and their values are held constant over all pixels, much like the
“uniform” variables used in 3D shading languages. The application in which a kernel is running will
provide some means of setting the parameters to a kernel.
All Pixel Bender programs must specify the version of Pixel Bender in which they are written, using
the languageVersion statement.
Coordinates
As noted above, a kernel is executed in parallel over all pixels of the output image, with exactly the
same parameter values for each pixel. The only thing that changes at each pixel is the current
output coordinate, accessible through the outCoord function. This function returns a value of type
float2 (a vector of two floats) giving the (x,y) coordinates of the center of the output pixel being
evaluated by the current invocation of the evaluatePixel function. The current output
coordinate varies across pixels of the image but is invariant during the lifetime of any particular
call. The coordinate system is set up so the x axis increases to the right and the y axis increases
downwards.
Pixels are not necessarily square; many video applications use non-square pixels. The size of the
pixels for any input or output image can be found using the pixelSize function.
We can use outCoord to define a kernel that produces a pattern in the output image, such as this
kernel, which renders a (non-anti-aliased) filled circle centered at the origin, with a specified color
and radius:
4
<languageVersion : 1.0;>
kernel RenderFilledCircle
{
parameter float radius;
parameter pixel4 color;
void evaluatePixel()
{
float2 coord_for_this_pixel = outCoord();
float cur_radius = length(coord_for_this_pixel);
if (cur_radius < radius)
result = color;
else
result = pixel4(0,0,0,0);
}
}
This kernel also demonstrates the use of the vector type float2 and the vector function length.
Pixel Bender includes a rich set of vector types and operations; for details, see “Pixel Bender
reference” on page 16.
Note that we have not referenced the size of the output image. In the Pixel Bender model, images
do not have a “size” per se—neither outputs nor inputs. Instead, each image is thought of as being
defined over an infinite plane of discrete pixel coordinates. The run-time machinery that invokes
the kernel determines the actual buffer sizes in which the pixels to be operated on are stored. This
lack of “size” has implications for filter design. For example, it is not possible to write a Pixel Bender
kernel that reflects an image around its “center,” because the center is undefined. Instead, the
kernel must reflect around the origin, or the line of reflection must be denoted explicitly by passing
it as a kernel parameter.
Correspondingly, Pixel Bender has only one, global coordinate system. The coordinate system is
square and uniform (isotropic, cartesian, and orthogonal), and all inputs and outputs are aligned to
it.
Since our images are made up of pixels, we need to know how these pixels are arranged in the
global coordinate system. Each image has an associated pixel size that defines the size of each
pixel in global coordinates. This allows for non-square pixels and effects like downsampling. The
pixel size for a given input or output image can be obtained with the pixelSize function. As a
convenient optimization, there is a method called pixelAspectRatio which returns the pixel
width divided by the pixel height for a given image.
The kernels defined so far have no image inputs. Such a kernel is termed a source or generator. A
practical example of a generator is a procedural texture routine; however, most kernels take one or
more image inputs.
5
Inputs and sampling
Input images to a kernel are declared as parameters of the appropriate image type. Input pixel
values are then accessed by passing the image parameter to one of several sampling functions,
along with a coordinate. The simplest possible, one-input/one-output kernel leaves the image
unchanged:
<languageVersion : 1.0;>
kernel Identity
{
input image4 source;
output pixel4 result;
void evaluatePixel()
{
result = sampleNearest( source, outCoord() );
}
}
The sampleNearest function accesses the given input image, denoted by the image parameter, at
the specified coordinate. As noted above, all input images are in the same coordinate system,
which also is the coordinate system of the output; therefore, sampling an input image at outCoord
returns the pixel value corresponding to (or “underneath”) the current output pixel.
The outCoord function returns coordinates in the global coordinate system, and the sampling
functions accept coordinates in the global coordinate system. To select adjacent pixels, the author
must use the pixel size of the input image. We can implement a kernel that averages each pixel
with its immediate left and right neighbors, as follows:
<languageVersion : 1.0;>
kernel HorizontalAverage
{
input image4 source;
output pixel4 result;
void evaluatePixel()
{
float2 coord = outCoord();
float2 hOffset = float2(pixelSize(source).x, 0.0);
While sampleNearest always returns the value of the pixel whose midpoint is nearest the given
coordinate, the sampleLinear function (which also may be called sample) performs bilinear
interpolation between the four pixels adjacent to the given coordinate. We expect to add calls for
more sophisticated filtering as the language evolves.
A kernel can take any number of input images, each of which can have a different number of
channels. The following kernel multiplies a four-channel image by a single-channel matte:
6
<languageVersion : 1.0;>
kernel MatteRGBA
{
input image4 source;
input image1 matte;
output pixel4 result;
void evaluatePixel()
{
pixel4 in_pixel = sampleNearest( source, outCoord() );
pixel1 matte_value = sampleNearest( matte, outCoord() );
As this example demonstrates, all the various sample functions return a pixel value of exactly the
same number of channels as the passed image.
• When sampling, you must use the pixel size to sample adjacent pixels.
• When doing bilinear sampling, you are sampling over an area of 1 x 1 pixels.
• Unless writing a filter for the explicit purpose of resampling, the pixel size of the first input
image will match the pixel size of the output image. In general, a filter should not try to
both resample and do another task; usually, resampling is done at the end of a chain of
filters, using an explicit resampling node.
• To get accurate results in all cases, you need to use the pixel size.
The pixelSize and pixelAspectRatio functions are available in Flash Player; however Flash
Player always uses 1 x 1 pixels.
Region functions
Region functions are not available when Pixel Bender is used in Flash Player.
In theory, Pixel Bender operates on infinite images; in reality, each image has a finite dimension,
known as the domain of definition or DOD, outside of which it is undefined. Even the domain of
definition may be much larger than the images actually being processed; for example, if rendering
one tile of a much larger document. To support practical and efficient image processing through
an arbitrary sequence of kernels, the run-time system must know which pixels actually take part in
any given computation. This is done through the specification of region functions within a Pixel
Bender kernel.
7
Region functions are optional. If none are supplied, the Pixel Bender run-time system assumes the
kernel is pointwise, meaning it accesses only one input pixel from each input, directly under the
current output pixel. Many kernels are pointwise, including the generators shown in prior sections
and the MatteRGBA kernel. The implication for the Pixel Bender run-time system is that the inputs
need only be exactly as large as the desired output. (The desired output size is a subset of the
infinite image plane, determined by the application, not the kernel writer. It may correspond to the
entire domain of definition—the “whole” image—or a single tile or other sub-region.)
Many useful image-processing operations are not pointwise; for example, the HorizontalAverage
kernel, defined above. This kernel accesses one pixel to the left and one pixel to the right of the
current output pixel. Correctly computing a given output rectangle of pixels requires an input
image exactly one pixel wider on each side than the desired output. If the Pixel Bender run-time
system does not know this, it will not supply extra columns of image data, and the pixels on the left
and right edges of the output region will be evaluated incorrectly. To solve this problem, we must
supply a needed function that tells Pixel Bender how many input pixels are required to compute a
given output region.
void evaluatePixel()
{
float2 coord = outCoord();
float2 hOffset = float2(pixelSize(source).x, 0.0);
region needed(
region output_region,
imageRef input_ref )
{
region result = output_region;
result = outset( result, float2(pixelSize(source).x, 0) );
return result;
}
}
Note the use of the outset function to operate on the region type. Regions cannot take part in
arithmetic expressions in the usual way, but a suite of custom functions is provided for dealing
with them.
8
The needed function always returns a region and takes the following arguments:
• output_region — The desired output region for which the needed region must be
evaluated
• input_ref — A reference to the input image for which the needed region is evaluated.
The output region (also known as the “desired” or “requested” region) is the portion of the output
image’s infinite plane that is to be computed; that is, the portion of the plane for which pixel values
will be evaluated by calling the kernel function. In other words, the output region is the “size” of
the output image that actually will be computed. The purpose of the needed function is to
determine the set of pixels, from each input, that must be accessed to compute all pixels of this
output region. The Pixel Bender run-time system then ensures that all these pixels are evaluated
and available for access through the corresponding image object. This region also is known as the
region of interest or ROI.
A kernel can have arbitrarily many inputs, each of which can access a different needed region for
computation of the same output region. For example, a kernel can transform each input
differently. For this reason, the needed function is passed a reference indicating the input for which
the ROI is to be computed. Since HorizontalAverage has only one input, the reference always
refers to the image named “source” and is ignored here.
The domains of definition of each input also are available during region computation. A needed
function may return any region, but only pixels inside the domain of definition actually are
computed; in other words, the ROI is always clipped to the domain of definition. Knowledge of the
domain of definition can lead to certain optimizations. For example, if a kernel mattes one source
against a mask, only that part of the source inside the mask domain of definition is needed, and
vice-versa. We can use this idea to write a needed function for the MatteRGBA kernel given above:
<languageVersion : 1.0;>
kernel MatteRGBA
{
input image4 source;
input image1 matte;
output pixel4 result;
region needed(
region output_region,
imageRef input_ref )
{
region result = output_region;
// clip to source
result = intersect( result, dod( source ) );
// and to mask
result = intersect( result, dod( matte ) );
return result;
}
void evaluatePixel()
{
pixel4 in_pixel = sampleNearest( source, outCoord() );
pixel1 matte_value = sampleNearest( matte, outCoord() );
9
The built-in function dod is supplied with either an image or an imageRef, and it returns the
domain of definition of that image in the global coordinate system. The needed function is not
required for correct operation of the kernel, as MatteRGBA is pointwise: it accesses only the current
input pixel for each input. Adding the needed function, however, allows the run-time system to
eliminate extra pixels from each input that will simply be masked away.
Finally, the needed function can access other kernel parameters simply by referring to them, much
as a member function can access member variables of the enclosing class. This is useful where the
region of interest varies based on a kernel parameter, such as a variable-size convolution:
<languageVersion : 1.0;>
kernel VariableHorizontalAverage
{
parameter int radius; // Measured in pixels
void evaluatePixel()
{
result = pixel4(0,0,0,0);
float2 hOffset = float2(pixelSize(source).x, 0.0);
region needed(
region output_region,
imageRef input_index )
{
region result = output_region;
result = outset( result, float2( radius * pixelSize( source ).x, 0 ) );
return result;
}
}
region changed(
region input_region,
imageRef input_index )
{
region result = input_region;
result = outset( result, float2( radius * pixelSize( source ).x, 0 ) );
return result;
}
This is not always the case, though. Generally, asymmetric kernels require a reflection between
needed and changed. Kernels that perform a geometric transformation of the image need to
invert the sense of the transform between these two functions.
10
If a changed function is not supplied, changes are assumed to be propagated in pointwise fashion.
This is almost always an error if a needed function must be supplied for correct operation. There are
exceptions, such as the MatteRGBA kernel discussed above, but generally needed and changed
should be supplied as a pair.
void evaluatePixel()
{
float2 coord_for_this_pixel = outCoord();
float cur_radius = length(coord_for_this_pixel);
if (cur_radius < radius)
result = color;
else
result = pixel4(0,0,0,0);
}
region generated()
{
float r = ceil(radius);
return region(float4(-r, -r, r, r));
}
}
A generated function has no required arguments but usually needs to access some kernel
parameters to compute its result. It uses the radius parameter to determine a bounding rectangle
for the output, which is then used to initialize a region object in (leftX, topY, rightX, bottomY)
form.
Although RenderFilledCircle takes no input images, generated may be required even for
kernels that process one or more inputs. Usually, this occurs when the kernel composites a
procedurally rendered object over an input; e.g., a paint stroke or lens flare. The generated
function must return a region that contains the rendered object; otherwise, the run-time system
will not know to expand the output buffer to include the entire object where it extends outside the
borders of the input image.
A kernel that theoretically produces an output over the entire infinite plane, like some types of
procedural texture generation, should return everywhere().
11
Conservative bounds
The region functions are not expected to return exact, pixel-perfect bounds for all cases. Often,
exact bounds are difficult or impossible to compute. Rather, the returned regions must be
conservative bounds: they must completely contain the ideal minimal set of pixels. The effect of
making the bounds too large is potentially wasted storage and computation. This can be very bad
if the bounds are grossly exaggerated, but the effect of making the bounds too small is worse:
incorrect computation.
• If the needed region is too small, needed pixels are not supplied, and portions of the input
image(s) are incorrectly represented to the kernel as black.
• If the changed region is too small, the domain of definition may not include all image data
(resulting in clipped images), and bad caches may be incorrectly held as valid.
• If the generated region is too small, the domain of definition may not include all image
data, resulting in clipped images.
There are cases where the true bounds of a result depend on image data; even here, bounds must
be conservative. The classic example of this is a displacement-map effect, which transforms an
image based on the pixel values of an input map. To express this effect in Pixel Bender, a maximum
possible displacement limit must be set—say, according to a user-controllable parameter—and
this limit used as a conservative radius in bounds computation.
Dependent values
Dependent values are not available when Pixel Bender is used in Flash Player.
Consider a convolution operation with a procedurally generated look-up table of pixel weights:
<languageVersion : 1.0;>
kernel GaussianBlur
{
input image4 source;
output pixel4 result;
void evaluatePixel()
{
const float sigma = 2.0;
float weight0;
float weight1;
float weight2;
float weight3;
float weight4;
This code works but has a problem: the look-up table is regenerated on every pixel. That negates
the point of a look-up table. We want to pre-compute this array once, then apply it to all pixels. We
12
could pass in the array as a kernel parameter, but then we would need code elsewhere to compute
it. Pixel Bender solves this problem through the use of dependent member variables and the
evaluateDependents function. Values declared as dependent are initialized within the body of
evaluateDependents, which is run once; thereafter, these values are considered read-only and are
held constant over all pixels as evaluatePixel is applied. The kernel above can be modified to run
this way, as follows:
<languageVersion : 1.0;>
kernel GaussianBlur
< namespace : "AIF test";
vendor : "Adobe";
version : 1;
>
{
dependent float weights[ 5 ];
void evaluateDependents()
{
const float sigma = 2.0;
void evaluatePixel()
{
// use weights here as read-only data
}
}
Pixel Bender 1.0 imposes several restrictions on dependent arrays. First, only arrays of floats are
supported. Also, all array sizes must be constants. Future versions of Pixel Bender will allow arrays
to be declared with a size based on kernel parameters, which will enable parameter-dependent
look-up table sizes.
Support functions
Support functions are not available when Pixel Bender is used in Flash Player.
Function libraries
Function libraries are not available when Pixel Bender is used in Flash Player.
To allow code re-use, Pixel Bender supports the definition of libraries of functions:
13
<languageVersion : 1.0;>
library Math
{
// Finds the real roots of a quadratic equation
// Returns the number of real roots
int quadratic(
float a,float b,float c,
out float root1,out float root2 )
{
int nRoots = 0;
float b2 = b * b;
float t = 4.0 * a * c;
if( b2 > t )
{
// Two roots
nRoots = 2;
float t1 = sqrt( b2 - t );
root1 = (-b + t1) / ( 2.0 * a );
root2 = (-b - t1) / ( 2.0 * a );
}
else if( b2 == t )
{
// One root
nRoots = 1;
root1 = -b / ( 2.0 * a );
}
return nRoots;
}
}
A library function does not have direct access to any parameters or dependents, although they can
be passed in as arguments.
A kernel or library can use a library by importing it and then calling its functions using the scoped
function call syntax:
<languageVersion : 1.0;>
kernel SmearedBlur
{
import Math;
input image4 src;
output pixel4 dst;
void evaluatePixel()
{
float a, b, c, root1, root2;
// Do something to set up a, b and c here
Libraries can also import other libraries; however, recursive importing of libraries is not allowed.
Like kernels, libraries must specify the version of Pixel Bender in which they are written, using the
languageVersion statement.
14
A more complex example
RotateAndComposite is a kernel that transforms one input and composites it on top of another. It
demonstrates the use of dependents to hold a pre-computed transformation matrix, multiple
inputs that are treated differently, and more complex region functions. Matrices are stored in
column-major order; for more detail, see “Pixel Bender reference" on page 16.
<languageVersion : 1.0;>
kernel RotateAndComposite
{
parameter float theta; // rotation angle
parameter float2 center; // rotation center
// rotate by theta
float3x3 rotate = float3x3(
cos(theta), sin(theta), 0,
-sin(theta), cos(theta), 0, 0, 0, 1 );
15
// needed function is different depending on which input region needed(
region output_region,
imageRef input_index )
{
region result;
Program structure
Each Pixel Bender program is specified by one string, containing a languageVersion statement
and one kernel.
where value is a floating-point value corresponding to the version of Pixel Bender in which this
program is written.
16
Kernels
A kernel contains:
• An evaluatePixel function
• Constant definitions
• Other functions
Names are text strings with no embedded spaces. Values are constants and must be integers or
strings delimited by double quotation marks; no other types are supported for the values.
kernel Sample
<
namespace : "Adobe Image Foundation";
vendor : "Adobe";
version : 1;
description: "My Filter";
>
{
void evaluatePixel()
…
}
17
A kernel_member is defined as follows. Details are in the next section.
[import_statement] | [kernel_parameter] | [kernel_dependent] |
[kernel_constant] | [input_image] | [output_pixel] | evaluatePixel_function |
[evaluateDependents_function] | [needed_function] | [changed_function] |
[generated_function] | [other_functions]
The C-preprocessor directives #define, #undef, #ifdef, and #if are provided to support
conditional compilation. We recommend const values for constant definition.
Kernel members
import_statement
An import_statement has this syntax:
import library_name;
All functions contained in the given library are then available to be called using the scoped
function call syntax.
kernel_parameter, kernel_dependent
A kernel_parameter or kernel_dependent has this syntax:
qualifier type name;
The qualifier specifies how the variable is set and used. It is required, and it must be one of the
following:
• parameter — Parameters are set before a kernel is executed and are read-only to kernel
functions. Parameters can be of any type except image and region. Float arrays used as
parameters must have sizes determined at compile time.
• dependent — Dependent variables are accessible within any kernel function, but can be
written to only in the evaluateDependents function. Float arrays used as dependents
must have sizes determined at compile time.
A kernel_parameter can have metadata attached to it. This metadata is made available after the
compilation and can help the client program understand what the parameter is for and what
reasonable values it can take.
If specified, metadata immediately follows the name of the variable and comprises a series of
name-value pairs enclosed in angled braces:
parameter type name
<
name1 : value1;
name2 : value2;
...
>;
18
The names are strings. The values are constants of any valid Pixel Bender type (for ints, floats and
bools the type is deduced automatically, for other types specify a constant of the correct type e.g.
float2( 1.0, -1.0 ) ), or a string delimited by double quotation marks; for example:
kernel_constant
A kernel constant has this syntax:
const type name = compile_time_expression;
input_image
An input image has this syntax:
input type name;
output_pixel
An output pixel is defined as follows:
output type name;
evaluatePixel_function
The evaluatePixel kernel function defines the processing to be performed, in parallel, at each
pixel. The evaluatePixel kernel function (and all functions that it calls) has read-only access to all
parameters and dependents, read access to all input images, and write access to all output pixels.
evaluatePixel is declared as follows:
void evaluatePixel()
{
statements
}
evaluateDependents_function
The evaluateDependents function also takes no arguments. dependent variables can be written
to only during execution of the evaluateDependents function or in functions called by
evaluateDependents during its execution. evaluateDependents is declared as follows:
19
void evaluateDependents()
{
statements
}
needed_function
A needed function returns a region. The function takes two fixed arguments. All region functions
can read parameters and dependents, but they cannot sample image inputs (although they can
call the dod function on images) and must return a region. needed is declared as follows:
region needed( region name, // requested output region
imageRef name) // reference to an image
{
statements
}
changed_function
A changed function is similar:
region changed(
region name, // changed input region
imageRef name ) // reference to an image
{
statements
}
generated_function
A generated function has no arguments, only a region return value:
region generated()
{
statements
}
other_functions
An arbitrary number of additional kernel functions can be defined. All of these take access
restrictions from their calling parent (i.e., only functions called from evaluateDependents can
write to dependent variables). The syntax is as follows:
return_type name([arguments])
{
statements
}
The out qualifier indicates that the argument essentially is a return value, uninitialized upon entry
to the function. The inout qualifier is similar but is initialized to the caller’s value on entry;
essentially, it is a pass-by-reference mechanism. Both must be supplied with values at the call site.
Otherwise, all arguments are passed by value. The default qualifier is in.
20
All functions are overloaded; that is, matched by argument types as well as names. Unlike C++, no
implicit type conversion is performed when matching overloaded functions. All functions must be
defined before calling; there are no forward declarations. Pixel Bender does not support recursive
function calls. Functions can be named according to the usual C conventions. All functions names
that start with an underscore (_) are reserved and cannot be used.
Statements
The only flow-control statements available when Pixel Bender is used in Flash Player are if and
else.
The following flow-control constructs are supported in Pixel Bender, with the usual C syntax:
if (scalar_expression) true_statement
if (scalar_expression) true_statement else false_statement
for (initializer; condition; incremental) statement
while (condition) statement
do statement while (condition);
break;
continue;
return expression;
Note that in evaluatePixel and functions called from evaluatePixel, Pixel Bender does not
support return statements inside the body of a conditional statement or loop.
Variables can be declared anywhere inside a function and have scope inside the enclosing set of
braces. As in C++, variables also can be declared inside the initializer of a for loop or the
conditional test of a while loop, but not within the conditional test of an if statement. Variables
can hide other variables of the same name in more outer scopes. The const qualifier can be
applied only if expression is a compile-time constant. Variables can be named according to the
usual C conventions. All variable names starting with an underscore (_) are reserved and cannot be
used.
As in C, a statement also can be a sequence of statements of the types above, inside braces:
{
statement
[statement...]
}
Types
Pixel Bender is strongly typed. There are no automatic conversions between types, with the single
exception of integral promotion during construction of floating-point vector and matrix types.
There are several classes of types, each defined over a particular set of operators and intrinsic
functions.
21
Scalar types
Pixel Bender supports bool, int, and float as basic types.
The int type must have at least 16 bits of precision (not including the sign), but an
implementation can use more than 16 bits. An implementation can convert an int to a float to
operate on it. When the result of an int operation (including a conversion from float) cannot be
represented as an int, the behavior is undefined.
The float type matches the IEEE single-precision floating-point definition for precision and
dynamic range. The precision of internal processing is not required to match the IEEE floating-
point specification for floating-point operations, but the guidelines for precision established by the
OpenGL 1.4 specification must be met.
A pixel type also is supplied, used to represent the value of one channel of an image:
pixel1
Note that the single-element pixel type is called pixel1, as opposed to simply pixel, to avoid
ambiguity: a single “pixel” usually is thought of as containing multiple channels. All bool, float,
int, and pixel types can participate in arithmetic operations; however, pixel arithmetic is special,
because the pixel type is somewhat abstract. Depending on compilation options, pixel channels
may be 8-bit integers, 16-bit integers, or 32-bit floating-point values. Arithmetic rules vary in each
of these formats; for example, in the 8-bit case, multiplication of two pixel types includes an
implicit division by 255.
bool, float, and int can be converted from one to another, using the usual C-style truncation and
promotion rules, with the following cast syntax:
type(expression)
For example:
int a = int(myfloat)
The pixel type can be converted only to (or from) float. Conversions from integer pixel types
produce floats in the range [0…1], whereas conversions from floats to integer pixel types are
implicitly rounded and clipped to [0…255] or [0…32768]. The underlying pixel arithmetic radix
essentially is a compile option and is unavailable to the kernel implementer.
All the usual arithmetic operators are defined over the scalar types, as described in “Operators” on
page 26. section.
Vector types
Pixel Bender supplies 2-, 3-, and 4-element vectors of the following scalar types:
float2 bool2 int2 pixel2
float3 bool3 int3 pixel3
float4 bool4 int4 pixel4
Vector types, including pixels, can be initialized using the following constructor syntax:
vector_type(element1 [, element2…])
22
For example:
float3(0.5, 0.6, 0.7)
This expression results in a value of the named type, which can be assigned to a variable or used
directly as an unnamed result.
• r,g,b,a
• x,y,z,w
• s,t,p,q
Pixel Bender also allows “swizzling” to select and re-order vector elements. For a vector value with
n elements, up to n indices from (r,g,b,a,x,y,z,w,s,t,p,q) can be specified following the dot operator.
The corresponding elements of the vector value are concatenated to form a new vector result with
as many elements as index specifiers. This syntax can be used to re-order, remove, or repeat
elements; for example:
float4 vec4;
Index specifiers also can be applied to variables on the left side of an assignment. In this case,
indices cannot be repeated. This functionality is used to implement write-masking. The correct
number of elements must be supplied on the right-hand side.
float3 vec3;
float2 vec2;
There is a potentially troublesome interaction between swizzling and the assignment operations.
Consider the following expression:
g.yz *= g.yy;
23
A naive expansion of this would look like the following:
g.y *= g.y;
g.z *= g.y;
The problem with this is that the value of g.y used in the second expression has been modified.
The correct expansion of the original statement is:
float2 temp = g.yz * g.yy;
g.yz = temp;
That is, the original value of g.y is used for both multiplications; g.y is not updated until after both
multiplications are done.
Conversions between vector types are possible using the scalar syntax described above, provided
the dimensions of the vectors are equal:
float3 fvec3;
int3 ivec3;
fvec3 = float3(ivec3);
Most scalar arithmetic operators are defined over vectors as operating component-wise. For
details, see “Operators” on page26.
Matrix types
The following matrix types are available:
float2x2
float3x3
float4x4
Matrix values can be generated using constructor syntax from float vectors describing the column
values, or floats indicating each element in column-major order, or a mixture of vectors and floats:
float2x2( float2, float2 )
float2x2( float, float,
float, float )
Matrices also can be initialized from a single float, which defines the elements on the leading
diagonal. All other elements are set to zero.
float2x2( float )
float3x3( float )
float4x4( float )
24
Matrix elements are accessed using double subscripts, column first:
matrix[ column ][ row ]
If the row subscript is omitted, a single column is selected, and the resulting type is a float vector of
the appropriate dimension:
matrix[ column ]
A small set of scalar operators are defined for matrices, which perform component-wise,
matrix/matrix, and matrix/vector operations. For details, see “Operators” on page26.
Region type
The region type is declared as follows:
region
A rectangular region can be constructed from a float4 representing the left, top, right, and
bottom bounds:
region( float4_bounds )
There are no operators defined for regions; instead, regions are manipulated through a set of
specialized functions (see below).
Image types
Pixel Bender 1.0 supports images of up to four channels.
image1
image2
image3
image4
Images cannot be constructed or used in expressions; however, they can be passed as arguments
to user-defined functions or passed as an argument to the dod function.
The imageRef type exists to allow the needed and changed functions to select the input image for
which they are being run. There are only two uses for an imageRef variable:
Arrays
Arrays are not available when Pixel Bender is used in Flash Player.
Pixel Bender 1.0 has limited support for arrays. Only the following arrays are allowed:
25
“Constant size” means the array size is a compile-time constant. There is no way for the size of the
array to depend on a kernel parameter. (Future versions of Pixel Bender will allow arrays to be
declared with a size based on kernel parameters, which will enable parameter-dependent look-up
table sizes.)
The result of accessing an array with a subscript less than 0 or greater than the declared size minus
1 is undefined.
Void
Functions that do not return a value must be declared with the void return type. There is no other
legal use for void within Pixel Bender.
Operators
Pixel Bender defines the following arithmetic operators over the scalar types, with their usual C
meanings, in order of highest to lowest precedence:
. Member selection
* / Multiply, divide
+ - Add, subtract
== != Equality
^^ Logical exclusive or
|| Logical inclusive or
?: Selection
= += -= *= /= Assignment
26
When Pixel Bender is used in Flash Player, the selection operator can be used only to select
between two constants or variables. You cannot place a general expression on the right-hand side
of the selection.
Regardless of whether short-circuit evaluation is in operation for “logical and,” “logical inclusive or”
is undefined. If you require short-circuit evaluation to be present (or absent), you must explicitly
code it.
The standard arithmetic operators (+, -, *, /) also can be used with combinations of vectors,
matrices, and scalars. If a binary operator is applied to two vector quantities, the operation behaves
as though it were applied to each component of the vector; for example:
float3 x, y, z;
z = x + y;
is equivalent to:
z[ 0 ] = x[ 0 ] + y[ 0 ];
z[ 1 ] = x[ 1 ] + y[ 1 ];
z[ 2 ] = x[ 2 ] + y[ 2 ];
is equivalent to:
x[ 0 ] = y[ 0 ] * w;
x[ 1 ] = y[ 1 ] * w;
x[ 2 ] = y[ 2 ] * w;
Important exceptions to this component-wise operation are multiplications between matrices and
multiplications between matrices and vectors. These perform standard linear algebraic
multiplications, not component-wise multiplications:
27
Built-in functions
Pixel Bender supports a variety of built-in functions over different types.
Mathematical functions
As with arithmetic operators, mathematical functions can be applied to vectors, in which case they
act in a component-wise fashion. Unless stated otherwise, all angles are measured in radians.
28
float exp( float x ) Returns ex.
float2 exp( float2 x )
float3 exp( float3 x )
float4 exp( float4 x )
If x > 0, returns 1
29
float min( float x, float y ) If x < y, returns x, otherwise
float2 min( float2 x, float2 y )
float3 min( float3 x, float3 y ) returns y.
float4 min( float4 x, float4 y )
30
Geometric functions
These functions operate on vectors as vectors, rather than treating each component of the vector
individually.
Since * performs algebraic matrix multiplication, the following function is provided to perform
component-wise multiplication:
matrixCompMult(float2x2 x, float2x2 y)
matrixCompMult(float3x3 x, float3x3 y)
matrixCompMult(float4x4 x, float4x4 y)
The following functions compare vectors component-wise and return a component-wise boolean
vector result of the same size:
31
bool4 greaterThanEqual(int4 x, int4 y) x >= y
bool2 greaterThanEqual(float2 x, float2 y)
bool3 greaterThanEqual(float3 x, float3 y)
bool4 greaterThanEqual(float4 x, float4 y)
Region functions
The opaque region type is manipulated through the following set of functions:
region nowhere() // empty region
region everywhere() // "infinite" region
region transform( float2x2 m, region r )// linear transform
region transform( float3x3 m, region r )// affine transform
region union( region a, region b)
region intersect( region a, region b )
region outset( region a, float2 amount )// expand each edge
region inset( region a, float2 amount ) // contract each edge
float4 bounds( region r ) // returns (leftX,topY,rightX,bottomY)
bool isEmpty( region r ) // returns true if the region is empty
32
Sampling functions
Each sampling function takes an image of a particular number of channels and returns a pixel with
the same number of channels. Pixel Bender supports several sampling calls:
pixel1 sample( image1 im, float2 v )
pixel2 sample( image2 im, float2 v )
pixel3 sample( image3 im, float2 v )
pixel4 sample( image4 im, float2 v )
pixel1 sampleLinear( image1 im, float2 v )
pixel2 sampleLinear( image2 im, float2 v )
pixel3 sampleLinear( image3 im, float2 v )
pixel4 sampleLinear( image4 im, float2 v )
pixel1 sampleNearest( image1 im, float2 v )
pixel2 sampleNearest( image2 im, float2 v )
pixel3 sampleNearest( image3 im, float2 v )
pixel4 sampleNearest( image4 im, float2 v )
sample and sampleLinear are identical; they handle coordinates not at pixel centers by
performing bilinear interpolation on the adjacent pixel values. sampleNearest performs nearest-
neighbor sampling. All pixels outside the image’s domain of definition are treated as transparent
black.
Intrinsics
Pixel Bender includes a few functions that allow access to the system’s compile- or run-time
properties.
float2 outCoord()
This call can be made only within the evaluatePixel function or a function called by the
evaluatePixel function. It returns the coordinate of the midpoint of the output pixel currently
being evaluated, as an (x,y) pair within a float2 object.
region dod( image1 )
region dod( image2 )
region dod( image3 )
region dod( image4 )
region dod( imageRef )
This call can be made only within the needed and changed functions. It returns the domain of
definition of the supplied image.
The pixel size and aspect ratio can be obtained from input and output images, using the following
functions:
float2 pixelSize( image1 )
float2 pixelSize( image2 )
float2 pixelSize( image3 )
float2 pixelSize( image4 )
float2 pixelSize( pixel1 )
float2 pixelSize( pixel2 )
float2 pixelSize( pixel3 )
float2 pixelSize( pixel4 )
33
float pixelAspectRatio( pixel2 )
float pixelAspectRatio( pixel3 )
float pixelAspectRatio( pixel4 )
Note that:
pixelAspectRatio( i ) == pixelSize( i ).x / pixelSize( i ).y
Preprocessor
A C-style preprocessor is available with the following keywords:
#if #ifdef defined
#endif #elif #define
Predefined symbols
Several predefined preprocessor symbols are available at compile time. If the compilation is done
on a machine with an ATI graphics card, every Pixel Bender program acts as though it is prefixed by
the following lines:
#define AIF_ATI 1
#define AIF_NVIDIA 0
If the compilation is done on a machine with an nVidia graphics card, the prefix looks like this:
#define AIF_ATI 0
#define AIF_NVIDIA 1
Both symbols always are defined. To mark card-specific code, use #if, not #ifdef.
If the compilation eventually will result in the production of byte code for Flash Player, every Pixel
Bender program acts as though it is prefixed by the following line:
#define AIF_FLASH_TARGET 1
If the compilation is for a non-Flash Player target, every Pixel Bender program acts as though it is
prefixed by the following line:
#define AIF_FLASH_TARGET 0
Implementation issues
Pixel Bender supports only 3- or 4-channel output images.
34
Summary of Flash Player restrictions
• pixelSize always returns (1.0, 1.0).
• The selection operator (?:) can be used only to select between two constants or variables.
35