ISSUE 1058

TU

TO

RIA

L

ISSUE 10 59

TU

TO

RIA

L

2D and 3D artists alike tend to be confused in trying to understand exactly what Pan N Tile is. This confusion actually comes with good reason, as it tends to fit somewhere between the 2D and 3D worlds. Pan N Tile (also sometimes called Track N Tile) originated with the idea that some 3D processes could be done in a 2D environment. This then evolved into the development of a 3D system within a 2D software package. One of the first products to take this approach was the Nuke high-end compositing system developed by D2 Software at Digital Domain. The concept of how this system works is actually quite simple. Where it can get confusing is in understanding the best way to create a Pan N Tile, what type of setup to do, and knowing how to make sure that what you have done was set up properly.

Since Pan N Tile is essentially a 3D process, 3D artists tend to have an easier time understanding it. However, for the 2D artist, Pan N Tile may be new territory. With this in mind, its helpful to go over the basicswhat Pan N Tile is, the basic differences between Pan N Tile and Track N Tile, how it works in real-world terms, and where and when it can be most useful in a production pipeline. This article covers those basics as well as the benefits and drawbacks of using a Pan N Tile system to create or re-create an environment, showing you how to set up a simple environment and test to see what resolution you are getting from your imagery once it goes through the 3D camera.

DEFINITIONThe concept of Pan N Tile is pretty straightforward. First, ask

yourself, what constitutes a three-dimensional background. This list can include: shadows, highlights, shading of objects from a light source, and the movement of one object in relation to another (also known as parallaxing or, sometimes, multiplaning). Most of these elements can be handled easily by a 3D package or some by a matte painter. But lets take a closer look at the last item, parallaxing, since it isnt as straightforward as the rest.

A simple example of parallaxing is when you are driving in a car and you watch the stationary objects on the side of the road pass by. As you watch things pass, like signs or a gas station, the farther away the object is from you, the less it appears to move. So, in essence, objects that are different distances away from

each other have the appearance of moving on multiple planes, or multiplaning. It stands to reason that if objects appear to move less the farther they are away from you, then at a certain distance, they will appear to stop moving altogether, and no longer give you the sensation of parallaxing. This is where Pan N Tile comes in.

Simply put, Pan N Tile is when an artist takes a still backgrounda snapshot of a setting sun at the horizon, for example, and animates it with a camera. Objects in the far distance will have no apparent sense of parallaxing, and because in real life, distant objects exhibit the same behavior, you can cheat this with a fl at 2D image. Another way to think of Pan N Tile is that, if a camera move is made up of pure rotations (no translations), then the background content can be exactly mimicked by a two-dimensional image, no matter what the distance from the camera to any pixel in the image. This enables a production pipeline to render a complex scene, have a matte painter augment the renders (or sweeten it), and then add in other non-paintable effects or character elements into the scene without sliding.

USAGEThe example above is simplistic. The 3D system in Nuke v4.5

gives you the power to explore and use this capability in many ways in a production environment. First, with Nuke v4.5 you are actually not limited to a single layer Pan N Tile system. You can re-create an environment that parallaxes by placing cards at different distances from the camera. Just as real world objects at different distances would move at different rates, so do the cards in Nuke. However (and this is a big however), to see the parallaxing, you would have to add translations to your camera. This is known as Track N Tile. Track N Tile works essentially the same way as Pan N Tile, with adding camera translations being the main difference.

You might also use a Track N Tile system when you have a long camera move (like a push-in or pull -out) and you are trying to maintain the resolution of the imagery as you get closer, revealing more detail. In this instance, you can create several different cards with varying amounts of detail that you would actually layer on top of each other, and only as you get closer with the camera do you need to turn on the cards that have the greater detail. So, as you move in, you are actually just revealing a card that, although it represents a small field of view, will have a higher level of resolution, creating the illusion that you are doing a continuous push-in. This technique works until you get to a point when you need to show parallaxing. At that point, you would probably let 3D geometry take over.

Ive used this technique on a few films. Remember The Time Machines scene where the character travels into the future and the camera pulls out from his workshop, into the sky, and eventually all the way to the moon. The shots where you see the detailed city transition to little specs of city lights, to

eventually giant land masses, was all connected via a Pan N Tile system.

Another example is Stealth. During the Big Suck sequence, when the character dives down in the city to drop a bomb onto a building, the cityscape was created with several layers of tiles that were photographed from a helicopter at different altitudes, and then handed off to a 3D building. For the purpose of this tutorial, however, I am going to limit our discussion and breakdown to a single layer, Pan N Tile (no translations) system.

These examples are just a few of the ways a Pan N Tile system can be used for production. The benefi ts in these cases typically come in the form of saved render time, speed, and interactivity. In a traditional pipeline, potentially hundreds of elements/layers would have to be rendered, and then re-rendered for each shot that shared the same background or every time a camera move had changed. The beauty of the Pan N Tile system is that you only have to render out your elements once, then render out your backgrounds for all of your shots from the Pan N Tile in Nuke, which can be done more effi ciently compared to most 3D renders.

The system is not without its weaknesses, and its not in any way designed to completely replace a 3D pipeline. The most common problem with a Pan N Tile system is that, ultimately, you are working with 2D images. If you have a camera move with a lot of translations (especially on more than one axis), then, unlike the real world, where you would see a fair amount of perspective change in your background (that is, different angles of a particular object as the camera moved by), in Pan N Tile the image will ultimately appear flat.

BUILDING A SIMPLE PAN N TILEThe basic process for creating a simple Pan N Tile is to set

up a single card, expand it into a multiple card setup, and apply a camera move to it. For this article, I will focus on setting up a Pan N Tile using photographed content, but explain some of the differences between this type of imagery and painted or 3D rendered sources.

Before you can set up the card (this applies to photographed imagery as well as 3D generated content), you need two key pieces of information: the focal length (or lens) used to photograph (or render) the frame, and the horizontal aperture (which is the physical width on the gate of the camera used to shoot the imagery). You use a standard aperture setting most of the time. A few examples of these are: Vista Vision, Full ap, etc. Note that a Vista Vision camera is 8 perf, which is identical to most still cameras (traditional, not digital), so they both would have the same horizontal aperture. If you are unsure of the horizontal aperture, check the data sheet that comes with your camera. You will not need the vertical number, as Nuke will determine it automatically by calculating the aspect ratio of the image.

Now that you have these two pieces of information, bring in a card object from the 3D menu, and enter the focal length (for our example its 50, and the horizontal aperture is 36.864 (fi gure 1). Next, bring in a 3D transform and connect it to the card. Set its translate on the Z-axis to a value of 1 (fi gure 2). Since you are going to use only the rotations of the camera move, distance doesnt matter. But for simplicity and troubleshooting, its easier if you set your cards exactly 1 unit away from the camera (imagine creating a one meter sphere that surrounds that camera).

Now, change the transformation order in the 3D transform to STR (figure 3 ). You want to be able to have any card you add to your setup move around the camera on this imaginary sphere, and changing the transformation order so that rotations are last will accomplish this. Next, add a camera and

WHATS A PAN N TILEAND WHAT CAN IT DO FOR YOU?

By Brian BegunLead Digital CompositorDigital DomainVenice, California

Figure 1Enter the focal length 50, and the horizontal aperture 36.864.

Figure 2Bring in a 3D transform and connect it to the card. Set its translate on the Z-axis to a value of -1.

Figure 3Change the transformation order in the 3D transform to STR.

Copyright 2007 DMG Publishing HDRI 3D Magazine

ISSUE 1060

TU

TO

RIA

L

ISSUE 10 61

TU

TO

RIA

L

a scanline render node to the script, then connect the render nodes camera input to the camera node and connect its obj/scn input to the 3D transform (figure 4 ).

Before importing or creating a camera move, for testing purposes, set the focal length and horizontal aperture of the camera to match the same values you entered in the card object (figure 5 ). Again, for testing purposes, add a constant color node (leave its color value set to black). Set its resolution so it matches the resolution of the input image that is connected to the card (figure 6 ). Now add a viewer to the render node, and a second view on the same viewer to the input image (figure 7). Toggle between them. You should see absolutely no change between the original input image and the resulting image that was rendered through the Pan N Tile camera. If they match, then you have successfully set up the Pan N Tile. If they dont match, go back and check the settings to make sure the information was input correctly.

By viewing your image through a camera with the same settings the original input image was photographed with and the same resolution, then you can be sure that when you change the Pan N Tile camera and resolution, your setup will operate correctly. At this point, if you are using a painted or 3D rendered background, and you just want to add a camera move to it, then you now have a functioning Pan N Tile setup. With this setup you can now re-photograph the tiles with a new shot camera at any focal length accurately. If, however, you have a photographed background, or have multiple tiles, read on.

Heres where Pan N Tile can get a little more complicated. Say, for example, you take a picture of a tall building, and you want to use it as a texture to map onto 3D geometry. You have a problem because you took the picture from the street looking up. Because you tilted the camera up to take the photo, you introduced some perspective (or keystoneing) into the photo. This would normally be fi ne if, when you go to re-use it as a Pan N Tile background, your camera happens to have the same tilt. But as a 3D texture, this isnt the case. If you dont remove this perspective before re-generating the image with a new camera or project it before rendering it, you will introduce compound perspective, which will not only be harder to troubleshoot, but will look wrong when you try to composite it with another element or render it. Ideally, you want to have imagery that has neutral perspective, whether it be for 3D textures or a Pan N Tile. Keep in mind that it is only possible to remove the perspective of objects at a particular

distance. If you have multiple objects at multiple distances, you wont be able to completely remove all perspective.

If you are dealing with a painted element or a 3D render, you can usually request that the artist either paint with no perspective or render perfectly perpendicular (or orthogonal) to the background. This may be difficult, impractical, or impossible with a photographed imageeither because it was shot hand-held, or the conditions didnt allow for a more controlled setup. Ideally you want to have as little perspective in your input image as possible; however, there is a limit as to how much can be removed.

When building a multiple card setup, start with a tile that was shot as close to perpendicular to the camera as possible. You also want to identify if there are any lines in the imagery that are supposed to be straight, like walls, fl oors, pillars, etc. Any of those elements would be good references to see how much the camera was tilted or panned when they were photographed. Now that you have identifi ed a starting tile, plug it into the card object that you loaded previously, as in Figure 8.

The next step is to bring in a grid node and connect it to the constant color that is plugged into the Bg input of the scanline renderer (fi gure 9). Then bring in a merge node, set the merge type to Plus, connect the A input to the grid, and connect the B side to the scanline renderer (fi gure 10). Connect your viewer to the merge node, and then set the grid settings so they match Figure 11. The reason to lay this grid over the image is to give you a point of reference for straight vertical and horizontal lines. Now adjust the X, Y, and Z rotations so that the lines of the building are straight. To adjust this first card, we only need to adjust Z because it was fairly perpendicular to the camera when it was shot. Once the horizontal part of the building we are using as a guide lines up to the grid, we are ready to work on the next card. Typically, I will build a setup one horizontal row at a time, a process that makes aligning a series of cards easier. For the next card, we will need to pan over the viewing camera about 15 degrees (using the Y rotation) to see where the intersection point of the two cards lives (figures 12 and 13, next page).

Now bring in the next image, add another card and 3D transform, and enter the same values you did when you set up the first card. At this point, you will add one additional node.

Figure 4Add a camera and a scanline render node to the script, then connect the render nodes camera input to the camera node and connect

its obj/scn input to the 3D transform.

Figure 5Set the focal length and horizontal aperture of the camera to match the same values you entered in the card object.

Figure 6Add a constant color node (leave its color value set to black). Set its resolution so it matches the resolution of the input image that is connected to the card. Additionally, you can set the proxy resolution of

your output image by setting it in the Constant color. The proxy resolution can be calculated by the following: 1544 *720/2048.

Figure 7Add a viewer to the render node, and a second view on the same viewer to the input image. Toggle between them. The image in

both views should match.

Figure 8You want to identify if there are any lines in the imagery that are supposed to be straight, like walls, fl oors, pillars, etc. These are good guides to use to adjust your card.

Figure 9Bring in a grid node and connect it to the constant color that is plugged into the Bg input of the scanline renderer.

Figure 10Bring in a merge node, set the merge type to Plus, connect the A input to the grid, and connect the B side to the scanline renderer.

Figure 11This grid over the image is to give you a point of reference for straight vertical and horizontal lines. Using the grid you can adjust the x, y, and z rotations to align the image with the grid.

Copyright 2007 DMG Publishing HDRI 3D Magazine

ISSUE 1062

TU

TO

RIA

L

ISSUE 10 63

TU

TO

RIA

L

Its a 3D merge. This node is used to combine several objects within one scene (figure 14 ).

Using only rotations in the transform of the new card, adjust all three rotation axes until the part of the new card that matches the first card, lines up. This is not a perfect process. Several factors will work against you when building a Pan N Tile using photographed elements. To begin with, photographed imagery has lens distortion and other lens aberrations, which will affect how well you will be able to line up your cards. There are ways to flatten out an image to reduce or remove this, but thats a topic for another article. If you can avoid having the images photographed with a really wide lens, youll avoid some issues, as such photographs will have an increased amount of distortion. Another complication is that since you are tiling flat images, you will be able to align only those parts of the image that are the same distance away from the original camera. So, for example, if you are trying to Pan N Tile two buildings and one is close, while the other is fairly distant, chances are good that only one or the other will line up, but not both since they are at different distances. If you need both of them to line up, you will probably have to create two separate setups, and then composite them together with a soft matte. Once the second tile is lined up, then, most of the time, you will need a soft matte to add the final touch of blending the cards together (figure 15 ). Since 3D renders are exact, you usually can avoid having to do soft blends, or have overlap, or have to deal with lens distortion. Pan N Tiling paintings (especially complicated ones) can be tricky, and can require a different approach.

Now that you have your second card set up, simply repeat the steps on the remaining cards. When you finish adding all of the cards, you should have a setup that looks like Figure 15. Notice that the cards look a little like a dome. If you think about it, it looks a bit like how it was originally photographedalmost like a slice of life.

After completing a Pan N Tile setup, you will want to test to see what resolution your output image will have based on the imagery that is on the 3D cards. In other words, you will want to test to see what kind of output quality you will get with the current lens/horizontal aperture settings you have in your camera. Select a checkerboard node from the image menu, set the square size to 512, and set the resolution to 1536x1024 (basically you want it to have the same resolution as your tiles) (figure 16 ).

Figure 12After adjusting the card values, notice how the architectural features line up with the grid.

Figure 13To make it easier to adjust the cards, we can add the same 17 degree tilt that is in the card to our viewing camera. Also, we will need to

pan over the viewing camera about 15 degrees (using the Y rotation) to see where the intersection point of the two cards lives, to add the

neighboring card.

Figure 14When adding the next card, its easier to start with copying the card and transform from the previous card to the next image, then

adjusting the rotations to move it into place.

Now replace your image tiles with the checkerboard node. NOTE: You can connect all your tiles into this one checkerboard image. Attach a viewer to the scanline renderer. Now count the number of squares in one column and count the number of squares in one row. Finally, multiply the number of squares in the column times 512 and separately multiply the number of squares in the row times 512. The resulting numbers are the pixel resolution of your image. This is based upon the 50mm lens that the camera is currently set to. Try

Figure 15Once the second tile is lined up, then, most of the time, you will need a soft matte to add the fi nal touch of blending the cards together. 3D

renders are the exception. They are exact, which means you can avoid having to do soft blends, or have overlap, or have to deal with lens distortion.

Figure 16Using the checkerboard image by temporarily swapping out your source imagery is a great way to test what your output quality (resolution) will be.

changing the lens to different lengths and then apply the same calculations to both the row and column. Notice that the longer the lens you have (higher number), the lower the resolution gets. Similarly, the resolution goes up when you make the lens wider (smaller number).

Now that youve built your setup and tested it, try experimenting with creating different camera moves (using rotations only) and different focal lengths. Or, you can try applying a camera move you already have (again only using rotations), and put a character or 3D object that was also rendered with the same camera over the Pan N Tile background, and voila! A composited shot.

Currently working as a Lead Digital Compositor at Digital Domain, Brian Begun has almost 10 years experience in Feature Film Digital Effects. Hes worked for several industry leading companies including: Tippett Studios, Rhythm and Hues, Digital Domain, and Pacifi c Ocean Post Digital Film. Brians work can be seen in numerous fi lms including: The Grinch, The Day After Tomorrow, I Robot, Stealth, and Clint Eastwoods new fi lm Flags of Our Fathers.

Copyright 2007 DMG Publishing HDRI 3D Magazine