View this PageEdit this PageUploads to this PageVersions of this PageHomeRecent ChangesSearchHelp Guide

Flash Textures

Interactive Flash Textures for Director 3D

Interactive, Animated Flash (SWF) Textures for Director 3D

Non-Interactive Flash textures too!

I set out to make Flash textures for 3D models in Director as part of DART. That means my methods had to be generalizable to any application, would end up being run in developer mode, and might even use an HMD with no mouse or keyboard for interaction. On the way to my final version, I discovered lots of different ways to do it but no single resource that explained it all. I hope this page can be that resource. I'll try to step you through how to decide what's best for you and I've provided Lingo and ActionScript source code. I used MX2004 versions of both Director and Flash. If you find this useful or find a mistake, please let me know! DART uses Methods 3 and 4 defined below because they're simply the most robust. You may be able to get away with interactive textures that are simpler than Method 3 but there are tradeoffs...
-- dave mccolgin, December 2004

Methods for Creating Animated SWF Textures
Interactive
Method When to Use
Make the SWF invisible and move it under the cursor according to the cursor position over the model You need robust interaction with the texture. The SWF can be as complex as you'd like and the interaction is smooth. This method works well in Shockwave, but can be difficult in projector and developer modes.
Move an invisible MIAW (movie-in-a-window) under the cursor that contains the SWF You have interactions like rollovers, and ActionScript as complex as you'd like but no response to clicks or keyboard. This gets around some of the problems above but sacrifices some interaction.
Put the SWF in a MIAW and have Director send it coordinates, mouse and keyboard events that ActionScript interprets You have DirectorMX2k4. You need interaction including clicks or keyboard events, but you're not publishing on the web. The methods to do this are more involved but it doesn't suffer the problems of the first 2 methods.
Non-Interactive
Method When to Use
Import the SWF into your cast, increment its posterFrame and take the member's image This is the easiest, most problem-free method of all. Use it when you have single-layer looped animations with no ActionScript.
Put the SWF on stage, take the stage image at the SWF sprite's rect Your SWF has multiple layers and/or some scripts. It's more than a tweened loop but has no interaction. This will work in Shockwave with some tweaks.
You could put the SWF in a MIAW... If your SWF is really non-interactive, there's no need to do this. Just stick it on the stage behind the 3Dworld. If you want to for some reason N.B. it won't work in Shockwave.

Method 1: Make the SWF invisible and move it under the cursor according to the cursor position over the model

Import the SWF into your cast and put it on stage (either at runtime with lingo or beforehand). I'll call it the flashSprite. Check its properties and make sure it is not direct-to-stage. That will allow it to be on top of the 3Dworld but invisible. Now you have to tie the model's texture to the image of the flashSprite and keep updating it to make it "animated." I put all of the following code on the the 3Dworld sprite. First, on beginSprite, create an image the size of the flashSprite and tie the texture to that image:

pImage = image(flashSprite.width,flashSprite.height,32)
--pMember is a reference to the cast member of the 3Dworld sprite
pTexture = pMember.newTexture("Flash Texture", #fromImageObject, pImage)
--pShader is the shader for whatever model you want to be interactive--use an existing shader if you want
pShader = pMember.newShader("flash_shader",#standard)
pShader.texture = pTexture

Now, on prepareFrame (or whenever you want) update pImage with the newest image of the flashSprite, and update pTexture:

pImage.copyPixels((the stage).image,pImage.rect,flashSprite.rect)
pTexture.image = pImage

Test it out, you now have an animated texture. Cool, huh? But not yet interactive. First, you have to detect whenever the cursor is over a 3D model.

pOffset = (the mouseLoc - (point(sprite(me.spriteNum).left, sprite(me.spriteNum).top)))
m = sprite(me.spriteNum).camera.modelsUnderLoc(pOffset, 1, #detailed)

'm' will be empty if you're not over any model. For this example, let's assume that you have only one model. That means if 'm' isn't empty, you're over the textured model, but if you have multiple models just test to make sure it's the right one. Now you need to translate the #uvcoords of the cursor over the texture into corresponding 2D coordinates on the source (the flashSprite). The method mMapPointToTexture does this. It was written by James Newton in 2001, and its full text is at the very bottom of this page. Finally, you move the flashSprite such that the cursor is over the same point on it as it is over the texture.

if m<>[] then

-- get 2D coords from the 3D #uvcoords
uv=mMapPointToTexture(m[1],point(sprite(flashSprite).width,sprite(flashSprite).height))
-- move the flashSprite
if not voidP(uv) then sprite(flashSprite).loc = (the mouseloc - uv) + sprite(flashSprite).member.regpoint

else

--there's no model under the cursor. move the flashSprite somewhere out of the way
--let's say there's enough whitespace between the stage origin and your 3Dworld to stick it at 0,0

sprite(flashSprite).loc = point(0,0)

end if

Ok, you're done. You now have an 'interactive texture.' Well, none of these are really interactive; in this case you're actually interacting with the flashSprite, you just can't see it because it isn't direct-to-stage (you remembered to turn off DTS, right?). But there are some problems you'll run in to. In order for copyPixels to return an image of the flashSprite, it has to be positioned on the screen (screen in developer mode, stage in projector mode). But because you're moving the sprite around, part of it may go off the stage/screen as you move over the texture. Also, you have to shove the flashSprite outside of the 3Dworld when you're not over the textured model or else it will continue to receive mouse events. Trust me, even if you dynamically swap its z-order it will still receive rollovers etc. So why is this bad? Well, any part of the flashSprite that's not in the viewable area, whether it's on the stage or not, will not get copied. If it's half-on, half-off, then the half that's visible will continue animating but the other half will not be replaced. Try scrolling your stage window around while the movie is playing to clip the flashSprite, you'll see. Is there a solution? Yes and no. In developer or projector modes, you'll have to have a visible stage buffer the size of the flashSprite all the way around the 3Dworld, and live with the fact that the flashSprite will be visible when it's in that buffer or use another method described on this page.

If you're publishing to the web, you can do the same but crop the excess stage using the web's document object model. Let's say your 3Dworld is 1024x768 and your flashSprite is 512x512. It can still look ideal in a web page by putting it in the following <div> element:

<div id=disp style="position:absolute; width:1536; height:1536; top:-512px; left:-512px; clip: rect(512px, 1024px, 1024px, 512px); ">

Alexandre Labedade came up with that solution here in December 2004. Working web example here, remember to right-click and drag to show the 3D. Unfortunately, I've found that this method is only interpreted correctly by Internet Explorer.


Method 2: Move an invisible MIAW (movie-in-a-window) under the cursor that contains the SWF

Director Help will tell you there's no such object as a MIAW, but it's a useful term. In this method, you'll create a small Director movie that simply contains the Flash SWF, and make them the exact same size. Then back in your main Director movie (the one with the 3Dworld), you'll bring up that first movie in a new window (hence MIAW) without ever making it visible, and move it around under the cursor such that the cursor is over the same point on the SWF in the MIAW as it is over the texture on the model. Before we begin, remember that because we never open() the window, it cannot receive clicks or keyboard input. Also, MIAWs cannot be used in Shockwave. On the other hand, it doesn't suffer the clipping problem that Method 1 does. So first, on the beginSprite of the 3Dworld:

pFlashWindow = window().new("flash")
pFlashWindow.fileName = "yourMIAW.dir"

As in Method 1, create the texture for your model, and set it to an image we'll repeatedly fill with the SWF, only now that SWF is in a MIAW.

Mwidth = pFlashWindow.rect[3]-pFlashWindow.rect[1]
Mheight = pFlashWindow.rect[4]-pFlashWindow.rect[2]
MbaseRect = rect(-Mwidth,-Mheight,0,0)
Mcenter = point( (Mwidth/2),(Mheight/2) )
pImage = image(Mwidth,Mheight,32)
--pMember is a reference to the cast member of the 3Dworld sprite
pTexture = pMember.newTexture("Flash Texture", #fromImageObject, pImage)
--pShader is the shader for whatever model you want to be interactive - use an existing one if you have it
pShader = pMember.newShader("flash_shader",#standard)
pShader.texture = pTexture

Now, on prepareFrame (or whenever you want) update pImage with the newest image of the pFlashWindow, and update pTexture:

pImage.copyPixels(pFlashWindow.image,pImage.rect, pImage.rect)
pTexture.image = pImage

Ok, now you get the uvcoords of the cursor's position on the texture and translate that into 2D coords for the MIAW. The method mMapPointToTexture does this, and was written by James Newton in 2001, and its code is at the very bottom of this page. Look at Method 1 for more comment on the code below.

pOffset = (the mouseLoc - (point(sprite(me.spriteNum).left, sprite(me.spriteNum).top)))
m = sprite(me.spriteNum).camera.modelsUnderLoc(pOffset, 1, #detailed)

Finally, you move the MIAW such that the cursor is over the same point on it as it is over the texture.

if m<>[] then

-- get 2D coords from the 3D #uvcoords
uv=mMapPointToTexture(m[1],point(Mwidth,Mheight))
-- move the pFlashWindow
if not voidP(uv) then
topLeft = (the mouseloc - uv) - Mcenter
pFlashWindow.drawRect = rect(topLeft[1],topLeft[2],Mwidth, Mheight)
end if

else

--there's no model under the cursor. move the pFlashWindow somewhere out of the way
pFlashWindow.drawRect = MbaseRect

end if

That's it. Very similar to Method 1 with some tradeoffs. Good for when your SWF has some rollover interactions and has some ActionScript that determines how it behaves. No problem with clipping in fullscreen developer mode or as a projector.


Method 3: Put the SWF in a MIAW and have Director send it coordinates/mouseevents that ActionScript interprets

You asked for it. This method is a hack but it works and it's what we ended up having to use. You'll have your SWF in its own same-size Director movie. In your .dir with the 3Dworld, you'll open it up but keep it invisible like in Method 2. But you won't move it around under the cursor. Instead, when your cursor is over the textured model, your main movie will send X,Y, and events (like a click or keypress) to your MIAW. In turn, your MIAW sends these parameters on to the SWF. You must have a main function in your ActionScript that takes in X,Y, and events, tests to see if you're within an interactive movieclip, and if so, calls the appropriate handlers. If you have rollOver/rollOut handlers for any movieclip, it must also track the last known X and Y and compare them so it 'manually' detects rollOvers and rollOuts. Some of this code is recycled from above and you can find more comments in Methods 1 and 2.

On beginSprite of the 3Dworld:

pFlashWindow = window().new("flash")
pFlashWindow.fileName = "yourMIAW.dir"
Mwidth = pFlashWindow.rect[3]-pFlashWindow.rect[1]
Mheight = pFlashWindow.rect[4]-pFlashWindow.rect[2]
MbaseRect = rect(-Mwidth,-Mheight,0,0)
Mcenter = point( (Mwidth/2),(Mheight/2) )
pImage = image(Mwidth,Mheight,32)
--pMember is a reference to the cast member of the 3Dworld sprite
pTexture = pMember.newTexture("Flash Texture", #fromImageObject, pImage)
--pShader is the shader for whatever model you want to be interactive - use an existing one if you have it
pShader = pMember.newShader("flash_shader",#standard)
pShader.texture = pTexture

Now, on prepareFrame (or whenever you want) update pImage with the newest image of the pFlashWindow, and update pTexture:

pImage.copyPixels(pFlashWindow.image,pImage.rect, pImage.rect)
pTexture.image = pImage

Ok, now you get the uvcoords of the cursor's position on the texture and translate that into 2D coords for the MIAW. The method mMapPointToTexture does this, and was written by James Newton in 2001, and its code is at the very bottom of this page.

pOffset = (the mouseLoc - (point(sprite(me.spriteNum).left, sprite(me.spriteNum).top)))
m = sprite(me.spriteNum).camera.modelsUnderLoc(pOffset, 1, #detailed)

if m<>[] then

-- get 2D coords from the 3D #uvcoords
uv=mMapPointToTexture(m[1],point(Mwidth,Mheight))
if not voidP(uv) then
-- tell the MIAW where the cursor is and that there's no click/keypress (false)
--you could do this over again in an on mouseUp handler and send (true)
--you could also make it the word 'click' or the letter of the key pressed and have the
--main ActionScript function figure out what to do with it

pFlashWindow.movie.sendFlash(uv[1],uv[2],false)
end if

end if

Not done yet. The MIAW has to have this code in a Movie Script:

on sendFlash mousex, mousey, clicked
sprite(flashSprite)._root.Main(mousex, mousey, clicked)
end sendFlash

Still not done! You need to configure any Flash movie a certain way in order for this to work. Obviously you need the 'Main' function that was just called above. You also need to define any interactive event handlers as functions so that they can be called arbitrarily. For example, normally you might put this on a my_mc:

on(release){
this._alpha -= 5;
}

Instead, define it in the root timeline's actions layer like this:

my_mc.onRelease = function(){
my_mc._alpha -= 5;
}

That way you can fire the onRelease function for that movieclip without an actual mouse click. The 'Main' function needs to take X and Y (and whatever else you want) from Director, run a hitTest() against every interactive element in the Flash movie, and call functions as appropriate. To debug, you can use "getURL(whatever)" to pass whatever to Director. You could have a Director movie script handler 'on getURL me, arg' that parses the data (Director can eval() it into strings, integers, lists, whatever). If you don't do that, then whatever you put in getURL will end up in the Director message window. An example of taking Director input and making rollOver and rollOut work for dart_mc is follows:

/*********************************
Flash template for interactive textures through a MIAW.
Director Augmented Reality Toolkit (DART)
Augmented Environments Laboratory (AEL)
Georgia Institute of Technology

Written December 2004 by Dave McColgin
*********************************/

/******************************
lastMouse will keep track of the previous x and y sent from
director. initialize it to someplace outside of all interactive
movieclips, buttons, etc.; it can even be negative.
******************************/
var lastMouse:Array = new Array(-5,-5);

/*********************************
the "main" function is where we receive coords from
director, test to see which interactive flash elements
might be affected, and call the appropriate commands.
tailor this function to your own applications!
N.B. if you have lots of interactive elements, you
could abstract the code for hit testing, put it directly
on the interactive movieclips, buttons, etc., and have
main simply call that function in each one. I've kept it
all in the _root movieclip to show the Big Picture.
*********************************/
function Main(x,y,directorClick){
//test for a hit against each interactive element
inClip=false;
inClipLast=false;

if(dart_mc.hitTest(x,y,false)){ inClip=true; }
if(dart_mc.hitTest(lastMouse[1],lastMouse[2],false)){ inClipLast=true; }
if(inClip){
if(directorClick){
//this mc has a release handler, so:
//if x,y are in the movieclip AND there was a click sent...
dart_mc.onRelease();
}
if(!inClipLast){
dart_mc.onRollOver();
}
} else if(inClipLast) { dart_mc.onRollOut(); }
inClip=false;
inClipLast=false;

//repeat for other interactive elements here...

//set the coords Director just sent to test against in next texture update
lastMouse[1]=x;
lastMouse[2]=y;

}

/********************
Below is where we defined handlers for all of our interactive elements
You could also put the handlers on their corresponding movieclips--in fact,
that's the normal way to do it--but then they can't be called arbitrarily,
like we need to.
********************/
dart_mc.onRollOver = function(){
dart_mc._width += 10;
dart_mc._height += 10;
}
dart_mc.onRollOut=function(){
dart_mc._width -= 10;
dart_mc._height -= 10;
}

Phew! A pain, but it works and there are situations where it comes in handy. With the main Director movie #uvcoord to 2D translation, the MIAW relay, and the specially formatted ActionScript, you can mimic the SWF receiving any handler you want. And, remember that in DirectorMX2k4, you can arbitrarily call any ActionScript function and read/write any ActionScript variable at any level as easy as this:

sprite(flashSprite).function(args)
or
flashTxtFormat = sprite(flashSprite).my_mc.my_txt.getTextFormat()

You've got a lot of options for one-off applications. Good luck!


Method 4: Import the SWF into your cast, increment its posterFrame and take the member's image

After Method 3, this one seems like a breath of fresh air. It's really simple. We progress the Flash movie while it sits in the cast and take its image from there. First, import your simple looped animation into the cast, and make a reference to it called flashMember. Now it starts off in a familiar way: on beginSprite of the 3Dworld, again we set the texture equal to an image we define:

flashMember = member("flash")
--what frame of the SWF to start on
pFlashPos = 1
pFlashCount = flashMember.frameCount
pImage = image(flashMember.width,flashMember.height,32)
--pMember is a reference to the cast member of the 3Dworld sprite
pTexture = pMember.newTexture("Flash Texture", #fromImageObject, pImage)
--pShader is the shader for whatever model you want to be interactive--use an existing shader if you want
pShader = pMember.newShader("flash_shader",#standard)
pShader.texture = pTexture

Now, on prepareFrame (or whenever you want) update pImage by incrementing the member's posterFrame--the 'thumbnail' of the movie that is viewable in the cast when it's in tile mode. Then take its image and update pTexture:

--progress the SWF one frame at a time
pFlashPos = (pFlashPos mod pFlashCount) + 1
flashMember.posterFrame = pFlashPos
pImage = flashMember.image
pTexture.image = pImage

Done. It's oh-so-easy and is the perfect solution for simple animations with no scripting or interaction. Works in Shockwave, projector, and developer modes. pTexture could also simply be set to flashMember.image without pImage. I needed pImage elsewhere in my code.


Method 5: Put the SWF on stage, take the stage image at the SWF sprite's rect

Use this only in very specific circumstances. Remember that the flashSprite will remain invisible even if it's on top of the 3Dworld as long as it is not direct-to-stage. However, any part of it that isn't onscreen won't update in the texture, so it has to remain fully onstage and fully under the 3Dworld. At the same time, even if you stick it underneath the 3Dworld, it will still receive mouse position so rollOvers and the like will fire when you don't want them to.

pImage = image(flashSprite.width,flashSprite.height,32)
--pMember is a reference to the cast member of the 3Dworld sprite
pTexture = pMember.newTexture("Flash Texture", #fromImageObject, pImage)
--pShader is the shader for whatever model you want to be interactive--use an existing shader if you want
pShader = pMember.newShader("flash_shader",#standard)
pShader.texture = pTexture

Now, on prepareFrame (or whenever you want) update pImage with the newest image of the flashSprite, and update pTexture:

pImage.copyPixels((the stage).image,pImage.rect,flashSprite.rect)
pTexture.image = pImage

Done. This will let you have some ActionScript (as long as it's not interactive) with no clipping problems.


Method 6: You could put the SWF in a MIAW...

But why? If your SWF is really non-interactive (no mouse/keyboard events; fully autonomous), there's no need to do this. Just stick it on the stage behind the 3Dworld like in Method 5. If it has some interactions, even simple rollOvers, you've got to go back to one of the first three methods. If you want to do this for some reason just remember it won't work in Shockwave because 'windows' don't exist.


James Newton's #uvcoord to 2D translation

-- MAP POINT TO TEXTURE --
--
-- © October 2001, James Newton <newton@planetb.fr>

on mMapPointToTexture(iSectList, anImage) ----------------------------
-- RETURNS the point in the Bitmap member used by the texture
-- defined implicitly in <iSectList>
--
-- INPUT:
-- <iSectList> should be a property list of the format:
--    [#model:             model("model name"),
--      #distance:         <float>,
--      #isectPosition: <vector>,
--      #isectNormal:    <vector>,
--      #meshID:            <integer>,
--      #faceID:            <integer>,
--      #vertices:       [<vector>, <vector>, <vector>],
--      #uvCoord:         [#u: <float>, #v: <float>]]
--
-- <anImage> can be VOID, an image object, or any type of object
--   with width and height properties (eg: propList)
--
-- When you use aCamera.modelsUnderLoc(aLoc, #detailed) or
-- aMember.modelsUnderRay(aPoint, aVector, #detailed), the result is
-- a linear list of lists with this format.
---------------------------------------------------------------------

    tModel      = iSectList. model
    tResource = tModel. resource

    isMesh = (tResource. type = #mesh )

    if not isMesh then
-- We need to use the mesh deform modifier to get texture data
       if not((tModel. modifier ). getPos ( #meshdeform )) then
-- Add mesh deform modifier... but remember to remove it later
          tModel. addModifier ( #meshdeform )
          isModified = TRUE
       end if
    end if

    tFace         = iSectList. faceID
    tUVCoord    = iSectList. uvCoord

-- Determine how the iSect data maps to this particular face
    if isMesh then
       tCoordList = tResource. textureCoordinateList
       tFaceList   = tResource. face [tFace]. textureCoordinates

    else
       tCoordList = tModel. meshdeform . mesh [iSectList. meshID ]. textureCoordinateList
       tFaceList   = tModel. meshdeform . mesh [iSectList. meshID ]. face [tFace]
       if isModified then
-- Remove the modifier now that it has done its job
          tModel. removeModifier ( #meshdeform )
       end if
    end if

    case anImage. ilk of
       #void :
          tWidth = 1
          tHeight = 1

       #point , #list :
          tWidth = anImage[ 1 ]
          tHeight = anImage[ 2 ]

       #image , #rect , #propList :
          tWidth = anImage. width
          tHeight = anImage. height
    end case

-- tCoordList will be a list of lists, each containing two floating-
-- point numbers between 0.0 and 1.0.   The first number defines the
-- relative horizontal position of a point in the texture, the
-- second number defines the relative vertical point.   The origin
-- point [0.0, 0.0] is in the bottom left hand corner.
--
-- tFaceList will be a list with three integer values [a, b, c]
-- These values determine which entry in tCoordList is used by the
-- chosen face.   The first entry <a> defines the origin.   The u
-- value increases from <a> to <b>.   Any point on the line between
-- <a> and <b> will have a v value of zero.   The v value increases
-- from <a> to <c>.   Any point on the line between <a> and <b> will
-- have a v value of zero.

-- Calculate the position of the points <a>, <b> and <c> within the
-- Bitmap member.

    tLocA = tCoordList[tFaceList[ 1 ]] -- [<float>, <float>]
    tLocA = point (tLocA[ 1 ] * tWidth, ( 1 - tLocA[ 2 ]) * tHeight)

    tLocB = tCoordList[tFaceList[ 2 ]] -- [<float>, <float>]
    tLocB = point (tLocB[ 1 ] * tWidth, ( 1 - tLocB[ 2 ]) * tHeight)

    tLocC = tCoordList[tFaceList[ 3 ]] -- [<float>, <float>]
    tLocC = point (tLocC[ 1 ] * tWidth, ( 1 - tLocC[ 2 ]) * tHeight)

    tUVector = (tLocB - tLocA) * tUVCoord.u -- actually a 2D point...
    tVVector = (tLocC - tLocA) * tUVCoord.v -- ... rather than a vector

-- Start from the origin <a>, move first in the u direction then in
-- the v direction to end up at the point in the texture

    return tLocA + tUVector + tVVector
end mMapPointToTexture


The End



Link to this Page