eyeon Development

Fusion: all the sourcey details

2010-12-16T01:55:00.001-05:00

Perhaps that should be, all the gourcey details.

The following shows some history of Fusion development, from early 1996, through to August 2010, visualised by gource, and spread over 11 minutes. It's hard to pick out any major milestones - or at least any that correlate with release dates - since the majority of development happens long before any particular release. There are some things I can point out though.

The number of tools and UI controls grow during 1996 and 1997, with the first release of Digital Fusion 1.0 for Windows NT late in 1996. July 1997 shows the tree explode with TIFF and JPEG libraries. While Digital Fusion had already been using them, this was when they were first placed under revision control. It also marks when development was first split between Australia, where all previous development had taken place, and Canada.

1998 shows an explosion of file format support, including QuickTime and OMF support, hardware support, Bins, a Render Manager, and the first version of Paint.

2000 shows the initial development of Particles and Text+, and in there somewhere is the beginnings of the non-grid flow view.

2002 shows DFScript becoming a real product, as part of Digital Fusion 4, along with some reference documentation for it. In there somewhere is the very beginnings of 3D too.

In 2004, there are some wide reaching changes, for Fusion 5 development, to do with transitioning from a binary flow to an ASCII comp format, and changing the plugin SDK from using 32-bit IDs extensively, to using string IDs. The 3D system is starting to mature. Text+ moves from Windows font system, to using freetype for font parsing, with custom character rendering.

By 2005, the basis of 3D has settled, FBX support is added, and development is continued through to the release of Fusion 5 later in the year.

Significant enhancements are made to 3D for 2006, and quite a lot of changes made to get Fusion working on Linux. Not seen here is all the wine updates, and custom wine development. A more advanced materials system based on Cg shaders begins here.

Script plugins, or fuses are introduced in 2008, while the new Cg based 3D material system grows...

2009 sees DoD/RoI and the 3D material system released in Fusion 6.0.

So that's 14.5 years of Fusion development in a nutshell.

Heads up (Updated: Almost here!)

2010-11-19T00:34:00.003-05:00

New build of Fusion is coming. Fair number of bugfixes in it.

Updated (16th Dec): Coming real soon now! There's a lot of bugfixes, but there's a few new features as well:

Fullscreen DirectX viewer (supports 3D Vision stereo, no Quadro needed!)
View mirroring (copies the main display view to an external view, like the new DX view)
Pick channels directly from 3D view (includes color, position, Z, UV, and rotation from normals)
Disk caching on mask tools
Huge 10x speed improvement in the Probe tool! :-)
Double-precision support in OpenCL
Improvements to polyline rendering, point cut/paste etc
and more...

3D Stereo with a 2D tool

2010-09-03T04:47:00.003-04:00

Here's a quick trick: you don't need Fusion's 3D engine to get stereo 3D. You can manage it with an instanced pair of 2D tools - so long as they do 3D rotation :-) All you need to do is produce two slightly-different viewpoints, and combine them. It can be faster to render, too.

The simplest method is to just rotate the view on the Y-axis a degree or two. This produces a nice stereo view centered on the axis of rotation, and anything in front of the axis will pop out from the screen. However, if we want a different effect, e.g. with the object inset into the screen further, we'll need to adjust the pivot point to be in front of the object.

Here's a quick example with Text+ (please excuse the ghastly colours - I needed something that worked with anaglyph glasses, and I have no taste). Since it's 3D internally, it's pretty easy to do this with. We need to adjust the pivot point on each tool, then rotate each inwards slightly so that they converge at the text itself. The two viewpoints are then combined with the Combiner tool, for easy display in Fusion's stereo view.

A quick note on how it's set up:

The instance of the Text+ tool has all the same settings, with a couple of de-instanced exceptions.
I used the Line Pivot Z control (in the Lines section of the Transform tab) to offset the text into the screen, since there's no pivot control in the Layout tab. This is equal in both tools.
Rotation has to be done after this stage, so Line Angle Y is used to do the rotation rather than Layout. There's a small rotation in the main tool, and an expression in the instanced tool rotates an equal amount in the other direction.
We also horizontally offset the two viewpoints from each other slightly to fine-tune the convergence point. This can done with the Center control on the Layout tab (and an expression on the instanced tool).
You can't adjust the other Layout controls without disturbing the stereo "rig", but Text+ allows you plenty of other options - Word- and Character-level transformations, multiple elements with their own offsets, the Follower modifier; there's plenty of scope for fun.

It's a hack, but who knows, you might find a use for it. Other tools you could do this with include DVE and 2D Particles. The latter even allows you to use a 3D camera tool to set the viewpoint, though you'd need a second camera for the instance.

OpenCL fuse manual

2010-07-29T03:40:00.006-04:00

Hm, been a while since the last post, hasn't it? Fate of most blogs I guess. Not entirely uneventful, what with Fusion 6.1 being released & all. Now that that's out of the way, I can divert a little mental energy elsewhere.

This is just a heads-up that I've posted an updated OpenCL fuse reference manual over on VFXpedia, so you've officially run out of excuses (except for the guy over there with last year's ATi card, you're off the hook). There are some example OpenCL fuses included with Fusion 6.1, and a quick walkthrough (could use a little revising) also on VFXpedia. I'm curious to see what people come up with, now that fuses can be even faster than built-in tools!

Next up, I'll show you how to do stereo 3D titles - without using the 3D engine (cue audience gasps). Also, should I be announcing things like the Build 667 point release here too, in case someone actually notices it here first? Anyone? Bueller? <FX: crickets>

Die 24fps, die!

2010-05-03T21:12:00.004-04:00

No, that's not German, I'm just tired of the fascination the industry has with 1920s technology. What is it about blurry, shuddering cinema that's so consistently attractive?

Edison, who first started using 35mm film in 1892, originally favoured 46fps - "anything less will strain the eye" - and his main rival's camera typically shot at 40fps. However, due to the slow and expensive film stocks of the day, directors frequently shot at slower speeds - often varying the speed within the same shoot! In 1915, a projectionists' handbook declared, "One of the highest functions of projection is to watch the screen and regulate the speed of projection [by hand-cranking] to synchronise with the speed of taking", though exhibitors regularly sped up the film to fit more reels in. Silent films often went as low as 16fps, but with the advent of "talkies" formats using optical sound strips on the film, cinematographers were forced to standardise on a consistent speed: 24fps.

In these enlightened days of digital soundtracks and digital video, we are of course not bound to any such limitation. We've hugely improved the sensitivity, resolution and grain of our film stocks and image sensors, and we relentlessly pursue the ideal of image quality. So why are we so quick to throw out our temporal quality (and degrade our spatial quality with motion blur) by deliberately shooting at less than half the framerate that video has been managing for years? It's like buying a 5D mkII camera and then leaving the photo size at only 1024 x 768.

Now in some cases we have no choice - we have to match 24fps film from other sources, or the projector can only show it at 24fps, and so be it, we're stuck with those. Yet even when we have the equipment to shoot squeaky-clean footage at 30fps or even 60fps with a fancy new digital camera, all too often we sabotage this with the ever-popular "film look", deliberately degrading our results and going to great lengths to increase their jerkiness, just to look more "cinematic".

Some may argue this is an artistic decision, like a soft filter or a bleach bypass, and of course it is. Low frame rate can likewise change the feel of a scene, increasing emotional impact. That impact is however somewhat lessened when it's used indiscriminately, with nothing to contrast it to and no thought of appropriateness - like blurring ALL your film, even the credits.

Others may say that audiences prefer their sweeping pans across a majestic landscape to be shuddering and/or blurred by motion, because it nostalgically reminds them of those other wonderful, shuddering, blurry moments they experienced in cinemas, years ago, when there was no other choice. I say, welcome to the 21st century, where sepia-toned photographs appear only in historical pieces and museums. We can do better now - we have been doing better in so many other ways; why are we still holding ourselves back? We spend millions on realistic special effects, then skimp on the framerate that could actually deliver that realism.

Still others may insist that our clients demand it, so we have to deliver - but they probably want to sell us their popular film-look plugin. Besides, how often have our clients ever known what they really wanted?

Enough already. Digital cameras and digital projectors mean that framerate is just another tool of the trade - lower it for that dramatic scene, by all means, but don't forget to crank it right up again for the majestic landscape pan. And for the credits.

Teasers

2010-04-07T22:55:00.003-04:00

I was going to throw up a few artfully-revealing screenshots of Fusion 6.1, to give people something to talk about, but now it looks like I won't have to. With NAB fast approaching, Marketing is going to take all the fun out of things, real soon now it seems.

Stereoscopic developments

2010-04-01T00:03:00.005-04:00

So, we've been watching the enthusiasm Hollywood has with stereoscopic video, and pondering ways to take it further. We think we've hit upon a big one: stereoscopic audio.

Adapting existing visual stereoscopy techniques has proved quite fruitful, and neatly avoids the extra expense and inconvenience of requiring more than one speaker. Here are a few approaches we've experimented with:

Anaglyph: By applying high-pass and low-pass filters to the left and right audio streams, respectively, the two streams can be mixed and played from a single speaker, without crosstalk. It is possible to direct them to the appropriate ear by means of specially-designed earmuffs, constructed from materials that only allow muffled or tinny sounds to the appropriate ear, and the brain naturally reconstructs these to form a "holographic" picture of the 3D audioscape, while still sounding as good as any Bose satellite audio system. Additionally, it is our opinion that a good set of earmuffs looks less dorky than red/cyan glasses, though admittedly we haven't consulted our spouses yet.

Time-division multiplexing: Similar to the popular LCD shutter glasses idea, this approach plays each audio stream alternately for brief periods, and a synchronised mechanical shutter apparatus on the earmuffs blocks and allows the audio into each ear in turn. When done at a high enough frequency, persistence-of-hearing lets the brain perceive the interrupted audio as almost continuous, and the "jackhammer effect" is soon tuned out and ignored. We did find that some care needed to be taken with the mechanical shutter design; early prototypes with hinged rather than sliding shutters had a tendency to generate sufficient lift that they needed to be tied down with a chin strap (though we feel there's some promise in a hybrid design where lift is carefully managed in order to reduce the otherwise burdensome weight of the mechanism).

Auto-stereoscopology: With a broad enough radiating surface, audio frequency emitters can use a "lenticular" approach, where tiny baffles direct the sound to specific regions assumed to be in the vicinity of the sole listener's ears. By our calculations, a 3 or 4 metre mylar-based electrostatic strip or ribbon should be adequate for good focus, and should still be potentially cheaper than an equivalent-sized Martin Logan once manufacturing scale is achieved. The use of nanotechnology or exotic materials when constructing the baffles isn't strictly necessary, but it wouldn't hurt either.

Audio polarisation: It's well-established that light can be polarised into horizontal and vertical orthogonal components, by blocking EM waves oscillating at other angles, and theatre 3D often uses this method. Audio waves oscillate longitudinally rather than laterally, so polarising them involves splitting the waves into forward and retrograde motion, and blocking any air particles moving in the wrong direction. This is not easy with traditional speaker cones, as by their design they must move both forwards and backwards. Luckily, the desired motion is easily achieved by using common household appliances instead; by using a speed-controlled fan for the left stream and a vacuum cleaner for the right stream, in conjunction with opposed one-way air valves on each ear, crosstalk can be kept to a minimum. The primary downside of this method is a tendency to cause listeners to lean to the right, so is probably best suited to political conservatives. Also, testers reported that sound in the left ear sucked less.

Come and see our most exciting stereo audio developments at our NAB stand (since you probably won't be able to hear them).

Grain's Anatomy

2010-03-29T18:02:00.020-04:00

Film grain has complex characteristics, both in spatial structure and in value/intensity. Many understand the basic principle behind how film works (in simple terms, light sensitive crystals - or grains - causing variations in density depending on the amount of exposure to light), and therefore why grain exists and why it tends to have the shape it does (lots of overlapping crystals/grains). What isn't immediately obvious to many is how the intensity of the grain is influenced by the nature of film.

The behaviour of film, and the variations in density - caused by the grains - can be measured and from that, complex graphs can be produced to somewhat quantify its nature, such as the following:

Each film stock type has its own specific characteristics, but given all the complexity, there's one main overriding factor: the basic log-exposure nature of film. Its influence is enough to almost overshadow every other aspect of the grain intensity.

Shown here is log data from the bottom section (the grey patches) of a ColorChecker chart:

You can see that the grain intensity (or variation in density) is similar in the dark and brighter areas, tending to a little higher in the dark areas (as per the graph above). What happens after a log to lin conversion?

It becomes immediately obvious that the grain intensity is now much greater in the brighter areas than in the dark areas. In effect, what happens is that in linear space, the intensity of the grain is directly proportional to the value it's being applied to. That is, a value twice as bright will have twice the intensity of grain. That's a bit of a generalisation, but it's an appropriate approximation to what goes on. Grain on a simple black to white linear gradient generally looks like:

Zooming in closer on the grain, you can see how the log nature expresses itself not just in the overall intensity, but in the basic shape of grain itself, here shown as log data:

After a log to lin conversion, that becomes:

You can see that even within the grain itself, after conversion to linear, the grain extends a lesser amount below the main/central value, and more above.

This behaviour has some implications. One issue that would occur is when a series of frames are averaged together to produce a clean plate with grain reduced/removed. If linear images (ones that have already had a log to lin conversion) are averaged together, since the grain extends more above and less below the main image value, the average will be higher than the real main/central image value. It is better to average the log images, where the variation in grain is linear, and then apply a log to lin conversion to the averaged result. The differences aren't huge, but they are definitely there, and could certainly affect black levels, where it's more likely to be noticed.

A lot of guys ignore the laugh

2010-03-22T03:27:00.003-04:00

And that's about standards.

I mean, if youre gonna get into the Evil League of Evil, you have to have a memorable laugh.

Loading a clipboard bitmap into Fusion

2010-03-18T01:45:00.003-04:00

Just a quicky, since it's something I've often wanted, and the subject came up on fusion-l recently - a console util to copy a bitmap on the system clipboard to a file, and a simple fuse to load it back into Fusion.

Download here.

Easier 3D LUTs

2010-03-17T02:52:00.012-04:00

I got some feedback from the boss about creating 3D LUTs, as described earlier: "too many steps". So I wrote a script to help out, and in a burst of imagination I called it "Create3DLUT". Thanks to the magic of automation, you can now create your 3D LUTs more easily:

Set up a chain of colour correction tools on the flow.
Select them all, and run the Create3DLUT script from the comp's Scripts menu.

Thanks to the magic of abbreviation, that list looks shorter, but you'll still be asked for a filename and some details about the LUT. And thanks to the magic of antedation, to make up a convenient word, I've already written the script and slipped it into your Scripts/Comp folder, so you don't even need to download it :-)

But what if you want to make a 3D LUT from a completely different program? Well, why are you even looking at a Fusion dev blog? But since you're here - all you need to do is create an image with a well-spaced range of colours, colour it with the method of your choice, read the pixels back and write their colour values to a file with an appropriate header. Sounds simple enough - and easy to do with a couple of fuses, LUTCubeCreator and LUTCubeAnalyzer.

Use LUTCubeCreator to create the special image, then save it and colour-correct it however you like. When done, load it back into Fusion, and feed it into the LUTCubeAnalyzer. Simply choose a LUT type and filename, click the Write button, and your new 3D LUT is created with a flourish and smile.

Visualising filter kernel shapes

2010-03-11T21:35:00.015-05:00

Here's a simple way you can visualise the shape of any convolution filter, such as a blur or defocus. Easy in Fusion, but the same approach works with most image processing apps.

In a new comp, start with a black background, about 64 x 64 is fine, and make it float32.
Add a single white pixel in the middle of it (I used Paint with the pixel brush).
Now add a tool that does filtering, such as Blur.
Normalise the image (in Fusion you can use the AutoGain tool, or just click the Show Normalised Image button on the image view toolbar, at the right).

You might want to zoom in a little, and probably increase the size of the Blur (200 is good), so you can see what's going on. You should see something like this:

So that's what a Gaussian kernel looks like, nice and smooth. Try turning HiQ on and off, and see how it changes. With HiQ off, it's not quite as smooth (but it's faster).

Now try setting the Blur to Multibox instead, and try different Num Passes settings (hover to see which is which):

You'll note that Passes = 1 is identical to Box, and Passes = 2 is identical to Bartlett. Beyond 3 or 4, the difference gets hard to pick.

But the fun doesn't end there! Instead of Blur, try a Defocus tool (turn off blooming by setting the Threshold to 1.0). Gaussian mode is similar to the Gaussian blur, but if you switch to Lens mode, you can clearly see what the NGon and Circle kernels look like:

Of course, filtering is done by other tools too. An obvious choice is the Filter tool - take a look at the Emboss, Sobel and Defocus filters. Or even a Transform (turn the Size up to 5 or so, and make sure HiQ is on):

In particular, note the darkened "lobes" on Catmull-Rom, Mitchell, Lanczos, Sinc and Bessel - these provide a sharpening effect. It's also interesting to try different windows on the Sinc and Bessel filters, to get an idea of their effect.

Looking for more? Try this same technique in other image processing apps, to see what their filters really look like.

Creating 3D LUTs

2010-03-04T01:30:00.006-05:00

As of, erm, 5.2? Fusion has supported 3D LUTs, in the image view and on the flow, natively (I think Rising Sun had a plugin for that before then). These go beyond ordinary (1D) LUTs and allow you to do more than simply adjust colour intensities, but to map colours to completely different colours, like making the image sepia, or changing all the chartreuse pixels in your image to a sort of pinky-russet.

In fact, pretty much any non-spatial colour correction can be done with a single 3D LUT, limited only by the range and accuracy of the LUT. They're fast too, at least when accelerated by modern GPU hardware. The catch; you had to find your own 3D LUT files, somewhere.

These days, it's possible to build your own 3D LUTs right within Fusion. There's a few ways to do that; easiest is to right-click on the image view, choose LUT..., then select Convert to 3D LUT. If you have a view LUT enabled, of any sort or combination (you did know you could use multiple view LUTs, didn't you?), then it will be converted into a 3D LUT file and exported to your LUTs: directory.

This works with macro LUTs too, so try this:

Set up a chain of colour correction tools on the flow; a single CC tool, a FileLUT tool with a LUT file in it, a chain of CCvs, Clrs and custom fuses, whatever.
Select them all and drag them to a Bin folder
Type "LUTs:Snazzy" into the pop-up Save File dialogue, and click Save.
Now you can choose "Snazzy" from the view's LUT menu! Enable the LUT, voila - but it's a little slow to update, because it's all rendered by the CPU.

Choose Convert to 3D LUT from the LUTs... context submenu, to get a file dialogue.
Type a filename with format extension, like "Snazzy LUT.3dl" or "ewww.alut3".
Choose the range and accuracy you'd like. Defaults are probably fine, so click OK.
Fusion creates the 3D LUT file, selects it in the view, and magically your very own fancy colouring is applied with all the sheer, ludicrous speed that your cheapo graphics hardware can provide (which I bet is actually pretty darn snappy).

So now you can hardware-accelerate those slow macro view LUTs you've been using. Plus of course, you can take Snazzy LUT.3dl and pass it on to your coworkers, so that they can get fast snazzy colour too and you can bask in their unbridled admiration.

No seriously

2010-03-04T01:14:00.003-05:00

This is the site where we talk a bit informally about Fusion from a developer's perspective. Maybe we'd like to point out something shiny in a recent release, maybe we'd like to clarify how something works internally, maybe we just feel like having a chat.

Though I have to say, anything we might accidentally let slip about potential new features should be taken with the traditional salt mine. Consider them unconfirmed rumours, even from us, because often we work on bits and pieces that don't make it into anything released (for any number of reasons). So don't go getting all excited, coz you'll probably just end up disappointed and resentful. We're just floating ideas, getting feedback, OK? No commitment implied.

So, anyway, on with the show, s'pose I better put up a post about something.

Obligatory Welcome

2010-03-04T01:10:00.003-05:00

First post!