Jump to content


Photo

Dual Quaternion Skinning [Example]

shader

  • Please log in to reply
19 replies to this topic

#1 Gamer3D

Gamer3D

    Human* me = this;

  • GMC Member
  • 1819 posts
  • Version:GM:Studio

Posted 24 September 2013 - 12:55 AM

This forum is, as of the time of writing, focused entirely too much on fragment (pixel) shaders, and is ignoring the possibilities offered by vertex shaders. This is silly, so I put about an hour into making a nice, pretty vertex shader to demonstrate what you're missing.

With that in mind, I made a dual-quaternion skinning shader. In simple terms, it facilitates skeletal animation without some of the artifacts caused by linear skinning.

Supports:Screenshot:
vjilsnircc4b9e9fg.jpg

Download GMZ (10.63 MiB)

Shaders:
Spoiler

How to use it:
  • Copy the vertex shader.
  • Discard the features you don't plan to use. If you're not planning to use something that requires the surface's tangent-space, for example, then get rid of that part of the code.
  • Code your fragment shader. It could be a simple passthrough shader; it could be normal-mapping, parallax mapping, etc.
  • Set up the custom vertex format. If you haven't modified the way I use vertex attributes, this will be 4 4-float vectors, with component-use specified in the comments in the vertex shader.
  • Set up your model. Each vertex will require the vertex's position (in the neutral pose), the indices of the bones it uses, the weights of the first 2 bones (the third is defined implicitly to force them to sum to 1), and (if you've left the shader as-is) the tangent and binormal.
  • (Recommended) Freeze your buffer after setting up your model. If your animations are skeletal, you shouldn't need to modify your model very often.
  • Set up the skeleton. Create 2 1-dimensional arrays, one for the real part of the dual quaternions, one for the dual part. Fill each with the appropriate quaternions. Note: Quaternions are in xyzw format, where xyz is the imaginary part.
  • Pass the arrays to the uniforms u_BoneQReal and u_BoneQDual, then render.
Textures in the example are from Kay's Blog.


EDIT (2013-12-14): The vertex shader is now rather full-featured. The fragment shader is not complete, but has an implementation of relief mapping. When I get the fragment shader and a full example done, I'll update the demo.
EDIT (2013-12-17): Relief-mapping bug fix. Added relief-mapping demo. Fragment shader/example still incomplete.
EDIT (2014-02-06): I'm moving on to greener pastures (C++ and SDL). I've uploaded my latest version of this project, which includes a script for setting the dual quaternions; all you need to know is how to use regular quaternions, which is very basic knowledge.

Edited by Gamer3D, 06 February 2014 - 06:31 PM.

  • 9
Fast Priority Queues - Game Maker's priority queues are O(n). Mine do everything that Game Maker's do, but in O(log n) time.
Dual-Quaternion Skinning - Modifying vertexes in GM is slow. This simple vertex shader does the job both quickly and well.

#2 jsorgeagames

jsorgeagames

    GMC Member

  • GMC Member
  • 710 posts
  • Version:GM:Studio

Posted 24 September 2013 - 02:21 AM

Really great stuff! Thanks for taking this forum in a direction it hasn't been yet.


  • 0

#3 slayer 64

slayer 64

    Slayer of gingers

  • GMC Member
  • 4301 posts
  • Version:GM:Studio

Posted 24 September 2013 - 02:53 AM

i don't like dealing with quaternions component by component. the shader's nice though.
  • 0

5y5rs3d.pngfg0UQNL.png


#4 Alvare

Alvare

    Not an administrator

  • GMC Member
  • 1812 posts
  • Version:None

Posted 24 September 2013 - 06:33 AM

@Gamer3D

I dunno anything about what you're talking about.. but you should continue working with shaders man. xD

I'll read trough the pdf you posted.

 

Also.. thank you for posting that site. Really good quality textures. :thumbsup:


Edited by Alvare, 18 December 2013 - 01:02 PM.

  • 0

#5 Gamer3D

Gamer3D

    Human* me = this;

  • GMC Member
  • 1819 posts
  • Version:GM:Studio

Posted 26 September 2013 - 04:04 AM

i don't like dealing with quaternions component by component. the shader's nice though.

You can manipulate the quaternions however you want, but you must eventually copy them to an array to pass them to the shader. Unless YoYo adds a shader_set_uniform_f_grid function, there's no other choice.

A word of warning: Avoid modifying the blend array in scripts, for performance reasons. GM doesn't pass arrays by reference (at least not if the function needs to change it), instead copying the entire array. For 64 dual quaternions, that copies 512 variables (doubled if GM copies the array twice) per manipulation, for about 32000 copies total.
If manipulated separately, this can be reduced to 512 extra copy operations, and fewer still if some bones are unused.
  • 1
Fast Priority Queues - Game Maker's priority queues are O(n). Mine do everything that Game Maker's do, but in O(log n) time.
Dual-Quaternion Skinning - Modifying vertexes in GM is slow. This simple vertex shader does the job both quickly and well.

#6 blackhawkrobbo

blackhawkrobbo

    GMC Member

  • GMC Member
  • 1121 posts
  • Version:GM:Studio

Posted 29 September 2013 - 05:06 PM

Would there be an easy way to implement this onto existing models?
  • 0

#7 KayVriend

KayVriend

    GMC Member

  • New Member
  • 1 posts
  • Version:Unknown

Posted 29 September 2013 - 09:54 PM

Also.. thank you for posting that site. Really good quality textures. :thumbsup:

 

Thanks! Much appreciated. I´m not a shader wiz, but cool idea!

 

Please let me know if you use/want to use some of my work so i can feature it on my blog, or if you have requests :).

 

Cheers!


  • 2

#8 Gamer3D

Gamer3D

    Human* me = this;

  • GMC Member
  • 1819 posts
  • Version:GM:Studio

Posted 30 September 2013 - 02:58 AM

Would there be an easy way to implement this onto existing models?

Yes and no. The tangent space computations are easy to automate, but unless your model format supports skinning (tracking which bones affect each vertex, and by how much), you will need to add the blend weights manually.

Something to be aware of is that the shader currently assumes that the tangent space is right-handed. If your UV mapping involves mirrored triangles (or triangles with mirrored UV coordinates), the normal will be computed incorrectly.
The shader can be modified to permit left-handed tangent spaces by adding additional vertex information indicating whether to negate the normal. Because the shader is currently using the entirety of GM's documented bandwidth, this information would necessarily need to be packed into pre-existing information, using a scheme similar to one of the following:
  • Negative blend weight (for first blend) indicates that normal should be negated. Ordering the blend weights in decreasing order guarantees uniqueness.
  • Negative blend index (for third blend) indicates that normal should be negated. Ordering the blend indices in increasing order guarantees uniqueness.

  • 0
Fast Priority Queues - Game Maker's priority queues are O(n). Mine do everything that Game Maker's do, but in O(log n) time.
Dual-Quaternion Skinning - Modifying vertexes in GM is slow. This simple vertex shader does the job both quickly and well.

#9 blackhawkrobbo

blackhawkrobbo

    GMC Member

  • GMC Member
  • 1121 posts
  • Version:GM:Studio

Posted 30 September 2013 - 12:35 PM

Okay, so that would be for an animated model, right? What about simple D3D models (e.g. imported from an .obj) that are rotated once, such as trees, benches, barrels etc. ?

My guess is that this should be easy. The other thing i'd like to add would be FPS weapons, say, a gun with normal mapping that changes as you rotate the character. Now i'm kind of a noob when talking about tangent spaces, blend weights and what not. Would you be able to create a simple example of some kind?

Thanks a ton in advance!
  • 0

#10 Gamer3D

Gamer3D

    Human* me = this;

  • GMC Member
  • 1819 posts
  • Version:GM:Studio

Posted 03 October 2013 - 04:59 AM

blackhawkrobbo, on 30 Sept 2013 - 08:21, said:
Okay, so that would be for an animated model, right? What about simple D3D models (e.g. imported from an .obj) that are rotated once, such as trees, benches, barrels etc. ?

Blend weights are only important for animated models. For a static model, this particular shader is overkill; you could (probably) remove half of its instructions, or more, and still have the important parts for static models (for a static model, all you really need to do is transform the position to view-space and rotate the tangent space).
 

blackhawkrobbo, on 30 Sept 2013 - 08:21, said:
My guess is that this should be easy. The other thing i'd like to add would be FPS weapons, say, a gun with normal mapping that changes as you rotate the character. Now i'm kind of a noob when talking about tangent spaces, blend weights and what not. Would you be able to create a simple example of some kind?

The example will need to wait until I complete a different project. When I have time, I intend to create such an example, as well as update the shader with a few things such as:
  • Support for both left and right-handed tangent spaces.
  • A better implementation of normal mapping. The current fragment shader is severely limited (it's just a placeholder, after all), and may not give proper results if the model has been scaled non-uniformly.
  • Possibly switching from the TBN matrix to its inverse.
  • Examples of shaders optimized for specific situations. For example, a separate vertex shader dedicated to static models and a vertex/fragment shader-pair for transformations consisting only of rotation, translation, and uniform scaling.
  • Examples of how to use tangent-space for various effects (Like normal mapping, parallax mapping, etc.), with discussions of exactly why it is used in that fashion.
EDIT (2013-12-17): I have crossed out all the features I have completed at this date.

Edited by Gamer3D, 18 December 2013 - 04:45 AM.

  • 2
Fast Priority Queues - Game Maker's priority queues are O(n). Mine do everything that Game Maker's do, but in O(log n) time.
Dual-Quaternion Skinning - Modifying vertexes in GM is slow. This simple vertex shader does the job both quickly and well.

#11 Alvare

Alvare

    Not an administrator

  • GMC Member
  • 1812 posts
  • Version:None

Posted 04 October 2013 - 01:57 PM

Parallax mapping! =D

Although.. it will make distant textures appear even more crispy when not using mipmapping.

 

There's also alot of work in getting all these shaders combined, right?


  • 0

#12 FrozenFireZFGC

FrozenFireZFGC

    GMC Member

  • New Member
  • 6 posts
  • Version:GM:Studio

Posted 01 November 2013 - 04:27 PM

Shoot, I need to animate 3D hair and to bend models to move/animate arms, legs, etc., and this looks like what I need to understand. *sigh* I really need to learn about shaders.

Even though I don't understand it enough to implement it as I need to, thanks for this. I'm sure I can learn something from it if I can get enough free time to learn about shaders.


  • 0

#13 Gamer3D

Gamer3D

    Human* me = this;

  • GMC Member
  • 1819 posts
  • Version:GM:Studio

Posted 18 December 2013 - 05:12 AM

I've gotten relief mapping to the point where I will show it off. It's still not a full example/tutorial, but the following ease-of-use improvements have been made:
  • The script shdr_skindq_set_blend has been provided to permit easy use of bones (it simplifies user effort to supplying a position and quaternion).
  • The work-in-progress script shdr_skindq_set automatically sets many variables, a process which includes extracting the camera location from its matrix.
  • The vertex shader now uses the preprocessor to allow easy removal of unused features. For example, if you're not using relief mapping, you can--by commenting out a single line--turn off the additional calculations required for the effect.
The remaining to-do list:
  • Demo, with examples of various effects and more comments on why each choice was made.
  • Choose additional effects to demonstrate. I've got normal mapping and relief mapping, but I might want to improve the whole thing by choosing a different lighting model, adding reflections, etc.
  • Fully comment the fragment shader.
  • Finalize some vertex-shader choices.
  • Release GFX-card accelerated relief map baking software. "Baking" additional visibility information permits martini-stepping, an improvement I made to relaxed cone-stepping, which is again an improvement over searching linearly.
  • Create command-line relief map baking software with support for batches.

Edited by Gamer3D, 18 December 2013 - 06:41 AM.

  • 3
Fast Priority Queues - Game Maker's priority queues are O(n). Mine do everything that Game Maker's do, but in O(log n) time.
Dual-Quaternion Skinning - Modifying vertexes in GM is slow. This simple vertex shader does the job both quickly and well.

#14 MishMash

MishMash

    GMC Member

  • GMC Member
  • 1124 posts
  • Version:GM:Studio

Posted 18 December 2013 - 09:44 PM

Wow that looks incredible! Have you tried this on other textures / shapes? Was just curious as to what performance is like :)! Great job!


  • 0

VitalitySig18thMar.png


#15 Gamer3D

Gamer3D

    Human* me = this;

  • GMC Member
  • 1819 posts
  • Version:GM:Studio

Posted 19 December 2013 - 12:46 AM

Wow that looks incredible! Have you tried this on other textures / shapes? Was just curious as to what performance is like :)! Great job!

I have tested the algorithm on other textures, including the classic lion sculpture. In each case, image quality was approximately as good as for the chesterfield texture.

The following benchmarks are created on an Ivy Bridge integrated graphics chip.

Performance on full detail:
  • Vertexes per second: 65075324 (102860996 without skinning)
  • Pixels per second: 292552704 (585105408 without relief mapping)
For reference, the default pass-through shader has the following performance:
  • Vertexes per second: 102860996
  • Pixels per second: 698351616

  • 0
Fast Priority Queues - Game Maker's priority queues are O(n). Mine do everything that Game Maker's do, but in O(log n) time.
Dual-Quaternion Skinning - Modifying vertexes in GM is slow. This simple vertex shader does the job both quickly and well.

#16 MishMash

MishMash

    GMC Member

  • GMC Member
  • 1124 posts
  • Version:GM:Studio

Posted 19 December 2013 - 02:59 AM

Ahh nice! I really can't wait for this to be fully featured, Just curious, do you know of any good software for generating relief maps? I tried a few i found online but they didn't seem to work.
Also, in the future, would you be willing to create a stand-alone demo showing off the relief mapping alone, without the DQ skinning, and just on a regular model. This is going to look incredible on things like walls and floors! 


  • 0

VitalitySig18thMar.png


#17 Gamer3D

Gamer3D

    Human* me = this;

  • GMC Member
  • 1819 posts
  • Version:GM:Studio

Posted 19 December 2013 - 04:08 AM

Ahh nice! I really can't wait for this to be fully featured, Just curious, do you know of any good software for generating relief maps? I tried a few i found online but they didn't seem to work.

Crazybump seems to be the gold standard. Not sure if free.

Also, simply importing a relief map is insufficient for use with the relief-mapping part of the shader. The ray-casting code I wrote uses additional information about the texture, so to use a relief map, you must first process it (I've made a tool for that, but its interface needs a bit of work before release) or switch to a linear search (there's a line in the fragment shader for you to uncomment).
 

Also, in the future, would you be willing to create a stand-alone demo showing off the relief mapping alone, without the DQ skinning, and just on a regular model. This is going to look incredible on things like walls and floors!

I'll add it to the to-do list. If you want to do this yourself, comment out the following line:
#define SKINDQ_BLEND_QUALITY 2
Once that line is removed, the shader will no longer perform any skinning operations. Currently, the defined uniforms and vertex attributes remain the same, but this isn't particularly difficult to alter. The reason I don't automatically disable them is that it would require changing a lot of non-shader code after each revision (need to remove requests for uniforms and alter the method used for building the vertexes).
  • 1
Fast Priority Queues - Game Maker's priority queues are O(n). Mine do everything that Game Maker's do, but in O(log n) time.
Dual-Quaternion Skinning - Modifying vertexes in GM is slow. This simple vertex shader does the job both quickly and well.

#18 MishMash

MishMash

    GMC Member

  • GMC Member
  • 1124 posts
  • Version:GM:Studio

Posted 19 December 2013 - 05:12 PM

 

 

Crazybump seems to be the gold standard. Not sure if free.

Also, simply importing a relief map is insufficient for use with the relief-mapping part of the shader. The ray-casting code I wrote uses additional information about the texture, so to use a relief map, you must first process it (I've made a tool for that, but its interface needs a bit of work before release) or switch to a linear search (there's a line in the fragment shader for you to uncomment).

Ahh k, yeah i use Crazy bump atm for normal maps, wasnt sure if the relief ones needed a different technique, Looking forward to being able to use your program.

Great job so far anyway!


  • 0

VitalitySig18thMar.png


#19 Gamer3D

Gamer3D

    Human* me = this;

  • GMC Member
  • 1819 posts
  • Version:GM:Studio

Posted 20 December 2013 - 08:24 PM

Here's the relief-map baking tool: Executable or Source Code

This tool takes a heightmap (white for details at the surface, black for details at maximum depth) and generates the data necessary for martini-stepping. To be more specific, it stores the following data:
  • Red channel: Depth map (full red for maximum depth, black for details at the surface).
  • Green channel: Square-root of slope of relaxed cone.
  • Blue channel: Cylinder radius, as a fraction of cone radius at one unit above cone vertex.
  • Alpha channel: Unused.

  • 2
Fast Priority Queues - Game Maker's priority queues are O(n). Mine do everything that Game Maker's do, but in O(log n) time.
Dual-Quaternion Skinning - Modifying vertexes in GM is slow. This simple vertex shader does the job both quickly and well.

#20 blackhawkrobbo

blackhawkrobbo

    GMC Member

  • GMC Member
  • 1121 posts
  • Version:GM:Studio

Posted 30 December 2013 - 05:13 PM

That is really cool, gamer, gonna check that out in a second :)


  • 0





Also tagged with one or more of these keywords: shader