Jump to content


Photo

[Complete]Fxaa Anti Aliasing

shader gm:studio

  • Please log in to reply
27 replies to this topic

#1 Greylore

Greylore

    GMC Member

  • GMC Member
  • 45 posts
  • Version:GM:Studio

Posted 06 October 2013 - 05:12 AM

Hello!  I'm pleased to announce the first ever FXAA method for GM:S.  It takes almost no performance to use, and it only requires that you use it in fullscreen.  There are variables in the pixel changer to change the resolution, but ATM it's hardcoded for 720P.  At the moment, my copy of GM:S is having trouble taking screenshots, so if anyone could UL some I'd greatly appriciate it!   Here are the shaders:

 

Vertex Shader

varying mat4 projMat;
varying vec2 texCoords;
varying vec3 vertPos;
attribute vec3 in_Position;                  // (x,y,z)
attribute vec4 in_Colour;                    // (r,g,b,a)
attribute vec2 in_TextureCoord;              // (u,v)
//attribute vec3 in_Normal;                  // (x,y,z)     unused in this shader.

varying vec2 v_vTexcoord;
varying vec4 v_vColour;

void main()
{
    vec4 object_space_pos = vec4( in_Position.x, in_Position.y, in_Position.z, 1.0);
    gl_Position = gm_Matrices[MATRIX_WORLD_VIEW_PROJECTION] * object_space_pos;
    projMat = gm_Matrices[MATRIX_WORLD_VIEW_PROJECTION];
    v_vColour = in_Colour;
    v_vTexcoord = in_TextureCoord;
    texCoords = v_vTexcoord;
    vertPos=in_Position;
}


Frag Shader:

 

//uniform vec2 frameBufSize = 16/9;
varying vec2 texCoords;

void main( void ) {
    vec2 frameBufSize = vec2(1280,720);
    
    float alpha = texture2D(gm_BaseTexture,texCoords).w;
    float FXAA_SPAN_MAX = 8.0;
    float FXAA_REDUCE_MUL = 1.0/8.0;
    float FXAA_REDUCE_MIN = 1.0/128.0;

    vec3 rgbNW=texture2D(gm_BaseTexture,texCoords+(vec2(-1.0,-1.0)/frameBufSize)).xyz;
    vec3 rgbNE=texture2D(gm_BaseTexture,texCoords+(vec2(1.0,-1.0)/frameBufSize)).xyz;
    vec3 rgbSW=texture2D(gm_BaseTexture,texCoords+(vec2(-1.0,1.0)/frameBufSize)).xyz;
    vec3 rgbSE=texture2D(gm_BaseTexture,texCoords+(vec2(1.0,1.0)/frameBufSize)).xyz;
    vec3 rgbM=texture2D(gm_BaseTexture,texCoords).xyz;

    vec3 luma=vec3(0.299, 0.587, 0.114);
    float lumaNW = dot(rgbNW, luma);
    float lumaNE = dot(rgbNE, luma);
    float lumaSW = dot(rgbSW, luma);
    float lumaSE = dot(rgbSE, luma);
    float lumaM  = dot(rgbM,  luma);

    float lumaMin = min(lumaM, min(min(lumaNW, lumaNE), min(lumaSW, lumaSE)));
    float lumaMax = max(lumaM, max(max(lumaNW, lumaNE), max(lumaSW, lumaSE)));

    vec2 dir;
    dir.x = -((lumaNW + lumaNE) - (lumaSW + lumaSE));
    dir.y =  ((lumaNW + lumaSW) - (lumaNE + lumaSE));

    float dirReduce = max(
        (lumaNW + lumaNE + lumaSW + lumaSE) * (0.25 * FXAA_REDUCE_MUL),
        FXAA_REDUCE_MIN);

    float rcpDirMin = 1.0/(min(abs(dir.x), abs(dir.y)) + dirReduce);

    dir = min(vec2( FXAA_SPAN_MAX,  FXAA_SPAN_MAX),
          max(vec2(-FXAA_SPAN_MAX, -FXAA_SPAN_MAX),
          dir * rcpDirMin)) / frameBufSize;

    vec3 rgbA = (1.0/2.0) * (
        texture2D(gm_BaseTexture, texCoords.xy + dir * (1.0/3.0 - 0.5)).xyz +
        texture2D(gm_BaseTexture, texCoords.xy + dir * (2.0/3.0 - 0.5)).xyz);
    vec3 rgbB = rgbA * (1.0/2.0) + (1.0/4.0) * (
        texture2D(gm_BaseTexture, texCoords.xy + dir * (0.0/3.0 - 0.5)).xyz +
        texture2D(gm_BaseTexture, texCoords.xy + dir * (3.0/3.0 - 0.5)).xyz);
    float lumaB = dot(rgbB, luma);

    if((lumaB < lumaMin) || (lumaB > lumaMax)){
        gl_FragColor.xyzw=vec4(rgbA.x,rgbA.y,rgbA.z,alpha);
    }else{
        gl_FragColor.xyzw=vec4(rgbB.x,rgbB.y,rgbB.z,alpha);
    }
}

 


  • 4

#2 Alvare

Alvare

    Allrounder

  • GMC Member
  • 1141 posts
  • Version:GM:Studio

Posted 06 October 2013 - 06:30 AM

Nice. I'll test this out.

 

EDIT -

I see a difference. Mostly on transparent textures.

 

It's just hard to see because I have to use a reduced resolution on 1080p. =D


Edited by THE_GAME_EDITOR, 06 October 2013 - 06:46 AM.

  • 0

#3 Gamer3D

Gamer3D

    Human* me = this;

  • GMC Member
  • 1819 posts
  • Version:GM:Studio

Posted 06 October 2013 - 01:36 PM

You've got an extra texture access that may not be optimized away. Here's a revision (I also put frameBufSize back as a uniform):
 
varying vec2 texCoords;

uniform vec2 frameBufSize;

const float FXAA_SPAN_MAX = 8.0;
const float FXAA_REDUCE_MUL = 1.0/8.0;
const float FXAA_REDUCE_MIN = 1.0/128.0;
 
void main( void ) {
    vec3 rgbNW=texture2D(gm_BaseTexture,texCoords+(vec2(-1.0,-1.0)/frameBufSize)).xyz;
    vec3 rgbNE=texture2D(gm_BaseTexture,texCoords+(vec2(1.0,-1.0)/frameBufSize)).xyz;
    vec3 rgbSW=texture2D(gm_BaseTexture,texCoords+(vec2(-1.0,1.0)/frameBufSize)).xyz;
    vec3 rgbSE=texture2D(gm_BaseTexture,texCoords+(vec2(1.0,1.0)/frameBufSize)).xyz;
    vec4 rgbaM=texture2D(gm_BaseTexture,texCoords);
 
    vec3 luma=vec3(0.299, 0.587, 0.114);
    float lumaNW = dot(rgbNW, luma);
    float lumaNE = dot(rgbNE, luma);
    float lumaSW = dot(rgbSW, luma);
    float lumaSE = dot(rgbSE, luma);
    float lumaM  = dot(rgbaM.xyz,  luma);
 
    float lumaMin = min(lumaM, min(min(lumaNW, lumaNE), min(lumaSW, lumaSE)));
    float lumaMax = max(lumaM, max(max(lumaNW, lumaNE), max(lumaSW, lumaSE)));
 
    vec2 dir;
    dir.x = -((lumaNW + lumaNE) - (lumaSW + lumaSE));
    dir.y =  ((lumaNW + lumaSW) - (lumaNE + lumaSE));
 
    float dirReduce = max(
        (lumaNW + lumaNE + lumaSW + lumaSE) * (0.25 * FXAA_REDUCE_MUL),
        FXAA_REDUCE_MIN);
 
    float rcpDirMin = 1.0/(min(abs(dir.x), abs(dir.y)) + dirReduce);
 
    dir = clamp(dir * rcpDirMin,
                vec2(-FXAA_SPAN_MAX),
                vec2(FXAA_SPAN_MAX)) / frameBufSize;
 
    vec3 rgbA = (1.0/2.0) * (
        texture2D(gm_BaseTexture, texCoords.xy + dir * (1.0/3.0 - 0.5)).xyz +
        texture2D(gm_BaseTexture, texCoords.xy + dir * (2.0/3.0 - 0.5)).xyz);
    vec3 rgbB = rgbA / -2.0 + (1.0/4.0) * (
        texture2D(gm_BaseTexture, texCoords.xy + dir * (0.0/3.0 - 0.5)).xyz +
        texture2D(gm_BaseTexture, texCoords.xy + dir * (3.0/3.0 - 0.5)).xyz);
    float lumaB = dot(rgbB + rgbA, luma);

//  In shaders, conditional evaluations are misleading, as both branches may be evaluated anyway.
    gl_FragColor.xyzw=vec4(rgbA + step(lumaMin, lumaB) * step(lumaB, lumaMax) * rgbB,rgbaM.w);
}
It's still not perfect, but it might perform just a tiny bit better.

Edited by Gamer3D, 06 October 2013 - 01:48 PM.

  • 0

#4 Alvare

Alvare

    Allrounder

  • GMC Member
  • 1141 posts
  • Version:GM:Studio

Posted 10 October 2013 - 02:02 PM

Lol.. I bet I'm doing this wrong.

I put this shader on each drawing..

 

It's supossed to be on one surface right?


  • 0

#5 Greylore

Greylore

    GMC Member

  • GMC Member
  • 45 posts
  • Version:GM:Studio

Posted 10 October 2013 - 06:17 PM

Yup.  You need to do this in a controller object (maybe obj_postproc)

 

 

Properties:

 

Depth:1000

 

Create event:

 

 

surf=surface_create(view_wview[0],view_hview[0])

view_surface_id[0]=surf

 

Draw GUI:

 

draw_surface_stretched(surf,view_wport[0],view_hport[0])


  • 1

#6 Alvare

Alvare

    Allrounder

  • GMC Member
  • 1141 posts
  • Version:GM:Studio

Posted 10 October 2013 - 06:54 PM

Yup.  You need to do this in a controller object (maybe obj_postproc)

 

 

Properties:

 

Depth:1000

 

Create event:

 

 

surf=surface_create(view_wview[0],view_hview[0])

view_surface_id[0]=surf

 

Draw GUI:

 

draw_surface_stretched(surf,view_wport[0],view_hport[0])

This makes me curious. =)

I'll try this and see how awesome this shader really is.

 

EDIT - Can't get it to work. xD


Edited by THE_GAME_EDITOR, 12 October 2013 - 09:54 AM.

  • 0

#7 darkwalker247

darkwalker247

    Fox

  • GMC Member
  • 1836 posts
  • Version:GM:Studio

Posted 11 October 2013 - 11:59 PM

So it's a post-processing sampling anti-alias shader?


  • 0

#8 Greylore

Greylore

    GMC Member

  • GMC Member
  • 45 posts
  • Version:GM:Studio

Posted 13 October 2013 - 10:38 PM

So it's a post-processing sampling anti-alias shader?

Yup.  You have to do it on the entire scene, and the scene must match the resolution specified in the fragment shader.


  • 0

#9 JstnPwll

JstnPwll

    GMC Member

  • New Member
  • 1 posts
  • Version:GM:Studio

Posted 11 February 2014 - 08:42 PM

I have added this to a test project and it worked fine on Windows. On Android, it's just showing a bunch of horizontal lines (in the colors of my sprites). Does anyone know why this is the case? Is there anything in this shader that isn't supported on Android?


  • 0

#10 RuinaImperi

RuinaImperi

    GMC Member

  • GMC Member
  • 427 posts
  • Version:GM:Studio

Posted 11 February 2014 - 11:22 PM

Could someone post comparison pictures? Can't try it out right now. Is the difference noticeable? How performance intensive is it?


  • 0

#11 Gamer3D

Gamer3D

    Human* me = this;

  • GMC Member
  • 1819 posts
  • Version:GM:Studio

Posted 12 February 2014 - 04:41 AM

Check the Wikipedia article for comparison pictures. FXAA is one of the fastest approximation methods for that level of quality.
  • 0

#12 MishMash

MishMash

    GMC Member

  • GMC Member
  • 827 posts
  • Version:GM8.1

Posted 14 February 2014 - 01:19 AM

Wow! This looks really good, Great job!


  • 0

#13 CoisadePro

CoisadePro

    GMC Member

  • New Member
  • 16 posts
  • Version:GM:Studio

Posted 04 March 2014 - 12:51 AM

Can someone help me with this? What I need to do to get this working? I've put all those codes like u said, but only what I got was a totally black screen. I'm using GM:S Standard with the latest updates.

 

Name: o_postprocess

 

Create:

surf=surface_create(view_wview[0],view_hview[0]);
view_surface_id[0]=surf;

Draw_GUI

shader_set(s_postprocess)
draw_surface_stretched(surf,view_wport[0],view_hport[0],view_wview[0],view_hview[0])
shader_reset();

Depth: 1000 (I tried both -1000 and 1000)

 

This is all the code of this object, and I enabled the use of views. Serious, I don't know anything about shaders. Of course, I know to code very well and I'm working with GM for about 3 years. If u can help me to make this work I will really apreciate it. Thank you all, Patrick.


Edited by CoisadePro, 04 March 2014 - 01:00 AM.

  • 0

#14 MishMash

MishMash

    GMC Member

  • GMC Member
  • 827 posts
  • Version:GM8.1

Posted 04 March 2014 - 04:04 PM

Are you sure its not working, it is a very subtle effect, so you may just not be noticing it (though it does make a difference overall). So long as you have imported the shader correctly, then it should work. Only other thing is, why are you also setting view_surface_id[0] to surf? All this will do is make the view use a custom surface rather than the one you are drawing. As you are drawing it anyway in draw_gui, theres no need for that.

 

If it's a black screen, its most likely got nothing to do with the shader, it appears to just be how you are setting up the surface. I don't think that setting the surface to the view surface will make things draw to that surface, I still think things always draw to the default surface, regardless, however setting a view surface just changes which one is drawn in the end (I may be wrong).

Try testing in a new project, and draw a few things to the surface in the draw event, and see if that works.


  • 0

#15 CoisadePro

CoisadePro

    GMC Member

  • New Member
  • 16 posts
  • Version:GM:Studio

Posted 04 March 2014 - 05:49 PM

Thanks MishMash, but can you make a working demo and put the download link here? Serious, I'm really new to surfaces and shaders and I can't get this working yet.


  • 0

#16 Alvare

Alvare

    Allrounder

  • GMC Member
  • 1141 posts
  • Version:GM:Studio

Posted 04 March 2014 - 11:01 PM

To be honest. I thought it was working but I haven't either. I know it's subtle but.. I dunno.


  • 0

#17 RuinaImperi

RuinaImperi

    GMC Member

  • GMC Member
  • 427 posts
  • Version:GM:Studio

Posted 08 March 2014 - 01:40 PM

Works nicely, thank you. It is subtle but after comparing it in PS it deblurs the whole thing quite a bit.


  • 0

#18 Alvare

Alvare

    Allrounder

  • GMC Member
  • 1141 posts
  • Version:GM:Studio

Posted 14 April 2014 - 11:32 AM

Got it working. Looks awesome.


  • 0

#19 CoisadePro

CoisadePro

    GMC Member

  • New Member
  • 16 posts
  • Version:GM:Studio

Posted 17 April 2014 - 05:52 PM

Can someone put a working demo gmz?


  • 0

#20 Alvare

Alvare

    Allrounder

  • GMC Member
  • 1141 posts
  • Version:GM:Studio

Posted 18 April 2014 - 08:28 AM

Can someone put a working demo gmz?

It's already implementable in one minute so I don't think anyone would make a example for this.

Just use the application surface to draw the shader on. That's all really.  :)


  • 0

#21 rage_77

rage_77

    GMC Member

  • New Member
  • 5 posts

Posted 26 April 2014 - 11:45 PM

Alvare , can you put a demo ? is hard for begginers  :sweat:


  • 0

#22 Alvare

Alvare

    Allrounder

  • GMC Member
  • 1141 posts
  • Version:GM:Studio

Posted 27 April 2014 - 03:53 PM

I will not make a example. I can however show you:

 

Create event:

global.gameWidth = display_get_width();
global.gameHeight = display_get_height();


window_set_fullscreen(true);
application_surface_draw_enable(false);
surface_resize(application_surface, display_get_width(), display_get_height()); 

DrawGui

shader_set(shader_fxaa)
draw_enable_alphablend(0)
draw_surface_stretched(application_surface,0,0,global.gameWidth,global.gameHeight);
draw_enable_alphablend(1)
shader_reset()

That's all you have to do.


  • 0

#23 rage_77

rage_77

    GMC Member

  • New Member
  • 5 posts

Posted 27 April 2014 - 06:13 PM

Thanks!! works fine  :thumbsup:


  • 0

#24 rage_77

rage_77

    GMC Member

  • New Member
  • 5 posts

Posted 27 April 2014 - 08:08 PM

some pics for compare 

sin-titulo-944575.png Subido en subir imagenes


  • 1

#25 Sirosky

Sirosky

    GMC Member

  • GMC Member
  • 1476 posts
  • Version:GM:Studio

Posted 27 April 2014 - 08:19 PM

Thanks rage. There is a difference but man it's subtle.


  • 0

#26 Pascal2851981

Pascal2851981

    GMC Member

  • GMC Member
  • 94 posts
  • Version:Unknown

Posted 08 May 2014 - 01:07 PM

Does it work on iOS Devices?


  • 0

#27 kebaplover

kebaplover

    GMC Member

  • GMC Member
  • 106 posts
  • Version:GM:Studio

Posted 08 May 2014 - 07:20 PM

Does it work on iOS Devices?

 
If the device supports shaders, yes.

 

 
I shortened it down a bit and made an example for it:

 

Spoiler

 

Download: FXAA.gmz, FXAA.exe


  • 0

#28 Pascal2851981

Pascal2851981

    GMC Member

  • GMC Member
  • 94 posts
  • Version:Unknown

Posted 09 May 2014 - 09:04 AM

 

Does it work on iOS Devices?

 
If the device supports shaders, yes.

 

 
I shortened it down a bit and made an example for it:

 

Spoiler

 

Download: FXAA.gmz, FXAA.exe

 

 

Thanks!!!

 

I tried it on an iPad 3 and it´s very laggy...

 

And there was no effect on vector graphics.


  • 0





0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users