40 replies to this topic

[Desert Dog]

http://www.scirra.co...alad-vs-stencyl

We're second, apparently.

They provided the .gmx of the files. It would appear they create a new object for each 'sprite'. Not exactly the fastest method on the planet. I actually can't load it for some reason.

I don't know much about Construct 2, but since the creators of it themselves did the test, I'd assume they optimised the heck out of it.

there is actually a topic of this on the Tigs forum here:
http://forums.tigsou...p?topic=22888.0

where one of the Scirra team(ScirraTom) is reading. He's a cool fella, if you have any questions about how they did the tests, ask him.

Anyway, time Yoyogames did their own benchmark. (I seem to recall Mike promising...)

And as for rules about no competing products.. yes, I know about the rule. Don't tell me about it. If a mod closes it, he closes it. I think the topic is well appropriate, however...

Edited by Desert Dog, 23 November 2011 - 04:16 AM.

[Dangerous_Dave]

I skipped the text and just looked at the pictures, but the final graph is interesting. Note that the huge construct bar is run through WebGL, not supported by GM and not supported by all browsers. So ignore that. GameMaker has come second in the test, and although it's only about 60% of the value of construct, you can bet it would be closer if they tested on more that just the number of sprites shown on screen.

The fact that they are starting to benchmark is fantastic news. People hate losing, so I'd expect that all the HTML5 products will work on closing the gap.

[Knuked]

Hmm, that's interesting! Each program has their pros and cons, I like GMHTML5 myself, but for $32 bucks, I'll give it a go just to see!

[icuurd12b42]

I would debate the validity of the benchmark. I mean they are not the same test. I am tempted to do some of my own.

[Desert Dog]

I would debate the validity of the benchmark. I mean they are not the same test. I am tempted to do some of my own.

Why do you think I posted it here? I want to see Yoyogames do some!!

I asked what method they used to draw their sprites, and their going to get back on that. But otherwise, would be cool to see a few GM users come up with some optimised methods of drawing sprites.(I'll assume tiles don't count!)

I just had a go with particles, and although it did seem to be faster than their tests, it wasn't by a whole big deal. You can read what I did in spoiler tags below, if anyone else is wanting to make a faster one.

Spoiler

init_particles()

Spoiler

global.p_system=part_system_create();

my_part=part_type_create();

part_type_sprite(my_part,sprite0,false,false,false);
part_type_alpha1(my_part,0.2);
part_type_life(my_part,100000,100000);

Object0

Spoiler

execute code:

init_particles();
test=0;

Step Event:

execute code:

part_particles_create(global.p_system,random(600),random(460),my_part,1);
test+=1;

Draw Event:

execute code:

draw_text(0,0,"FPS = " +string(fps) + "Number" + string(test));

[True Valhalla]

Creating a new object for each sprite is just idiotic. No one can call these results fair.

This is very obviously biased.

[Nocturne]

Guys, you don't expect the company that makes the software to create un-biased tests do you? In what world do you live in!?! When I see INDEPENDANT blogs/sites doing these tests (and far more rigorously too) then I may sit up and take notice...

[True Valhalla]

Exactly.

[iSeiren]

I've used construct 2, and sorry for swearing but its a load of ****.

And remember, Game Maker HTML5 is still in beta.

[YellowAfterlife]

Their test is not more than useless.
What it checks, is instance creation time. For that, behaviour-driven Construct obviously wins (with barely any code being generated for dummy objects). In comparison, GameMaker initializes some variables and processes them each step (else you don't get magical local variables for everything!). Probably similar thing applies to GameSalad.
At back end, all HTML5 game creation tools use canvas.context.drawImage in 2D canvas. There just isn't other way.

For their comparison with Stencyl, I cannot even say how stupid that is - Stencyl enables Box2D physics for every object, causing a slower creation time and processing at each step, unless object (actor) is configured to.

Of more to Flash, since version 11 hardware-accelerated rendering is available - (obviously) beats WebGL, because of running quite closer to hardware (and not being interpreted language, since AS3).

You can also say that Scirra's post serves solely purpose of making them 'look better than everything else', because they do not include stats of free HTML5 game making programs & frameworks like Tululoo or Canvaslord. It'd be hard to say "But with WebGL, it is much wiser to buy our software than use free alternatives", don't you think?

Desert Dog: Tiles should have aproximately same effect. For mocking purposes, you could create an example where images are drawn into surface

[iSeiren]

Guys, I wouldn't worry too much about this. It's incredibly easy to beat their scores if we really wanted to. Their test is "noddy" in the extreme, and the method used biased towards how their product works. We prefer to base tests on real games as it's what it's all about anyway. We aren't going to be drawn into a "speed" war as we know that with full scripting we can easily render much higher numbers and produce an identical result by simply printing stuff from within a single object, on top of this, with full scripting, we can easily "cheat" by using surfaces and the like to produce even better looking results with little effort - it's meaningless.

If someone makes a game that we feel should run okay and it's not, we'll happily look at it, poke it with a stick and see if we can help, but so far everything people have tried to do, has been fine. The most notable exception has been some games which use tens of thousands of particles, and the basic canvas will just never do that.

So don't worry about "this kind" of performance, worry about making great games because that's all that really matters.

Mike, what browser do you suggest to run GM:HTML5 games on android? With the stock browser, opera and firefox - games run really slowly.

Any ideas?

[iSeiren]

Yeah, bundled browsers on Android are crap. Opera Mobile is by far the best. Massive speed boost on the devices we tried.

Will try and get back to you.

[paul23]

Guys, I wouldn't worry too much about this. It's incredibly easy to beat their scores if we really wanted to. Their test is "noddy" in the extreme, and the method used biased towards how their product works. We prefer to base tests on real games as it's what it's all about anyway. We aren't going to be drawn into a "speed" war as we know that with full scripting we can easily render much higher numbers and produce an identical result by simply printing stuff from within a single object, on top of this, with full scripting, we can easily "cheat" by using surfaces and the like to produce even better looking results with little effort - it's meaningless.

If someone makes a game that we feel should run okay and it's not, we'll happily look at it, poke it with a stick and see if we can help, but so far everything people have tried to do, has been fine. The most notable exception has been some games which use tens of thousands of particles, and the basic canvas will just never do that.

So don't worry about "this kind" of performance, worry about making great games because that's all that really matters.

Well the tests give some good insight to the performance difference in webgl .

As for the test, just for fun I ran it through a profiler (not that I can say much looking at the obfuscated source ><):
1 function takes around 26% of the execution time, though then again it probably can't be optimized at all: average execution time is 0.032ms.

Profiling also showed me clearly where a lot of potential goes to: most individual time per function call goes to functions being called each step. I moved the instance creation code from the step event to a alarm event (called every other step) to identify the difference between drawing & instance creation.
I see quite a few functions which started (at instance count = 0) at a speed of about 0.05 ms in the first steps. However it grows exponentially to 3500 ms at 200 instance count (notice I've got a debugger + profiler + quite a few other programs open). - This isn't what you expect from drawing (expecting it to grow linearly). It looks like behaviour I expect to see with collision checking.

[iSeiren]

With room speed set at 30, and on a fairly decent android tablet. A simple test of a box with on-screen controls that moves the box around runs at 10fps.

[iSeiren]

That was opera mobile, firefox ran at 60fps for like 5 seconds then dropped to 5.

[Desert Dog]

Good post Yellow, makes a lot of sense to me.

Ashley from Scirra clarified the method he meant.

Construct 2 doesn't separate objects and sprites like GM does. A Sprite in Construct 2 is an object which contains its own sprite. We wanted to make the test fair so we took the same approach with all the tools. [keep reading]

So in their blog post when they say 'sprite' it isn't meaning just an image! However, it is apparent that it's very light-weight compared to a GM object(and all that other stuff), and that would seem to be the point they proved.

I suppose one could argue having a light-weight 'base' unit gives you an advantage when devving for something like iPhones&other slow devices.

[millzyman]

I just tried their program and i had the following thoughts (literally what i thought while using it):

1st Minute: Oh okay its jammed, have to start it again
1st Minute: Where do i add an object
1st Minute: Oh found it, had to right click
2nd Minute: No sprites come with it? Go to the sprites included from Game Maker
2nd Minute: Hmm... add some physics i guess
3rd Minute: Add a floor and make it solid
3rd Minute: Where do i test my game?
4th Minute: Oh in a html browser - good good
4th Minute: Nothings loading...
5th Minute: Nothings loading...
6th Minute: This is stupid [exit program]

I will say this, they advertise that it requires no programming, they forgot to mention that it is very unuser friendly and you have to go on a treasure hunt to find anything. It may claim that it is 'faster' but at least Game Maker is usable. Result: Game Maker makes Construct 2 look lousy and unprofessional.

[YellowAfterlife]

millzyman: Your opinion is also biased. If you have expected to 'get hang' of program in 6 minutes, you might be living in wrong reality.

iSeiren: "fairly decent android tablet" is to be classified. From my personal exprience (tests) with the following game (on-screen controls; basic game; basic framework) I can say that different Android devices can run HTML5 games at different maximum speed, somewhat relvant to screen redrawing (slow buffer update?). Thus, adding frame skip (N steps per screen redraw) can grant a decent performance boost.

Desert Dog: point of that post isn't quite clear then - objects are rarely used as idle sprites in anything apart Construct (in GameMaker, you'd use tiles or surfaces; Stencyl supports tiles as well). I'd say, everyone have their likes - some would use GameMaker for specific game, some would use Construct or other program, and other people (not as many?) would code game from scratch to make sure that everything works as they want it to.

paul23: sudden expotential performance loss can be caused by debugger itself - if you have used tracing (printing to console), so-called trace overflow may occur (when debugger is running out of power to handle data passed to it, normally lagging out itself and object of inspection).