Jump to content


Photo

Construct 2 vs GameMaker vs gamesalad, vs..


  • Please log in to reply
40 replies to this topic

#1 Desert Dog

Desert Dog

    GMC Member

  • GMC Elder
  • 6409 posts
  • Version:Unknown

Posted 23 November 2011 - 04:15 AM

http://www.scirra.co...alad-vs-stencyl

We're second, apparently.

They provided the .gmx of the files. It would appear they create a new object for each 'sprite'. Not exactly the fastest method on the planet. I actually can't load it for some reason.

I don't know much about Construct 2, but since the creators of it themselves did the test, I'd assume they optimised the heck out of it. :P

there is actually a topic of this on the Tigs forum here:
http://forums.tigsou...p?topic=22888.0

where one of the Scirra team(ScirraTom) is reading. He's a cool fella, if you have any questions about how they did the tests, ask him.

Anyway, time Yoyogames did their own benchmark. (I seem to recall Mike promising...)

And as for rules about no competing products.. yes, I know about the rule. Don't tell me about it. If a mod closes it, he closes it. I think the topic is well appropriate, however...

Edited by Desert Dog, 23 November 2011 - 04:16 AM.

  • 1

#2 Dangerous_Dave

Dangerous_Dave

    GMC Member

  • Global Moderators
  • 9413 posts
  • Version:Unknown

Posted 23 November 2011 - 04:33 AM

I skipped the text and just looked at the pictures, but the final graph is interesting. Note that the huge construct bar is run through WebGL, not supported by GM and not supported by all browsers. So ignore that. GameMaker has come second in the test, and although it's only about 60% of the value of construct, you can bet it would be closer if they tested on more that just the number of sprites shown on screen.

The fact that they are starting to benchmark is fantastic news. People hate losing, so I'd expect that all the HTML5 products will work on closing the gap.
  • 0

#3 Knuked

Knuked

    GMC Member

  • New Member
  • 242 posts
  • Version:GM:HTML5

Posted 23 November 2011 - 05:04 AM

Hmm, that's interesting! Each program has their pros and cons, I like GMHTML5 myself, but for $32 bucks, I'll give it a go just to see!
  • 0

#4 icuurd12b42

icuurd12b42

    Self Formed Sentient

  • GMC Elder
  • 16041 posts
  • Version:GM:Studio

Posted 23 November 2011 - 06:30 AM

I would debate the validity of the benchmark. I mean they are not the same test. I am tempted to do some of my own.
  • 0

#5 Desert Dog

Desert Dog

    GMC Member

  • GMC Elder
  • 6409 posts
  • Version:Unknown

Posted 23 November 2011 - 06:49 AM

I would debate the validity of the benchmark. I mean they are not the same test. I am tempted to do some of my own.


Why do you think I posted it here? :P I want to see Yoyogames do some!!

I asked what method they used to draw their sprites, and their going to get back on that. But otherwise, would be cool to see a few GM users come up with some optimised methods of drawing sprites.(I'll assume tiles don't count!)

I just had a go with particles, and although it did seem to be faster than their tests, it wasn't by a whole big deal. You can read what I did in spoiler tags below, if anyone else is wanting to make a faster one.
Spoiler

  • 0

#6 True Valhalla

True Valhalla

    ಠ_ಠ

  • GMC Member
  • 5277 posts
  • Version:Unknown

Posted 23 November 2011 - 07:57 AM

Creating a new object for each sprite is just idiotic. No one can call these results fair.

This is very obviously biased.
  • 0

#7 Nocturne

Nocturne

    Nocturne Games

  • Administrators
  • 21978 posts
  • Version:GM:Studio

Posted 23 November 2011 - 08:19 AM

Guys, you don't expect the company that makes the software to create un-biased tests do you? In what world do you live in!?! When I see INDEPENDANT blogs/sites doing these tests (and far more rigorously too) then I may sit up and take notice...
  • 0

#8 True Valhalla

True Valhalla

    ಠ_ಠ

  • GMC Member
  • 5277 posts
  • Version:Unknown

Posted 23 November 2011 - 10:19 AM

Exactly.
  • 0

#9 iSeiren

iSeiren

    www.worrall.pw

  • GMC Member
  • 613 posts
  • Version:GM:Studio

Posted 23 November 2011 - 07:14 PM

I've used construct 2, and sorry for swearing but its a load of ****.

And remember, Game Maker HTML5 is still in beta.
  • 0

#10 YellowAfterlife

YellowAfterlife

    GMC Member

  • Global Moderators
  • 4001 posts
  • Version:GM:Studio

Posted 23 November 2011 - 08:00 PM

Their test is not more than useless.
What it checks, is instance creation time. For that, behaviour-driven Construct obviously wins (with barely any code being generated for dummy objects). In comparison, GameMaker initializes some variables and processes them each step (else you don't get magical local variables for everything!). Probably similar thing applies to GameSalad.
At back end, all HTML5 game creation tools use canvas.context.drawImage in 2D canvas. There just isn't other way.

For their comparison with Stencyl, I cannot even say how stupid that is - Stencyl enables Box2D physics for every object, causing a slower creation time and processing at each step, unless object (actor) is configured to.

Of more to Flash, since version 11 hardware-accelerated rendering is available - (obviously) beats WebGL, because of running quite closer to hardware (and not being interpreted language, since AS3).

You can also say that Scirra's post serves solely purpose of making them 'look better than everything else', because they do not include stats of free HTML5 game making programs & frameworks like Tululoo or Canvaslord. It'd be hard to say "But with WebGL, it is much wiser to buy our software than use free alternatives", don't you think?

Desert Dog: Tiles should have aproximately same effect. For mocking purposes, you could create an example where images are drawn into surface :P
  • 1

#11

  • Guests

Posted 23 November 2011 - 08:44 PM

Guys, I wouldn't worry too much about this. It's incredibly easy to beat their scores if we really wanted to. Their test is "noddy" in the extreme, and the method used biased towards how their product works. We prefer to base tests on real games as it's what it's all about anyway. We aren't going to be drawn into a "speed" war as we know that with full scripting we can easily render much higher numbers and produce an identical result by simply printing stuff from within a single object, on top of this, with full scripting, we can easily "cheat" by using surfaces and the like to produce even better looking results with little effort - it's meaningless.

If someone makes a game that we feel should run okay and it's not, we'll happily look at it, poke it with a stick and see if we can help, but so far everything people have tried to do, has been fine. The most notable exception has been some games which use tens of thousands of particles, and the basic canvas will just never do that.

So don't worry about "this kind" of performance, worry about making great games because that's all that really matters.

#12 iSeiren

iSeiren

    www.worrall.pw

  • GMC Member
  • 613 posts
  • Version:GM:Studio

Posted 23 November 2011 - 09:01 PM

Guys, I wouldn't worry too much about this. It's incredibly easy to beat their scores if we really wanted to. Their test is "noddy" in the extreme, and the method used biased towards how their product works. We prefer to base tests on real games as it's what it's all about anyway. We aren't going to be drawn into a "speed" war as we know that with full scripting we can easily render much higher numbers and produce an identical result by simply printing stuff from within a single object, on top of this, with full scripting, we can easily "cheat" by using surfaces and the like to produce even better looking results with little effort - it's meaningless.

If someone makes a game that we feel should run okay and it's not, we'll happily look at it, poke it with a stick and see if we can help, but so far everything people have tried to do, has been fine. The most notable exception has been some games which use tens of thousands of particles, and the basic canvas will just never do that.

So don't worry about "this kind" of performance, worry about making great games because that's all that really matters.


Mike, what browser do you suggest to run GM:HTML5 games on android? With the stock browser, opera and firefox - games run really slowly.

Any ideas?
  • 0

#13

  • Guests

Posted 23 November 2011 - 09:06 PM

Yeah, bundled browsers on Android are crap. Opera Mobile is by far the best. Massive speed boost on the devices we tried.

#14 iSeiren

iSeiren

    www.worrall.pw

  • GMC Member
  • 613 posts
  • Version:GM:Studio

Posted 23 November 2011 - 09:18 PM

Yeah, bundled browsers on Android are crap. Opera Mobile is by far the best. Massive speed boost on the devices we tried.


Will try and get back to you.
  • 0

#15 paul23

paul23

    GMC Member

  • Global Moderators
  • 3857 posts
  • Version:GM:Studio

Posted 23 November 2011 - 09:35 PM

Guys, I wouldn't worry too much about this. It's incredibly easy to beat their scores if we really wanted to. Their test is "noddy" in the extreme, and the method used biased towards how their product works. We prefer to base tests on real games as it's what it's all about anyway. We aren't going to be drawn into a "speed" war as we know that with full scripting we can easily render much higher numbers and produce an identical result by simply printing stuff from within a single object, on top of this, with full scripting, we can easily "cheat" by using surfaces and the like to produce even better looking results with little effort - it's meaningless.

If someone makes a game that we feel should run okay and it's not, we'll happily look at it, poke it with a stick and see if we can help, but so far everything people have tried to do, has been fine. The most notable exception has been some games which use tens of thousands of particles, and the basic canvas will just never do that.

So don't worry about "this kind" of performance, worry about making great games because that's all that really matters.

Well the tests give some good insight to the performance difference in webgl :P.

As for the test, just for fun I ran it through a profiler (not that I can say much looking at the obfuscated source ><):
1 function takes around 26% of the execution time, though then again it probably can't be optimized at all: average execution time is 0.032ms.

Profiling also showed me clearly where a lot of potential goes to: most individual time per function call goes to functions being called each step. I moved the instance creation code from the step event to a alarm event (called every other step) to identify the difference between drawing & instance creation.
I see quite a few functions which started (at instance count = 0) at a speed of about 0.05 ms in the first steps. However it grows exponentially to 3500 ms at 200 instance count (notice I've got a debugger + profiler + quite a few other programs open). - This isn't what you expect from drawing (expecting it to grow linearly). It looks like behaviour I expect to see with collision checking.
  • 0

#16 iSeiren

iSeiren

    www.worrall.pw

  • GMC Member
  • 613 posts
  • Version:GM:Studio

Posted 23 November 2011 - 09:37 PM

With room speed set at 30, and on a fairly decent android tablet. A simple test of a box with on-screen controls that moves the box around runs at 10fps.
  • 0

#17 iSeiren

iSeiren

    www.worrall.pw

  • GMC Member
  • 613 posts
  • Version:GM:Studio

Posted 23 November 2011 - 09:46 PM

That was opera mobile, firefox ran at 60fps for like 5 seconds then dropped to 5.
  • 0

#18 Desert Dog

Desert Dog

    GMC Member

  • GMC Elder
  • 6409 posts
  • Version:Unknown

Posted 23 November 2011 - 09:47 PM

Good post Yellow, makes a lot of sense to me.

Ashley from Scirra clarified the method he meant.

Construct 2 doesn't separate objects and sprites like GM does. A Sprite in Construct 2 is an object which contains its own sprite. We wanted to make the test fair so we took the same approach with all the tools. [keep reading]


So in their blog post when they say 'sprite' it isn't meaning just an image! However, it is apparent that it's very light-weight compared to a GM object(and all that other stuff), and that would seem to be the point they proved.

I suppose one could argue having a light-weight 'base' unit gives you an advantage when devving for something like iPhones&other slow devices.
  • 1

#19 millzyman

millzyman

    GMC Member

  • GMC Member
  • 572 posts
  • Version:Unknown

Posted 23 November 2011 - 09:53 PM

I just tried their program and i had the following thoughts (literally what i thought while using it):

1st Minute: Oh okay its jammed, have to start it again
1st Minute: Where do i add an object
1st Minute: Oh found it, had to right click
2nd Minute: No sprites come with it? Go to the sprites included from Game Maker :P
2nd Minute: Hmm... add some physics i guess
3rd Minute: Add a floor and make it solid
3rd Minute: Where do i test my game?
4th Minute: Oh in a html browser - good good
4th Minute: Nothings loading...
5th Minute: Nothings loading...
6th Minute: This is stupid [exit program]



I will say this, they advertise that it requires no programming, they forgot to mention that it is very unuser friendly and you have to go on a treasure hunt to find anything. It may claim that it is 'faster' but at least Game Maker is usable. Result: Game Maker makes Construct 2 look lousy and unprofessional.
  • 1

#20 YellowAfterlife

YellowAfterlife

    GMC Member

  • Global Moderators
  • 4001 posts
  • Version:GM:Studio

Posted 23 November 2011 - 10:35 PM

millzyman: Your opinion is also biased. If you have expected to 'get hang' of program in 6 minutes, you might be living in wrong reality.

iSeiren: "fairly decent android tablet" is to be classified. From my personal exprience (tests) with the following game (on-screen controls; basic game; basic framework) I can say that different Android devices can run HTML5 games at different maximum speed, somewhat relvant to screen redrawing (slow buffer update?). Thus, adding frame skip (N steps per screen redraw) can grant a decent performance boost.

Desert Dog: point of that post isn't quite clear then - objects are rarely used as idle sprites in anything apart Construct (in GameMaker, you'd use tiles or surfaces; Stencyl supports tiles as well). I'd say, everyone have their likes - some would use GameMaker for specific game, some would use Construct or other program, and other people (not as many?) would code game from scratch to make sure that everything works as they want it to.

paul23: sudden expotential performance loss can be caused by debugger itself - if you have used tracing (printing to console), so-called trace overflow may occur (when debugger is running out of power to handle data passed to it, normally lagging out itself and object of inspection).
  • 1

#21 paul23

paul23

    GMC Member

  • Global Moderators
  • 3857 posts
  • Version:GM:Studio

Posted 23 November 2011 - 10:58 PM

You might very well be true about that: further testing didn't show this exploding increase in time.
  • 0

#22 Rusky

Rusky

    GMC Member

  • GMC Member
  • 2492 posts

Posted 24 November 2011 - 01:50 AM

In general, these benchmarks, especially with how close GM and Construct (with canvas) were, don't mean too much, like Mike said- individual games will have a lot of opportunities to optimize away the kinds of things that slowed GM down here.

The test is also somewhat biased toward how C2's objects work, but that doesn't make it meaningless. A behavior system, besides being a faster baseline by including less, is able to sustain that advantage in real games by removing unused variables/updates/etc.

Besides performance, behaviors are also much more flexible than GM's single inheritance parenting. I would really like to see something similar in GameMaker.
  • 0

#23

  • Guests

Posted 24 November 2011 - 08:28 AM

Well, as I said... it's a meaningless test. If you really want to beat it, I'm pretty sure if you simple create a couple of arrays with X and Y coords, then print them all from within a single loop, you'll get exactly the same result, but it should be much faster than even theirs. Really... take benchmarks with a massive pinch of salt - even ours. They are designed to test a single aspect of a single program, and that's fine if they way to profile an area of their program, but not only does it depend on all programs doing things the same way (which clearly isn't so), but it's so easy to circumvent the test but make it look the same that benchmark wars are pointless.

Lastly... remember making games is all about fun, and you usually cheat to make things "look" like like it's doing much more. This is true on even the most powerful of systems, and we believe that this kind of developer lead performance is only possible with full scripting, as that gives you the developer the best chance to produce the effects you want without the engine getting in the way.

LOL... I could easily draw something that looked and behave identical, but would draw one more sprite to a surface each frame and then draw the single surface each loop. That would mean I have 1 object, and 2 sprites drawing a frame - no matter HOW many it looked like. Results matter, not how you code them. pointless.

#24 Smarty

Smarty

    GMC Member

  • GMC Elder
  • 7470 posts
  • Version:GM:Studio

Posted 24 November 2011 - 08:37 AM

So the people who made Construct did a performance test of Construct against other game development tools and amazingly, Construct concludes that Construct is better than their competitors.

What do you mean, biased?
  • 0

#25 millzyman

millzyman

    GMC Member

  • GMC Member
  • 572 posts
  • Version:Unknown

Posted 24 November 2011 - 09:28 AM

millzyman: Your opinion is also biased. If you have expected to 'get hang' of program in 6 minutes, you might be living in wrong reality.


Your right, i was making a point - probably to the wrong people. I was exaggerating and taking out of context what they said. Besides to make a balanced opinion could you not just add two oppositely baised opinions and add them together :P (of course just joking)
  • 0

#26

  • Guests

Posted 24 November 2011 - 11:41 AM

What do you mean, biased?


I mean that whenever you make a product, you're obviously going to promote it on what you think are it's strengths, and try and make others look bad because that's how you sell products. Just as we focus on GML because we believe it's the best and fastest way of doing things, they believe their way is best. I know we can do that test better if we tried, but they they obviously aren't going to spend ages trying to get the best performance out of GameMaker because it simply isn't in their interest.

#27 Smarty

Smarty

    GMC Member

  • GMC Elder
  • 7470 posts
  • Version:GM:Studio

Posted 24 November 2011 - 11:48 AM

Mike... I know. I think your sarcasm meter is broken. :whistle:
  • 3

#28

  • Guests

Posted 24 November 2011 - 11:53 AM

Mike... I know. I think your sarcasm meter is broken. :whistle:

DOH.... I'd add you to my list of death, but I'm sure your on there several times already - damn you! :biggrin:
(I really need some sleep... :confused: )

#29 NakedPaulToast

NakedPaulToast

    GM Studio/Mac/Win

  • GMC Member
  • 8288 posts
  • Version:GM:Studio

Posted 16 December 2011 - 09:18 PM

I'll probably regret bringing up this benchmark again, but since GM:HTML5 now supports WebGL I thought I would revisit.

Let me preface this; their benchmark, as is mine; is complete hogwash. Often benchmarks can have some validity to them, I believe a benchmark between GM, Construct, Stencyl, whatever simply isn't possible. These engines have their similarities but their differences and implementations of components make comparisons completely unreasonable.

On my machine (I7, 12 Gig RAM) using their benchmark before the rendered images fell under 30 FPS.
Their Construct benchmark using Chrome with WebGL rendered ~23,000 sprites per step before falling under 30FPS
The GM Benchmark created by Scirra using Chrome rendered ~4,400 sprites (and instances).

This is the comparison that caused them to pat themselves on the back and post the following disingenuous comment.

We won't embarrass them by comparing their result to our WebGL renderer!


So let's re-run the benchmark with GM:HTML5 - WebGL enabled - it rendered 13,000 images before falling under 30FPS.

A huge improvement, but Construct is still running 1.76 times faster than GM:HTML5

OK so let's take Mike Dailley's advice.

If you really want to beat it, I'm pretty sure if you simple create a couple of arrays with X and Y coords, then print them all from within a single loop, you'll get exactly the same result, but it should be much faster than even theirs.


That's what I did. I changed the benchmark, so instead of creating new instances, I populated two arrays saving the x,y values then rendered the increasing number of sprites within a for-loop in the draw event.

My results:
~ 48,000 images rendered before performance dropped under 30FPS

That's four times faster than Scirra Construct's rendering.

So what can we learn from the following:
Benchmarks between such dissimilar engines simply isn't reasonable.
Benchmarks between GM:HTML5 and GM:HTML5 (WebGL) are reasonable seeing a throughput increase of 300% (though that doesn't just represent the rendering improvement).

But most importantly some of the best ways to increase performance was just to do things differently and focus on optimization. Simply by drawing the same sprite multiple times instead of creating multiple instances performance increased by 370%.

Oh yeah, and WebGL is F^&%@#@ fast.

Edited by NakedPaulToast, 16 December 2011 - 09:32 PM.

  • 3

#30 Nocturne

Nocturne

    Nocturne Games

  • Administrators
  • 21978 posts
  • Version:GM:Studio

Posted 16 December 2011 - 09:27 PM

<SNIP>
But most importantly some of the best ways to increase performance was just to do things differently and focus on optimization. Simply by drawing the same sprite multiple times instead of creating multiple instances performance increased by 370%.

Oh yeah, and WebGL is F^&%@#@ fast.


This... Probably the most important aspect of programing for GM:Html5 and it can't be stressed enough!
  • 0




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users