251 replies to this topic

[uuf6429]

Why would you build a whole compiler anyway? Wouldn't C++/C# export be enough?

Depends on use and need. That is, a small native compiler vs shipping C++/C# runtimes + SDKs + the compiler/translator, distributing some of them may also require a license, which is costly and mostly impractical.

This is what I'd like to understand better. Mark has made similar comments, and I certainly believe him.

But what is it about GM's architecture specifically that makes compilation such a challenge? What makes it different from other compiled applications?

The bytecode supports a different type of interpreter then machine-code with the machine (obviously). There would be lots of shortcuts in the interpreter which in case of compiled code, it would require more code to be written. Simply put, byte code and native code are not 1:1.

I think 'complete rewrite' is a bit of an overstatement. The editor can be kept more or less without changes because all it really does is produce a file containing all the resource descriptors and associated data.

I was talking about the runner, sorry for not being clear.

Contrarary to popular belief, the compiled executables are still large.

I can assure you all that GM's size is mostly because of half of the VCL embedded in it.
When I wrote up this image converter dll, I had to use some parts of the ImageList functionality. It increased the dll by 300Kb (graphics unit only)! (I ended up not including the unit and put some custom code instead.)
GM uses little parts of different units, causing it to become so large. As an example, it only uses a single Form and draws stuff on it without any use of GDI/Delphi VCL so, using the Forms unit is a huge overkill.
Delphi is great in many aspects, but size is quite a problem with Delphi when it comes to VCL. VCL is made for apps using loads of controls/components and detailed GUI elements, none of which required in GM.

I've not used BB, and have no idea what it does, however as I said the engine becomes considerably (in hundreds of kilobytes) smaller by using a dedicated one, rather then relying on a bulk of ready-made stuff.

Critics may point out intelligent, conditional compilation where the compiler doesn't build whole units into the final executable, but rather use what's needed. Pascal examples:

program IntelligentCompilation;

implementation

procedure ShowHello; // this is compiled
begin
  writeln('hello');
end;

procedure ShowBye; // this is not
begin
  writeln('bye');
end;

begin
  ShowHello;
end.

program ConditionalCompilation;

implementation

begin
  {$IFDEF WIN32}
  writeln('Welcome to windows!'); // compiles on windows but not others
  {$ELSE}
  writeln('Error, unknown operating system.'); // compiles on others but not windows
  {$ENDIF}
end.

I've still yet to see why this doesn't always work as intended in the case of Delphi.

Kind regards,
Chris.

Edited by uuf6429, 17 July 2009 - 08:35 AM.

[KC LC]

I can assure you all that GM's size is mostly because of half of the VCL embedded in it.
[snip]
Delphi is great in many aspects, but size is quite a problem with Delphi when it comes to VCL. VCL is made for apps using loads of controls/components and detailed GUI elements, none of which required in GM.

That may be a bit over-stated. But I agree that's a dilemma lots of developers face. They want to add functionality with 3rd-party libraries. And sometimes they have to embed features they don't need, just to get the ones they want.

But this seems like a red herring. It's a challenge developers face with any library -- not just Borland's VCL. And it's an issue whether or not GM is compiled.

[uuf6429]

Using a different library (like KOL), or making their own, should decrease the overall size. This may not be too hard - Delphi comes with all of the VCL's sources.
I admit I'm veering out of topic, since doing the above decreases the size of the interpreter too. However, with compilation (as GearGod suggested), particular parts of the library may be left out - producing an even smaller executable.

I'll show you a demo of what I'm talking about (just for the fun of it). So stay tuned

Edit: Here you go. http://covac-softwar...L vs WinAPI.zip VCL => 384Kb || WinAPI => 16Kb

Edited by uuf6429, 17 July 2009 - 06:00 PM.

[GearGOD]

However, with compilation (as GearGod suggested), particular parts of the library may be left out - producing an even smaller executable.

I'm pretty sure what I actually said is that there won't be much change in final size because virtually all components of the engine are put to use in any game.
You also don't need to do any compilation to do selective linking. DLLs are a much simpler approach. So I think we can leave the 'smaller filesize' argument as reduced to 'maybe 300k smaller'.

[smart_monkey]

I think Mark should move game maker away from a stand-alone executable into something more flexible. Even if it was not compiled but interpreted, compilation of executable would be faster (as it would be smaller), possibly more advanced DLL scripts, if you didnt use some features in your game (ie. Built-in sound) then the sound DLL wont be needed lowering total size of all contents.

Maybe if Game Maker was designed like some other game designing studio's.

[uuf6429]

I'm pretty sure what I actually said is that there won't be much change in final size because virtually all components of the engine are put to use in any game.

My bad, miss-quoted you. Maybe it was icuurd? I admit I don't remember much.

You also don't need to do any compilation to do selective linking. DLLs are a much simpler approach. So I think we can leave the 'smaller filesize' argument as reduced to 'maybe 300k smaller'.

Fine by me, I didn't promise any miracles

[TheMagicNumber]

VCL => 384Kb || WinAPI => 16Kb

You do know, they could use VCL in GML, it'd help many people (that want UI), while making that extra 400kB a bit more useful. This could still be done somewhat easily in GM, while it's interpreted. That includes customizing UI elements and even event handling.

So lets get back on topic here, we are talking about Compiling GML.

So ENIGMA can "compile" GML, and all it really does is translate the GML to C++. Now, compiling GML will have advantages (like speed, security, etc) but will also have disadvantages, losing a lot of the dynamic stuff. The compiled game cannot deal with GML directly as it would require the GML to either be compiled (to machine code, may not be easy), or interpreted.

So, GML would most likely lose a lot of features if it were compiled. Things like adding objects at runtime, execute_* functions, etc.

[MitchGraham]

Those dynamic functions are designed to link the expert to the novice, providing easy ways for us to make "one-time" scripts that perform an action (by making a custom object). Without this dynamic approach, game maker will become less easy to use to some extent. This is the inverse to YoYoGames desires.

The only likely thing we'll be seeing is trimming of the language, cutting away some less used features and scripts. We may also see re-writes of some key features which are used every day, to squeeze every piece of speed possible out of them. This will provide the size and speed boost you desire, and it much easier on YoYoGames behalf. It also pleases them, as novices will have less to learn, making it more friendly, and encouraging more people to buy it.

However, as it looks for the next 6 years, or maybe just half a dozen, YoYoGames will not have the manpower or reason to write a real compiler, nor is it required. The most logical choice on their behalf is actually to release the runner to a select few, one at a time. After getting slightly familiarized, these select few would be able to increase or improve a certain aspect of the runner. People like GearGOD, Uuf, Yourself, and many others- the ones who really know what they're talking about. This is YoYoGames smartest move, but also a move they probably will not take.

[uuf6429]

Things like adding objects at runtime

Thought you should know that that is not true:

for i:=0 to n do TObject.Create;

If you where referring to classes, they can be emulated by using a base class (eg, a class having basic GM object stuff like x,y,direction,sprite_index etc) and adding stuff to it.

I think that using loose translation, compilation gives a load of features not yet available to GM, such as adding/using data types GM doesn't use at present.

Edited by uuf6429, 19 July 2009 - 12:15 PM.

[TheMagicNumber]

Things like adding objects at runtime

Thought you should know that that is not true:

for i:=0 to n do TObject.Create;

If you where referring to classes, they can be emulated by using a base class (eg, a class having basic GM object stuff like x,y,direction,sprite_index etc) and adding stuff to it.

The objects would be hardcoded if compiled, so adding them at runtime is just hard. Now, you'd be unable to make that object useful. You can't really set its events at runtime, this makes it useless.

And in your example code, that is more like instance_create. You are creating more instances of TObject, not making a new class ("Object").

Edited by TheMagicNumber, 19 July 2009 - 01:02 PM.

[DarkSentinel]

Is adding objects at runtime even useful? I've only ever heard of it being used as a workaround for GM's limitations.

[TheMagicNumber]

It can be useful in certain cases, such as having external objects (or editable objects).

[the_mysterious_gamer]

Right now, the main area I think we should discuss is whether the way we use GM would be changed for the worse by compilation. I don't see how compilation would require any changes to the current GML architecture, including dynamic scripting (as I discussed in my first post). If someone would also make a coherent argument as to why as good runtime checking as there is currently would not be possible, that would be helpful.

The objects would be hardcoded if compiled, so adding them at runtime is just hard. Now, you'd be unable to make that object useful. You can't really set its events at runtime, this makes it useless.

Objects are just abstractions, like variables, when it comes down to the assembly, there's nothing there that actually defines them, and therefore there isn't anything you need to add to have dynamically created ones. When they are created at runtime, it will create a new structure in a global table of objects, which holds information such as variables, events, size in memory. For events, a JIT compiler would emit the machine code directly into RAM at runtime as a new function, and the pointer to this data is held in the objects info structure. For variables, memory is allocated on the heap and pointed to in another table.

[LoopStan]

I don't know if I should contribute with this subject on the matter : Enigma. I am going to briefly say a few words about it.
I believe this is something we could take a glance at as "Compiling Gm". From what I know, it is compiling gm. Open your game in LGM, and with the extension compile it. I don't know if it is the smartest thing to post this, or if its even allowed in a topic as such, but here you go. If Enigma can do it (for the most part) then we are already a step ahead of where we think we are.

[[UTS]ShadowX]

I don't know if I should contribute with this subject on the matter : Enigma.  I am going to briefly say a few words about it.
I believe this is something we could take a glance at as "Compiling Gm".  From what I know, it is compiling gm.  Open your game in LGM, and with the extension compile it.  I don't know if it is the smartest thing to post this, or if its even allowed in a topic as such, but here you go. If Enigma can do it (for the most part) then we are already a step ahead of where we think we are.

However, Enigma goes through loops and bounds and is completely recoding GM's functions. Enigma is essentially only converting the code to C++ and having the compiler go from there. Rather than a true compiler, it's merely exploiting GM's similar to C++ syntax. It does not compile GML. It compiles C++. For example, this valid GML script would turn in to doo-doo the minute it hit the C++ compiler.

x = true
if x and 1
game_end()

The only thing Enigma does is add a backbone that supplies a var data-type, a game framework (instantiates objects, etc), and replicate (only some at the moment) all of GM's functions. While it is still an ingenious idea and ultimately works, it is not a true GML compiler.

[Clam]

However, Enigma goes through loops and bounds and is completely recoding GM's functions. Enigma is essentially only converting the code to C++ and having the compiler go from there. Rather than a true compiler, it's merely exploiting GM's similar to C++ syntax. It does not compile GML. It compiles C++. For example, this valid GML script would turn in to doo-doo the minute it hit the C++ compiler.

If by "turn into doo-doo" you mean "not work" then you might want to check your claims. Enigma does quite a bit more than just blindly copy GML and hope it's been written as valid C++. Your example would be parsed and translated into C++ fine.

[[UTS]ShadowX]

However, Enigma goes through loops and bounds and is completely recoding GM's functions. Enigma is essentially only converting the code to C++ and having the compiler go from there. Rather than a true compiler, it's merely exploiting GM's similar to C++ syntax. It does not compile GML. It compiles C++. For example, this valid GML script would turn in to doo-doo the minute it hit the C++ compiler.

If by "turn into doo-doo" you mean "not work" then you might want to check your claims. Enigma does quite a bit more than just blindly copy GML and hope it's been written as valid C++. Your example would be parsed and translated into C++ fine.

When I tried out Enigma it was unable to use a script like that, I would get an error by simply not having a semi-colon; however that was a good while ago. If it is able to do so now then it does follow GML's syntax, and my statement is no longer valid. Back when I used it it had no input support; I had to manually place the C++ code in the source. By all means it is an actual GM compiler, but it's still long from finished.

One of my favorite features of Enigma however would have to be using C++ code inline with the GML.

[jmd]

Enigma is essentially only converting the code to C++ and having the compiler go from there. Rather than a true compiler, it's merely exploiting GM's similar to C++ syntax. It does not compile GML. It compiles C++.

I have never used Enigma and probably won't but would like to clear something up, I don't know what you class a true compiler but even tools such as GCC only translate a given language into a lower level language (assembly language) and then call the assembler and linker to generate the binary.

If Game Maker was ever to compile I would imagine this is how it would be done. I doubt that Game Maker would ever generate binary files itself as this would be incredibly hard work. GM could also take advantage of the massive effort put into tools such as GCC like the code optimisation. Generating C++ output and running it through a optimising compiler would probably be far more beneficial to us as users and the people working on the compiler, it would also make it alot easier to produce binaries for different architectures and operating systems. Outputting a language like C++ would also help the when interacting with the third party libraries it would reduce the complexity of the code generation greatly.

I do believe that creating a compiler for Game Maker is not as great a benefit as it seems. I know I havn't seen the GM architecture but I can imagine it would have to be changed in some way to allow for better use as a compiler. Also turning Game Maker into a compiler suggests it would/should become more of a proffesional tool and I can see many ways in which we use Game Maker that would have to change to reflect this new status, I myself have ran into quite a few ways the interface is limited.

I think there would be many easier ways Game Maker executables could become better such as the runner becoming more efficient (which I believe it is). Maybe even using tuned up graphics systems optimized for Game Maker and many other ways, I'm sure you guys here on the GMC have a few thorts on this matter.

I'm not claiming to be an expert in computer science and I don't know everything that goes on with Game Maker as I am more of an on-looker but I can't see compilation really being the best thing for Game Maker.

Thanks
Joe Doherty

If anyone can see errors in my post please put me right I havnt been around the GMC in a while and may have overlooked alot of whats been going on in GM development

[uuf6429]

The objects would be hardcoded if compiled, so adding them at runtime is just hard. Now, you'd be unable to make that object useful. You can't really set its events at runtime, this makes it useless.

And in your example code, that is more like instance_create. You are creating more instances of TObject, not making a new class ("Object").

You can make new classes as well, at least in Delphi. You can even store classes and move them around just as if they were objects (aka instances).
As to setting their events, it's also possible. The only thing you're missing is execute_string, so you'd have to hard-code events. But you can switch between different events easily.

If someone would also make a coherent argument as to why as good runtime checking as there is currently would not be possible, that would be helpful.

That's just a misconception. With the right translation method, even runtime checking is very possible, maybe even better then there is currently.This misconception comes from the fact that programmers keep coding stuff badly, to the extent that others make it sound good (it's even slipped into a couple of standards). Example, someone may "index" sprites to sounds by using their id's. Since both IDs are numbers, drawing sprite 1 and playing sound 1 works on both IDs. However, in compiled languages, the IDs doesn't work this way (unless the translator attempts to force this). A valid example would be making a game with no resources except one room and the following room initialization code:

if(sprite_add(...)==sound_add(...)) show_message('They match');

The above shows "match" in all GM versions except a theoretic compiled one.

I have never used Enigma and probably won't but would like to clear something up, I don't know what you class a true compiler but even tools such as GCC only translate a given language into a lower level language (assembly language) and then call the assembler and linker to generate the binary.

You should look up the difference between a compiler and a translator. Enigma is a translator, GCC is a compiler.

I'm not claiming to be an expert in computer science and I don't know everything that goes on with Game Maker as I am more of an on-looker but I can't see compilation really being the best thing for Game Maker.

Even if GM isn't worth to be compiled, I believe this topic should keep on going because a lot of features available with compilation could be implemented into GM as is.

Kind regards,
Chris.

[jmd]

You should look up the difference between a compiler and a translator. Enigma is a translator, GCC is a compiler.

Yes you are right but that's only terminology, the principal of the tools is still the same, take one language and translate into another.

I do agree with you however that alot can be taken from the principles of compiler design and applied throughout Game Gaker but I would imagine at this late stage alot of the areas have already been covered/implemented.

It would be very nice to see a flow diagram of how Game Maker generates executables it would help with more accurate discussion but I doubt we will ever see that.
To be honest without an full idea on how GM internals work there isn't very much discussion we can have apart from discussing general compiler design/implementation.

Regards
Joe Doherty