Jump to content


Photo

GMAPI v0.0.7 2012/09/28


  • Please log in to reply
84 replies to this topic

#61 SyncViews

SyncViews

    GMC Member

  • GMC Member
  • 392 posts

Posted 12 August 2012 - 05:56 PM

Apparently in my own code I only ever called a script without arguments so never ran into this before.

From what I can tell looking at the script_execute disassembly, it seems to expect argcnt+1 elements in the argument array for some reason. It creates a new (stack allocated) argument array, and then runs a loop argcnt times copying the arguments into the new array, starting from args[1], which naturally over runs the original array and crashes. So I guess GM has an extra empty value on the ends of its argument arrays for some reason, cant think how it would possibly be useful and such an element is referenced by none of the fixed-argument functions as far as I can see (but then they often seem to not even bother checking the argument count or types).

Not got time to really work on it till tomorrow night, but if you want to try something on the off chance I found these 2 internal functions in there and have a pretty good idea what they do.
getScript
void *getScriptPtr = 0x004048B8;
__asm
{
    mov eax, scriptId
    call getScriptPtr
    mov scriptPtr, eax
}
scriptExecute
void *scriptExecutePtr = 0x00588AF8
__asm
{
    ;arguments, do not include the scriptId, there the args the script sees in argument0, argument1, etc
    push argcnt
    push args
    push 15 ;max args? was hardcoded
    push 1; really no idea this time, again gm hardcoded the value
    push pretvar; A gm::Value* for the return value
    eax = scriptPtr
    edx = self; Instance*
    ecx = other; Instance*
}



The other thing to be aware of with any of the methods to call back into GML is that if it then calls a dll again (doesn't need to be yours, or any GMAPI extension, just anything that would use external_call) it will overwrite the stored self and other pointers, possibly to different instances, so either don't depend on self and other being the same afterwards, or save/restore them yourself (and in the event someone thinks multi threading is a good idea, then that gets even more complicated).


EDIT:

Also you could add the extra element yourselves:
    gm::Value args[] = {scr, arg1, arg2, gm::Value()};
    retstr = gm::script_execute(3, args).str;

Edited by SyncViews, 12 August 2012 - 06:57 PM.

  • 0

#62 icuurd12b42

icuurd12b42

    Self Formed Sentient

  • Global Moderators
  • 14389 posts
  • Version:GM:Studio

Posted 13 August 2012 - 02:00 AM

Well I was going by what it said here: http://wiki.codebloc...th_Code::Blocks
If I just create an executable project (e.g. console application), I can just press F8 (Debug->Start) and it will run the exe and stop at any breakpoints etc. But with the GM exe I cant get it to actually start anything (Build->Run works with the "Set program arguments" thing, but then there is no debugger attached)


Building to ensure sources are up-to-date
Build succeeded
Selecting target:
Debug
Adding source dir: E:\Projects\GMAPI\Test\
Adding source dir: E:\Projects\GMAPI\Test\
Changing directory to: ../GCC/Debug/
Adding file: ..\..\UnitTest.exe
Starting debugger:
done
Registered new type: wxString
Registered new type: STL String
Registered new type: STL Vector
Setting breakpoints
Debugger name and version: GNU gdb 6.8


Is there some C++ IDE using GCC on Windows that does just work for debugging GM DLL's (i.e. exe is not something it can build from source, you need to tell it to just run the exe)? I know a lot of people don't like to use MSVC, so I guess they worked this out already...


And that's when I use MSVC, now at 2010. I also installed the new code::blocks a few days ago. I setup my system so it compiles with both code::blocks and MCVC. I use MSVC to compile and debug. I love the re-compile while debugging... I use code::blocks for the release compile.
  • 0

#63 SyncViews

SyncViews

    GMC Member

  • GMC Member
  • 392 posts

Posted 13 August 2012 - 03:12 PM

script_execute should be fixed in the svn now. I added an empty value to all the gm::call overloads, although I still don't really see why GM seems to expect it, almost looks like a bug in script_execute (looping once too many times), but then why add an empty value as a "solution"?

I tried GCC some more, apparently nightly code blocks and whichever the latest release minGW was have a debugger that will actually work. It says there was a segmentation fault in gm::DelphiModifiableString<gm::ValueString>::release (2nd in callstack, first was "0018E4DE ??()"). I tried putting a breakpoint there and stepping into releaseStr, the result was hanging UnitTest.exe such that Code::Blocks or task manager could not terminate it... I guess the "0018E4DE" came from the stack getting corrupted, but how it can break so badly just by trying to step into the function....

Also some other bits, see the SVN log.

Edited by SyncViews, 13 August 2012 - 03:24 PM.

  • 0

#64 Houdini

Houdini

    GMC Member

  • New Member
  • 195 posts

Posted 13 August 2012 - 07:19 PM

SyncViews,

I was able to completely replace the old GMAPI with your new version in GMOgre3D and everything is running great. Really appreciate all the hard work you put into this project!

Something that you may want to consider adding in the future: the old GMAPI had a way to replace GML functions with C++ functions in the DLL itself, so calling a GML function from within GML would instead call the specified DLL function. I was looking into using this in GMOgre3D way back and found a ~25% speed improvement in GM 8.0 when calling GMOgre3D functions using this technique. Don't know hard something like this would be to add, but I thought I'd put the bug in your ear.

- Houdini
  • 0

#65 SyncViews

SyncViews

    GMC Member

  • GMC Member
  • 392 posts

Posted 14 August 2012 - 12:29 AM

Well I have no idea how the old GMAPI did it. The performance boost I suspect came from skipping the complex "translation" code of external_call.

Only thing I can think of is perhaps the array I found script_execute using is an array of function pointers (as opposed to pointers to say GML byte code) that can be replaced to let you call them like normal scripts.

The other option I can think of is to mod external_define and external_call into having some new call types that don't do the whole parameter translation thing, and instead just call the function with the gm::Value parameter array and returns


Either way, like most other features, perhaps even more so, it would be many times easier for yoyo to have done it given they have some nice source code to work with...


EDIT: Very big bit of code behind script_execute, spent over an hour and only done a very small part of the main function it calls. Interestingly its looking like the argument access in scripts actually access a global 16 element array, not some kind of "stack" values. A lot of the code I looked at seems to be the copying of existing values to the stack, new values to the array, zeroing out unused arguments, and then restoring it all.

Wonder how GM::Threads ever worked, seems even less thread safe than CPython, and they used a global lock so only one thread could execute the interpreter at a time....



Certainly makes the idea of getting GM to call a DLL function seem a lot more difficult, looking more and more like I'm going to find it just puts a pointer to some byte code somewhere and jumps into the interpreter code...

Edited by SyncViews, 14 August 2012 - 01:31 AM.

  • 0

#66 icuurd12b42

icuurd12b42

    Self Formed Sentient

  • Global Moderators
  • 14389 posts
  • Version:GM:Studio

Posted 14 August 2012 - 04:38 AM

This topic is great guys. I'm working on a dll for our game so I'm keeping an eye on the progress and any discussion about the cool things you can do...
  • 0

#67 SyncViews

SyncViews

    GMC Member

  • GMC Member
  • 392 posts

Posted 14 August 2012 - 02:44 PM

Well worked on it a bit more, but for the most part just ended up with more questions than answers, seems the "script objects" are pretty large, but I have little idea as to the purpose of most of them.

Certainly seems to be lots more code then I suspect is actually required, but then that sums up most of GM's code I have looked at with objects larger than they need to be, excessive copying, poorly thought out use of string encodings, etc etc...

Going through code like this is likly to take forever, since unlike the other "utility" functions, this goes right into GM's interpreter stuff at some point. Might be worth finding the memory values for the Script and Script2 objects and seeing if some common values come up (e.g. number of used arguments, references to globals, etc)
dword ptr ds:[686B10h]  = Script **scripts
Script * scr = scripts[script_id]
struct Script
	int u00
	int u04
	Script2 *scr2
	...?
struct Script2
	...
	int u04
	bool1 b08
	...
	int u10
	...
	Value u48
	int u60
	bool1 b64
	...?

Edited by SyncViews, 14 August 2012 - 02:46 PM.

  • 0

#68 ChaosMaker

ChaosMaker

    GMC Member

  • GMC Member
  • 288 posts
  • Version:GM:HTML5

Posted 15 August 2012 - 05:29 AM

interesting... there are any version to MinGW ?

im using codeblocks because

visual c++ is so much large to download with 3g...





  • 0

#69 SyncViews

SyncViews

    GMC Member

  • GMC Member
  • 392 posts

Posted 15 August 2012 - 10:06 AM

Well personally I think its worth it, especially for the debugger and ui polish.

Essentially the 2 things that make the Code::Blocks / minGW build broken right now, so if you want to use minGW you will have to find out what its issue is... Best I got is to make the debugger hang somewhere around the issue so that I need to restart Code::Blocks.... so its likly a small error on my part, but why on earth should that break the debugger...

Edited by SyncViews, 15 August 2012 - 10:10 AM.

  • 0

#70 daspirit

daspirit

    The Quiet

  • GMC Member
  • 500 posts
  • Version:GM:Studio

Posted 25 August 2012 - 12:15 AM

This is some high quality stuff. I haven't tested it but I've read the documentation. Everything seems to be done professional, I commend you to that. A few questions:

  • What is missing before a major release is finished?
  • What's with all those _pads in the Instance class? Are those simply variables that you haven't figured out, or are they variables unused by GameMaker?
  • Won't setting the Runtime Library under Code Generation to Multi-threaded (instead of the default VSC++ Multi-threaded DLL) allow you use the DLL without the need for the redistributable? Sorry if I'm wrong.

Again, props to you for making this. I hope you continue working on this, as it'll prove very important to many projects.
  • 0

#71 SyncViews

SyncViews

    GMC Member

  • GMC Member
  • 392 posts

Posted 25 August 2012 - 04:52 PM

Well mostly now just getting mingw to work.


There bytes that I was not able to account for. Some may be used by GM for something, others may just be to get other objects to be correctly aligned.


Well you could possibly do. The main issue is I have not designed GMAPI's interface to be safe if it and the modules using it get a separate heap, it might be safe but if it is that's just by chance.

Also I personally use the DLL version though out my projects code base since with several DLL's it adds up to more than the 600KB I need to redistribute the CRT anyway (using a merge module included in a msi installer), for GMAPI.dll alone its over 100KB to add the static CRT.
  • 0

#72 SyncViews

SyncViews

    GMC Member

  • GMC Member
  • 392 posts

Posted 30 August 2012 - 05:41 PM

Did anyone have luck getting themselves a mingw build? Not got my desktop right now since moved, but if not I think that will be the priority next week, and try and get a stable release that people can use.
  • 0

#73 icuurd12b42

icuurd12b42

    Self Formed Sentient

  • Global Moderators
  • 14389 posts
  • Version:GM:Studio

Posted 30 August 2012 - 11:24 PM

Did anyone have luck getting themselves a mingw build? Not got my desktop right now since moved, but if not I think that will be the priority next week, and try and get a stable release that people can use.


I did not have luck building the source in Dev Studio

fatal error C1083: Cannot open include file: 'd3d8.h': No such file or directory

I can tell looking as the AssemblyVc.cpp file, some changes will be needed to have it compile with the assembler. The mingw/Code::blocks inline assembler is different from masm...
  • 0

#74 SyncViews

SyncViews

    GMC Member

  • GMC Member
  • 392 posts

Posted 31 August 2012 - 05:25 PM

What about the AssemblyGcc file? I think that is on SVN at least.

d3d8.h is a Microsoft DirectX file, think I put macros ont he latest SVN so you can avoid it since the DirectX8 SDK is certainly not on most dev machines these days.
  • 0

#75 icuurd12b42

icuurd12b42

    Self Formed Sentient

  • Global Moderators
  • 14389 posts
  • Version:GM:Studio

Posted 31 August 2012 - 08:07 PM

What about the AssemblyGcc file? I think that is on SVN at least.

It's not in your distribution package (gmapi-source-0.0.6). I'll check with the SVN folders later when I get a chance.
  • 0

#76 SyncViews

SyncViews

    GMC Member

  • GMC Member
  • 392 posts

Posted 15 September 2012 - 10:23 AM

You have any luck with that?
Also are there any changes/improvements people want in the API's for existing functionality?
  • 0

#77 ChaosMaker

ChaosMaker

    GMC Member

  • GMC Member
  • 288 posts
  • Version:GM:HTML5

Posted 15 September 2012 - 03:02 PM

1>------ Build started: Project: Test, Configuration: Debug Win32 ------
1> Variables.cpp
1> Obj.cpp
1> Main.cpp
1> InstanceArray.cpp
1> D3d.cpp
1>c:\program files\microsoft visual studio 10.0\vc\include\gmapi\d3d.hpp(25): fatal error C1083: Cannot open include file: 'd3d8.h': No such file or directory
1> Generating Code...
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========





ill need DXSDK ?





  • 0

#78 SyncViews

SyncViews

    GMC Member

  • GMC Member
  • 392 posts

Posted 15 September 2012 - 10:28 PM

For your own extensions you should be able to define GMAPI_NO_D3D to prevent it from trying to include the D3D8 headers (which require an old SDK, since recent SDK's do not include D3D8). For the next release I think I will make it opt in though since that seems the more common use case (GMAPI_USE_D3D ?).

Compiling GMAPI itself without D3D may work, but it is not something I have tested.
  • 0

#79 icuurd12b42

icuurd12b42

    Self Formed Sentient

  • Global Moderators
  • 14389 posts
  • Version:GM:Studio

Posted 16 September 2012 - 02:18 AM

In Studio 2010 it's a simple matter to right click the file and exclude it from getting compiled.

Do any GM function require this to be included? would it remove the 3d functions from the API?
  • 0

#80 SyncViews

SyncViews

    GMC Member

  • GMC Member
  • 392 posts

Posted 16 September 2012 - 10:02 AM

Well I guess D3d.cpp, but simply excluding that cpp wont work. Also D3d.hpp and Texture.hpp can try to include the D3D headers. When using GMAPI, you should be able to just define GMAPI_NO_D3D to prevent it at present.

I guess I could support building GMAPI itself without D3D components, if lots of people want to embed it into there projects or use modified versions rather than prebuilt ones (please be careful with the hook/related core code though, the intention is to not prevent different extensions being used in the same game)

Edited by SyncViews, 16 September 2012 - 10:06 AM.

  • 0




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users