84 replies to this topic

[SyncViews]

This library provides other library writers an API to directly access various features of GM.

Only for VC++ for now, and only tested on the 10.0 compiler (2010) with GM 8.1.141. Some older 8.1 GM versions should work, back perhaps to when get_function_address was added.

You need to call the gm::init method with the address of get_function_address, as in the test dll. After that the other functionality should work (creating gm::Value objects, calling gm functions, etc.).

Doxygen Documentation

Current Features

• Call GM methods.
• Access some built in variables for the self and other instances.
• Access user defined variables.
• Supports multiple GM 8.1 versions.
• Provide the ability to call all Game Maker functions (any missing?).
• Provide the ability to get an instance pointer from an instance id.
• Access to some GM resources, such as sprites and textures.

Planned for 1.0.0

• Support common toolsets.
• Support all commonly used GM versions.
• Ensure everything works when mutliple versions of the GMAPI built by different toolsets with different configurations are present in a single game, everything works.

Downloads

Source 0.0.7: 7z zip
Binaries 0.0.7: 7z zip
Binaries archives include headers, and compiled builds for dll/static, debug/release. Requires VC++ Runtimes)
SVN
Get older versions from http://willnewbery.com/gmapi/

Change List
0.0.7 - 2012-09-28
Direct3D8 header usage is now opt-in. Define GMAPI_USE_D3D, else the interfaces will be typed to void.
Added access to GM's Sprite and Texture objects, including the IDirect3DTexture8 objects used.
gm::call adds an extra empty argument since it appears to be expected by certain GM functions.
Changed CPU instructions inserted into the start of external_call (HOOK_VERSION changed to 3).
Added some files for Code::Blocks and GCC and corrected some issues. These platforms are still unsupported.

0.0.6 - 2012-08-07
Added access to GM's IDirect3DDevice8*.
Added "unsigned argcnt, Value *args" overloads to some GM functions.
Completed code to handle multiple GMAPI instances in a single process (when 2 or more extensions use different GMAPI builds in a single game).
Added the RefString class to accept the readonly borrowed ref strings from GM.
Added the ability to get and set existing user defined variables and array elements (just zero initialise them in GML first or such).

0.0.5 - 2012-07-29
Rewrote all the string interfacing code. String objects are now used rather than a pointer and a few utility methods.
Wrote a simple GM unit test program to go with the Test project.

0.0.4 - 2011-10-17
Added particle functions.
Added d3d functions.
Added more comments to public header files.
Added some more gm::Instance fields.
Added functionality to get instance pointer from an id.
Corrected self and other instance pointer register use.

0.0.3 - 2011-09-29
Fixed mistake with releaseStr
Moved all assembly to single file

0.0.2 - 2011-09-29
Initial public version

Edited by SyncViews, 28 September 2012 - 10:00 PM.

[Ember]

You made a typo on number nine. :3

And if you remake the API for GM, I will bow down to you.

[SyncViews]

Well point 5 is the real killer right now. Since my assembler isnt so good as to pick out exactly which function I need out of deep callstacks as well as working out all the stack and register parameters. From taking a look at an chr(65) call though, I really have to wonder how good this new Unicode support is, the process in my test case literally seemed to go along the lines of:

• Create a 1 element UTF-16 String
• Do some magic to pick a ANSI code page. Picked a value out of global memory, and then did some math and comparisons with it. There seems to be a lot of code here, but ultimatly it decided it didnt like any of a number of hardcoded options, and called the Win32 GetACP API... (I feel I'm missing somthing on the whole point here, since it seems to me that makes any value > 127 useless to us anyway since we dont know what chr(150) for example will give us?)
• Use MultiByteToWideChar with the ACP it picked, the UTF-16 String, and my char code
• Create a new UTF8 string from the UTF16 string (havnt managed to dig through this, I imagen it comes to a call to WideCharToMultiByte with the CP_UTF8 constant.)
• Delete the UTF-16 string
• Return the UTF-8 string

Which is hardly the simplest thing for me to find the correct internal function and params. The other string functions dont really get much better, with constant encoding conversions... Don't suppose anyone here can read a little of x86 assembler and decode all the jump and mov instructions?

Edited by SyncViews, 15 September 2011 - 07:41 PM.

[amd42]

Found a few string functions (8.1.135). The assembly for the string functions is somewhat hard-to-understand, so it's possible that I may have missed something or misunderstood the inputs/outputs.

• 0x408350 converts a UTF16 string to a GM string. On input, edx = pointer to the null-terminated UTF16 string, and eax = memory location to store the resulting pointer to. If the value pointed to by eax holds something other than 0, the runner will decrease the reference count at that location.
• 0x408D00 allocates a new GM UTF16 string structure. On input, eax = string length, and on output, eax = pointer to the first character.
• 0x4072D8 decreases the reference count of a (UTF16?) GM string and frees it if it becomes 0. On input, eax = memory location of the pointer to the first character of the string. On output, the value pointed to by eax will be set to 0 regardless of whether or not the string was actually freed.

As far as working around the updater, I started working on some functions for scanning for function signatures a while ago but I can't find the source at the moment. I'll send it to you if I can find it.

[SyncViews]

Well I found that it seems there compiler doesnt optimise with custom calling conventions, so you can work out a bit from there:

Wikipedia x86 callign conventions:

Caller clean-up

Borland fastcall

Evaluating arguments from left to right, it passes three arguments via EAX, EDX, ECX. Remaining arguments are pushed onto the stack, also left to right.[5]

It is the default calling convention of Borland Delphi, where it is known as register.

Calls into the Win32 API's naturally use __stdcall and use a jump table. A good compiler will also show you the symbol (You might need to get the files off the MS Symbol Server)

Going from some of the assembler I found it seems safe to change the codepage and element size (Im thinking thats what unknown2 field in the string structure is right now) after allocated, and the decref seems safe with any string. So only 2 functions need to be found, the allocate string with length function, and the decref function.

I'm thinking these will both be in the Delphi runtime, and so the code shouldnt change between minor GM versions, just the locations, so should be possible to scan the process memory to locate them (everything but address would be the same).


I also found there is an array like struture of Delphi TTypeInfo objects that seemed in the versions I tested to always start at GetModuleAddress(0) + 0x1000. Not sure if this is useful or not, the old GMAPI seemed to use the TTypeInfo for the string class for somthing. I found some information on freepascals RTTI that seems to line up with the structures content (lots of those nasty variable size things).

Edited by SyncViews, 18 September 2011 - 12:20 PM.

[amd42]

Found the function that allocates a UTF8 string. It's at 0x407388 in v8.1.135. On input, eax = string length, and edx = code page (or 0 for the default). On output, eax = pointer to the first character of the allocated string.

There doesn't appear to be any sort of "global allocator" function that works for both UTF-8 and UTF-16 because Delphi uses different types for the different string sizes (AnsiString for UTF8 and UnicodeString for UTF16). If you really wanted to, I suppose you could make your own by calling Delphi's malloc at 0x404324 with eax = allocation size. The decref function that I found earlier should definitely work with both string sizes though.

I also found a white paper related to Unicode support in Delphi. If you scroll to page 9, you can find the details on the internal string implementation, and it indeed matches the struct you've mapped and also says that unknown2 is the char size as you had guessed.

[SyncViews]

I'll take a look at finding those in other versions to check my theory that there not changing every build (apart from addresses).

I'm working on a simple memory scanner with support for "unknown" bytes (where the addresses are located) to search for the location of a function. I think the chance of two functions being exactly the same apart from jump and call addresses is very small. If these core functions are staying the same between minor versions, it should work.

[SyncViews]

Ok, I checked your results against mine and they seem to line up. I had the string release function, and some of those allocate ones I annotated but didnt pin down as the basic string creation functions

I'm going to start with just the essentails, the creation and destruction of UTF8 strings. The right hand column is the memory contents for the assembly, with wildcards added. I think this should locate the desired functions in any GM version. Ive only had time to compare to a limited extent against a few versions and havnt written the search code yet, so if anyone has got some time to check these against different GM versions that would be very useful.

DelphiString *delphiNewUtf8(length, codepage=0)
00407388  test        eax,eax                       85 C0
0040738A  jle         004073CF                      7E ??
0040738C  push        eax                           50
0040738D  add         eax,0Eh                       C0 0E
00407390  jo          004073CA                      70 ??
00407392  and         eax,0FFFFFFFEh                83 E0 FE
00407395  push        edx                           52
00407396  push        eax                           50
00407397  call        00404324                      E8 ?? ?? ?? ??
0040739C  pop         edx                           5A
0040739D  pop         ecx                           59
0040739E  mov         word ptr [edx+eax-2],0        66 C7 44 02 FE 00 00
004073A5  add         eax,0Ch                       83 C0 0C
004073A8  pop         edx                           5A
004073A9  mov         dword ptr [eax-4],edx         89 50 FC
004073AC  mov         dword ptr [eax-8],1           C6 40 F8 01 00 00 00
004073B3  test        ecx,ecx                       85 C9
004073B5  jne         004073BD                      75 ??
004073B7  mov         ecx,dword ptr ds:[6868D0h]    8B 0D ?? ?? ?? ??
004073BD  mov         edx,ecx                       89 CA
004073BF  mov         word ptr [eax-0Ch],dx         66 89 50 F4
004073C3  mov         word ptr [eax-0Ah],1          66 C7 40 F6 01 00
004073C9  ret                                       C3
004073CA  jmp         00405EA4                      E9 ?? ?? ?? ??
004073CF  xor         eax,eax                       31 C0
004073D1  ret                                       C3

void delphiFreeStr(DelphiString *str)
004072D8  mov         edx,dword ptr [eax]           8B 10
004072DA  test        edx,edx                       85 D2
004072DC  je          004072FA                      74 ??
004072DE  mov         dword ptr [eax],0             C7 00 00 00 00
004072E4  mov         ecx,dword ptr [edx-8]         8B 4A F8
004072E7  dec         ecx                           49
004072E8  jl          004072FA                      7C ??
004072EA  lock dec    dword ptr [edx-8]             F0 FF 4A F8
004072EE  jne         004072FA                      75 ??
004072F0  push        eax                           50
004072F1  lea         eax,[edx-0Ch]                 8D 42 F4
004072F4  call        00404340                      E8 ?? ?? ?? ??
004072F9  pop         eax                           58
004072FA  ret                                       C3

Edited by SyncViews, 28 September 2011 - 07:38 PM.

[icuurd12b42]

Good, I do hope people continue the GMAPI stuff for 8.1 and above. I have a few dead dlls waiting for this

[SyncViews]

Was about one click away from giving everyone a working version last night, and figured Id test the release build... So now I'm releasing 0.0.2 Only for VC++ for now, and only tested on the 10.0 compiler (2010) with GM 8.1.135. Some older 8.1 GM versions should work, back perhaps to when get_function_address was added (I might look at supporting 6/7 in the future, however I lack installed full copies of those and YoYo seems to have made them unavailable).

You need to call the gm::init method with the address of get_function_address, as in the test dll. After that the other functionality should work (creating gm::Value objects, calling gm functions, etc).


Source 0.0.2
Binaries 0.0.2 (Includes headers, and binaries for dll/static, debug/release. Requires VC++ Runtimes)
SVN

[icuurd12b42]

I dont compile with MSVC anymore. I hope someone will convert to code::blocks

[SyncViews]

Well if somone can give a quick rundown on getting the assembly to compile on GCC, I dont think there are that many other things to deal with (I'm assuming here any C/C++ programmer can make there own project/makefiles as required ).

Edited by SyncViews, 29 September 2011 - 03:40 PM.

[SyncViews]

The assembly blocks are:

    __declspec(naked) void hookFunction()
    {
        __asm
        {
            ;save stuff
            push eax
            push edx
            
            call getShared
            
            pop edx
            mov dword ptr [eax]Shared.self, edx
            pop edx
            mov dword ptr [eax]Shared.other, edx
            
            ;overwritten code
            push        ebp  
            mov         ebp,esp  
            add         esp,0FFFFFE64h  
            ;push        ebx  
            ;push        esi  
            ;push        edi  
            jmp hookReturn
        }
    }

    static Value docall(Function f, Instance *self, Instance *other, unsigned argcnt, Value *args)
    {
        Value retv;
        Value *retvptr = &retv;
        __asm
        {
            push args
            push argcnt
            push retvptr
            mov eax, other
            mov edx, self
            mov ecx, argcnt
            call f
        }
        return retv;
    }

Need to take a closer look at this one anyway...

    GMAPI_DLL void releaseStr(char *gmstr)
    {
        if(gmstr)
        {
            /*__asm
            {
                mov eax, gmstr
                call delphiReleaseStr
            }*/
        }
    }

    GMAPI_DLL char *newStr(unsigned len)
    {
        __asm
        {
            mov     esi, len
            mov     edi, 0
            mov     eax, len
            mov     ecx, 0
            mov     ebx, 0
            mov     edx, 0
            call    delphiNewUtf8
            ret
        }
    }

Naturally I could put these into there own assembly files (allthough I kind of like inline asm being able to easily access C++ types and namespaces), however I recall the gnu assembler uses different syntax as well (to masm), so im not sure that solves anything.

[icuurd12b42]

Right... the inline assembly is cool but in code::block the method is different. I would put all the asm in a few inline function in a separate file where all your asm is located and can be modified easily

[SyncViews]

I moved the assembly to its own file and fixed the string release code. See first post.

[Houdini]

Just wanted to throw out my support for this project. There are a TON of libraries that can no longer be maintained due to the changes in GM 8.x which breaks the old GMAPI lib. Also, don't forget to add support for viewing/modifying custom variables in objects (as opposed to just built in variables).

Keep up the good work SyncViews!

- Houdini

Edited by Houdini, 06 October 2011 - 03:31 PM.

[Ember]

Just wanted to throw out my support for this project. There are a TON of libraries that can no longer be maintained due to the changes in GM 8.x which breaks the old GMAPI lib. Also, don't forget to add support for viewing/modifying custom variables in objects (as opposed to just built in variables).

Keep up the good work SyncViews!

- Houdini

Like GMOgre3D D:

Best extension ever.

Edited by Ember, 06 October 2011 - 04:01 PM.

[SyncViews]

Also, don't forget to add support for viewing/modifying custom variables in objects (as opposed to just built in variables).

You have the variable_* functions in GM to access such things for now, including builtin vars I havnt located yet.

[Houdini]

You have the variable_* functions in GM to access such things for now, including builtin vars I havnt located yet.

Hmmm, I was not aware of that. Looks like I'll need to download the latest 8.1 and check out some of the new additions!

- Houdini

[Houdini]

Also, don't forget to add support for viewing/modifying custom variables in objects (as opposed to just built in variables).

You have the variable_* functions in GM to access such things for now, including builtin vars I havnt located yet.

Ah, I looked into this briefly and while it would work I don't believe it would be suitable for my needs. I haven't ran tests yet in GM8.1, but I ran extensive tests in GM 7 and calling any kind of GM function was EXTREMELY slow. I believe directly changing/retrieving variables directly via the old GMAPI lib was much quicker than using the variable_* functions.

Guess I need to setup a new test bed to see if this any of this has changed in GM 8...

- Houdini