Jump to content


Photo

GmkSplitter


  • Please log in to reply
21 replies to this topic

#1 Medo42

Medo42

    GMC Member

  • GMC Member
  • 322 posts

Posted 11 June 2011 - 10:31 AM

This exists for a while already, but I noticed that I never introduced it here. Note that this is not an extension or dll, but rather a standalone tool that offers functionality not provided by Game Maker itself.

GmkSplitter is primarily a tool for collaboration. It does not offer any collaboration functions itself, but it enables you to use existing version control tools like git, mercurial and bazaar for Game Maker games. These tools allow you to keep track of all the old versions your project went through, to find out who changed what, but most importantly, to work on different branches of development and later merge the changes together.

The problem in using these version control tools with .gmk files is that .gmk files are big binary blobs. Version control tools are usually written to work with text files, so they will not be able to understand the changes to a .gmk file between versions. That is essential for many of the functions offered by these tools though.

What GmkSplitter does is this: It converts between GM source files (.gmk or .gm81) and an SCM friendly "file tree" format. The file tree format is a directory which looks a bit like the folder structure of your resources in Game Maker, and there are separate files for e.g. Global Game Settings and Game Information, and subfolders for Sprites, Sounds, Objects and so on. The information for each resource is stored in an individual .xml file. Sounds store the original sound file, and Sprites are stored as several .png files (one for each frame of the animation).

This file tree format can then be managed with the usual version control tools, as I mentioned above. The workflow is a bit more complicated than using these tools normally: To work on the source and build it, you convert it back to a .gmk and edit it normally. Before you can commit, you first delete the old file tree directory and recreate it from the new .gmk.

Almost all information is preserved. The most notable exception are Triggers, because they are not properly supported by the library which I use for reading and writing the Game Maker source files. Action libraries are not yet handled either. Apart from that, some bits of information are purposely not stored because they are usually not needed and would cause unneccessary conflicts in version control. For that reason, the timestamp and the ID for every resource are lost in the conversion, except for the ID of objects, which is important for the execution order of events.

In the Gang Garrison 2 project, we have been using this tool in combination with Git for almost a year now, and I tested that it can convert source files as big as that of Mushroom Kingdom Fusion. You can take a look at the Gang Garrison 2 source tree in our repository on Github, to get a feel for what it looks like: https://github.com/M...ster/Source/gg2

You can download the tool at https://github.com/M...itter/downloads
It is a console tool which requires Java. The general usage for the exe version is:
gmksplit.exe input output
If you want to convert from Game Maker source to the file tree format, input is a .gmk or .gm81 file and output is the name of a directory that does not yet exists. For converting the other way around, input is the name of a source tree directory and output is the name of the .gmk or .gm81 file that should be created. For example, to split gg2.gmk into a directory called gg2:
gmksplit.exe gg2.gmk gg2
Not that difficult, huh?

I will try to answer questions about the tool and how it can be used with version control systems. I will not explain how to use git, mergurial or bazaar, there are perfectly good tutorials, mailing lists, IRC channels, documentation pages and other ways of getting help.

Edited by Medo42, 09 December 2011 - 05:13 PM.

  • 1

#2 Revel

Revel

    ɹǝqɯǝɯ ɔɯƃ

  • GMC Member
  • 4935 posts
  • Version:GM8

Posted 11 June 2011 - 04:32 PM

The most notable exception are Triggers, because they are not properly supported by the library which I use for reading and writing the Game Maker source files.


Are you using the x2048 (Zach's) library?

Since he didn't post the source you can't patch it yourself, but ill let him know that your having problems with triggers


EDIT: Nevermind, it looks like you used java. I missed that :unsure:

Otherwise, this took looks extremely useful :)

Edited by ugriffin, 11 June 2011 - 09:32 PM.
GM decompiler... same website.

  • 0

#3 ugriffin

ugriffin

    Idiot

  • GMC Elder
  • 1591 posts
  • Version:Mac

Posted 11 June 2011 - 05:59 PM

Not sure if this is breaking the rules, but me thinks it is.

Send me a PM if you think otherwise.

Topic locked.


Awright, had a discussion with the admins, and this is sort of not breaking the rules.

Sort of.

So keep stuff within the rules, people. :)

Topic unlocked.
  • 0

#4 slam drago

slam drago

    The slam drag

  • New Member
  • 422 posts
  • Version:GM:Studio

Posted 11 June 2011 - 09:09 PM

I seen this on yoyogames. I think its kind of useless.
  • 0
My newest website. Check it out!
I support Cryset Games. Make money playing and making games!
(\__/)
(='.'=) This is Bunny. Copy and paste Bunny into your
(")_(") signature to help him gain world domination.
My avatar is best viewed in the lighter GMC theme.

#5 cspotcode

cspotcode

    GMC Member

  • New Member
  • 3 posts

Posted 11 June 2011 - 11:23 PM

I seen this on yoyogames. I think its kind of useless.

If you don't have experience using version control then you probably won't understand why this tool is incredibly useful.
  • 1

#6 Skarik

Skarik

    GMC member

  • GMC Member
  • 1068 posts
  • Version:GM:Studio

Posted 20 July 2011 - 10:58 PM

I am log in to say thank and bye

This thing does awesome muchly

I do have but one request on this, however: an argument to load the images in a way that is GM6/7 compatible. That just means exporting no transparency in the image but saving it in the XML file.

Unfortunately, I am too much of a lazy ass to shell out 30 bucks for GM8 ((that's like a week's worth of food, man!) although that will probably change within the next thirty minutes) and I also use black for many of the outlines for some of my character's portraits. Problem is, guess what BG color is used for a PNG that's exported with this program? Yeah, that's right, BLACK. OH MY GOSH NOCI I CAN SEE THROUGH YOUR BOOBS

Besides that, everything works great. Deliciously so. I just had my little 10 year old bro apply tilesets for about 10 rooms and then commit it to SVN. Delicious. Tantalizingly so.

*Hold up, that may be a problem with LGM163. PERFORMING MORE SCIENCE

**NOPE I THINK ITS A COMBO. A WOMBO COMBO. NOT, NOT REALLY. LGM164 DOES THE SAME THING, JUST WORSE.
PLEASE FIX OR ELSE I'LL BE TEMPTED TO WASTE 30 DOLLARS INSTEAD OF MAKING MORE ROOMS
AND EVERYBODY KNOWS MORE ROOMS IS BETTER THAN SPENDING THIRTY DOLLARS
UNLESS IT'S A SANDWICH
THAT'D BETTER BE A DAMN GOOD SANDWICH

***DAMN IT YOU FORCED ME TO BUY GM8 I HOPE YOU'RE HAPPY

Edited by Skarik, 21 July 2011 - 04:00 AM.

  • 0

#7 Mayhem Games

Mayhem Games

    Proud Kiwi

  • GMC Member
  • 959 posts
  • Version:GM:Studio

Posted 25 July 2011 - 08:29 AM

I have been using this for awhile, i even used it on my Game Maker to Java attempt (Since my GMK decompiler failed =D). It is more usefull than people make it out to be but it is limited to a few projects.
  • 0

For a long time it puzzled me how something so expensive, so leading edge, could be so useless, and then it occurred to me that a computer is a stupid machine with the ability to do incredibly smart things, while computer programmers are smart people with the ability to do incredibly stupid things. They are, in short, a dangerously perfect match. ~ Bill Bryson


#8 Medo42

Medo42

    GMC Member

  • GMC Member
  • 322 posts

Posted 25 July 2011 - 06:46 PM

I am log in to say thank and bye

This thing does awesome muchly

I do have but one request on this, however: an argument to load the images in a way that is GM6/7 compatible. That just means exporting no transparency in the image but saving it in the XML file.

Unfortunately, I am too much of a lazy ass to shell out 30 bucks for GM8 ((that's like a week's worth of food, man!) although that will probably change within the next thirty minutes) and I also use black for many of the outlines for some of my character's portraits. Problem is, guess what BG color is used for a PNG that's exported with this program? Yeah, that's right, BLACK. OH MY GOSH NOCI I CAN SEE THROUGH YOUR BOOBS

Besides that, everything works great. Deliciously so. I just had my little 10 year old bro apply tilesets for about 10 rooms and then commit it to SVN. Delicious. Tantalizingly so.

*Hold up, that may be a problem with LGM163. PERFORMING MORE SCIENCE

**NOPE I THINK ITS A COMBO. A WOMBO COMBO. NOT, NOT REALLY. LGM164 DOES THE SAME THING, JUST WORSE.
PLEASE FIX OR ELSE I'LL BE TEMPTED TO WASTE 30 DOLLARS INSTEAD OF MAKING MORE ROOMS
AND EVERYBODY KNOWS MORE ROOMS IS BETTER THAN SPENDING THIRTY DOLLARS
UNLESS IT'S A SANDWICH
THAT'D BETTER BE A DAMN GOOD SANDWICH

***DAMN IT YOU FORCED ME TO BUY GM8 I HOPE YOU'RE HAPPY

Hi there. I didn't even see that you posted more until now, since you only edited your post.
If i understand you right, you have an image with one "transparent" color, which is saved as an actual transparent png, but when importing it back to GM the png transparency is removed again and you always get a black background, right? I could try to find out where the original color goes missing, since it is possible to keep the colors intact in the transparent regions of a PNG. Would that be a good solution in your case?
  • 0

#9 Medo42

Medo42

    GMC Member

  • GMC Member
  • 322 posts

Posted 13 August 2011 - 05:53 PM

Well, I didn't address the sprite issue yet, but I released v0.14 now which modifies the handling of character encodings. .gm81 files are now treated as UTF-8 encoded and written this way as well, which matches the behaviour of the current GM 8.1 version. Older source files are now read/written with the system default charset, which matches what older versions of GM do. Previous versions of GmkSplitter always assumed and wrote ISO8859-1.

As long as you only use ASCII characters in your game, nothing changes for you. Otherwise GmkSplitter now behaves more like GM itself.
Download as always at https://github.com/M...itter/downloads
  • 0

#10 Medo42

Medo42

    GMC Member

  • GMC Member
  • 322 posts

Posted 15 August 2011 - 08:32 PM

WARNING: Version 0.14 has a bug that causes output .gm81 files to be corrupted if they contain strings with unicode characters. Sorry about that, I should have tested more carefully. On the upside, only one unlucky user downloaded that version. I'll upload a fixed version later today.
  • 0

#11 Medo42

Medo42

    GMC Member

  • GMC Member
  • 322 posts

Posted 15 August 2011 - 10:18 PM

WARNING: Version 0.14 has a bug that causes output .gm81 files to be corrupted if they contain strings with unicode characters. Sorry about that, I should have tested more carefully. On the upside, only one unlucky user downloaded that version. I'll upload a fixed version later today.

Released v0.15 to fix the problem. Download at https://github.com/M...itter/downloads as always.
  • 0

#12 Medo42

Medo42

    GMC Member

  • GMC Member
  • 322 posts

Posted 15 November 2011 - 12:52 AM

Yoyo Games are starting to adopt a new "Team Collaboration" ready format with their Game Maker HTML5, which is called GMX. This is an XML based file tree format which looks quite similar to what GmkSplitter produces. In other words, they finally support what GmkSplitter has been able to do for about a year now :)

Sensible decision, that. I hope GMX will make GmkSplitter unnecessary in the long run. As long as you have to pay extra to get this new format though, GmkSplitter will still have its place.

You can get an impression of the GMX format here (bottom of the first post):
http://gmc.yoyogames...howtopic=517146

Edited by Medo42, 15 November 2011 - 12:59 AM.

  • 0

#13 Medo42

Medo42

    GMC Member

  • GMC Member
  • 322 posts

Posted 08 January 2012 - 04:29 PM

Released v0.16

The primary resource groups (Sprites, Objects, Extension Packages etc.) are now determined by type instead of by name when reading .gmk files. This only really makes a difference for people using alternative tools to manipulate their .gmks because Game Maker always uses the same names for them.

Additionally, the splitter uses the latest version of LateralGM now, which might fix a problem somebody had with color being lost in transparent areas of sprites.

Download on Github as always.

The modified LateralGM version used for this project which contains a few custom fixes and improvements is now on Github as well. You can find it here: https://github.com/Medo42/LateralGM
  • 0

#14 wnsrn3436

wnsrn3436

    GMC Member

  • GMC Member
  • 194 posts
  • Version:GM:Studio

Posted 02 March 2012 - 08:56 AM

I love it!!!
Did not know you made ​​it!
I was looking for this a long time.
This will be helpful in my project.
Thank you. :laugh:

My Project : Inline Script Converter (Algorithm Project)

Edited by wnsrn3436, 02 March 2012 - 09:11 AM.

  • 0

#15 IceMetalPunk

IceMetalPunk

    InfiniteIMPerfection

  • GMC Elder
  • 9603 posts
  • Version:GM:Studio

Posted 05 March 2012 - 02:15 AM

This is pretty cool, even for people like me who don't use version control systems. I've played with it for only about an hour and a half (using Ether to simplify the XML parsing), and already I've gotten a system that loads all the room information in. I'm working on backgrounds now, then sprites, and sounds, and objects, etc.

What will I do with that information? I don't know; but I love that I can get it! :D Thanks for making this.

-IMP
  • 0

:GM123: Are you an artist who likes creating original creature designs? Maybe you can help out with Elementa here! Give it a look; you might like the idea :)

:bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny:


#16 Bleed

Bleed

    Chevalier

  • GMC Member
  • 763 posts
  • Version:GM:Studio

Posted 27 March 2012 - 06:22 PM

I see many possible uses for this nifty application, working at the moment to add .gm81 support to my VEditor with this, hope it works.
Question though : you say the end-user needs to have java installed but if i only use the gmksplit.exe should work without java right?

Edited by Bleed, 27 March 2012 - 06:46 PM.

  • 0

rlztjp_zpsoqffaixe.jpg


#17 IsmAvatar

IsmAvatar

    Good Samaritan

  • GMC Member
  • 2411 posts
  • Version:GM8

Posted 27 March 2012 - 08:14 PM

Bleed, I think you misunderstand something.

The "end-user" is the person playing your game. They do *not* need to have Java installed, unless your game somehow needs to use GmkSplitter during gameplay, in which case, you have a really weird game...

You, the game developer, need to have Java installed to use GmkSplitter, however. I'm pretty sure it's required, for you, the developer, regardless of if you use the "gmksplit.exe". The exe is just a simple wrapper for the Java application. It does not (as far as I know) include Java, nor does it bypass the need for java. All it does is provide an .exe, as most windows users are accustomed to running, to wrap the Java application in.
  • 0
Posted Image LateralGM is an IDE for editing GM files on Linux, Mac, and Windows.
We also house the GM File Format Documentation and other projects of that nature.
IsmAvatar.com | Examples | Scripts | Particles | DLLs | Credit Me
Email | Registered

#18 Bleed

Bleed

    Chevalier

  • GMC Member
  • 763 posts
  • Version:GM:Studio

Posted 27 March 2012 - 09:08 PM

Actually "end-user" is precisely the word i wanted to use, as i've said i intend to include this app with VEditor (which is a room editor for GameMaker, not exactly a game) as a .gm81 interpreter so yes it will use GmkSplitter at runtime to read .gm81 files and parse various properties to VEditor and since the application will end up with various "other" users, hence the end-user, i was wondering if they will need java installed, but thank you for already clearing that out.

Sorry for the misunderstanding and thank you again.

Edited by Bleed, 27 March 2012 - 09:15 PM.

  • 0

rlztjp_zpsoqffaixe.jpg


#19 TheGameMakerGuide

TheGameMakerGuide

    GMC Member

  • GMC Member
  • 52 posts
  • Version:GM:Studio

Posted 20 December 2012 - 03:04 AM

This tool is great and the only one of its kind, thanks :thumbsup:/>
  • 0
TGMG_BANNER.png
Check out my Game Maker video tutorials at: youtube.com/TheGameMakerGuide

#20 TheGameMakerGuide

TheGameMakerGuide

    GMC Member

  • GMC Member
  • 52 posts
  • Version:GM:Studio

Posted 30 May 2013 - 05:56 PM

I've been using GMkSplitter in a project for a few months without any problems, now the project file grew up to 4MB and since then I'm getting the following error message:
 
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
        at java.util.Arrays.copyOf(Unknown Source)
        at java.io.ByteArrayOutputStream.grow(Unknown Source)
        at java.io.ByteArrayOutputStream.ensureCapacity(Unknown Source)
        at java.io.ByteArrayOutputStream.write(Unknown Source)
        at org.lateralgm.file.StreamEncoder.write(StreamEncoder.java:63)
        at org.lateralgm.file.GmStreamEncoder.write(GmStreamEncoder.java:77)
        at org.lateralgm.file.GmStreamEncoder.writeBGRAImage(GmStreamEncoder.jav
a:238)
        at org.lateralgm.file.GmFileWriter.writeBackgrounds(GmFileWriter.java:45
5)
        at org.lateralgm.file.GmFileWriter.writeGmFile(GmFileWriter.java:116)
        at com.ganggarrison.gmdec.GmkSplitter.compose(GmkSplitter.java:135)
        at com.ganggarrison.gmdec.GmkSplitter.main(GmkSplitter.java:76)
 
Any ideas?
 
Thanks!
 
 
 
[EDIT]
 
Problem solved, I was using a stupidly large background: 8000x4000 pixels, discarded it and now everything is working fine again. I guess there must be some memory leak in the background routines in GmkSplitter, so I'll try not to use large backgrounds as a workaround.
 
Thanks!

Edited by TheGameMakerGuide, 30 May 2013 - 07:22 PM.

  • 0
TGMG_BANNER.png
Check out my Game Maker video tutorials at: youtube.com/TheGameMakerGuide

#21 Medo42

Medo42

    GMC Member

  • GMC Member
  • 322 posts

Posted 30 May 2013 - 07:52 PM

Maybe not a memory leak, but the tool definitely isn't optimized for low memory use. I'm not wild on editing LateralGM toward that end either though, so I'm glad it is working out for you again. A different possibility would be to increase the maximum Java heap size with a commandline parameter.

 

The images are stored uncompressed in memory - so that 8000x4000 background works out to ~128MB. Then to write them to Game Maker format, we need to swivel the order of colors around and we request an array with all the pixel data for that, which takes up another 128MB (though those are discarded again once the image is written).


  • 1

#22 TheGameMakerGuide

TheGameMakerGuide

    GMC Member

  • GMC Member
  • 52 posts
  • Version:GM:Studio

Posted 03 June 2013 - 03:49 PM

I increased the Java heap size to 2GB with the following parameter Xmx2048m, but it didn't work, that's why I thought the problem could be a memory leak. Your GmkSplitter is a great tool, really very useful. Thanks.

 

 


Edited by TheGameMakerGuide, 03 June 2013 - 03:53 PM.

  • 0
TGMG_BANNER.png
Check out my Game Maker video tutorials at: youtube.com/TheGameMakerGuide