Jump to content


Photo

Gmfilesystem


  • Please log in to reply
283 replies to this topic

#1 paul23

paul23

    GMC Member

  • Global Moderators
  • 4147 posts
  • Version:GM:Studio

Posted 17 January 2013 - 02:29 PM

GMFileSystem
28/08/2014: version 1.5, the sky is the limit?


Written by: Paul Weijtens
Download: GoogleCode - Project home (source etc)
Dropbox extension (gex)
manual (pdf)

Size: 1.8 Mb (packed) 3.1 Mb (unpacked)
Platforms: Windows (depends on a dll, though everything is written in a portable manner so someone with a mac, contact me!)
License: New BSD license
Rationale
Now that GameMaker moved into a sandbox environment a gab has been created for those who like to do file management with GameMaker. This extension is build to fill the gab. As such one of the core components is to make the functions as transparent as possible; the functions work nearly identical to GM's build in. This means no helpfile is necessary, the help comes from the GM manual.



Currently supported:
  • *new* Unicode text
  • All file_text_* functions
  • All file_bin_* functions
  • Ability to add sprites, backgrounds & sounds
  • Conversion from and to all common image formats
  • Ability to export resources & save screenshots
  • All filesystem functions (and directory exist/create)
  • Extra binary functions to write larger pieces of data
  • file functions to check filestream state
  • All ini functions
  • Ability to handle multiple ini files at the same time
  • Lightning fast XML file functions.
Planned version
These are in my personal order of importance.
  • All functions in the filesystem area (file_copy, file_delete etc) (1.1)
  • Better failed-reading handling (I should actually allow you to fix problems) (1.2.1)
  • resources (sprite_add(), background_add() etc) (1.2)
  • ini_files (1.3)
  • xml_files (1.4)
  • YAML & JSON support
  • .... (open for suggestions)
What download
GMFilesystem has several downloads - and all older versions are still available. All downloads you can find at the google-code page (select all downloads instead of current ones to see old versions). Basically you should always use the newest version, and also download the newest manual (manual might be a few version behind though).

Spoiler


Spoiler


The latest version is 1.5 Downloaded from dropbox


Change log
Version 1.5.0
  • Ability to load files with "unicode"/wide names
  • For text files the ablity to read & write unicode text.
  • UTF8, UTF16, UTF32 supported completely - note that a single "character" may consist of multiple bytes
  • Unicode is convert from/to UTF8 on the fly for gamemaker
  • Automatic line breaking handled in text, set the default
Spoiler
Help?
I don't own a mac, so if someone thinks this extension is useful and wishes to create it for the mac, don't hesitate to contact me; I'd love to help you create a dylib version. Also I tried to keep it locale agnostic, however I don't really know how well it works with unicode. As far as I could test reading/writing data from/to files worked also for unicode files, about the filenames I have my doubts though, so any information would be kind there.
As always generic ideas for improvement are more than welcome, I'm still wondering how I will make the extension handle loading of image data (and passing it to GM).

Last but not least, if you use GMFileSystem & find it useful it would be kind if you leave a note here; The more people show they like/need this, the more likely I am to make progress!
  • 24

#2 wnsrn3436

wnsrn3436

    GMC Member

  • GMC Member
  • 194 posts
  • Version:GM:Studio

Posted 18 January 2013 - 06:54 AM

Good. I always need it. :thumbsup:
I'm going to use this in my studio games. And I hope to quickly add the ini system.

Ps, How about add a buffer function?

Edited by wnsrn3436, 18 January 2013 - 06:56 AM.

  • 0

#3 paul23

paul23

    GMC Member

  • Global Moderators
  • 4147 posts
  • Version:GM:Studio

Posted 18 January 2013 - 10:42 AM

Good. I always need it. :thumbsup:/>
I'm going to use this in my studio games. And I hope to quickly add the ini system.

Ps, How about add a buffer function?

The goal is to mimic GM's file io, so that anyone can simply download this extensions, replace his functions with FS_ in front of it and then it works exactly the same as gamemaker. (I went to go as far as to add the inconsistencies from gm to the file_find functionality).


Adding extra functionality is not a priority at this moment - though I might actually do that. However it might make sense to actually read a binary file into a buffer. Another interesting thing I might look into is the boost.asio library; asynchronous input/output should make sure that even when writing megabytes to the hard disk your program still runs.
  • 0

#4 TheSnidr

TheSnidr

    Heavy metal viking dentist

  • GMC Elder
  • 3345 posts
  • Version:GM:Studio

Posted 18 January 2013 - 10:49 AM

Wow, this is very useful! I might actually use this for my level editor, for loading in resources and levels and saving them in the actual game's directory - having to keep it all in the sandbox is a pain in the butt.
  • 2

#5 paul23

paul23

    GMC Member

  • Global Moderators
  • 4147 posts
  • Version:GM:Studio

Posted 23 January 2013 - 01:46 AM

Update 1.2 - Resource manipulating & basic extensions.

What's new
I've added functions to load new images (and sounds) from "anywhere". The image files (for backgrounds & sprites) can be either png or jpeg, later support for TIFF & bmp files is planned. Audio files have to be of MP3 format. The new functions require some space (around 350 Kb), though it is compiled in such a way they can actually be excluded from the project without excluding the core functionality. (see more about this later). What the extension does for images is convert the file to png, copy(?!) it to a temporary file, in a location where GM can read it. Then with the normal GM function the resource is included inside gamemaker.

Copy, isn't that slow?
Yes... No... And it doesn't matter.
To understand this, we have to notice two distinct ways the script can run: either the file is already a png, and no converting is necessary, or the file is a jpeg & it has to be converted.
When no converting is necessary, what the DLL does is create a hard-link in the working directory to the original file, a hard-link can be compared a bit with shortcuts (though any application will see just the file not the shortcut). As such no data has to be written to the physical hard-drive & only the operating system has to update some indices. This is very, very fast (around 7 milliseconds per file). It is also largely independent of the file size. - (There's a small catch, if you add the same file more than 1023 times without cleaning up the data inbetween it can't create more hard-links and it falls back on actually copying, which can be a factor 10 times as slow).
In the other case the file has to be converted from jpeg to png first & then written back to the hard drive. In this case also the hard drive speed isn't the driving factor. The bottleneck in this code is the converting from files, this has to be done on the pixel level and each pixel has to be checked. Typically this is 10 times as slow as actually writing the data to a the disk. On my (very old) pc a file of 1280*800 pixels took on average 500 milliseconds to convert & save. Of which around 50 were to actually write data on the disk. This method is (as you could expect) totally size dependent, smaller files are also near instant.

Extending an extension
The other update is "support" for extensions. This meant that only parts of the file can be included, while excluding other parts. Each part will get a distinct name, currently 2 names are used "GMFile" & "GMImage". To exclude one of those parts simply delete the file(s) that contain this name. This will allow anyone to use GMFileSystem extension in a perfect manner, without increasing filesize where not necessary. To give an overview:
GMFile       : GMImage
size        : 200 Kb       : 360 Kb
typical uses: text files   : sprites
            : binary files : backgrounds
            : filesystem   : sounds
files       : GMFile.dll   : GMImage.dll
            :              : GMImage.gml

  • 2

#6 paul23

paul23

    GMC Member

  • Global Moderators
  • 4147 posts
  • Version:GM:Studio

Posted 29 January 2013 - 02:44 PM

So I think I've stumbled on around 10-20 bugs, strange behaviour or simply missing core features in gamemaker itself over the course of the project. You would almost start to think that external resources are considered an "unwanted child" from yoyogames. Getting the last (exporting data) few features to work has been a real pain. Notice the "strange behaviour" described as standard in this report. The solution works flawlessly, but it is something you have to be careful off - and something this extension won't do for you!


Well another new week, another new release:
What's New?
Version 1.2.1 A lot of changes behind the schemes happened between 1.2 & 1.2.1. The biggest change is probably the swap from using boost::gil for image conversion to opencv. This change sped up the conversion by a factor 4, with peaks noted as 10 times as fast as boost::gil. An example conversion from a 1280*800 jpeg image to png took previously 500 milliseconds (half a second), with the new version and openCV it takes less than 120 milliseconds - almost 5 times as fast. While I was improving the speed of conversion I've also increase the png->png file copy efficiency, shaving 2 more milliseconds there (actually the efficiency is gained by improved checking if a file is a png file. It also enabled support for a lot more formats (bmp, tiff, jp2, ppm etc etc). The downside is a larger file size (from 350Kb to 2.5 Mb). Here you can see the change in a handy table:
             GMImage (1.2.0) | GMResource (1.2.1)
size       : 360 Kb          | 2537 Kb
speed      : ~500 ms         | ~120 ms
(1280*800) :                 |
features   : -sound          | -sounds
           : -png            | -png
           : -jpeg           | -jpeg
           : -Loading only   | -bmp
                             | -tiff
                             | -Loading & saving
Another big update is the change from using Host-A.net as prime hosting site to code.google.com project homepage. This allows me to actually share the source efficiently, and if you wish you might even compile the source yourself, using not-yet-released features. (Or and as with all free software, everyone if free to start committing updates). For now host-a.net will be used as a backup host.. Though in the future I might completely remove the code from there.
The third big change is the release of an official documentation, this documentation will probably always lag behind, though when major features are added I will update the documentation PDF
As a side project I have created a python script that automatically reads gml-scripts (a header in the script & a list of files described by a mask). And then creates an "extension part" (part can easily be copy-pasted in the extension.gmx file with a text editor).

There are also quite some changes in the feature lists
-Image saving is now working
-Image loading from more file types
-Setting the error states from text/binary file reading operations possible
  • 0

#7 iSeiren

iSeiren

    www.worrall.pw

  • GMC Member
  • 633 posts
  • Version:GM:Studio

Posted 29 January 2013 - 05:08 PM

I'll look at it later on, but from what i've read it looks great.
  • 0

Professional Mobile app Developer


#8 Belial

Belial

    GMC Member

  • GMC Member
  • 8 posts
  • Version:GM:Studio

Posted 02 February 2013 - 09:08 AM

this is awesome! thank you for this gex. it's a shame that gm studio not can handle this itself.
  • 0

#9 TeamSteeve

TeamSteeve

    GMC Member

  • GMC Member
  • 1359 posts
  • Version:GM:Studio

Posted 04 February 2013 - 02:21 AM

This is fantastic. I definitely will use this. :thumbsup:
FS_directory_exists() always returns true. :confused: You might want to fix that.

Also, I don't really know how the licensing works. I've never really used extensions much and I'm not familiar with what it all means. Can you explain in "layman's terms" please?
  • 0

TeamSteeveBanner.png


#10 paul23

paul23

    GMC Member

  • Global Moderators
  • 4147 posts
  • Version:GM:Studio

Posted 04 February 2013 - 02:59 AM

This is fantastic. I definitely will use this. :thumbsup:/>/>
FS_directory_exists() always returns true. :confused:/>/> You might want to fix that.

Also, I don't really know how the licensing works. I've never really used extensions much and I'm not familiar with what it all means. Can you explain in "layman's terms" please?

Well I use the BSD license - that is a weak-copyleft license. Copyleft means that I particularly say that anyone is (and should) be able to modify & redistribute the actual extension. When someone uses a weak copyleft license (as opposed to a strong copyleft license like GPL) it means that when you export it you are not forced to make your modification also open source, your modified GMFileSystem can be (as long as it has a different name - credits this) closed source copy of this.
Some licenses I use are:

GPL - strong copyleft, this license also indicates that anything that uses the dll (or extension) also must be licensed under GPL (thus also must be open source)
LGPL - strong copyleft, though when using a dll-extension it means that only the dll-extension source has to be made available. Not the source of the rest of the game. For scripts LGPL & GPL would be actually equal licenses
BSD/MIT - weak copyleft, means you don't have to distribute the original source

In the license I state you have to give a notice that you use this along with a copy of the "license" somewhere in the documentation/about field/ending credits. (Actually stating "GMFileSystem - copyright 2013, Paul Weijtens" somewhere in credits or extra included file would be enough for me). It also states that this projects comes as it comes: I do not say it works, nor can you make me responsible if any of your customers seem to find this dll not working...
  • 0

#11 TeamSteeve

TeamSteeve

    GMC Member

  • GMC Member
  • 1359 posts
  • Version:GM:Studio

Posted 04 February 2013 - 03:38 PM

Okay so for future reference, if a license states that I have to have a copy with it somewhere with the game, I could just copy the license text (e.g. your license from here: http://opensource.or...es/BSD-3-Clause) and include it in a license.txt file, right?

Thanks heaps for the reply and the awesome extension! :thanks:
  • 0

TeamSteeveBanner.png


#12 paul23

paul23

    GMC Member

  • Global Moderators
  • 4147 posts
  • Version:GM:Studio

Posted 04 February 2013 - 03:54 PM

Okay so for future reference, if a license states that I have to have a copy with it somewhere with the game, I could just copy the license text (e.g. your license from here: http://opensource.or...es/BSD-3-Clause) and include it in a license.txt file, right?

Thanks heaps for the reply and the awesome extension! :thanks:/>

Well the exact license itself is actually included inside the extension - in the license field of the extension - (the link at google is just show what the license holds):

Copyright © 2013, Paul Weijtens
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of Paul Weijtens nor the
names of its contributors may be used to endorse or promote products
derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL Paul Weijtens BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Basically what you linked with the fields filled in :P.
  • 0

#13 TeamSteeve

TeamSteeve

    GMC Member

  • GMC Member
  • 1359 posts
  • Version:GM:Studio

Posted 04 February 2013 - 04:06 PM

Awesome, thanks again. I'm exporting a program from GM:S with your extension as I'm typing! :medieval:
  • 0

TeamSteeveBanner.png


#14 paul23

paul23

    GMC Member

  • Global Moderators
  • 4147 posts
  • Version:GM:Studio

Posted 08 February 2013 - 12:14 PM

FS_directory_exists() always returns true. :confused: You might want to fix that.


I really can't seem to reproduce that bug. ie show_message(string(FS_directory_exists("c:\winddddows"))) shows "0" for me. Can you tell me what you used to create this bug? (Ow and please also verify that the string you use is non existsing.. Other than that: ini files have just been added to the working parts. The manual will be updated later..

What's New?
Version 1.3
Added ini-file support in version 1.3. Ini files have the same advantage as text files over GM:S (and in this case: all version of GM); you can use, write and read, ini files from anywhere. The ini file functions have also been made completely transparent. Which meant that for the standard functions only a single ini file can be opened at the same time. (Though unlike GM opening a new ini file while an old file is open does not result in a memory leak, the old file is the closed & saved).
However as this is quite an arbitrary thing I decided to add another set of ini-functions. Those functions have the extension *_ext. They also take always 1 extra argument: a handle to the ini file. (This handle is returned by the function ini_open_ext("filename")). This make ini-files even more alike to handling text files. Though with the huge difference that ini-files are only written to the disk after calling "close" function. Ow and don't mix _ext() and normal ini file functions: the result is undefined behaviour.
Ini-files are a third packages that works completely separate of GMFile & GMResource. It is about 240Kb in size.

So to update the table of included.
            | GMFile       : GMImage     : GMIni
------------|-------------- ------------- -----------
size        | 200 Kb       : 2540 Kb     : 240Kb
------------|-------------- ------------- -----------
typical uses| text files   : sprites     : ini files
            | binary files : backgrounds :
            | filesystem   : sounds      :
------------|-------------- ------------- -----------
files       | GMFile.dll   : GMImage.dll : GMIni.dll
            |              : GMImage.gml :

  • 1

#15 wnsrn3436

wnsrn3436

    GMC Member

  • GMC Member
  • 194 posts
  • Version:GM:Studio

Posted 08 February 2013 - 01:51 PM

Is a very good update. (I've always wanted ini function) :happy:
However, why you not do dll files compressed?

The DLL to UPX DLL results.
  • GMResource.dll, 2591744 -> 857600 (33.09%)
  • GMFile.dll, 204800 -> 81408 (39.75%)
  • GMIni.dll, 243712 -> 87552 (35.92%)
Compress this library, there is a problem?

Edited by wnsrn3436, 08 February 2013 - 01:54 PM.

  • 0

#16 paul23

paul23

    GMC Member

  • Global Moderators
  • 4147 posts
  • Version:GM:Studio

Posted 08 February 2013 - 02:27 PM

Hmm because I didn't know about it. Besides that point: well I'm not too sure about upx; as I've read reports of some anti virus programs report such dlls as a thread. I don't really wish to put this extra mark for other programmers to take care of. There are also several other downsides, like the fact a compressed dll has to be loaded completely into the memory, parts can't be removed and dynamically reloaded by windows (though none that actually influence GMFilesystem).

But I know these virus scanners might be changed (updated etc) the last few years & no longer cause the problems, so if a programmer wishes to use he CAN now compress the dlls himself. (Though I doubt the difference matters when one actually distributes the game as zip/installer).

Bottom line: I don't think compressing the dll is worth the effort.
  • 0

#17 TeamSteeve

TeamSteeve

    GMC Member

  • GMC Member
  • 1359 posts
  • Version:GM:Studio

Posted 10 February 2013 - 10:00 PM

FS_directory_exists() always returns true. :confused:/>/> You might want to fix that.


I really can't seem to reproduce that bug. ie show_message(string(FS_directory_exists("c:\winddddows"))) shows "0" for me. Can you tell me what you used to create this bug? (Ow and please also verify that the string you use is non existsing.. Other than that: ini files have just been added to the working parts. The manual will be updated later..

Ah yes, sorry it doesn't always return true. The problem was that I was using it to test whether something found using FS_find_file_ functions was a directory or not.
FS_directory_exists() returns true even if the string is a file and not a directory. I've found that FS_file_exists() returns true even if the string is a directory too. I wanted to iterate through all the directories in a specified location but FS_file_find_... will always find regular files as well as directories if I use fa_directory, so I need to be able to test if what I've found is indeed a directory. ;)/>

Edited by TeamSteeve, 10 February 2013 - 10:01 PM.

  • 0

TeamSteeveBanner.png


#18 paul23

paul23

    GMC Member

  • Global Moderators
  • 4147 posts
  • Version:GM:Studio

Posted 10 February 2013 - 11:47 PM

FS_directory_exists() always returns true. :confused:/> You might want to fix that.


I really can't seem to reproduce that bug. ie show_message(string(FS_directory_exists("c:\winddddows"))) shows "0" for me. Can you tell me what you used to create this bug? (Ow and please also verify that the string you use is non existsing.. Other than that: ini files have just been added to the working parts. The manual will be updated later..

Ah yes, sorry it doesn't always return true. The problem was that I was using it to test whether something found using FS_find_file_ functions was a directory or not.
FS_directory_exists() returns true even if the string is a file and not a directory. I've found that FS_file_exists() returns true even if the string is a directory too. I wanted to iterate through all the directories in a specified location but FS_file_find_... will always find regular files as well as directories if I use fa_directory, so I need to be able to test if what I've found is indeed a directory. ;)/>

Ow ye, should've specified that; the underlying functions fore fs_*_exists() are the same.. This is because the windows & possix api functions also have the same functions for these 2.. Should've actually made a check to see if the filetype is correct (directory or file), will do that in a small update. Though current solution would be to actually test a file for correct type:

(FS_file_attributes("FILENAME") & fa_directory == fa_directory) //whether file is directory
(FS_file_attributes("FILENAME") & fa_directory == 0) //whether file is not a directory
This method should also be at least as fast as actually checking for file existence.
  • 0

#19 paul23

paul23

    GMC Member

  • Global Moderators
  • 4147 posts
  • Version:GM:Studio

Posted 18 February 2013 - 02:16 PM

Update took a bit longer than expected, but anyways, a new version with a bugfix (First one :P), and some more functionality. As of now GMFileSystem Officially supports everything Gamemaker does and more. (Regarding external file management). Still looking how I should add XML-file reading (SAX, or DOM model etc), YAML & JSON.

What's New?
Version 1.3.1
This update brought a few extra functions:
  • fixed bug regarding directories/files both reported in "directory_exists" & "file_exists". Those functions now also check if the pointed "file" is a directory or file respectively.
  • Added support reading/writing real values in text documents (file_text_read_real, file_text_write_real)
  • Added locale support (locale defaults now to system locale) instead of C-locale as GMFileSystem did in 1.3 and earlier
  • Added some low level text file reading options to have more fine grained control

  • 0

#20 Nediradesigns

Nediradesigns

    GMC Member

  • GMC Member
  • 97 posts
  • Version:GM:Studio

Posted 18 February 2013 - 02:56 PM

Sorry, but I am not understanding what use this extension is over gamemaker. I see people are excited over this, but can someone maybe explain better why this is so great, what you would be using for exactly, and how this could be used as a level editor?

Thank you
  • 0

#21 paul23

paul23

    GMC Member

  • Global Moderators
  • 4147 posts
  • Version:GM:Studio

Posted 19 February 2013 - 11:26 AM

The basic use for this extension is when GM's sandboxed environment is too limited for you game/application.

Now the question is when is the sandbox limiting your game (too much)?

There are many reasons you might wish to improve this but first look into how the sandbox limits you. The most obvious limitation is that you can only save in %localappdata%/<gamename>. The only ways to save outside this is by a "backdoor" - or at the very least a method that does not work neat with a game's workflow (get_save_filename dialog). The localappdata is good for internal files, however the location is difficult to find for gamers, and even harder to remember.
Reading is done from several places, the localappdata folder, the working directory and a temporary directory. However the working directory is NOT the location of the project, it differs depending on whether you launch the game through ide (in which case it is the GM:studio executable directory) or launch the game as standalone (in which case the working directory is the program directory). The temporary directory is also created differently whether you run through studio or as standalone. On top of that it is impossible to deriviate between the three folders. So naming collisions can happen.


Now what does this imply for your projects? First thing one should notice is the hidden save data folder. This means that any saved information is "hidden" from the user, and smooth screenshots or other output can not be done, except by giving a save file dialog. Gamers do not want this, mostly they expect screenshots to be saved in the my documents or pictures folder, probably in a subfolder for your game.

Another problem is when you have multiple executable working together to create a "program". For example your map editor is a different executable. In this case it is impossible to access the information from the main game in your map editor. Unless of course you expect users to manually find the %localappdata% folder through a open file dialog.

This brings me to the next point; you more often than not in a map editor like users to save their maps in a folder -of their choosing- so the mapfiles can be exported. Or at least, you do not wish the map-data to be hidden in an "obscure" folder. In such a case you should of course use "get_save_filename" to specify the area in which you like to save your data - which is possible with gamemaker. However what is not possible is if you open an existing file (after restarting the map editor) to simply use a "save" button, you HAVE to use get_save_filename to add the file are to the writeable areas.
Another thing you can not create is for example a "recent files" list, since many of those maps would point to an area outside "%localappdata%" - you HAVE to use an open-file-dialog before you can load a file.

Finally a thing some people would like is to be able to open a map-file with your editor by double clicking the file. This is impossible with GM:Studio (Again unless you store the files in the %localappdata% folder.




There is also a second side to this extension: it extends the functions from gamemaker.
Gamemaker can currently only load png images for sprites & backgrounds - this extension allows you to load many formats (Notable exception are gif image though). The extension also extends quite a bit on the file handling options. Especially the binary file functions are extended to make them actually useful, but also the text file functions have now some extended use. (Getting more information if an error occurs during reading when for example the storage medium is removed, and to read a file character-by-character instead of only line-by-line). One can also use the extension when you wish to open multiple ini files at once.


Bottom line is

Use this extension when Gamemaker's filesystem is too limited. There are many, many improvements this extension has over gamemaker's filesystem probably more you can count.
  • 0

#22 paul23

paul23

    GMC Member

  • Global Moderators
  • 4147 posts
  • Version:GM:Studio

Posted 04 March 2013 - 07:46 PM

Huge patch, to go to update 1.4. XML files are now supported - some feature gamemaker is lacking. Though I'm not really fond of the interface provided yet, it's hard to improve the readability as xml files are simply more difficult to use than ini files. To match this the manual also gets improvement a lot, added sections for XML & INI files. A future update to gamemaker studio (structures) might actually improve the extension by a lot and make it again a slightly more easy to maintain. Xml files are read into a DOM at file load (similar to how ini files are handled by the extension as well as gamemaker itself), and the changes you do are only saved to the harddrive when you close the file.
The underlying library used for this is tinyxml2...

With this update everything I originally planned is inside the extension, now I'll look into more data storage formats such as json and yaml. Also ideas for improvement are more than welcome, especially to the core functionality I'm always looking at how to make it more easy.

What's New?
Version 1.4
  • Added XML file handling (~50 functions)
  • D3D_model loading/saving added
To update the table of included parts with GMXML:
            | GMFile       : GMImage     : GMIni     : GMXML     |
------------|-------------- ------------- ----------- -----------|
size        | 200 Kb       : 2540 Kb     : 240Kb     : 190Kb     |
------------|-------------- ------------- ----------- -----------|
typical uses| text files   : sprites     : ini files : xml files |
            | binary files : backgrounds :           :           |
            | filesystem   : sounds      :           :           |
------------|-------------- ------------- ----------- -----------|
files       | GMFile.dll   : GMImage.dll : GMIni.dll : GMXML.dll |
            |              : GMImage.gml :           :           |

  • 1

#23 mhermann

mhermann

    GMC Member

  • GMC Member
  • 12 posts
  • Version:GM:Studio

Posted 07 March 2013 - 05:08 AM

Was just testing the ini functions and ran into a couple issues. (Could be user error)

1. When using the FS_ini_close() function it is expecting an argument and I am not sure why.

2. Having a basic problem when trying to open an ini file in a sub directory in the working directory.

This works:
x = FS_ini_open_ext(working_directory + "\cyclops.ini");
name = FS_ini_read_string_ext(x,'character','name','default');
FS_ini_close_ext(x);
show_message(name);

This does not:
x = FS_ini_open_ext(working_directory + "\char_ini\cyclops.ini");
name = FS_ini_read_string_ext(x,'character','name','default');
FS_ini_close_ext(x);
show_message(name);

The directory and files exist. Here is the error message:

Runtime Error!
Program: This application has requested the Runtime to terminateit in an unusual way.
Please contact the application's support team for more information.

Edited by mhermann, 07 March 2013 - 05:54 AM.

  • 0

#24 paul23

paul23

    GMC Member

  • Global Moderators
  • 4147 posts
  • Version:GM:Studio

Posted 07 March 2013 - 10:47 AM

Was just testing the ini functions and ran into a couple issues. (Could be user error)

1. When using the FS_ini_close() function it is expecting an argument and I am not sure why.

2. Having a basic problem when trying to open an ini file in a sub directory in the working directory.

This works:

x = FS_ini_open_ext(working_directory + "\cyclops.ini");
name = FS_ini_read_string_ext(x,'character','name','default');
FS_ini_close_ext(x);
show_message(name);

This does not:
x = FS_ini_open_ext(working_directory + "\char_ini\cyclops.ini");
name = FS_ini_read_string_ext(x,'character','name','default');
FS_ini_close_ext(x);
show_message(name);

The directory and files exist. Here is the error message:

Runtime Error!
Program: This application has requested the Runtime to terminateit in an unusual way.
Please contact the application's support team for more information.

Oops, FS_ini_close indeed should not ask for an argument; Simply a mistake creating the extension. I will update this soon (if you wish you can manually remove the requirement for an argument in the extension package - the function is correctly defined inside the DLL).


@the second problem: you do know that "working_directory" will point to the temporary directory?? NOT the application directory.. IE for me the string would point to:
C:\Users\user\AppData\Local\Temp\gm_ttt_36017\char_ini\cyclops.ini

I think the problem is caused because the folder in the temporary files does not exist. The engine can create a new ini-file if it is not existing, I however forgot to include code to create also subfolders if they are not there. Could you verify this by calling the following code:
show_message(working_directory + "\char_ini\cyclops.ini"); //shows the actual directory you are working in
show_message(FS_directory_exists(working_directory + "\char_ini")); //"1" if subdir exists, "0" if not exists

It is currently near "impossible" to programmatically get the working directory (application directory or even file directory). You will simply have to store the "exact" location of your project in a variable somewhere. (NOTE you can use %appdata% and other environment variables if you wish). I've filed a suggestion for gamemaker to include such information here.

Nonetheless, it should not halt with a runtime error, so it is a bug and I'll fix it soon, though i wonder how to fix it to be completely frank. The filesystem will also "fail" to open a file when the file is inside a non existing folder. It then fails by returning a bad index (-1). - this is the same behaviour as GM8.1 emitted - and the default behaviour for C++ std::basic_fstream objects. So to keep it in line with these, ini files will in the future also fail and return a bad code when opening a file inside a non existing folder. - The way to work around this is to first create all sub-directories with FS_directory_create(...)

Thank you for reporting these problems :)/>.
  • 0

#25 mhermann

mhermann

    GMC Member

  • GMC Member
  • 12 posts
  • Version:GM:Studio

Posted 07 March 2013 - 02:47 PM

I was indeed putting my files in the wrong place. I was thinking this would allow me to do that. I think I misunderstood when reading through. I told you it could be user error :)

Thanks for fixing. Great extension.
  • 0

#26 Floofpaws

Floofpaws

    Fox

  • GMC Member
  • 2161 posts
  • Version:Unknown

Posted 09 March 2013 - 04:12 AM

Yay, I can continue work on Arc/Lite Studio Edition now :D
  • 0

#27 icuurd12b42

icuurd12b42

    Self Formed Sentient

  • GMC Elder
  • 18181 posts
  • Version:GM:Studio

Posted 09 March 2013 - 04:30 AM

Wow, there is a lot of good stuff in there!
  • 0

gmcbanner.pnggmcbanner_tools.png

ICU Live Tutoring Through Slack or Skype | My Tools Page follow.png

I FRANTICALLY MADE MY 18000 POST TOPIC BEFORE MIKE ANNOUNCED A DELAY...
Now I'm squirming not to hit that reply button


#28 Floofpaws

Floofpaws

    Fox

  • GMC Member
  • 2161 posts
  • Version:Unknown

Posted 09 March 2013 - 04:45 AM

I've never used extensions on Studio before, so this is probably a stupid question. But why do I keep getting this error?
Error on load
Unable to find function FS_background_replace
I installed the extension correctly. (I think)
  • 0

#29 paul23

paul23

    GMC Member

  • Global Moderators
  • 4147 posts
  • Version:GM:Studio

Posted 09 March 2013 - 02:46 PM

I've never used extensions on Studio before, so this is probably a stupid question. But why do I keep getting this error?

Error on load
Unable to find function FS_background_replace
I installed the extension correctly. (I think)

That seems weird. I just tested it and it seems that gm:studio no longer can handle script-based extensions (.gml extensions).. That is quite a deal-breaker, and I can only recommended reverting GM:studio to a lot of versions back (pre 700 it worked).. I'll test this further and hopefully yoyogames can fix this ASAP.

I'm going to upload a version with all "scripts" outside the extension, so one can simply use a script-version for the time being. Scripts won't have the argument hints or auto completion though..
  • 0

#30 Floofpaws

Floofpaws

    Fox

  • GMC Member
  • 2161 posts
  • Version:Unknown

Posted 09 March 2013 - 03:20 PM

I've never used extensions on Studio before, so this is probably a stupid question. But why do I keep getting this error?

Error on load
Unable to find function FS_background_replace
I installed the extension correctly. (I think)

That seems weird. I just tested it and it seems that gm:studio no longer can handle script-based extensions (.gml extensions).. That is quite a deal-breaker, and I can only recommended reverting GM:studio to a lot of versions back (pre 700 it worked).. I'll test this further and hopefully yoyogames can fix this ASAP.

I'm going to upload a version with all "scripts" outside the extension, so one can simply use a script-version for the time being. Scripts won't have the argument hints or auto completion though..

Oh. Okay. I'm more used to using DLLs like that, anyway.

Edited by darkwalker247, 09 March 2013 - 03:21 PM.

  • 0

#31 paul23

paul23

    GMC Member

  • Global Moderators
  • 4147 posts
  • Version:GM:Studio

Posted 09 March 2013 - 04:36 PM

Ok I've uploaded a bug report to yoyogames, you can find it here. Now let's just pray they fix it soon, keeping many different versions with gamemaker for such things is NOT nice (I'm running then into the bugs/limitations of gm's version control <_< ).

I've also fixed a few bugs, but most importantly, I added even more download options for you guys. These options can probably be neglected by 90% of the people using it, however I've added an extension version called "debug". This extension version simply has all debug symbols included (and optimization removed) in the dlls. This means that you could use visual studio to debug when a bug appears. - And even better, I added quite some messages to give some hard-error messages when you try to open non existing files or bad pointers; no longer failing with some hidden error codes.

Combined with the "no gml versions" this lead to 4 simultanously updated extensions.


What's New?
Version 1.4
  • Catching some more errors with bad or non existing xml & ini files to prevent runtime errors.
  • Added debug versions of all downloads
  • Added no-gml-extension version to circumvent a bug in gm:studio

  • 0

#32 sagez

sagez

    GMC Member

  • GMC Member
  • 43 posts

Posted 16 March 2013 - 10:35 PM

You are a life saver mate.

And the extra features are the icing on the cake!
  • 0

#33 j4m1ef0rd

j4m1ef0rd

    GMC Member

  • GMC Member
  • 74 posts
  • Version:GM:Studio

Posted 30 March 2013 - 09:08 AM

GMfilesystem wroks perfect! i love it, i thought i was gonna have to go back to GM8 to make the software im developing atm. you saved my ass :)

Thank you so much!! this was exactly what i was looking for! i owe you big time man. if there is anything i can do, don't hesitate to ask! 

- Jamie Ford


  • 0

sig_zps773ace79.png


#34 Hugo_Peters

Hugo_Peters

    herp derp

  • GMC Member
  • 493 posts
  • Version:GM:Studio

Posted 12 April 2013 - 02:52 PM

This looks amazing!
The only problem is; it doesn't do anything!
This might be caused by the "nogml" bug, but I've downloaded and installed the nogml version, however, the nogml version only includes the ResourceDLL functions as a separate .gml file.

Also, in the extensions window it shows that the .gex is on version 1.2.1...?


  • 0

Posted Image


#35 paul23

paul23

    GMC Member

  • Global Moderators
  • 4147 posts
  • Version:GM:Studio

Posted 12 April 2013 - 03:08 PM

Just tested it myself but all the downloads contain a ".gex".. The "nogml" functions simply are an archive containing both the gex & a separate .gml file. BOTH do you have to include in your game for it to work. (Which is what the bug is about, gml code inside an extension do not work)

If you did it correctly you should see some 10-ish scripts (from the gml file) AND an extension in your resource tree. This extension should contain a 50-100 functions. (separated over 4 files).

The version number on the gex I indeed forgot to update - that's minor "bug" that should have no effect at all.
  • 0

#36 Hugo_Peters

Hugo_Peters

    herp derp

  • GMC Member
  • 493 posts
  • Version:GM:Studio

Posted 12 April 2013 - 03:29 PM

Just tested it myself but all the downloads contain a ".gex".. The "nogml" functions simply are an archive containing both the gex & a separate .gml file. BOTH do you have to include in your game for it to work. (Which is what the bug is about, gml code inside an extension do not work)

If you did it correctly you should see some 10-ish scripts (from the gml file) AND an extension in your resource tree. This extension should contain a 50-100 functions. (separated over 4 files).

The version number on the gex I indeed forgot to update - that's minor "bug" that should have no effect at all.

Yes, I did all that.

When I called the function FS_file_exists() with a path I knew existed ("C:\tdtprj.ico") it returned 0...

 

Also, calling functions like FS_file_text_open_append("blabla.txt"); won't do anything.

 

Any idea what's going on?


  • 0

Posted Image


#37 iSeiren

iSeiren

    www.worrall.pw

  • GMC Member
  • 633 posts
  • Version:GM:Studio

Posted 12 April 2013 - 05:36 PM

Sorry, this may be because im tired... so slap me if i sound stupid...

 

Will this work with HTML5? To save to the web server?


  • 0

Professional Mobile app Developer


#38 paul23

paul23

    GMC Member

  • Global Moderators
  • 4147 posts
  • Version:GM:Studio

Posted 12 April 2013 - 06:00 PM


Just tested it myself but all the downloads contain a ".gex".. The "nogml" functions simply are an archive containing both the gex & a separate .gml file. BOTH do you have to include in your game for it to work. (Which is what the bug is about, gml code inside an extension do not work)

If you did it correctly you should see some 10-ish scripts (from the gml file) AND an extension in your resource tree. This extension should contain a 50-100 functions. (separated over 4 files).

The version number on the gex I indeed forgot to update - that's minor "bug" that should have no effect at all.

Yes, I did all that.
When I called the function FS_file_exists() with a path I knew existed ("C:\tdtprj.ico") it returned 0...
 
Also, calling functions like FS_file_text_open_append("blabla.txt"); won't do anything.
 
Any idea what's going on?


uh to be frank: no.. I tested it in like all cases - only thing I can suggest now is pming me your (test) file so I can verify this.


@iSeiren: it relys on dlls... So no. Nor would this ever be possible, javascript is executed at the USER pc NOT at the server.
  • 0

#39 iSeiren

iSeiren

    www.worrall.pw

  • GMC Member
  • 633 posts
  • Version:GM:Studio

Posted 12 April 2013 - 06:48 PM

So my extension that i made, using php is the only way?

Damn, need to figure out how to save pictures with php xD


  • 0

Professional Mobile app Developer


#40 Bleed

Bleed

    Chevalier

  • GMC Member
  • 763 posts
  • Version:GM:Studio

Posted 13 April 2013 - 09:52 PM

Whoah, this sounds too good to be true...noob question though, is this only for studio, or will it work with 8.1 also?


  • 0

rlztjp_zpsoqffaixe.jpg


#41 ZinkMaxwell

ZinkMaxwell

    GMC Member

  • GMC Member
  • 13 posts

Posted 14 April 2013 - 07:23 PM

It is currently near "impossible" to programmatically get the working directory (application directory or even file directory). You will simply have to store the "exact" location of your project in a variable somewhere. (NOTE you can use %appdata% and other environment variables if you wish). I've filed a suggestion for gamemaker to include such information here.

 

Thought I'd add in that this extension is awesome.

 

Just to make sure I understand this correctly, there's currently no way to get the location of the game/application/exe direcotry?  I want to have "Folder exe is in\Levels\MyLevel\level.lvl".  This allows users to add custom music and tilesets to the level editor (by searching for files named "Folder exe is in\Levels\MyLevel\CustomMusic.mp3" etc.).  Without the exe directory I would have to hard code the location, which would mean custom resources wouldn't work if users want the game folder in a "Games" folder or "Program Files" or anywhere else but what I chose.


Edited by ZinkMaxwell, 14 April 2013 - 07:24 PM.

  • 0

#42 paul23

paul23

    GMC Member

  • Global Moderators
  • 4147 posts
  • Version:GM:Studio

Posted 14 April 2013 - 09:26 PM

Well there *are* several ways (as I've explained in some other posts) - but those ways depend on your preferences and the way you build your game - as such there is no single way this extension can get the directories.



First of all you could simply use a relative directory ("/Levels/MyLevel/CustomMusic.mp3") - this will default to the working directories, though I'm unsure when and how GM sets the working directories (in previous versions of GM <8 save_get_filename for example could set the working directory to the newly user chosen directory). In the current version of GM the working directory seems more or less stable (but here you go into tricky territory dlls like ogre typically change it) - apart from the fact that this working directory gets set to GM:STUDIO install directory when running through the IDE (testing).

Another posibility is to use the first command line parameter (parameter_string(0)). In windows this defaults mostly to the "opening executable" (mostly as when you have a different filemanager installed than windows explorer things might change - but that could be considered bad behaviour of the file manager, it is not a requirement by windows to set this program parameter but it has become a facto de standard). However once again when running through the IDE your game is not a stand alone executable, so the first parameter once again points to the GM:STUDIO directory. ALSO when creating the single executable this parameter is also not set correctly. (ow also note that this points to the executable, so you need to take the parent directory to get the game directory).

A third option is to use an installer, and set either an environment variable (bad) or variable in the registry (standard method for programs) with the installer. This also facilitates "repairs" and "uninstall" or "remove user data". However this requires an installer or manually setting the variable for testing.



These options show that for testing a different approach has to be taken than for distribution. My recommendation hence is to:
1) create a global variable "progdir"
2) for testing hardcode this variable progdir to the game (data) directory
3) for distribution use one of above methods to set this variable progdir
4) use progdir as "root"
  • 0

#43 Bleed

Bleed

    Chevalier

  • GMC Member
  • 763 posts
  • Version:GM:Studio

Posted 14 April 2013 - 10:08 PM

Hmm, lemme rephrase myself : something as ESSENTIAL as this does not support 8.1 ?!

Sacrilege!!! lol

 

Anyway, this is a powerful extension, too bad i don't care much for Studio.

 

Good job, as always...looking forward to more updates.


Edited by Bleed, 14 April 2013 - 10:09 PM.

  • 0

rlztjp_zpsoqffaixe.jpg


#44 paul23

paul23

    GMC Member

  • Global Moderators
  • 4147 posts
  • Version:GM:Studio

Posted 14 April 2013 - 10:12 PM

This project started to give all GM8 filesystem capability to GM:Studio; but with that also some extra features were added :P. I don't have the luxury of having too much time on my hand to also test it for GM8.1. However I can't see why the gex should not work, I won't give you any guarantees but all functionality is more or less equal regarding extensions. Besides you would only use the INI/XML/binary data structure functions anyways, and those are quite straight forward (all file data is kept inside the DLL-thread).
  • 0

#45 rickster

rickster

    GMC Member

  • GMC Member
  • 150 posts
  • Version:GM:Studio

Posted 17 April 2013 - 10:08 PM

This is exactly what I was looking for, sadly it does not work for me!  I have tried everything, but every file I load fails, and it says none of my files exist.  This is some sample code I made to test it out:

 

name = get_string('Open file: ','filename')

if !FS_file_exists(name)
{
  show_message('That file does not exist!')
}

file = FS_file_text_open_read(name)

if !FS_file_good(file)
{
  show_message('Load failed!')
}

 

This should show no errors if the file exists and loads, but it always throws both errors, saying that the file does not exist AND that it could not be loaded.  It does not matter the file.  I've tried full file paths, partial file paths, even loading files in the same directory as the game.  Nothing seems to work, even though I am POSITIVE all of the files exist.  To make sure, I put a file directly in the C drive and entered "C:\file.txt" into that sample code, and it still said it did not exist.

 

Am I missing something?  There is no error message associated with this, so I have no way of showing it's not working, it just doesn't think any of my files exist.  Could this be caused by some sort of virus protection or the game not having permissions?

 

Any help would be greatly appreciated,

 - Rickster


  • 0

uprep.png           upvote%20button%202.png

 

Credit  |  StewartB  |  GMComicified 19 - Signatures


#46 paul23

paul23

    GMC Member

  • Global Moderators
  • 4147 posts
  • Version:GM:Studio

Posted 17 April 2013 - 10:33 PM

Hmm seems more people show this problem lately - which is quite "weird" as I don't recall any major changes here... tomorrow I'll take a complete thorough look at potential problems with the winapi.


Are you by any chance using windows 8 in the JS/tablet mode? - Or actually what version of windows do you run? Also for real help an example project is the best thing you can send me?
  • 0

#47 rickster

rickster

    GMC Member

  • GMC Member
  • 150 posts
  • Version:GM:Studio

Posted 18 April 2013 - 01:41 AM

I threw something together to help show my problem.

 

screen.png

 

This little tool checks five things as you type:

 

 - Does GM see the file?  (file_exists())

 - Does GMFileSystem see the file?  (FS_file_exists())

 - Is the file good?  (FS_file_good())

 - Did opening the file fail?  (FS_file_fail())  Will show green check if it did NOT fail.

 - Is the file bad?  (FS_file_bad())  Will show green check if it is NOT bad.

 

Here's a .gmz.  Open it up and start typing.  If it sees a valid file the X's will change to checks.

 

I actually noticed something else strange.  FS_file_fail() and FS_file_bad() seem to always return false, resulting in green checks in their spaces.  FS_file_good(), however, also returns false.  So nothing is bad and nothing failed, yet it isn't "good".  In my screenshot you can see that GM says the file exists (you can see it in the background), yet FS does not agree.

 

Since GM is sandboxed, the GM file_exists() function only works if the file is where the game is (I think).

 

Good luck, I have no idea at this point.

 - Rickster

 

(Oh, I'm running Windows 7 Home Premium, 64 bit version.  Nothing with Windows 8, don't even have that!   :P)


Edited by rickster, 20 April 2013 - 05:25 AM.

  • 0

uprep.png           upvote%20button%202.png

 

Credit  |  StewartB  |  GMComicified 19 - Signatures


#48 mhermann

mhermann

    GMC Member

  • GMC Member
  • 12 posts
  • Version:GM:Studio

Posted 18 April 2013 - 01:59 AM

Found a small issue with FS_ini_read_string_ext().

 

If the ini file looks like this:

 

[Match 1 Round 1]
winningteam = 2
p1.name = KFM

 

It can read winningteam just fine, but it has a problem with p1.name. It does not like the period. IF I use the default ini_read_string it works fine. Unfortunately I do not have control of how the ini file that I am reading from is written.

 

Otherwise I am loving this extention.


  • 0

#49 Wumpus

Wumpus

    GMC Member

  • GMC Member
  • 8 posts

Posted 19 April 2013 - 07:51 PM

That seems to be a wonderful extension. Thank you very much for your work. Unfortunatley, I have the same problem as others and I cannot get it to work. I tried Rickster's .gmz on my 64-bit Windows 7 system and ran into exaclty the same issues.


  • 0

#50 paul23

paul23

    GMC Member

  • Global Moderators
  • 4147 posts
  • Version:GM:Studio

Posted 20 April 2013 - 11:33 AM

With tests I & rickster did, I found that the dlls functions are (most probably) not being called at all - as if gamemaker can not load the dll & simply silently fail/return "0" in each function. Now the problem becomes "why is the dll not being called"... The file GMZ below shows some test can be shown if the function is being called.

 

 

the test

Simply compile (run) the file, and then type "test.txt" (a file that OUGHT to be exist)... Now if the file works correctly it should show something like the following for an existing file:

testlccw.png

and during typing -or when giving an non existing file-:

testvj.png

Especially mark the file open numbers: it gives a positive (0 included) number when a file succeeds in opening, and a negative "-1" value when the file fails to open. However when opening multiple files at once they can never ever be the same number. (so you'll see "0" & "1" and never two times "0").

 

 

However as with rickster -and I suppose everyone who has problems both file_open numbers stay at "0" - this means that the dll function is not called at all - or at least the returned value of the dll function is not taken by gamemaker correctly... This is very difficult for me to debug and I'll look into fixing it. I would be really happy if you could show the results of tests here.

 

On a related the note, I fixed two bugs regarding ini-files already, when trying to open a nonexisting ini file the ini file ought to be created - and dots in ini file sections/keys are not supported (the underlying algorithms do require a delimiter to mark en of keys internally - however I should've used "=" sign).


  • 0