75 replies to this topic

[icuurd12b42]

Download GMBinFile.zip
These scripts will ease the burden of saving/loading complex data like a real, a buffer or a string to a binary file as well as providing encryption/decryption as the file is created, updated and read... That is, no interim file is created that people can snoop.

Credit:
Davve For the Big Endian addition and current compile.

Example Here

Function List
BigEndian versions of the functions added with the suffix BE

GMBINInit
GMBINUnload

GMBINOpenFileRead
GMBINOpenFileReadPW
GMBINOpenFileWrite
GMBINOpenFileWritePW
GMBINOpenFileReadWrite
GMBINOpenFileReadWritePW
GMBINCloseFile

GMBINReadByte
GMBINReadUByte
GMBINReadByteChar
GMBINReadShort
GMBINReadUShort
GMBINReadInt
GMBINReadUInt
GMBINReadLong
GMBINReadULong
GMBINReadFloat
GMBINReadDouble
GMBINReadReal
GMBINReadBuffer
GMBINReadString
GMBINExtractFile <NEW>

GMBINWriteByte
GMBINWriteByteChar
GMBINWriteShort
GMBINWriteInt
GMBINWriteLong
GMBINWriteFloat
GMBINWriteDouble
GMBINWriteReal
GMBINWriteBuffer
GMBINWriteString
GMBINInsertFile <NEW>

GMBINIsEOF
GMBINIsError
GMBINGetPosition
GMBINGetActualSize
GMBINSeekBOF
GMBINSeekEOF
GMBINSeekFromCurrent
GMBINSetPosition

GMBINPackBits
GMBINUnpackBit

*NOTE: I have not tested my read/write (int dword float double and real versions) against binary files created by other application; the byte order may not be standard (reversed). However, byte and buffer should be 100% compatible.

If you are wondering why there is no WriteUxxx, It's because the system does not care if the value is signed or not while saving. The important thing to remember is to use ReadUxxx if you wrote a positive value packed in the type you specified and the positive value would be greater than the signed range...
GMBINWriteShort(hf,65535);
v = GMBINReadUShort(hf); //(0-65535)
Signed/Unsigned and Data Range Info

Buffer and String are different... Buffer reads/writes a buffer of a length specified by the caller and can be used to blindly read anything.... Where strings are saved with a header telling the size of the string, and read back according to that size. This should allow saving GM strings that have chr(0) in them.

I have not benchmarked against GM's file_bin functions, but the simple fact that you can write data other than bytes should make the dll faster and easier to use.

The just in time encryption is a bonus you should appreciate.

It's also a good replacement for file_text

To use, copy the dll to you project folder, merge the file with your game and replace your gm file io function calls with the included scripts.

The included room tests all the functions in the room code (button).

UPDATE
Now you can package multiple files in one binary file using
GMBINInsertFile - The function adds an entire file to the opened binary file. All you need is specify the source file.
GMBINExtractFile - The function extracts an entire file from the opened binary file. All you need is specify the destination file.
You are in charge of the file naming and what to do with the extracted files... You can use a temp file and delete it when done.
example
hf = GMBINFileOpenWrite("t.bin")
GMBINWriteShort(hf,2);
GMBINWriteString(hf,"File1.bmp");
GMBINInsertFile(hf,"File1.bmp");
GMBINWriteString(hf,"File2.bmp");
GMBINInsertFile(hf,"File2.bmp");
GMBINfileClose(hf);

hf = GMBINFileOpenRead("t.bin")
repeat(GMBINReadShort(hf))
{
filename = GMBINReadString(hf);
GMBINExtractFile(hf,filename);
}
GMBINfileClose(hf);

When part of the bin file, the data is encrypted if using password encryption.

Cheers!

Edited by icuurd12b42, 25 May 2011 - 04:12 AM.

[icuurd12b42]

Added Shorts (not the kind you wear)

Fixed a hang on buffers and string of 0 length
Added source cpp file.

Changed the API and trimmed the fat from the dll.

Added an unsigned reads for byte and all integer types

Edited by icuurd12b42, 21 August 2008 - 03:59 AM.

[IceMetalPunk]

I do have a question, based on just reading the first post (haven't downloaded yet). What exactly is the UByte? An unsigned byte?

I didn't know there were signed bytes. Would those just be a single byte, with the first bit as a sign bit? So values between -127 and 127?

-IMP

[icuurd12b42]

I do have a question, based on just reading the first post (haven't downloaded yet). What exactly is the UByte? An unsigned byte?

I didn't know there were signed bytes. Would those just be a single byte, with the first bit as a sign bit? So values between -127 and 127?

-IMP

-128 to 127 to be exact.

unsigned char is 0 to 255

It's still a single byte (8 bits), using the left most bit that defines the sign as part of the actual number instead.


It's the same deal with short (-32768 to 32767) or unsigned short (0 to 65535)

int -2147483648 to 2147483647 or 0 to 4294967295

long -9223372036854775808 to 9223372036854775807 or 0 to 18446744073709551615

If your integer number is never negative, it a good way to save space while still having a respectable range. Like if your nuber will never be more than 255, a byte read with Read UByte will do. the write functions really don't care if the value is signed or not.

For GM's object_index or sound index or any other resource indext, It is very unlikely the index will be > 65535 so a unsigned short will do to save a map...

A float (32 bits), on the other hand can store a normal double (64 bits) (GM Real), loosing half it's size, range and precision, but in most case, it would still be within normal range used in games... It's unlikely you will have a game related value larger than the range of a float... (Thinking about speed and color and position and alpha values)

The BitPacker allows to pack several 0/1 values in a single byte.

All these types are used to save on space and read/write time.

You can get away with using Write/ReadReal if you don't want to go through all the trouble.

Edited by icuurd12b42, 15 November 2008 - 10:23 AM.

[Markolainen]

I never heard of bin files in GM before.
A quick question: Is this like saving and loading ini files?

[icuurd12b42]

I never heard of bin files in GM before.
A quick question: Is this like saving and loading ini files?

no. It's like file_bin_open but adding the abilities you get with file_text_open. Bin files in GM only save and read bytes (1 byte at a time) and having your own scripts to save anything else is slow using gml.

[icuurd12b42]

UPDATE
Now you can package multiple files in one binary file using
GMBINInsertFile - The function adds an entire file to the opened binary file. All you need is specify the source file.
GMBINExtractFile - The function extracts an entire file from the opened binary file. All you need is specify the destination file.
You are in charge of the file naming and what to do with the extracted files... You can use a temp file and delete it when done.
example
hf = GMBINFileOpenWrite("t.bin")
GMBINWriteShort(hf,2);
GMBINWriteString(hf,"File1.bmp");
GMBINInsertFile(hf,"File1.bmp");
GMBINWriteString(hf,"File2.bmp");
GMBINInsertFile(hf,"File2.bmp");
GMBINfileClose(hf);

hf = GMBINFileOpenRead("t.bin")
repeat(GMBINReadShort(hf))
{
filename = GMBINReadString(hf);
GMBINExtractFile(hf,filename);
}
GMBINfileClose(hf);

When part of the bin file, the data is encrypted if using password encryption.

Cheers!

[Skarik]

This is ridiculous. Right when I was about to program a little copying script, an insert file function is added. Pah. Stupid coincidences. I've have too many of them in one year.

[icuurd12b42]

Here is an example of a simple save and read. name is the filename....

//var hf; hf= GMBINOpenFileWritePW(name,"password");
var hf; hf= GMBINOpenFileWrite(name);


if(!hf) 
{
	show_message("Error opening file for writing:##" + 
		string_replace( name, "\",  "\ "))
	return "";
}

GMBINWriteBuffer(hf,"GMPINBALLv001");
GMBINWriteShort(hf,instance_number(NodeGroup));
with(NodeGroup)
{
	GMBINWriteString(hf,m_Name);
	GMBINWriteFloat(hf,m_origx);
	GMBINWriteFloat(hf,m_origy);
	var np; np = 0;
	if(path_exists(m_path))
		np = path_get_number(m_path);
	GMBINWriteShort(hf,np);
	if(np)
	{
		GMBINWriteByte(hf,path_get_precision(m_path));
		GMBINWriteByte(hf,path_get_kind(m_path));
		GMBINWriteByte(hf,path_get_closed(m_path));
		var i; i = 0;
		repeat(np)
		{
			GMBINWriteFloat(hf,path_get_point_x(m_path,i));
			GMBINWriteFloat(hf,path_get_point_y(m_path,i));
			GMBINWriteFloat(hf,path_get_point_speed(m_path,i));
			i+=1;
		}
	}
}
if(GMBINIsError(hf))
{
	show_message("An error occured writing the file:##" +
	string_replace( name, "\",  "\ "))
}
GMBINCloseFile(hf);

//var hf; hf= GMBINOpenFileReadPW(name,"password");
var hf; hf= GMBINOpenFileRead(name);


if(!hf) 
{
	show_message("Error opening file for reading:##" + 
		string_replace( name, "\",  "\ "))
	return "";
}

if(GMBINReadBuffer(hf,string_length("GMPINBALLv001")) <> "GMPINBALLv001")
{
	show_message("File is not compatible:##" + 
		string_replace( name, "\",  "\ "))
	GMBINCloseFile(hf);
	return "";
}
var n; n = GMBINReadShort(hf);
repeat(n)
with(instance_create(0,0,NodeGroup))
{
	m_Name = GMBINReadString(hf);
	m_origx = GMBINReadFloat(hf);
	m_origy = GMBINReadFloat(hf);
	var pn; pn = GMBINReadShort(hf);
	if(pn)
	{
		path_set_precision(m_path,GMBINReadByte(hf));
		path_set_kind(m_path,GMBINReadByte(hf));
		path_set_closed(m_path,GMBINReadByte(hf));
		repeat(pn)
		{
			path_add_point(m_path,GMBINReadFloat(hf),GMBINReadFloat(hf),GMBINReadFloat(hf))
		}
	}
}
if(GMBINIsError(hf))
{
	show_message("An error occured reading the file:##" +
	string_replace( name, "\",  "\ "))

}
GMBINCloseFile(hf);

[Alexander_Q]

Hey Icuurd. I was thrilled when I tried this out and it worked first time . I have a question re: passwords, or several questions. Firstly, what is the best way to distribute the pass? Say in an online game model where the client will request your encrypted files... He needs to have the password before he can process them.

In 39dll there is a function to encrypt the buffer when sending, but even then one needs a password to decrypt at the other end. What is the best/safest way to get the encryption password to the client?

This is perhaps a little bit out of the scope of this topic, but it has to do with using the encryption effectively I guess.

Another question - the password is attached to the files. Say I setup my project where all of the write functions used a variable "pass" for the password... This would change the result of an MD5 hash check with the previous password if I were to change "pass", yes?

I'm thinking contingencies, if someone were to get hold of the encryption password, it would be very easy for me to quickly alter all of the files with a new password and then make all of the clients files fail hash checks.

But this is problematic - I intend to encrypt every file, so having to redistribute everything for the sake of changing the password is a little dramatic. What do you suggest?

*Considers multiple passwords

I've never thought about this kind of stuff before, so forgive me if I'm on completely the wrong track

FURTHER QUESTIONS-----------------------------------------------------------------------------

Consider this script:

//Write Tiles
num=tile_add(0,0,0,0,0,0,0,0);
tile_delete(num);
for (i=10000000;i<=num;i+=1) {
	if tile_exists(i) {
		GMBINWriteString(fileid,background_get_name(tile_get_background(i)));
		GMBINWriteShort(fileid,tile_get_left(i));
		GMBINWriteShort(fileid,tile_get_top(i));
		GMBINWriteByte(fileid,tile_get_width(i));
		GMBINWriteByte(fileid,tile_get_height(i));
		GMBINWriteShort(fileid,tile_get_x(i));
		GMBINWriteShort(fileid,tile_get_y(i));
		GMBINWriteInt(fileid,tile_get_depth(i));
	}
}
//Write objects
with (all) {
	if (!persistent) {   
		GMBINWriteShort(other.fileid,x);
		GMBINWriteShort(other.fileid,y);
		GMBINWriteString(other.fileid,object_get_name(object_index));
	}
}

This presents a problem for me when reading - the number of objects/tiles in the resulting file is not fixed, and upon reading the script won't know when to stop reading tiles and start reading objects. At first I put in a "*" delimeter, but because it is reading specific values of specific lengths it will not know when to check for the delimeter.

One thing that may work is the following: As the for loop iterates, record the increase in position of the binary file and count each tile. When done, seek back to the beginning and place the tilecount, so upon first arriving the script knows how many tiles there will be.

For example, inside each iteration of the tile writer:

binpos+=4+string_length(background_get_name(tile_get_background(i)))+2+2+1+1+2+2+4;
		count+=1;

And at the end:

GMBINSeekFromCurrent(fileid,-binpos);
GMBINWriteShort(fileid,count);
GMBINSeekFromCurrent(fileid,binpos-2);

Same with objects and so on. What do you think? Will this work? Is there an easier way?

ONE LAST THING-------------------------------------------------------

WriteString with a byte head instead of an int head? Is this something that ever gets done? It would save a lot of space (I really don't want to make my own heads and use buffer).

Edited by Alexander_Q, 26 November 2008 - 07:22 PM.

[icuurd12b42]

Yes, different password change the entire file content so the hash would be defferent from the original.

For passing allong passwords, the only safe way would be to find a public/private key system... All your clients would have the public key while you encrypt with a private key...


So, you would encrypt the password string used by the encrypted files with a such a public key system to send along passwords.

You could pair the encrypted file with the public key used right in the file... Using the file packager API I just added...

Say the password is "Test"

You encrypt the data file using "Test".

You encrypt the password "Test" using the public key method

You bundle it to another file, called MainArchive or the final name of the file...

Example
hf = GMBINOpenFileWritePW("actualdatafile.map", "Test"))
save the map
GMBINfileClose(hf);

hf = GMBINOpenFileWrite("Finalfile.dat")
GMBINWriteString(hf,EncryptUsingPublicKey("Test"))
GMBINInsertFile(hf,"actualdatafile.map");
GMBINfileClose(hf);

or even add another layer here by also encrypting the archive LOL that's overkill though... And would really be slow

hf = GMBINOpenFileWritePW("Finalfile.dat","CheapLevel1ProtectionPassword")
GMBINWriteString(hf,EncryptUsingPrivateKey("Test"))
GMBINInsertFile(hf,"actualdatafile.map");
GMBINfileClose(hf);


to open the file

hf = GMBINOpenFileRead("Finalfile.dat")
password = DecryptUsingPublicKey(GMBINReadString(hf));
GMBINExtractFile(hf,"actualdatafile.map");
GMBINfileClose(hf);
hf = GMBINOpenFileReadPW("actualdatafile.map",password)
//read the map
GMBINfileClose(hf);
file_delete("actualdatafile.map") //optional


I really should see if I can find a public key system. Perhaps I can add it to the api in this dll.

[Alexander_Q]

Hmmmmmmmmm... I think I need to re-read that a few times But in the meantime, problem:
http://gmc.yoyogames...es/rte-list.gif

show_message("Doing tiles")
	num=GMBINReadUShort(fileid);
	show_message("Number is "+string(num));
	for(i=0;i<num;i+=1) {
		show_message("Doing tile no. "+string(i));
		room_tile_add(room_id,
		ds_map_find_value(map_backgrounds,GMBINReadString(fileid)),
		GMBINReadUShort(fileid),
		GMBINReadUShort(fileid),
		GMBINReadUByte(fileid),
		GMBINReadUByte(fileid),
		GMBINReadUShort(fileid),
		GMBINReadUShort(fileid),
		GMBINReadInt(fileid)
		);
		show_message("Done a tile")
	}

The code gets the "num" correctly (which means my seeking and inserting method worked, yay!) but after it displays "show_message("Doing tile no. "+string(i));" it hangs before getting to the end of the iteration. Somewhere in the middle of room_tile_add there's a problem. No errors get thrown and it's not an infinite loop issue, so is there perhaps something going on in the dll?

EDIT:

At the top of the for loop I put:

for(i=0;i<num;i+=1) {
		show_message("Doing tile no. "+string(i));
		
		test=GMBINReadString(fileid);
		
		show_message(test);

...

And it fails right there during that read string. I know it's up to the right spot and the next value is a string. What could be the problem? Note: It fails on the very next value after the first value in the file that was "inserted" using a relative seek.

Edited by Alexander_Q, 26 November 2008 - 08:43 PM.

[icuurd12b42]

Hmmmmmmmmm... I think I need to re-read that a few times But in the meantime, problem:
http://gmc.yoyogames...es/rte-list.gif

show_message("Doing tiles")
	num=GMBINReadUShort(fileid);
	show_message("Number is "+string(num));
	for(i=0;i<num;i+=1) {
		show_message("Doing tile no. "+string(i));
		room_tile_add(room_id,
		ds_map_find_value(map_backgrounds,GMBINReadString(fileid)),
		GMBINReadUShort(fileid),
		GMBINReadUShort(fileid),
		GMBINReadUByte(fileid),
		GMBINReadUByte(fileid),
		GMBINReadUShort(fileid),
		GMBINReadUShort(fileid),
		GMBINReadInt(fileid)
		);
		show_message("Done a tile")
	}

The code gets the "num" correctly (which means my seeking and inserting method worked, yay!) but after it displays "show_message("Doing tile no. "+string(i));" it hangs before getting to the end of the iteration. Somewhere in the middle of room_tile_add there's a problem. No errors get thrown and it's not an infinite loop issue, so is there perhaps something going on in the dll?

EDIT:

At the top of the for loop I put:

for(i=0;i<num;i+=1) {
		show_message("Doing tile no. "+string(i));
		
		test=GMBINReadString(fileid);
		
		show_message(test);

...

And it fails right there during that read string. I know it's up to the right spot and the next value is a string. What could be the problem? Note: It fails on the very next value after the first value in the file that was "inserted" using a relative seek.

I'd have to see your write script, your read must match your write... If you read a string when the file position is not on a string, it will try to read the data as it would a string... And crash because it's not a string, having read an eroneous string size... This is pretty much the same story whith any language...


By the way, if you seek and write, it does not insert, it overwrites...

[Alexander_Q]

!

Well there's the problem right there. I'm seeking and overwriting the header for the string which is to come next. Damn. Damn damn damn.

Damn.

(Damn.)

It was such a good idea.... How shall I delimit my information? Is there any way to "insert" values? (Problem mentioned in edit of post #10).

[Rexhunter99]

icuurd12b42 you are my saviour! *hugs* I've been looking for a Buffer ability! I so desperetly needed it for my Carnivores 3D editor because the model format .CAR saves it's WAV's and TGA's into the file without the header and GM is ****house with it's loops as we all know.

Again, Thank you so much!

[icuurd12b42]

OK, for the tile, loop twice, once to count, a second to save (you could use a ds_list in the first loop). Save the count to the file... so when you read you know how many you have...

There is no method to insert data in the middle of a file, aside from loading the whole thing and resaving it after inserting data using your application...

As for the byte as string heather... I was sure I used a short... Oh well. too late, I can't touch that without breaking peoples files if they update...

If you really want a single byte
Change the read string (Actually, make your own)
//script MyReadString(filehandle,string)
return GMBINReadBuffer(argument0,max(0,GMBINReadUByte(arg
ument0)));

And add your own write string
//script MyWriteString(filehandle,string)
var sz; sz = min(255,string_length(argument1));
GMBINWriteByte(argument0,sz);
GMBINWriteBuffer(argument0,argument1);


I use a UINT in the DLL. I could/should have used USHORT instead... Byte is simply too small to be useful and people would have complained the moment their string would be bigger than 255 characters.

[Alexander_Q]

Yeah, ushort would have made more sense. I'll take your advice and try and make my own write/read strings. Cheers!

Guess what - I solved my problem without needing to do two loops. It's simple. Before the for loop I add in a ushort of value 0. Then why I seek back it will have something to write over, and the string header won't be overwritten. Also, if there are NO tiles, then when the room is importing it will be told tile number is zero and know just to skip that part of importing.

This is a great DLL. Thankyou very much for drawing my attention to it.

[icuurd12b42]

Yeah, ushort would have made more sense. I'll take your advice and try and make my own write/read strings. Cheers!

Guess what - I solved my problem without needing to do two loops. It's simple. Before the for loop I add in a ushort of value 0. Then why I seek back it will have something to write over, and the string header won't be overwritten. Also, if there are NO tiles, then when the room is importing it will be told tile number is zero and know just to skip that part of importing.

This is a great DLL. Thankyou very much for drawing my attention to it.

That's smart... I would never have thought of doing that.

I got my hands on a few RSA Public/Private key systems... It took the whole day to find one that did not crash... The shear size of the numeric value needed for safe encryption is too much for the standard data types in C... So all the sample I found would crash (most of them were simply examples of the method using small values). In real application they would fails to generate the keys... Anyway, I have to strip out a few things from one that uses a huge number class. Should take a while though.


I will not encrypt files with it, but strings in the file... Because the resulting size of the file and the time needed to decrypt would be too great... So, Using the password encryption that exists (it's prety secure) and adding the ability to RSA string will allow for transmitting the password in the file...


Glad you like the dll.

[Alexander_Q]

That's good news. I'll keep my eyes on this topic. This makes everything so much easier because I'm not constantly double checking how many times to do readln and making sure I don't have strings on the same text line as reals and other such annoyances with file_text_write etc.

[TheMyst]

Necro post, I know, but I think that this DLL may have solved all of my problems with my games level format. You are awesome man!

EDIT: Icurrd, if I ever meet you in real life, remind me to buy you a drink. I officially love this DLL.

Edited by TheMyst, 05 April 2009 - 12:57 AM.

[icuurd12b42]

Necro post, I know, but I think that this DLL may have solved all of my problems with my games level format. You are awesome man!

EDIT: Icurrd, if I ever meet you in real life, remind me to buy you a drink. I officially love this DLL.

Thanks!

[uuf6429]

Signed and unsigned stuff usually have their own names (not always "signed int" and "unsigned int").
Example: integers are -2147483648..2147483647 (signed 32 bit)
While: cardinals are 0..4294967295 (unsigned 32-bit)
And: byte (0..255) and shortint (-128..127) are both 8-bit

Kind regards,
Chris.

[icuurd12b42]

Signed and unsigned stuff usually have their own names (not always "signed int" and "unsigned int").
Example: integers are -2147483648..2147483647 (signed 32 bit)
While: cardinals are 0..4294967295 (unsigned 32-bit)
And: byte (0..255) and shortint (-128..127) are both 8-bit

Kind regards,
Chris.

short int is 16 bits. not 8 (on 32 bit computers/applications) -32K to +32K
byte is -127 to 128 signed, 0-255 unsigned

[uuf6429]

I guess that depends on the compiler? My delphi 7 documentation says:

PRE

Type		Range			Format

Shortint	-128..127		signed 8-bit

Smallint	-32768..32767		signed 16-bit

Longint		-2147483648..2147483647	signed 32-bit

Int64		-2^63..2^63-1		signed 64-bit

Byte		0..255			unsigned 8-bit

Word		0..65535		unsigned 16-bit

Longword	0..4294967295		unsigned 32-bit

Kind regards,
Chris.

Edited by uuf6429, 06 April 2009 - 03:57 PM.

[icuurd12b42]

I guess that depends on the compiler? My delphi 7 documentation says:

PRE

Type		Range			Format
Shortint	-128..127		signed 8-bit
Smallint	-32768..32767		signed 16-bit
Longint		-2147483648..2147483647	signed 32-bit
Int64		-2^63..2^63-1		signed 64-bit
Byte		0..255			unsigned 8-bit
Word		0..65535		unsigned 16-bit
Longword	0..4294967295		unsigned 32-bit

Kind regards,
Chris.

I see

PRE

Delphy		c++ win32 compiler

Shortint	= char (byte)
Smallint	= short int
Longint		= int
Int64		= long
Byte		= unsigned char
Word		= unsigned short
Longword	= unsigned int
??		= unsigned long (Do you have a 0 to 2^64) (Do I have this option, I don't think so either)
??		= float (16 bit floating point or is it 32?)
??		= double (32 bit floating point or is it 64?)

Edited by icuurd12b42, 06 April 2009 - 06:54 PM.

[uuf6429]

First, it's "Delphi"

Secondly:

PRE

Type		Range						Digits		Size in bytes
Real48		2.9 x 10^-39 .. 1.7 x 10^38			11-12		6
Single		1.5 x 10^-45 .. 3.4 x 10^38			7-8		4
Double		5.0 x 10^-324 .. 1.7 x 10^308			15-16		8
Extended	3.6 x 10^-4951 .. 1.1 x 10^4932			19-20		10
Comp		-2^63+1 .. 2^63 -1				19-20		8
Currency	-922337203685477.5808.. 922337203685477.5807	19-20		8

As to unsigned long, no there sin't anything likw that (to my knowledge). But in newer versions of Delphi (2005+) you can specify "long" with normal types which increase the size, eg: "long Cardinal". I'm not sure if it works like this.
The name for this data type would probably be LongWord64 or Cardinal64 (longword==cardinal).

Also, question about your dll. I assume that there's a limit on the values? Say, I can't use all the features of saving an int64 since GM supports up to an integer. How do you fix this?

Kind regards,
Chris.

Edited by uuf6429, 08 April 2009 - 09:30 AM.

[icuurd12b42]

Also, question about your dll. I assume that there's a limit on the values? Say, I can't use all the features of saving an int64 since GM supports up to an integer. How do you fix this?

Kind regards,
Chris.

Try it, you'll see my easter egg...

No, you can't use long, I disabled it (in the gmk function) and suggest you use double or real instead which is the upper limit of GM anyway. The Writelong will cause abnormal behaviour once you reach a certain high value in the type casting part of the code when it tries to cast the double into a long. The type cast fails in a really bad way (crash) which is odd because all the other type casts work even with extreme values. I may have circomvented this crash in the dll, I can't remember though, I am too lazy to check, by typecasting to a DWORD beforehand to limit the value. So essentialy, the upper limit of a non fractional number is 32 bits then you have to use the double version... Hope that makes sense.

[uuf6429]

This [dll] would be very useful for general use rather then personal use (eg, using it to write binary files like bmp's etc).
In which case, taking full advantage of writing capabilities is important.

My idea is passing numbers (integers and floats) around as strings. So the dll can even write int64 using write(file,'3425464654') which is not possible in GM. For this to work, you just need to convert strings to numbers on the dll side, eg (Delphi):

funtion WriteInt64(FileHandle:Double; Value:PChar):Double; stdcall;
var v:Int64;
begin
  v:=StrToInt(Value); // converts string to integer
  TFileStream(Trunc(FileHandle)).WriteBuffer(^v,SizeOf(v)); // writes the int64, FileHandle is a pointer to file
			 // stream class and since pointers can't be floats, trunc must be used which returns integral part
  Result:=GMTRUE; // a double constant for GM's bools
end;

Kind regards,
Chris.

Edited by uuf6429, 08 April 2009 - 11:45 AM.

[icuurd12b42]

This [dll] would be very useful for general use rather then personal use (eg, using it to write binary files like bmp's etc).
In which case, taking full advantage of writing capabilities is important.

My idea is passing numbers (integers and floats) around as strings. So the dll can even write int64 using write(file,'3425464654') which is not possible in GM. For this to work, you just need to convert strings to numbers on the dll side, eg (Delphi):

funtion WriteInt64(FileHandle:Double; Value:PChar):Double; stdcall;
var v:Int64;
begin
  v:=StrToInt(Value); // converts string to integer
  TFileStream(Trunc(FileHandle)).WriteBuffer(^v,SizeOf(v)); // writes the int64, FileHandle is a pointer to file
			 // stream class and since pointers can't be floats, trunc must be used which returns integral part
  Result:=GMTRUE; // a double constant for GM's bools
end;

Kind regards,
Chris.

Most file formats use int 32 numbers so you should be able to use the dll for general use. It would be faster to perform 2 readint and reassemble it yourself if you ever encounter a file format that saved a long value...

Also, you have to be aware that the byte order of the numbers int or float in a file depends on what the programmer decided... Some save each byte from left to right some right to left....

The source is included if you every need to flip it arround or add extra features.

[Skarik]

I get problems with this when my files approach the 2 meg size. It seems to stop writing after a while. Is it me or this?

[icuurd12b42]

I can't say, I got 20MB files going myself. Maybe you found a bug in one of the features. What are you using?

[Skarik]

I can't say, I got 20MB files going myself. Maybe you found a bug in one of the features. What are you using?

A butt load of write strings.

Actually, I comment those out - well, most of them, I may only have about 4 or 5 in there now, or even just one - and it works fine. My problem is with my trigger system, which I use all strings when setting up the events before the game starts. I usually have around a hundred objects per level as well now.


But yes, a buttload of write strings.

[icuurd12b42]

Weird because this test works... 25 MB file

GMBINInit()


show_message("Making list")
ds = ds_list_create();
repeat(1000000)
ds_list_add(ds,
choose(
"aaaaaaaaaaaaaaaaaaaaaaaa",
"bbbbbbbbbbbbbb",
"ccccccccccccccccc",
"dddddddddddddddddddddddddddddddddddddd",
"eeeeeeeeeeee",
"ffffffffffffffffffffffffffff"))
show_message("Saving")
hf = GMBINOpenFileWrite("test.tst");
i = 0;
repeat(1000000)
{
GMBINWriteString(hf,ds_list_find_value(ds,i))
i+=1;
err = GMBINIsError(hf)
if(err)
{
show_message("Write Error " + string(err))
GMBINCloseFile(hf);
exit;
}
}
GMBINCloseFile(hf);

show_message("Reading/Comparing")
hf = GMBINOpenFileRead("test.tst");
i = 0;
repeat(1000000)
{
s = GMBINReadString(hf,)
err = GMBINIsError(hf)
if(err)
{
show_message("Read Error " + string(err))
GMBINCloseFile(hf);
exit;
}
if(ds_list_find_value(ds,i) != s)
{
show_message("Compare failed Item: " + string(i))
show_message(s +"!=#" + ds_list_find_value(ds,i))
GMBINCloseFile(hf);
exit;
}
i+=1;
}
GMBINCloseFile(hf);

show_message("Done. All Clear.")



Can you try it on your pc?

Edited by icuurd12b42, 01 July 2009 - 01:22 AM.

[Skarik]

It took ten minutes, but I get an all clear. Hold on, let me check my file format.


Edit:
Okay, here's what I have in the middle of my file format, after objects, but before the environment and specular settings. It's commented out right now, since it's useless:

// _-_LINKS_-_
// Write the object link block start

// Linker database
STCKExportWriteBlock(3); // Writes a byte of 3
with (object_base) {
	if (name != "") {
		STCKExportWriteNameIndex(id);
	}
}
STCKExportWriteNameIndex(-1);

This is my STCKExportWriteNameIndex function:

if (argument0 == -1) {
	GMBINWriteInt(global.stck_export_file,argument0);
}
/*else
{
	GMBINWriteInt(global.stck_export_file,argument0);
	GMBINWriteString(global.stck_import_file,argument0.name);
}*/

And it works as long as that part is commented out.

Here's the reader part, simple loop:

GMBINReadByte(global.stck_import_file); // Reads the three

if (global.level_vs == 2)
{
	exit;
}

var new_link, verify;
verify = true;
// Linker database
while (verify) {

	new_link = GMBINReadInt(global.stck_import_file);
	//show_message("link bla"+string(new_link));
	if (new_link != -1)
	{
		GMBINReadString(global.stck_import_file);
	}
	else
	{
		verify = false;
	}
}

verify = true;
// link type, object1, object2
while (verify) {
	
	new_link = GMBINReadUByte(global.stck_import_file);
	//show_message("link2 bla"+string(new_link));
	if (new_link != 255)
	{
		GMBINReadInt(global.stck_import_file); //cause
		GMBINReadInt(global.stck_import_file); //target
	}
	else
	{
		verify = false;
	}
}

Dunno what's wrong, I always thought it was the strings, 'cause it like dies at around that point, just a random 134 int pops out of nowhere. No idea. I've got number signs and a buttload of symbols everywhere, but that shouldn't make a difference, looking at the source code. And my length of strings are definitely under the size of an UINT type. Huh, no idea. It may be a problem on my part. Weird....

Edited by Skarik, 01 July 2009 - 02:25 AM.

[icuurd12b42]

OK, if it dies, it's because your writing and your reading no longer match, like you forget to read something you wrote and now the next read call takes reads in crap data.


in the case where you would read a string from the wrong spot, it readS the (WRONG) string size, say a gazilion bytes to read and proceeds trying to alocate and read those bytes...

I can't see anything wrong with the snippets you posted. I suspect you have a mismatch read somwhere else though... Found it

BTW

//if (argument0 == -1) {
// GMBINWriteInt(global.stck_export_file,argument0);
//}
//else
//{
// GMBINWriteInt(global.stck_export_file,argument0);
GMBINWriteString(global.stck_import_file,argument0
.name);
//}





// Linker database
STCKExportWriteBlock(3); // Writes a byte of 3
with (object_base) {
//if (name != "") {
STCKExportWriteNameIndex(id);
//}
}
//Why is this here See, you wrote a valid name above and now you still write -1, here is your mismatch
//it should have been in an else under the if above
//STCKExportWriteNameIndex(-1);





var new_link, verify;
verify = true;
// Linker database
while (verify) {

//new_link = GMBINReadInt(global.stck_import_file);
//show_message("link bla"+string(new_link));
//if (new_link != -1)
//{
//you foget to stuff the read data in a variable here
name = GMBINReadString(global.stck_import_file);
//}
//else
//{
// verify = false;
//}
verify = (name == "")
}

[edit] never mind, I think I screwed up understanding your code... the -1 is for verifying the end of the read loop... anyway check for mismatch write/reads...

Edited by icuurd12b42, 01 July 2009 - 03:06 AM.

[JudgeSpear]

I apologize for resurrecting an old thread, but I thought I should bring attention to an issue I have with using GMBinFile.dll in Game Maker 8 under Windows 7. The DLL initialization script does not want to work properly when running a GM8 game under Windows 7. I get a whole bunch of "Error defining an external function" messages on my initialization object's Game Start event.

Game Maker 7 games run GMBinFile.dll with no problems.

[icuurd12b42]

FIXED!

Sorry, I dont know what went wrong here. I think I updated with a bad file I was tesing stuff (that did not work) with.

At least that is what was going on here... so many versions. I hope it works for you. I also recompiled with another compiler.

Edited by icuurd12b42, 29 January 2010 - 02:56 AM.

[IceMetalPunk]

I may have found another bug (or else I'm doing something wrong)...

I'm using this DLL with GM8 to handle saving and loading of a custom file format, which includes the packing of some images and an MP3. The packing is irrelevant, though.

The "magic number" at the very beginning of the file is the 5-character buffer ")RYTH". The entire file saves fine (I checked), but when I go to load it, the program crashes. Not even an unexpected error, just a complete Windows error "This program has stopped working".

I've narrowed the crashing point down to the simple reading of the magic number...which simply reads a 5-character buffer. I can't figure out why this would be crashing...can you?

-IMP

*EDIT* Did a quick test. The DLL works fine in GM7, it only causes this crash in GM8...

Edited by IceMetalPunk, 31 January 2010 - 08:36 PM.

[icuurd12b42]

I may have found another bug (or else I'm doing something wrong)...

I'm using this DLL with GM8 to handle saving and loading of a custom file format, which includes the packing of some images and an MP3. The packing is irrelevant, though.

The "magic number" at the very beginning of the file is the 5-character buffer ")RYTH". The entire file saves fine (I checked), but when I go to load it, the program crashes. Not even an unexpected error, just a complete Windows error "This program has stopped working".

I've narrowed the crashing point down to the simple reading of the magic number...which simply reads a 5-character buffer. I can't figure out why this would be crashing...can you?

-IMP

*EDIT* Did a quick test. The DLL works fine in GM7, it only causes this crash in GM8...

You'll have to send me a simple setup for me to debug.

[IceMetalPunk]

In trying to make an example file with the bug, everything works fine. So it seems the issue is some conflict between something in my code and your DLL...though I can't for the life of me figure out what that could be.

I'll PM you the .GMK of the game I'm working on, since that's the only file with which I have the problem.

*EDIT* You can ignore the PM I sent you; I found the problem . I had reversed the order of arguments to my own scripts, and thus was passing a real value as a filename... I'll pretend this never happened if you all will .

-IMP

Edited by IceMetalPunk, 01 February 2010 - 06:25 PM.

[icuurd12b42]

In trying to make an example file with the bug, everything works fine. So it seems the issue is some conflict between something in my code and your DLL...though I can't for the life of me figure out what that could be.

I'll PM you the .GMK of the game I'm working on, since that's the only file with which I have the problem.

*EDIT* You can ignore the PM I sent you; I found the problem . I had reversed the order of arguments to my own scripts, and thus was passing a real value as a filename... I'll pretend this never happened if you all will .

-IMP

You shoul check the result of the OpenFile call. it did result in 0 (FAILED)
You should not call any read function if it failed. reading buffer on file handle 0 is where the crash happens.
For speeds sake, I do not validated every single read byte/buffer/int/etc calls so make sure the file handle is valid (hf != 0). You have to do it <once> before doing your entire file reading code

Edited by icuurd12b42, 01 February 2010 - 06:54 PM.

[brett14]

Do you mind if I take some of the code from your source, and put it into my dll? You'll be given credit.

Anyways, great DLL I use it alot for saving data.

[icuurd12b42]

Do you mind if I take some of the code from your source, and put it into my dll? You'll be given credit.

Anyways, great DLL I use it alot for saving data.

I dont mind. the source is there for you to use...

[brett14]

thnx.

[JudgeSpear]

One person testing a game I made using GMBinFile.dll reported the same issues I had earlier on in 32-bit Windows 7, now fixed:

The DLL initialization script does not want to work properly when running a GM8 game under Windows 7. I get a whole bunch of "Error defining an external function" messages on my initialization object's Game Start event.

This person reported this issue trying to run the game on a 64-bit Windows 7 computer.

[icuurd12b42]

One person testing a game I made using GMBinFile.dll reported the same issues I had earlier on in 32-bit Windows 7, now fixed:

The DLL initialization script does not want to work properly when running a GM8 game under Windows 7. I get a whole bunch of "Error defining an external function" messages on my initialization object's Game Start event.

This person reported this issue trying to run the game on a 64-bit Windows 7 computer.

So the new version work? Yes?

[JudgeSpear]

So the new version work? Yes?

It works perfectly in 32-bit Windows 7 computers.

One friend with a 64-bit Windows 7 computer encountered the same problems I had reported earlier in 32-bit Windows 7 before you released the new version. So now, it seems that the DLL does not want to initialize in Game Maker 8 games running in 64-bit Windows 7 computers.

I'd like to thank you for the swift response earlier. I did not expect a new build of the DLL that quickly.

[brett14]

@JudgeSpear
I have Win7 (32bit) and GM8 with no problems at all using this dll.

@Icuurd12b42
I am making a level editor, and using this to save the data. When saving you encrypt the file with a password. Then the player is able to include the map in his external resources that he made. He loads it with a password. However there is one problem, if the WRONG password is given the game will pop you up errors and just shut down. So in my level editor if you load a map to edit it and type the wrong password in, the game crashes without warning. I'm curious if there is someway that you can tell if the user inputted the correct password, and if they didn't return an error value. The built in error detecting function does not work as the game crashes.

Edited by brett14, 09 February 2010 - 12:15 AM.

[icuurd12b42]

@JudgeSpear
I have Win7 (32bit) and GM8 with no problems at all using this dll.

@Icuurd12b42
I am making a level editor, and using this to save the data. When saving you encrypt the file with a password. Then the player is able to include the map in his external resources that he made. He loads it with a password. However there is one problem, if the WRONG password is given the game will pop you up errors and just shut down. So in my level editor if you load a map to edit it and type the wrong password in, the game crashes without warning. I'm curious if there is someway that you can tell if the user inputted the correct password, and if they didn't return an error value. The built in error detecting function does not work as the game crashes.

you must check the result of the open file... if the wrong password is provided the file handle return will be 0.

var hf;
hf = GMBINOpenFileReadPW("test.bin","password")
if(hf == 0)
{
show_message("Wrong password, or error opening the file");
exit;
}
read the file with hf...

iserror will not work on 0 file handle either.

ALSO, looking at the code, it looks like I made the mistake of not returning failure if you try to open a file using a password where the file is not encryped... It return the file handle, however, since I read 6 bytes (the tag to indicate it's PW encoded), the file pointer is already 6 bytes in the file. which wiil cause crashes in your gml, reading data from the wrong file buffer location....

ALSO, there is not check if you open without a password and the file has a password... So you would read garbage, causing a problem on GM's end... like
v = GMBINReadDouble(hf); //v has a garbage value


As for windows 7 64bits and GM8... I cant test that... Failure to initialise the dll means it could not load the dll in memory or failed to associate to a function. Is the dll in your project of game directory?

Also, gm7 and <possibly> gm8 have intermitent troubles loading ANY dll. it may be what happened to your user. tell him to shut down and restart the computer.

Edited by icuurd12b42, 09 February 2010 - 03:06 AM.

[Krisando]

Could you add a function to read an inputted about of bytes into it's integral form.

For example reading the three bytes of values "12^2, 34^1, 57^0", would show "795193":
variable = GMBINReadBytes(3);
show_message(string(variable));

Also do you know the formula for storing the mantissa in a 4-bit float value? "using bitwise operators"