Jump to content


Photo

Simple External Resource (archive) Manager


  • Please log in to reply
34 replies to this topic

#1 Revel

Revel

    ɹǝqɯǝɯ ɔɯƃ

  • GMC Member
  • 4935 posts
  • Version:GM8

Posted 20 June 2009 - 12:50 AM

I have been working on an external resource manager called "Cypher". It's very powerful but it can take a while to understand how it works. Basically you can open an archive, add files, Extract files and re-write the archive. Each file you add to the archive can have its own individual encryption password, but you can also encrypt the whole package with another password to add dual security.


Here is a basic list of the functions. Please see the .GMK for more information:
Cypher_Init(); Initiates the Engine. Make sure to use this before calling any other function.
Cypher_Open(Fname,Password); Open an archive. If you are creating a new archive, set both arguments as an empty string.
Cypher_AddFile(PkgID,Fname,Password,InternalFname); Adds a file to an archive. 
Cypher_Write(PkgID,Fname,Password); Writes the archive
Cypher_Extract(PkgID,File,Password,ExtractLocation); Extracts a File
Cypher_Close(PkgID); Closes a package and frees memory.
Cypher_FileCount(PkgID); Gets the number of files in a package
Cypher_GetFname(PkgID, FileID); Gets a filename of a file in the package
Cypher_RemoveFile(PkgID,Fname); Deletes a file from a package


I included a well commented example in the GMK. I hope the API is easy enough to understand!

Download


Please tell me your suggestions/comments!


Edit: Performance test!

1000 Sprites with each sprite encrypted and the whole package encrypted: 12 seconds (Packaging)

Edited by Revel, 25 June 2009 - 03:45 PM.

  • 0

#2 Recreate

Recreate

    Furry

  • GMC Member
  • 2992 posts
  • Version:GM:Studio

Posted 20 June 2009 - 01:13 AM

GM6, Please.
  • 0

If the post that you are reading was created prior to 2011. For the safety of the general public, It is not to be regarded under any circumstances.
Please don't ask me to join your group at anything.


#3 Revel

Revel

    ɹǝqɯǝɯ ɔɯƃ

  • GMC Member
  • 4935 posts
  • Version:GM8

Posted 20 June 2009 - 01:23 AM

Use LGM or something to convert it.
  • 0

#4 Recreate

Recreate

    Furry

  • GMC Member
  • 2992 posts
  • Version:GM:Studio

Posted 20 June 2009 - 01:44 AM

Use LGM or something to convert it.

I can't find the LGM anymore.
  • 0

If the post that you are reading was created prior to 2011. For the safety of the general public, It is not to be regarded under any circumstances.
Please don't ask me to join your group at anything.


#5 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 20 June 2009 - 02:02 AM

Recreate, there is this website that is made for situations like that. Google. Simply go to it and type in "Lateral Game Maker", it will find it for you.

Anyways, Revel, it's nice to see you got it done after all the speed issues.
  • 0

#6 Marchal_Mig12

Marchal_Mig12

    The Rhouan

  • GMC Member
  • 1182 posts
  • Version:GM:Studio

Posted 22 June 2009 - 02:22 AM

This new DLL is awesome. It is a must for every game. New features are very useful and are easy to use. Congratz and well done.
  • 0

#7 Krisando

Krisando

    GMC Member

  • New Member
  • 1351 posts

Posted 24 June 2009 - 03:15 AM

Compression could be nice =]
  • 0

Posted Image


#8 Revel

Revel

    ɹǝqɯǝɯ ɔɯƃ

  • GMC Member
  • 4935 posts
  • Version:GM8

Posted 24 June 2009 - 03:25 AM

Yea, I wanted to add that. Once the current version is 100% stable, then I will find a good compression algorithm and add it in for sure!
  • 0

#9 freaked

freaked

    freak up!

  • New Member
  • 890 posts

Posted 24 June 2009 - 04:20 AM

Go for zlib.
Easy to implement.
  • 0

#10 Revel

Revel

    ɹǝqɯǝɯ ɔɯƃ

  • GMC Member
  • 4935 posts
  • Version:GM8

Posted 24 June 2009 - 05:48 PM

I found a fatal bug in the DLL that pretty much rendered it useless because the DLL did not work with more than 1 file. I never really tested it with more than 1 file (silly me!) but I have found the fatal bug which brings this DLL into a stable state. Hopefully with this bug out of the way, Ill get more replies..::):

Download


NOTE* You will need to re-write any archives, because the bug was in the creation of the archives, which effected the reading.

Time to add compression :P


EDIT: Compression is done.. I just need to do some testing to make sure it works fully.


EDIT2:
Compression is done. New Link Here. The DLL size is now about 300kb rather than the 1.3mb before.

Edited by Revel, 25 June 2009 - 03:46 AM.

  • 0

#11 Revel

Revel

    ɹǝqɯǝɯ ɔɯƃ

  • GMC Member
  • 4935 posts
  • Version:GM8

Posted 26 June 2009 - 06:44 PM

Disregard this :D

Edited by Revel, 27 June 2009 - 06:12 PM.

  • 0

#12 Marchal_Mig12

Marchal_Mig12

    The Rhouan

  • GMC Member
  • 1182 posts
  • Version:GM:Studio

Posted 09 July 2009 - 12:30 AM

Could it be possible to archive more than one file at time?
  • 0

#13 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 09 July 2009 - 02:06 AM

You can use Game Maker's file finding functions to make a script that adds the files in a directory. (*Cough* Revel forgot these functions for this DLL, I didn't.)
  • 0

#14 Marchal_Mig12

Marchal_Mig12

    The Rhouan

  • GMC Member
  • 1182 posts
  • Version:GM:Studio

Posted 09 July 2009 - 02:08 AM

It looks like I cant archive and extract multiple files. It keeps running into an unexpected error.
  • 0

#15 Revel

Revel

    ɹǝqɯǝɯ ɔɯƃ

  • GMC Member
  • 4935 posts
  • Version:GM8

Posted 09 July 2009 - 02:11 AM

It looks like I cant archive and extract multiple files. It keeps running into an unexpected error.

Can you please post or PM me your GML code?

If you wish to keep your code private, could you please explain what your doing?
  • 0

#16 Recreate

Recreate

    Furry

  • GMC Member
  • 2992 posts
  • Version:GM:Studio

Posted 14 July 2009 - 04:21 PM

This Seems Just like what i am looking for!
Thanks :) (its for the thing in my signature, long story)
Edit: It seems that the packed file is about 600Bytes larger, No big deal, Just letting you know.

Edited by Recreate, 14 July 2009 - 04:34 PM.

  • 0

If the post that you are reading was created prior to 2011. For the safety of the general public, It is not to be regarded under any circumstances.
Please don't ask me to join your group at anything.


#17 Revel

Revel

    ɹǝqɯǝɯ ɔɯƃ

  • GMC Member
  • 4935 posts
  • Version:GM8

Posted 14 July 2009 - 07:37 PM

Edit: It seems that the packed file is about 600Bytes larger, No big deal, Just letting you know.


Interesting... Ill check it out :) Someone also mentioned to me that the compression wasn't working properly.


Edit:

During my tests, I packages about 6.5 mb of data, and the output file was exactly 6.5 mb. I tried compressing the file BEFORE the encryption step and the output file was then 7.3mb...

However, the compression seems to work when I just use images.


It looks like the compression works better if you don't apply a final password to your archive.

Edited by Revel, 15 July 2009 - 01:48 AM.

  • 0

#18 slayer 64

slayer 64

    Slayer of gingers

  • GMC Member
  • 4301 posts
  • Version:GM:Studio

Posted 16 July 2009 - 01:46 AM

this looks nice. i would use it to keep people from stealing game resources. i made a function i think you should add, and improve on.
cypher_extract_sprite
//
//  Usage:
//	  sprite_index=cypher_extract_sprite(EXTRACT,"Sprite.bmp","SpritesPassword");
//
//  Arguments:
//	  extract
//	  file path
//	  password
//
//  returns:
//	  sprite extracted
//

var spr;
Cypher_Extract(argument0,argument1,argument2,"temp.bmp");
spr=sprite_add("temp.bmp",0,0,1,0,0,0,0);
file_delete("temp.bmp");
return spr;

  • 0

5y5rs3d.pngfg0UQNL.png


#19 Brother156

Brother156

    GMC Member

  • Validating
  • 498 posts

Posted 16 July 2009 - 06:21 AM

This is nice! You could create a compressing program with this! I migh try it, could I?



#20 nick1965

nick1965

    GMC Member

  • New Member
  • 88 posts

Posted 30 July 2009 - 06:08 PM

Very good! Well, it still doesn't solve the problem though, and I don't think anyone but YYG / Mark could fix the problem- not being able to load the files straight into the game. All external archives have that same problem, and they get around it by creating the file in unencrypted format, loading it into GM using 'sprite_add', and 'sound_add', then deleting the files... This leads to some serious security issues...
Anyways, I really liked the dual encryption. :)

~Nickl
  • 0

#21 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 30 July 2009 - 06:26 PM

Since when do exposed sprites cause serious security issues? Now, GMAPI would probably be able to load it directly, but this doesn't prevent people from using GMAPI to reverse this? They could also take screenshots and recreate them.
  • 0

#22 Razon

Razon

    GMC Member

  • GMC Member
  • 268 posts
  • Version:GM:Studio

Posted 05 August 2009 - 05:44 PM

The file creation and modified dates are lost when they're stored and extracted, you need to fix this- also would be nice if there was an option of wether or not it should store them.


And exposed sprites aren't a security issue, you just don't want people getting at the graphics. While it's true that you can take graphics out of game, you can't get them raw say with png alpha and such without them being blended into the environment and being a mess. He may have been referring to masks and such being externally loaded, but you should md5 check any files before they're loaded into the game anyway.

Edited by RazonEX, 05 August 2009 - 05:50 PM.

  • 0

#23 Yambam

Yambam

    GMC Member

  • GMC Member
  • 647 posts
  • Version:GM8

Posted 06 August 2009 - 08:28 AM

this looks nice. i would use it to keep people from stealing game resources. i made a function i think you should add, and improve on.
cypher_extract_sprite

//
//  Usage:
//	  sprite_index=cypher_extract_sprite(EXTRACT,"Sprite.bmp","SpritesPassword");
//
//  Arguments:
//	  extract
//	  file path
//	  password
//
//  returns:
//	  sprite extracted
//

var spr;
Cypher_Extract(argument0,argument1,argument2,"temp.bmp");
spr=sprite_add("temp.bmp",0,0,1,0,0,0,0);
file_delete("temp.bmp");
return spr;

I think this is a beter function
//
//  Usage:
//	  sprite_index=cypher_extract_sprite(EXTRACT,"Sprite.bmp","SpritesPassword");
//
//  Arguments:
//	  extract
//	  file path
//	  password
//
//  returns:
//	  sprite extracted
//

var spr;
Cypher_Extract(argument0,argument1,argument2,"temp.bmp");
if spr=sprite_add("temp.bmp",0,0,1,0,0,0,0);
  file_delete("temp.bmp");
return spr;
this function wil directly delete the 'temp.bmp' file when the sprite is added ;)

Edited by Yambam, 06 August 2009 - 08:30 AM.

  • 0

#24 LoopStan

LoopStan

    North-See Developer

  • GMC Member
  • 1398 posts

Posted 19 August 2009 - 07:41 PM

Don't forget the part where each sprite should have a different handle. Easily done. I want to say gratulations on this because it works very well and I am going to be using it for all my projects until someone makes a better one (which is going to be hard). Thank you very much for this :)
  • 0

Posted Image


#25 Revel

Revel

    ɹǝqɯǝɯ ɔɯƃ

  • GMC Member
  • 4935 posts
  • Version:GM8

Posted 20 August 2009 - 01:51 AM

I'm currently working on a new version of Cypher (Complete rewrite).

If you have any feature requests, please post here.
  • 0

#26 MDiaz

MDiaz

    GMC Member

  • New Member
  • 23 posts

Posted 23 August 2009 - 09:31 AM

Alright, so I think I'm using this wrong or something. I'm simply trying to load 2 sprites that I added into a PAK file a made. The PAK only has 2 sprites. It's taking awfully long time to load and sometimes freezes and crashes. I know everything is correctly coded, because it was working perfectly with 1 sprite. So now I tried doing it with 2 sprites and it doesn't work. Adding to PAK file is good, I didn't get an errors from that. I think the extracting part is where I made a mistake or something.

Here is the code I used:

//Extract the PAK

//Re-Open the PAK.
EXT = Cypher_Open("bin\Sprite.CYN","Cyanide");

//Extract the sprite.
Cypher_Extract(EXT,"sprite\test.png","SprCyanide","bin\temp\test.png");

//Add the sprite to the game.
spr = sprite_add("bin\temp\test.png",0,0,1,0,0,0,0);

_sprworked = 1;

//Encrypt PAK
Cypher_Write(EXT,"bin\Sprite.CYN","Cyanide");

//Extract the sprite.
Cypher_Extract(EXT,"sprite\test2.png","SprCyanide","bin\temp\test2.png");

//Add the sprite to the game.
spr2 = sprite_add("bin\temp\test2.png",0,0,1,0,0,0,0);

_sprworked2 = 1;

//Encrypt PAK
Cypher_Write(EXT,"bin\Sprite.CYN","Cyanide");

//Close the PAK.
Cypher_Close(EXT);

I don't understand, if I did something wrong or what?
Thanks in advance.
  • 0

#27 Rusky

Rusky

    GMC Member

  • GMC Member
  • 2492 posts
  • Version:Unknown

Posted 28 August 2009 - 02:05 AM

Encryption and compression don't work well together. Encryption often makes things bigger, and compression works best when things have patterns, which encryption generally gets rid of.

@MDiaz: Why are you using Cypher_Write after extracting each file? You're not changing anything in the archive so it's not needed, and the archive is still open when you do it, which may be the reason for the crash.
  • 0

#28 Lithium

Lithium

    Spitfire Member

  • GMC Member
  • 1477 posts
  • Version:Unknown

Posted 28 August 2009 - 11:34 AM

Other resource DLLs allow you to write strings in the file in addition to adding files. I'm suggesting that for the new version.
  • 0

E1JdWqS.png

FCO now in Alpha testing phase

Sign up on the forums for further information:

http://fco2.forumotion.cc/

 

#29 kburkhart84

kburkhart84

    Firehammer Games

  • GMC Member
  • 2026 posts
  • Version:GM:Studio

Posted 08 September 2009 - 06:56 AM

This was the best looking resource manager I could find. Would it be a problem if I "redid" this dll into an extension. I would possibly do d&d actions along with gml functions. In fact, I think I would also like to create a "helper" program that allows easy creation of said archives. Do I have permission??

EDIT**

I now have an extension working. All it does though is create gml functions from the dll functions by including the extension automagically. I still have work to do though.

If I don't get permission to distribute this extension, I atleast intend to use it for myself, since I don't like messing with dlls.

Edited by kburkhart84, 09 September 2009 - 03:11 AM.

  • 0

My KBInput system is now on the marketplace here.  It wraps up nice and tight GMStudio's input system into a few function calls making a user configurable input system that works the same regardless of what inputs the player has chosen including keyboard, mouse buttons, and gamepad/joysticks using DInput/XInput.  The support forum topic for it is here.


#30 Advokara

Advokara

    GMC Member

  • New Member
  • 116 posts

Posted 22 November 2009 - 10:16 AM

Apparently doesn't work on Windows 7

I ran the example and it says "Unexpected error"

EDIT: I tried it again and it worked... :D I must have done sothing wrong lol

Edited by Advokara, 22 November 2009 - 10:20 AM.

  • 0

Current Project: Craftbox Version 0.5


#31 FmMan3

FmMan3

    GMC Member

  • GMC Member
  • 189 posts
  • Version:GM:Studio

Posted 22 November 2009 - 10:50 PM

Hey, man.

LOVE the DLL, you asked for requests for your rewrite of the DLL. A function I'd like to see is a "Cypher_AddDirectory" and Cypher_ExtractDirectory"
It would definitly be faster to write out a full directory that you wish to package.

Cheers for this awesome DLL!

Apparently doesn't work on Windows 7

I ran the example and it says "Unexpected error"

EDIT: I tried it again and it worked... :D I must have done sothing wrong lol


I am currently running Windows 7 Ultimate using GM8 RC2, it runs perfectly fine for me.

Edited by FmMan3, 22 November 2009 - 10:52 PM.

  • 0
Signatures are nothing but a fad.

#32 Advokara

Advokara

    GMC Member

  • New Member
  • 116 posts

Posted 23 November 2009 - 01:34 AM

Lol, yaeh.... I am experiencing errors elsewhere lol... The "Unexpected Error" ... Is there some sort of limit on this DLL? I mean, can you open two packages at once?

EDIT: ONCE AGAIN... :D It was my error. Haha. I got it working :3, just in case anyone cares. I'm making a Myst Fan Engine / Game. ;) Allows the entirety of hotspots and images to be packed into one file.

Edited by Advokara, 23 November 2009 - 04:11 AM.

  • 0

Current Project: Craftbox Version 0.5


#33 Razon

Razon

    GMC Member

  • GMC Member
  • 268 posts
  • Version:GM:Studio

Posted 06 January 2010 - 07:20 PM

Sent a crash issue with GM8 by PM 2 weeks back, hopefully you can get it fully working with the new GM version soon- it was tested under Windows XP SP2 Home.

Edited by Razon, 06 January 2010 - 08:26 PM.

  • 0

#34 MasterMind007

MasterMind007

    MasterMind Prod.

  • GMC Member
  • 543 posts

Posted 30 August 2010 - 06:46 PM

Does it work with all files or just sprites? If it works with all, it won't work with a text file.
  • 0
Projects:
The Dragon's Reign - A MMORPG - DELAYED
Posted Image

#35 Harbinger

Harbinger

    GMC Member

  • GMC Member
  • 131 posts
  • Version:GM:Studio

Posted 06 May 2014 - 05:27 PM

This is quite an impressive DLL resource manger 

 

do you think you can add a new function call "Cypher_Addsprite", to make it easier to add an encrypted file dirctly into the game?

 

Example:

/*
Cypher_addsprite(PkgID,Password,File,InternalFname);
    Cuptures the ID from a file from a package that can be assign to a varible.
Arguments:
          [0]PkgID: Package file name
          [1]Fname: The path of the file
          [2]Password: password of the file
          [3]InternalFname: internal Filename      
                             
Return:
Returns 0 if the function failed. Returns internal file ID if success.



*/
var spr;
 
spr=Cypher_addsprite("Game_Data","game_data/spr_player.png","a_creative_password","spr_player.png")
 
with  (obj_player) {sprite_index=spr}
 

Edited by Harbinger, 06 May 2014 - 05:29 PM.

  • 0