Jump to content


Photo

Arch Dll [ External Archives ]


  • Please log in to reply
58 replies to this topic

#1 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 24 June 2009 - 06:46 PM

After creating the PAK Archive DLL, I got a few requests for additions into it. I've made this new archive DLL that uses a different format and should have what was requested before, and maybe even more. The topic title and description handle the poin of the description so just stop reading this if you're bored of it. The scripts included contain more information on the functions, this is not hard to use once you get the hang of it. Simple example included.

I highly suggest you do not use this, it was poorly written and may not work as expected.

Download: Host-A

Notes:
- This doesn't require any ZLib DLLs.
- Tested in GM6 and GM7 (should work in GM8).
- Internal filenames are case sensitive (different cases don't match).
- Adding a file with the same internal name will replace it.
- The directory functions extract all files that begin with a string, an empty internal path string will extract all files.

Edited by TheMagicNumber, 04 September 2011 - 11:14 PM.

  • 2

#2 IceMetalPunk

IceMetalPunk

    InfiniteIMPerfection

  • GMC Elder
  • 9603 posts
  • Version:GM:Studio

Posted 24 June 2009 - 09:07 PM

I would love this...if it worked. It seems the arch_file_exists() function halts execution of the calling script somehow.

I have this in my script, with the debug show_messages exactly as shown:

show_message("Opening archive...");
arch_open(file);

show_message("Checking existence...");
if (!arch_file_exists("SaveData.sdf")) {
show_message("Save Data File Does Not Exist");

arch_close();
show_message("Closed Archive");

return false;
}
show_message("Save Data File Does Exist");

When I run this script (there's more after it, of course; this is just the relevant part), I get the messages up to "Checking Existence...". After that, I get no messages, neither the "Does Not Exist" nor the "Does Exist". The script just stops there, but the game does not freeze. It continues on as if the script was terminated prematurely.

Any ideas what would cause this?

-IMP ;) :)
  • 0

:GM123: Are you an artist who likes creating original creature designs? Maybe you can help out with Elementa here! Give it a look; you might like the idea :)

:bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny:


#3 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 24 June 2009 - 10:21 PM

I've just tested the function in scripts, I failed to replicate your problem.

By the way, I tested it in both GM6 and GM7.

Could you please PM me an example of this problem with your specs? I found a small bug that could have been the problem, I've fixed it and reuploaded it. Please redownload it. :)
  • 2

#4 IceMetalPunk

IceMetalPunk

    InfiniteIMPerfection

  • GMC Elder
  • 9603 posts
  • Version:GM:Studio

Posted 25 June 2009 - 02:30 AM

This occurred in one of my projects... I can make a quick example and send it to you later today (I'm a bit busy at the moment).

At that time I'll also redownload and try the new DLL.

-IMP ;) :)
  • 0

:GM123: Are you an artist who likes creating original creature designs? Maybe you can help out with Elementa here! Give it a look; you might like the idea :)

:bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny:


#5 Revel

Revel

    ɹǝqɯǝɯ ɔɯƃ

  • GMC Member
  • 4935 posts
  • Version:GM8

Posted 25 June 2009 - 03:47 AM

Are you planning on adding encryption?
  • 0

#6 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 25 June 2009 - 02:22 PM

I might add encryption, but only after I find out what is crashing IMP's script.

I have added try..except statements around all functions, they should show an error if any (I suspect an access violation). Although, the problem may have been fixed, too.

Edited by GMREC1111, 25 June 2009 - 02:22 PM.

  • 2

#7 coolist

coolist

    The Coolist

  • New Member
  • 913 posts

Posted 25 June 2009 - 02:46 PM

Can it store directories too? Like directories inside directories?
  • 0

#8 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 25 June 2009 - 02:49 PM

Yes, but there won't be any directory specific functions. The internal file names can contain any character you want (just not null, as GM passes a C-string to DLLs). The names can also be very long, too (65,000 characters max :)).

Basically it doesn't support directories but you use it like it does.

Sorry, misread. Coolist, I started it. Done.

Edited by GMREC1111, 26 June 2009 - 07:53 PM.

  • 2

#9 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 27 June 2009 - 03:02 PM

Thanks. :D

I've added functions for adding directories and extracting them, along with encryption.
  • 2

#10 Aragon1029

Aragon1029

    GMC Member

  • New Member
  • 940 posts

Posted 01 July 2009 - 11:39 PM

Pretty nice DLL. I made a DLL similar to this awhile ago for a game which needed some external sprites. Still pretty useful.
  • 0


Rave Breakout! A new game for iOS devices!
Ever wondered how many lines of code your game has?
Guys, I have a great idea, let's protect the public through censorship.


#11 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 05 July 2009 - 08:52 PM

Thanks.

Still pretty useful.

But nobody seems to think of that...
  • 2

#12 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 14 July 2009 - 09:14 PM

Bump, can I seriously get some suggestions for this? What am I doing wrong? D:
  • 2

#13 fire_soul453

fire_soul453

    aarons video games

  • GMC Member
  • 542 posts

Posted 16 July 2009 - 05:01 PM

Looks awesome. Can it only open the archives you make with it? License?

Anyway probably will use!
  • 0
Game maker Top sites
I'm sick of paypal. From now on I'm not buying indie games unless I can use something else...

#14 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 16 July 2009 - 09:40 PM

It only supports my archives, and if you are asking for a license for the DLL's use, it would be free to use with optional credit (it's appreciated very much).
  • 2

#15 Krisando

Krisando

    GMC Member

  • New Member
  • 1351 posts

Posted 19 July 2009 - 12:16 AM

Wow great dll, umm is there a compression level you can set when you compress a file?
  • 0

Posted Image


#16 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 19 July 2009 - 01:09 AM

There are no options for the compression level as of now, I guess I can add them now. Thanks for the "suggestion."

Added compression levels.

Edited by TheMagicNumber, 19 July 2009 - 03:27 AM.

  • 2

#17 Marchal_Mig12

Marchal_Mig12

    The Rhouan

  • GMC Member
  • 1182 posts
  • Version:GM:Studio

Posted 19 July 2009 - 09:24 PM

Is it me or there is no difference between each compression level. I tried 3 and 1. Level 1 Gave me a 2.67MG packed file and level 3 gave me a 2.66MG packed file.

Is it me or...?

Great dll though :GM071:

EDIT: If I save on an existing file name, does it overwrite the old one?

Edited by Marchal_Mig12, 19 July 2009 - 09:25 PM.

  • 0

#18 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 19 July 2009 - 11:56 PM

The compression levels are fine, it's just the content you are compressing. Some data just can't be compressed as well as other data (compressed data can't really get compressed more).

Thank you. :GM071:

If you save the archive to an existing file, it will (or should, I'll test this) overwrite it. If you want to edit the content, open the archive and edit it, then save it.

Edited by TheMagicNumber, 19 July 2009 - 11:58 PM.

  • 2

#19 Marchal_Mig12

Marchal_Mig12

    The Rhouan

  • GMC Member
  • 1182 posts
  • Version:GM:Studio

Posted 21 July 2009 - 12:59 AM

How about adding .zip format to your dll?
  • 0

#20 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 21 July 2009 - 03:20 AM

Sorry but no, is there any specific reason you want ZIP compatibility in my DLL. I could use suggestions. :P
  • 1

#21 Revel

Revel

    ɹǝqɯǝɯ ɔɯƃ

  • GMC Member
  • 4935 posts
  • Version:GM8

Posted 21 July 2009 - 04:36 AM

I noticed that in my Archiving DLL, the compression doesn't work very good on Mp3 files, and I'm sure the same problem exists here. I would recommend having one archive for sound and one for images (sprites), so that way you can make the sprites package super compressed while the mp3 files can be not-so compressed.


@Marchal_Mig12 I'm sure their are a lot of other DLL's on this forum for "zip" files. I think he wants to mainly focus on his own archive format before adding other formats.

Edited by Revel, 21 July 2009 - 04:37 AM.

  • 0

#22 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 21 July 2009 - 12:58 PM

Revel, MP3 files are already compressed. This means that it would have less data to compress, meaning less results. This goes for other compressed files, too, they simply are already compressed.
  • 1

#23 Marchal_Mig12

Marchal_Mig12

    The Rhouan

  • GMC Member
  • 1182 posts
  • Version:GM:Studio

Posted 21 July 2009 - 01:03 PM

Sorry but no, is there any specific reason you want ZIP compatibility in my DLL. I could use suggestions.


Well there is no specific reason but it would be nice to not use 2Dlls in a game that does relatively the same thing:)
  • 0

#24 fire_soul453

fire_soul453

    aarons video games

  • GMC Member
  • 542 posts

Posted 21 July 2009 - 10:19 PM

Very nice! Does the old version compress at the highest level?
Also is there a way to extract all files? (ones the game doesn't know exists yet)
Or can it only extract specified files?

Oh and if your going to add another format then don't bother with zip, use 7zip or rar. (If you ever add more, but no need for another format.)
  • 0
Game maker Top sites
I'm sick of paypal. From now on I'm not buying indie games unless I can use something else...

#25 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 22 July 2009 - 01:31 AM

Well there is no specific reason but it would be nice to not use 2Dlls in a game that does relatively the same thing:)

Ok, thanks anyways. :)

Very nice! Does the old version compress at the highest level?
Also is there a way to extract all files? (ones the game doesn't know exists yet)
Or can it only extract specified files?

Oh and if your going to add another format then don't bother with zip, use 7zip or rar. (If you ever add more, but no need for another format.)

The old version used the default compression level, there is no way to extract all files. But you can extract all if all files are in a folder (in the archive), so partially.

And I'm still not planning on adding any other format(s). There's a small chance I may.
  • 1

#26 fire_soul453

fire_soul453

    aarons video games

  • GMC Member
  • 542 posts

Posted 25 July 2009 - 09:35 PM

I like the better compression and extracting file from a folder solves the problem.
  • 0
Game maker Top sites
I'm sick of paypal. From now on I'm not buying indie games unless I can use something else...

#27 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 29 July 2009 - 12:45 AM

extracting file from a folder solves the problem.

Okay. :P I just noticed you can extract all files in the archive by specifying the internal folder (using arch_file_extract_dir) with the internal path set to an empty string. This is because the "folders" are basically filenames with the internal path in front of them. So really, you can use arch_file_extract_dir to extract all files that begin with a string.

Any suggestions? Comments? Requests? :P

Edited by TheMagicNumber, 29 July 2009 - 03:08 AM.

  • 1

#28 fire_soul453

fire_soul453

    aarons video games

  • GMC Member
  • 542 posts

Posted 29 July 2009 - 05:04 PM

When I use a different code to decrypt the archive it still decrypts fine. Can you not encrypt the actual archive?
  • 0
Game maker Top sites
I'm sick of paypal. From now on I'm not buying indie games unless I can use something else...

#29 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 29 July 2009 - 05:47 PM

The encryption is per file, the archive is not encrypted. I just tried decrypting with an invalid key, it did not work. The DLL will not tell you if the key is invalid, but the data will be invalid. Basically the archived files are compressed and or encrypted, the archive will always be in the same format.

I'm pretty sure you misread the description for the encryption functions.
  • 1

#30 fire_soul453

fire_soul453

    aarons video games

  • GMC Member
  • 542 posts

Posted 31 July 2009 - 08:40 PM

Oh ok. I was trying encrypt the actual archive file.
  • 0
Game maker Top sites
I'm sick of paypal. From now on I'm not buying indie games unless I can use something else...

#31 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 02 August 2009 - 01:29 AM

The function names and description can both tell you a lot of what the function does. Just read it. :)
  • 1

#32 jobt

jobt

    GMC Member

  • New Member
  • 307 posts

Posted 02 August 2009 - 10:24 PM

One thing, when I use arch_file_add_dir() with all subdirectories, the subdirectories are gone when the files are extracted again (or maybe I'm doing something wrong). It would be nice if all files kept within their directories when extracted.
When I add a file with an internal directory like 'Directory\file.txt' and I want to extract this file to 'Directory\file.txt' (the Directory folder does not exist yet), an error pops up (can't find the path). I can extract into directories when these directories are already created.
Besides, a very nice system, I will definitely use it if there is some sort of a solution for these problems.
cheers,
jobt

Edited by jobt, 02 August 2009 - 10:25 PM.

  • 0

#33 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 02 August 2009 - 11:13 PM

I'll fix the directory creation soon, it wont be hard. :)
  • 1

#34 jobt

jobt

    GMC Member

  • New Member
  • 307 posts

Posted 03 August 2009 - 07:57 AM

I'll fix the directory creation soon, it wont be hard. :P

Thanks, really looking forward to it, I'm going to store about 120 MB of mediafiles in those archives :)

Edited by jobt, 03 August 2009 - 10:06 AM.

  • 0

#35 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 04 August 2009 - 12:21 AM

Please don't, try to split them up. It'll seriously take a load off the DLL, and you should have fewer issues. Try making them around 40MB each.

Updated the DLL, extracting files will create all directories needed.

Edited by TheMagicNumber, 04 August 2009 - 02:52 AM.

  • 1

#36 jobt

jobt

    GMC Member

  • New Member
  • 307 posts

Posted 04 August 2009 - 08:11 AM

Please don't, try to split them up. It'll seriously take a load off the DLL, and you should have fewer issues. Try making them around 40MB each.

ofcourse, they won't be over 30MB each I think. One question, when I extract for example 1MB from a 30MB archive, will it be as fast (or almost as fast) as extracting 1MB from a 5MB archive? If not, I will have to split them up even more.
Thanks for fixing directory creation =]

Edited by jobt, 04 August 2009 - 08:12 AM.

  • 0

#37 jobt

jobt

    GMC Member

  • New Member
  • 307 posts

Posted 04 August 2009 - 08:40 AM

Arch Error:
EInOutError: Unable to create directory

when using the following code:
[codebox]arch_init()
arch_set_comp(comp_best)
arch_file_add("2.wav","s4.wav", true)
arch_save("Archive.arch")
arch_file_extract("3.wav","s4.wav", true)
arch_close()
arch_free()
game_end()[/codebox]
In fact, I get this with every file I try to compress, every compression quality, even without compression. Did I just make a very stupid mistake or is it a bug?

EDIT: Even with the un-edited example gm6 I get this error, there must me something wrong with the updated dll...

Edited by jobt, 04 August 2009 - 08:54 AM.

  • 0

#38 jobt

jobt

    GMC Member

  • New Member
  • 307 posts

Posted 04 August 2009 - 11:53 AM

Man that was weird. I just tested again (after a reboot) and everything works fine now... I don't have a clue what the problem was :lol:
Any ideas? I don't want this problem to occur when someone is trying to play my game...
  • 0

#39 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 04 August 2009 - 02:35 PM

@jobt: Posted Image

And I should have that bug fixed now.
  • 1

#40 jobt

jobt

    GMC Member

  • New Member
  • 307 posts

Posted 04 August 2009 - 06:17 PM

excuse me, will edit from now on :lol:

a little request:
if I want to take some encrypted files out of an archive, I have to decrypt the files, then extract them, and then encrypt them again. It would be much faster if there was an option to extract those files and then decrypt the extracted files (or decrypt a copy of the file inside the archive, extract the copy, then delete the copy) , so that you do not need to encrypt them again when you're finished extracting.

Edited by jobt, 04 August 2009 - 06:17 PM.

  • 0

#41 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 04 August 2009 - 10:13 PM

Honestly, you shouldn't be encrypting large files. So re-encrypting the little files is practical. Then again, you shouldn't be modifying resource files, use other archives for those little files.
  • 1

#42 jobt

jobt

    GMC Member

  • New Member
  • 307 posts

Posted 05 August 2009 - 09:40 AM

OK, then I'll use encryption for my script files only. But then my resource files are unprotected. Isn't there a way to use password protection for an entire archive? Always less safe than encryption but better than nothing I think?
  • 0

#43 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 05 August 2009 - 11:35 AM

I may add passwords later, but what I really want to do is improve the extracting.
  • 1

#44 jobt

jobt

    GMC Member

  • New Member
  • 307 posts

Posted 12 August 2009 - 04:38 PM

Back from a short holiday, are there any new features/improvements? I'm now at the point I want to integrate this dll with the level editor I made, so if everything works fine and all known bugs are fixed I'd like to know so that I can use it =)
  • 0

#45 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 12 August 2009 - 06:06 PM

Nope, nothing new. I don't think I will add my own features anymore, I will try and complete requests (if you can get me into it).
  • 1

#46 jobt

jobt

    GMC Member

  • New Member
  • 307 posts

Posted 15 August 2009 - 09:53 AM

Well I'd really like to use a simple password protection, because encryption will result in very annoying load times. Extraction is fast enough, at least as fast as all alternatives =)
I hope you have time to make password protection, it would be really useful, otherwise I have to find another method for protecting the files of my project, but I'd really like to use this dll.
  • 0

#47 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 15 August 2009 - 12:18 PM

How exactly would I add passwords, is what I'm thinking.
  • 1

#48 jobt

jobt

    GMC Member

  • New Member
  • 307 posts

Posted 15 August 2009 - 09:28 PM

Well, when saving an archive, you can set a password for that entire archive. Then you need that password when you want to load that archive again. Very simple, but always better than nothing. I've seen methods that use a password for every file inside the archive, but a password for the entire archive is according to me just as effective, because when you can hack one password by decompiling an exe, you can also find passwords for each file. Anyways, I don't want to use a password system to make my files completely safe from hackers/thieves or something, I just want it to be sure not everyone who uses this dll can extract my archives in a very simple way =)

Edited by jobt, 15 August 2009 - 09:29 PM.

  • 0

#49 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 15 August 2009 - 10:11 PM

If you just put the password in the EXE, someone can get the password just by opening the archive in Notepad, or even better, a hex editor.
  • 1

#50 Revel

Revel

    ɹǝqɯǝɯ ɔɯƃ

  • GMC Member
  • 4935 posts
  • Version:GM8

Posted 15 August 2009 - 11:56 PM

If you just put the password in the EXE, someone can get the password just by opening the archive in Notepad, or even better, a hex editor.



Unless you encrypted the password with another (constant) password, or some other randomly generated password, but someone would eventually crack that.
  • 0