Jump to content


Photo

Pak Archive Dll


  • Please log in to reply
60 replies to this topic

#1 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 29 December 2008 - 08:36 PM

A PAK file is a type of archive that was used in Quake games to store data. This DLL allows you to edit those files or even make and use your own in your game. It simply stores data (or files) inside of a single file.

I highly suggest you do not use this even if it works. It's poorly written and not supported.

Download: Host-A
Archive Manager: Host-A

Special thanks to Uuf6429 for pretty much remaking the unit. I fixed some issues and made the DLL. Also re-fixed them. And then I fixed some bugs. He also made most of the archive manager, a long time ago. I just had the time to fix it up for public use (more user friendly, can drop files into archive).

Read before posting questions:
You do NOT need Quake or anything, it just allows you to use them (PAK Files) in your game easily.
You do NOT need Quake PAK files, make your own!
It does NOT encrypt/compress/decompress archives automatically, you must do it yourself.

Edited by TheMagicNumber, 04 September 2011 - 11:06 PM.

  • 2

#2 posva

posva

    GMC Member

  • New Member
  • 610 posts
  • Version:GM8

Posted 29 December 2008 - 09:19 PM

it's a good idea! i didn't try it beacuse it doesn't work on GM7. I recommend you to put the time it use for packing and unpacking, will be great if it use encryption and compression!
  • 0

#3 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 29 December 2008 - 10:24 PM

Thanks! I won't add compression, encryption may be added though. What did you mean here:

I recommend you to put the time it use for packing and unpacking

Like, get the time it took to unpack? Or progress?

Edited by GMREC1111, 29 December 2008 - 10:25 PM.

  • 2

#4 BrokenGlass

BrokenGlass

    GMC Member

  • New Member
  • 524 posts

Posted 29 December 2008 - 10:29 PM

What
  • 0

#5 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 29 December 2008 - 10:31 PM

What

That was pointless...
  • 2

#6 BrokenGlass

BrokenGlass

    GMC Member

  • New Member
  • 524 posts

Posted 29 December 2008 - 10:33 PM

No, i was just wondering what of use this would be.
  • 0

#7 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 29 December 2008 - 10:37 PM

Hehe, multiple uses. Too many to list.

Edited by GMREC1111, 29 December 2008 - 10:37 PM.

  • 2

#8 BrokenGlass

BrokenGlass

    GMC Member

  • New Member
  • 524 posts

Posted 29 December 2008 - 10:37 PM

Looks nice, should add compression too ^.^
  • 0

#9 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 29 December 2008 - 10:47 PM

I got a Huffman unit I could add to the DLL, you would need to call it on the file and it will not be threaded, just adding it on.

Added Huffman compression/decompression functions.

Edited by GMREC1111, 29 December 2008 - 11:14 PM.

  • 2

#10 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 30 December 2008 - 03:23 AM

Update
Added simple encryption, will take suggestions on a more secure way. Currently using XOR with single character keys.

Link in first post.
  • 2

#11 uuf6429

uuf6429

    Covac Software

  • New Member
  • 2522 posts
  • Version:Unknown

Posted 30 December 2008 - 10:36 AM

You can for example pack all of your game files using this into one file and then unpack them when the game starts. Sort of like GM's include files system, but with some added features, such as you can make it unpack a particular file (which is faster the unpacking all of them, as GM does).
Also, considering it's fully compiled, it's faster then GM's file functions.

By the way, on my home pc, I tried it on some example files not noticing one of them was an update 200Mb large. I took only a few seconds to pack/unpack it.

GMREC, an interesting feature you may add (and which is also a bit of a challenge) is to make threaded unpacking with progress. Just an idea though ;)

Edited by uuf6429, 30 December 2008 - 10:41 AM.

  • 0

#12 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 30 December 2008 - 01:28 PM

Maybe, but I'm not too experienced with threading.

You can even use this to contain update packages where they are compressed and you have them installed so hey don't need to download the entire game ;)

Fixed for GM7

Edited by GMREC1111, 30 December 2008 - 09:19 PM.

  • 2

#13 Sindarin

Sindarin

    Indie Game Developer

  • New Member
  • 1644 posts
  • Version:GM:HTML5

Posted 31 December 2008 - 09:53 AM

Actually PAK are compressed files and folders. Most commercial games use that 'format' which is actually a different compression algorithm encrypted. They are nice for storing external resources and organizing them transparently.
  • 0
GM 8.1 / GM:HTML5 User

#14 score_under

score_under

    Least kawaii

  • GMC Member
  • 1321 posts

Posted 31 December 2008 - 03:50 PM

Currently using XOR with single character keys.

If you want multiple character keys, it's not much different,
Single:
chr2=chr^key;

Multiple
if(index>strlen(key))
index=0;
chr2=chr^key[index];
index++;
  • 0

Anti-Decompiler for GM6.1 to GM8.1.91! :GM8_new: [Main skin by Sindarin]
Discontinued.

decimal2.png
^ Signature image because it's been sorta empty since the old host died

If you need to contact me, I still get notification emails from PMs.


#15 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 31 December 2008 - 04:36 PM

Ok, I'll do that later. (Working on something else)

It isn't C++, it is Delphi. "^" is the symbol for a pointer. XOR is "xor" :D

Edited by GMREC1111, 01 January 2009 - 12:15 AM.

  • 2

#16 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 02 January 2009 - 05:03 PM

7z compression, and I'm happy.

Umm, no. Huffman ftw.
  • 2

#17 SonicMX

SonicMX

    GMC Member

  • New Member
  • 30 posts

Posted 02 January 2009 - 05:21 PM

It's a bit more complicated than I thought, (or not really what I wanted?)

When you want to add a file to a pak, you just can add strings (or texts)?
I don't understand the way the DLL does it. if you want to add a file, why you have to enter the data next? Can't the DLL read the whole data from the file and then storage it?
  • 0

#18 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 02 January 2009 - 09:35 PM

I could add a function to add a file from a file. It will be doing the same thing and will just be easier on your side <_<

It will be done soon, multiple character keys for encryption will come with it. (Done, just need release)

Edited by GMREC1111, 07 February 2009 - 01:18 PM.

  • 2

#19 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 05 January 2009 - 08:40 PM

Dam u.

no u

Anybody have requests, comments, or problems with it? (It can't be that useless)
  • 1

#20 MatrixQuare

MatrixQuare

    Matrix Square Entertainment Corp.

  • GMC Member
  • 772 posts
  • Version:Unknown

Posted 07 January 2009 - 08:44 AM

Uh... It can only store plain text contents?

If that's the case, then yes, it is pretty useless.

I tried to enter a file name for the content string, but all it did was simply store the filename as a text that could be outputted back... What's the point of that?
  • 0

#21 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 07 January 2009 - 12:22 PM

All files are plain text, did you not know that? You could load the file into a string but generally people would ask me to add it, I am currently doing that and it's not working out too well. (String turns out blank)
  • 1

#22 uuf6429

uuf6429

    Covac Software

  • New Member
  • 2522 posts
  • Version:Unknown

Posted 07 January 2009 - 06:24 PM

MatrixQuare, you really look like an idiot you know.
pak_init();
// load file data
var i,data,file;
data="";
file=file_bin_open("image.bmp",0);
size=file_bin_size(file);
for(i=0; i<size; i+=1){
 file_bin_seek(file,i);
 data+=chr(file_bin_read_byte(file));
 // you may show percent progress using (i/size*100)
 // then use screen_repaint() to show changes
}
file_bin_close(file);
// add file to pak
pak_file_add("TheImage.bmp",data);
// save pak file
pak_save("test.mopak");
pak_free();

  • 0

#23 MatrixQuare

MatrixQuare

    Matrix Square Entertainment Corp.

  • GMC Member
  • 772 posts
  • Version:Unknown

Posted 07 January 2009 - 10:20 PM

All files are plain text, did you not know that?

Haha? Nice Joke? I don't even need to point out how wrong you are.

@uuf: I'm well aware that you can use GM's binary functions to store/load files on run-time, but it's slow. The whole point of using DLL's to pack things is for SPEED.

I really wonder who's the idiot here :).
  • 0

#24 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 07 January 2009 - 10:28 PM

I'll still say you, you may also be lazy. This came up as a request question previously and you should read a topic before posting or you may start some topic over again. The discussion is quite short, too...

Anyways, I am working on it.

Edited by GMREC1111, 07 January 2009 - 10:40 PM.

  • 1

#25 MatrixQuare

MatrixQuare

    Matrix Square Entertainment Corp.

  • GMC Member
  • 772 posts
  • Version:Unknown

Posted 07 January 2009 - 10:35 PM

I did read the entire thread, there's only 27 posts (28, including this one).

I appreciate that you're answering someone else's question, but just because I don't happen to like the implementation doesn't mean I'm an idiot. At least I'm intelligent enough to realize that using GM's built in binary functions to store/retrieve files on the fly is going to be slow.

And if you mean by "lazy" that I don't like inefficient methods of doing things, then that's your opinion; I couldn't care less.
  • 0

#26 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 07 January 2009 - 10:44 PM

It's a bit more complicated than I thought, (or not really what I wanted?)

When you want to add a file to a pak, you just can add strings (or texts)?
I don't understand the way the DLL does it. if you want to add a file, why you have to enter the data next? Can't the DLL read the whole data from the file and then storage it?

I could add a function to add a file from a file. It will be doing the same thing and will just be easier on your side :)

So you did read that?

I also do know GM's binary file functions are slow, they weren't added very well and can only read a single byte at a time.
  • 1

#27 MatrixQuare

MatrixQuare

    Matrix Square Entertainment Corp.

  • GMC Member
  • 772 posts
  • Version:Unknown

Posted 07 January 2009 - 11:18 PM

Yes, I did read that, and I also read you're currently trying to implement direct file packing/unpacking.

Good luck, then :).
  • 0

#28 uuf6429

uuf6429

    Covac Software

  • New Member
  • 2522 posts
  • Version:Unknown

Posted 09 January 2009 - 01:35 PM

Uh... It can only store plain text contents?

If that's the case, then yes, it is pretty useless.

I tried to enter a file name for the content string, but all it did was simply store the filename as a text that could be outputted back... What's the point of that?

I replied to that statement, which does make you look like an idiot.

@uuf: I'm well aware that you can use GM's binary functions to store/load files on run-time, but it's slow. The whole point of using DLL's to pack things is for SPEED.

You're contradicting yourself. If you don't do something because of speed it doesn't mean it's impossible.
What you are saying is completely stupid because, rather then using GM's functions, you could save binary data from a different source, such as from an online/sockets dll transmiting a file or something.

How would you do it otherwise? You'd have to save to a file and then use this.

Now, I really wonder who is.
  • 0

#29 score_under

score_under

    Least kawaii

  • GMC Member
  • 1321 posts

Posted 09 January 2009 - 06:12 PM

All files are plain text, did you not know that?

Haha? Nice Joke? I don't even need to point out how wrong you are.

@uuf: I'm well aware that you can use GM's binary functions to store/load files on run-time, but it's slow. The whole point of using DLL's to pack things is for SPEED.

I really wonder who's the idiot here :lol:.

You.

I know the term "Plain Text" is a very bad one to use in this case. I also notice the lack of any evidence, hand-waved by "I don't even need to point out how wrong you are", on your part.

Define plain text. How far into the character set can you go until it is not "plain text"? Are we talking ASCII, EBCDIC, Unicode, UTF-8, Shift-JIS?

Edited by score_under, 09 January 2009 - 06:18 PM.

  • 0

Anti-Decompiler for GM6.1 to GM8.1.91! :GM8_new: [Main skin by Sindarin]
Discontinued.

decimal2.png
^ Signature image because it's been sorta empty since the old host died

If you need to contact me, I still get notification emails from PMs.


#30 MatrixQuare

MatrixQuare

    Matrix Square Entertainment Corp.

  • GMC Member
  • 772 posts
  • Version:Unknown

Posted 13 January 2009 - 12:19 AM

You're contradicting yourself. If you don't do something because of speed it doesn't mean it's impossible.

And you're failing at elementary reading--show me where I said it's impossible to do anything.

So someone here is an idiot and can't read... Of course, I'm not saying who--wouldn't wanna get wacked by the mods :( (Sarcasm there).

I know the term "Plain Text" is a very bad one to use in this case.

That's a fair argument--however, it's generally accepted that "Plain Text" refers to ASCII and/or EBCDIC, anything else is no longer "plain" since it requires Encoding; on top of that, GM doesn't even support anything beyond ASCII-confined plain-text so it should be a simple given in this context.
  • 0

#31 death-droid

death-droid

    GMC Member

  • GMC Member
  • 2600 posts

Posted 13 January 2009 - 01:36 PM

:) sucks it doesn't work in GM7
  • 0

Posted Image


#32 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 13 January 2009 - 01:55 PM

Is it the person who is posting off topic, speaking to one person in a topic without PMing...

If you post here it better be directly related to the DLL and not some fight over who's an idiot and all this.

:) sucks it doesn't work in GM7

Haha, it does...you just need to use the proper scripts. I forget if I ever did upload so...I'll check. I will reupload the new version with the updated encryption soon too.

Updated Encryption and the add from file function is in it but not ready, don't use it.

Edited by GMREC1111, 13 January 2009 - 02:41 PM.

  • 1

#33 uuf6429

uuf6429

    Covac Software

  • New Member
  • 2522 posts
  • Version:Unknown

Posted 14 January 2009 - 05:54 PM

And you're failing at elementary reading--show me where I said it's impossible to do anything.

Uh... It can only store plain text contents?


That's a fair argument--however, it's generally accepted that "Plain Text" refers to ASCII and/or EBCDIC, anything else is no longer "plain" since it requires Encoding; on top of that, GM doesn't even support anything beyond ASCII-confined plain-text so it should be a simple given in this context.

You can effectively read anything beyond ASCII as "plain text". Of course, your app has to do the "encoding" itself. But, a a byte is always a byte. It only depends on it's use.
You can if you want read utf8 (for example) with GM, and show a character you want by parsing it. Of course, at this point it looks nothing less like "plain text".

You're the idiot here. If something supports the basics (ASCII) it can effectively also support anything else including jis or utf.

Point is, you can read a file or pass file data even in non-ascii and it is stored and retrieveable in the same way, without changing anything. Problems occur when you try read the file.

Of course, if you haven't got into this yourself, this dll is for storing files not to en/decode non-ansi ones. This is could be a potential feature of the dll, but it isn't by any means the core feature.

Edited by uuf6429, 14 January 2009 - 05:58 PM.

  • 0

#34 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 14 January 2009 - 09:58 PM

Uuf6429, I am having the stupidest problem with loading a files' contents into a string...I will send the code soon.
  • 1

#35 uuf6429

uuf6429

    Covac Software

  • New Member
  • 2522 posts
  • Version:Unknown

Posted 15 January 2009 - 02:49 PM

function ReadAll(FileName:String):String;					 // read file into a string

begin

  if FileExists(FileName) then begin

	with TFileStream.Create(FileName,fmOpenRead+fmShareDenyNone) do

	begin

	  try

		SetLength(Result,Size);

		Read(Pointer(Result)^,Size);

	  except

		Result:='';

		Free;

		raise;

	  end;

	  Free;

	end;

  end else begin

	Result:='';

  end;

end;
Easy as pie! ;)
  • 0

#36 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 15 January 2009 - 11:15 PM

Yumm! Thanks, I should have used TFileStream too.

Added, working on saving to file as well.
  • 1

#37 Schalk

Schalk

    GMC Member

  • GMC Member
  • 487 posts
  • Version:GM:HTML5

Posted 15 January 2009 - 11:28 PM

7z compression, and I'm happy.

Umm, no. Huffman ftw.


Huffman is what is used in some zip compressions isn't it? I read a bit about it and how it works by using character frequency. Would this be hard to implement though?
  • 0

Game Maker HTML5 Extensions
Mouse Extension | HTML Extension


#38 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 15 January 2009 - 11:38 PM

Huffman is used for RAR compression, I don't know about ZIP. I do know that Huffman on it's own is slow and the reason WinRAR isn't open source is cause they got an epic implementation of it. Fastest that I know of.

I may be wrong, I'm no expert.
  • 1

#39 Schalk

Schalk

    GMC Member

  • GMC Member
  • 487 posts
  • Version:GM:HTML5

Posted 16 January 2009 - 12:18 AM

Huffman coding today is often used as a "back-end" to some other compression method. DEFLATE (PKZIP's algorithm) and multimedia codecs such as JPEG and MP3 have a front-end model and quantization followed by Huffman coding.

From the reliable Wikipedia
  • 0

Game Maker HTML5 Extensions
Mouse Extension | HTML Extension


#40 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 30 January 2009 - 01:36 AM

Update:
- Saving to Files is Complete!
- Loading from Files is Complete!

Please notify me of any crashes (i.e. "An unexpected error has occurred when running the game") or bugs such as files not writing fully/encryption issue.
  • 1

#41 ShingingDB

ShingingDB

    GMC Member

  • New Member
  • 27 posts

Posted 19 April 2009 - 07:07 PM

Wait, so how would you use this? Cause I've tried to customly add new files into the .pak but it always seem to just overwrite everything I had in it before >.>
  • 0

#42 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 19 April 2009 - 07:31 PM

To add a file to it you load the PAK then add to it, don't execute AddFile and then save without loading the old PAK.

Warning: This seems to crash on binary files, I have not found any way to fix this.

Edited by GMREC1111, 10 May 2009 - 11:32 AM.

  • 1

#43 ShingingDB

ShingingDB

    GMC Member

  • New Member
  • 27 posts

Posted 19 April 2009 - 07:52 PM

To add a file to it you load the PAK then add to it, don't execute AddFile and then save without loading the old PAK.

Warning: This seems to crash on binary files, I have not found any way to fix this.


Alright I think I got it working. Thanks :D But how can I use GM to read files off the .pak file?
  • 0

#44 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 19 April 2009 - 09:06 PM

Using my DLL, you can do it. Look at the scripts, I am sure I commented them all (like I always do), if not, the name should help.
  • 1

#45 ShingingDB

ShingingDB

    GMC Member

  • New Member
  • 27 posts

Posted 19 April 2009 - 09:48 PM

Oh alright thanks! :]

Edit: Oh what's the difference between "pak_file_add", "pak_file_add_from_file", and "pak_file_save_to_file"?

Edited by ShingingDB, 19 April 2009 - 09:54 PM.

  • 0

#46 uuf6429

uuf6429

    Covac Software

  • New Member
  • 2522 posts
  • Version:Unknown

Posted 19 April 2009 - 10:37 PM

pak_file_add - Add file to the current pak file (in memory). The added file's data is passed as a string.
pak_file_add_from_file - Add file to the current pak file (in memory). The added file's data is read from a real file.
pak_file_save_to_file - Save the data of a file in a pak file (in memory) to a file on the disk.

Regards,
Chris.
  • 0

#47 Mister_M

Mister_M

    GMC Member

  • New Member
  • 499 posts

Posted 22 April 2009 - 11:10 AM

Put more detail on how to use each and every function. The comments you put are understandable only to yourself...
Im working on a pak file - I want to add a file to the pak file. Why do I need to put two arguments?

Oh, and by the way, the example you provided wont work.

EDIT: It fails to load any pak file.
This is what I tried:
pak_init(true);
   show_message("Initiated");
   pak_load(working_directory + "\AFs.pak");
   show_message("Loaded AFs.pak");
   pak_file_save_to_file(working_directory + "\Files.txt", "Files.txt");
   show_message("Saved Files.txt");
   file = file_text_open_read("Files.txt");
   while (!file_text_eof(file))
   {

	  filename = file_text_read_string(file);
	  show_message("File found: " + filename);
	  file_text_readln(file);
	  pak_file_save_to_file(working_directory + "\" + filename, filename);
	  show_message("Saved " + filename);
   }
   pak_reset();
   show_message("
Reset complete");
   pak_free();
   show_message("Free complete");
   game_end();
The pak file exists. I get a message "Initiated" and I dont get the second message after the loading.

Edited by Mister_M, 22 April 2009 - 11:26 AM.

  • 0

#48 uuf6429

uuf6429

    Covac Software

  • New Member
  • 2522 posts
  • Version:Unknown

Posted 22 April 2009 - 12:35 PM

I think you switched the arguments for:
pak_file_save_to_file
  • 0

#49 TheMagicNumber

TheMagicNumber

    GMC Member

  • GMC Member
  • 5247 posts
  • Version:Unknown

Posted 22 April 2009 - 07:55 PM

Actually, uuf6429, I think I used the wrong PAK unit. I am sure this is the one you created and I never edited, the one where I fixed that infinite loop. I will need to either find the newer one or redo the changes.

The problem should only occur when you have an "empty" PAK, meaning it has no files in it. Correction, it crashes on an empty file.

Edited by GMREC1111, 22 April 2009 - 08:08 PM.

  • 2

#50 IceMetalPunk

IceMetalPunk

    InfiniteIMPerfection

  • GMC Elder
  • 9603 posts
  • Version:GM:Studio

Posted 24 April 2009 - 05:38 AM

I'm having an issue with this.

Everything works fine, but when I try to unpack a .BMP, nothing gets created. The file in the .PAK is called "8.bmp". I checked pak_file_index("8.bmp") and it returns 0, so the file is there. However, after calling pak_file_save_to_file(working_directory+"/8.bmp","8.bmp"), no file is created. I even tried swapping the arguments, but nothing works. I'm not getting any errors, either.

Any ideas?

-IMP :P ;)
  • 0

:GM123: Are you an artist who likes creating original creature designs? Maybe you can help out with Elementa here! Give it a look; you might like the idea :)

:bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny: :excl: :bunny: