Jump to content


Photo

Problem with file_text_read_string?


  • Please log in to reply
10 replies to this topic

#1 RPSR1994

RPSR1994

    GMC Member

  • GMC Member
  • 330 posts
  • Version:GM8

Posted 04 March 2012 - 05:02 PM

Hello,

I wanted to transport information from a file to another and so I opened the file, read some strings, and saved them in the other file.

However there was a problem.

I want to know if there is a maximum size for the string read or if there are characters it doesn't read.

The situation was:
I opened a PNG file, read the strings, and wrote them in another file, but there were characters that weren't written.
In normal .txt files it worked.
Is it because I'm using the file_text_* functions? If so, are there any functions that do this?

Is there a DLL or something similar that allows me to read anything, from any type of file?

BTW, when I write file_open_read() instead of file_text_open_read() function, GM also highlights it, but the function doesn't appear in the help file. Why so?

Thanks.

Edited by RPSR1994, 04 March 2012 - 05:03 PM.

  • 0

#2 FiLkAtA

FiLkAtA

    GMC Member

  • GMC Member
  • 511 posts
  • Version:GM:Studio

Posted 04 March 2012 - 05:20 PM

Binary files might be what you're looking for. From the manual:


In rare situations you might need to read data from binary files. The following low-level routines exist for this:



file_bin_open(fname,mod) Opens the file with the indicated name. The mode indicates what can be done with the file: 0 = reading, 1 = writing, 2 = both reading and writing). When the file does not exist it is created. The function returns the id of the file that must be used in the other functions. You can open multiple files at the same time (32 max). Don't forget to close them once you are finished with them.
file_bin_rewrite(fileid) Rewrites the file with the given file id, that is, clears it and starts writing at the start.
file_bin_close(fileid) Closes the file with the given file id.
file_bin_size(fileid) Returns the size (in bytes) of the file with the given file id.
file_bin_position(fileid) Returns the current position (in bytes; 0 is the first position) of the file with the given file id.
file_bin_seek(fileid,pos) Moves the current position of the file to the indicated position. To append to a file move the position to the size of the file before writing.
file_bin_write_byte(fileid,byte) Writes a byte of data to the file with the given file id.
file_bin_read_byte(fileid) Reads a byte of data from the file and returns this.


  • 1

#3 RPSR1994

RPSR1994

    GMC Member

  • GMC Member
  • 330 posts
  • Version:GM8

Posted 04 March 2012 - 05:26 PM

Binary files might be what you're looking for. From the manual:


In rare situations you might need to read data from binary files. The following low-level routines exist for this:



file_bin_open(fname,mod) Opens the file with the indicated name. The mode indicates what can be done with the file: 0 = reading, 1 = writing, 2 = both reading and writing). When the file does not exist it is created. The function returns the id of the file that must be used in the other functions. You can open multiple files at the same time (32 max). Don't forget to close them once you are finished with them.
file_bin_rewrite(fileid) Rewrites the file with the given file id, that is, clears it and starts writing at the start.
file_bin_close(fileid) Closes the file with the given file id.
file_bin_size(fileid) Returns the size (in bytes) of the file with the given file id.
file_bin_position(fileid) Returns the current position (in bytes; 0 is the first position) of the file with the given file id.
file_bin_seek(fileid,pos) Moves the current position of the file to the indicated position. To append to a file move the position to the size of the file before writing.
file_bin_write_byte(fileid,byte) Writes a byte of data to the file with the given file id.
file_bin_read_byte(fileid) Reads a byte of data from the file and returns this.


Using those functions, how would I read from files like:
  • .png
  • .doc
  • .txt
  • .ppt
  • .jpg
  • .gif
  • other common windows files?

  • 0

#4 IceMetalPunk

IceMetalPunk

    InfiniteIMPerfection

  • Retired Staff
  • 9314 posts
  • Version:Unknown

Posted 05 March 2012 - 01:47 AM

When you use the binary functions, the format of the file doesn't matter at all. Whatever the file is, you read it the same way: byte by byte.

So basically, you'd open it up, then put some file_bin_read_byte() calls inside a while() loop that ends when you've reached the end of the file.

file = file_bin_open("filename_goes_here.ext", 0); /* 0=read, 1=overwrite, 2=read/write (doesn't clear file automatically) */

/* Loop until we've reached the end of the file */
while (file_bin_position(file) < file_bin_size(file)) {
  byte = file_bin_read_byte(file);
  /* Now you have the next byte value, from 0 to 255. Do whatever you want with it. */
}

file_bin_close(file);

What you do with those byte values is up to you. If you're trying to get any real information out of them, you'll need to read up on the file format for whatever file type you're reading.

-IMP
  • 1

#5 RPSR1994

RPSR1994

    GMC Member

  • GMC Member
  • 330 posts
  • Version:GM8

Posted 05 March 2012 - 02:50 PM

When you use the binary functions, the format of the file doesn't matter at all. Whatever the file is, you read it the same way: byte by byte.

So basically, you'd open it up, then put some file_bin_read_byte() calls inside a while() loop that ends when you've reached the end of the file.

file = file_bin_open("filename_goes_here.ext", 0); /* 0=read, 1=overwrite, 2=read/write (doesn't clear file automatically) */

/* Loop until we've reached the end of the file */
while (file_bin_position(file) < file_bin_size(file)) {
  byte = file_bin_read_byte(file);
  /* Now you have the next byte value, from 0 to 255. Do whatever you want with it. */
}

file_bin_close(file);

What you do with those byte values is up to you. If you're trying to get any real information out of them, you'll need to read up on the file format for whatever file type you're reading.

-IMP


I've applied that while() principle with a for() loop and it worked. It reads the file exactly how it is. However, what I'm doing is moving the information from one file to another, and on a 2,1 MB file I miraclously had a 100kb/s copying velocity (never done it again however), but in a 68 MB file I only had a 20kb/s velocity (O.O it would never end...)...

Are there any functions (DLL or not) that act like the file_bin_* functions but way faster? Thanks
  • 0

#6 IceMetalPunk

IceMetalPunk

    InfiniteIMPerfection

  • Retired Staff
  • 9314 posts
  • Version:Unknown

Posted 05 March 2012 - 07:49 PM

Yeah, GM is not very fast with its file I/O. I made a file-reading DLL once that reads an entire file into memory at once (it read a 2.5MB file in less than a second; I didn't test more than that), but it only worked well with text-based files and didn't have any write capability.

One question: are you doing anything with the information besides just copying it? If not, why not use the file_copy(file1, file2) function that GM already has?

-IMP
  • 0

#7 icuurd12b42

icuurd12b42

    Self Formed Sentient

  • Global Moderators
  • 14446 posts
  • Version:GM:Studio

Posted 05 March 2012 - 09:00 PM

GMBinaryfile in my tools for binary functions
  • 1

#8 RPSR1994

RPSR1994

    GMC Member

  • GMC Member
  • 330 posts
  • Version:GM8

Posted 06 March 2012 - 02:57 PM

Yeah, GM is not very fast with its file I/O. I made a file-reading DLL once that reads an entire file into memory at once (it read a 2.5MB file in less than a second; I didn't test more than that), but it only worked well with text-based files and didn't have any write capability.

One question: are you doing anything with the information besides just copying it? If not, why not use the file_copy(file1, file2) function that GM already has?

-IMP

I don't use file_copy() because I'm not merely copying it, I'm inserting the information into another file

GMBinaryfile in my tools for binary functions

I've tried it and it indeed is faster! Thanks :biggrin:

Btw, is there any way of calculating what is the best copying speed for the computer executing the program? I don't want to "overheat" any computer, making the program execute slower and hence actually copying slower, just because in my computer it was ok.
  • 0

#9 icuurd12b42

icuurd12b42

    Self Formed Sentient

  • Global Moderators
  • 14446 posts
  • Version:GM:Studio

Posted 06 March 2012 - 08:33 PM

Btw, is there any way of calculating what is the best copying speed for the computer executing the program? I don't want to "overheat" any computer, making the program execute slower and hence actually copying slower, just because in my computer it was ok.


The best way would be the using the file system. the copy command (a batch file calling it if not possible directly) which does have the ability to append files together if that is what you want to do. The process will run in it's own thread.

If you want to use code to copy, then you would want to do the copying in chunks large enough to improve drive speed and small enough not to disturb memory.

AKA, you dont want to do it byte by byte in GM


So what is the propose of this feature?
  • 0

#10 RPSR1994

RPSR1994

    GMC Member

  • GMC Member
  • 330 posts
  • Version:GM8

Posted 09 March 2012 - 11:39 AM

The best way would be the using the file system. the copy command (a batch file calling it if not possible directly) which does have the ability to append files together if that is what you want to do. The process will run in it's own thread.

If you want to use code to copy, then you would want to do the copying in chunks large enough to improve drive speed and small enough not to disturb memory.

AKA, you dont want to do it byte by byte in GM


So what is the propose of this feature?

I don't know how to use batch files :/

How large should the chunk of code be then? Just to be safe I have the code to read one byte and I repeat it 100 or 1000 times each step....Of course that it was written just as a temporary one since it is very slow.

The purpose is a kind of a file vault application.
  • 0

#11 icuurd12b42

icuurd12b42

    Self Formed Sentient

  • Global Moderators
  • 14446 posts
  • Version:GM:Studio

Posted 09 March 2012 - 08:56 PM

For a file vault application, GMBinaryFile has a file insertion system

Add file to fault
hf = GMBINOpenFileWrite(filename);
GMBINInsertFile(hf, "image1.png");
GMBINInsertFile(hf, "image2.png");
GMBINInsertFile(hf, "image3.png");
GMBINCloseFile(hf);

extract file from vault
hf = GMBINOpenFileRead(filename);
GMBINExtractFile(hf, "image1.png");
GMBINExtractFile(hf, "image2.png");
GMBINExtractFile(hf, "image3.png");
GMBINCloseFile(hf);

of course you would want to make it smart enough to know what the file name was. you can mix the insertion with all the extra information you need
hf = GMBINOpenFileWrite(destfilename);
GMBINWriteInt(hf,numberoffiles)
i = 0;
repeat(numberoffiles)
{
    GMBINWriteString(hf,filename[i]);
    GMBINInsertFile(hf, filename[i]);
    i+=1;
}
GMBINCloseFile(hf);

hf = GMBINOpenFileRead(filename);
numberoffiles = GMBINReadInt(hf)
i = 0;
repeat(numberoffiles)
{
    filename[i] = GMBINReadString(hf);
    GMBINExtractFile(hf, filename[i]);
   i+=1;
}
GMBINCloseFile(hf);

where filename[] is the array of files and numberoffiles is the size of the array.

as for the chunk method, though the file insert trumps that, it would be

hdest = GMBINOpenFileWrite(destfilename);
for each file to insert, set sourcefilename
{
    hsrc = GMBINOpenFileWrite(sourcefilename);
    while(!GMBINIsEOF(hsrc))
    {
        data = GMBINReadBuffer(hsrc,SIZE);
        GMBINWriteBuffer(hdest,data,SIZE);
    }
    GMBINCloseFile(hsrc);
}
GMBINCloseFile(hdest);

where SIZE is 1024 (1k), 2048(2k), 4096(4k) and so on. you would have to test which size is optimal. I did omit the end of the file where the size of the read would be less that the size left. You would need to track what you read vs what is left and adjust the size for the last read/write
  • 1




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users