Create File.dat in c

[cgm2k7]

olá pessoal bon dia!

Venho aqui mais uma vez pedir ajuda...

Estou criando um projeto, que preciso criar uma criar um arquivo binário com varais imagens dentro desse arquivo,

e no cabeçalho desse arquivo tem que informar o tamanho de o inicio e fim de cada imagem.
Exemplo de como ficaria o arquivo em hex: 89 44 41 54 87 22 00 00 3D 67 00 00 2C 89 00 00
os 1ª 4 caracteres seria o extensão do arquivo ".DAT" os 4 próximos seria o tamanho da imagem, e os 4 próximos seria
o ponteiro de inicio da imagem, e os 4 próximo seria o ponteiro do fim da imagem...
E assim com todas as imagens dentro desse arquivo.DATA
https://ibb.co/ghj1WK
https://www.imagemhost.com.br/image/2jf0B
se alguém poder me ajuda desde já agradeço...

[jonnin]

you can write a binary file like this. You may have to cast things to unsigned char*, I can't remember, I mix c and c++ as I wrote a hybrid for a long time.

but the basic idea is you use fwrite to write raw bytes (unsigned char) to the file, either as an integer (think of it as 4 bytes for a 32 bit int, 8 for 64, ..) or a string or a chunk of image data (array of unsigned char), and so on. Just write in the order you need the data. I highly recommend you make the strings a fixed length.

if you need to read a binary file, eg your image data, use fread, it works pretty much the same way.

 int main()  
 {

FILE * fp;
fp = fopen("x.bin", "wb+");
unsigned char* image[10000000];
char dat[20] = "name.dat";

//fread(image, image_file_name, image_size, 1, file_pointer2);

int size = 100*100;
fwrite(dat,20,1,fp);  //this will have unused junk bytes after the valid string data. 
fwrite(&size,sizeof(int),1,fp);
fwrite(image,10000000,1,fp); //write the bytes to the file. 
fclose(fp);
  return 0;
}

[kbw]

The idea is you fill in a struct, then write it a file in binary mode.

The compiler will normally align fields in the struct on a boundary of 2 or 8 bytes, But you'll need to force 1 byte alignment.
http://www.catb.org/esr/structure-packing/
https://msdn.microsoft.com/en-us/library/2e70t5y1.aspx
https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.cbclx01/pragma_pack.htm

For your example:

#include <stdint.h>

struct hdr {
    char signature[4];  // .DAT
    uint4_t image_size;
    uint4_t image_start;
    uint4_t image_end;
};

You fill in that struct and write it with fwrite() or write(), depending on how you opened the file.

Let me know if you need more ...

[tpb]

Question translated from the Portugese:

I am creating a project in which I need to create a binary file with various images inside that file, The header of that file has to inform the size of the beginning and end of each image.

Example of how the file would look in hex:
89 44 41 54 87 22 00 00 3D 67 00 00 2C 89 00 00

The 1st 4 characters would be the extension of the file ".DAT" the next 4 would be the size of the image, and the 4 next would be the start pointer of the image, and the next 4 would be the end pointer of the image. And so with all the images inside that file.DATA

https://ibb.co/ghj1WK
https://www.imagemhost.com.br/image/2jf0B

Why do you need an "end pointer" if you are storing the "start pointer" and a "size"? Isn't the end just the start plus the size?

It should be noted that you are storing the values in "little endian" order, i.e., least-significant byte first.
BTW, a period '.' is 2E in hex, not 89.

#include <iostream>
#include <iomanip>
#include <fstream>
#include <stdint.h>

int main() {
    uint32_t id=0x5441442E, len=300, start=16, end=316;

    std::ofstream fout("file.dat", std::ios::binary);
    fout.write((char*)&id, 4);
    fout.write((char*)&len, 4);
    fout.write((char*)&start, 4);
    fout.write((char*)&end, 4);
    fout.close();

    std::ifstream fin("file.dat", std::ios::binary);
    fin.read((char*)&id, 4);
    fin.read((char*)&len, 4);
    fin.read((char*)&start, 4);
    fin.read((char*)&end, 4);
    fin.close();

    std::cout << std::hex << std::uppercase << std::setfill('0');
    using std::setw;
    std::cout << setw(8) << id    << '\n'
              << setw(8) << len   << '\n'
              << setw(8) << start << '\n'
              << setw(8) << end   << "\n\n";

    fin.open("file.dat", std::ios::binary);
    char ch;
    while (fin.read(&ch, 1))
        std::cout << setw(2) << (int)(unsigned char)ch << ' ';
    std::cout << '\n';
    fin.close();
}

5441442E
0000012C
00000010
0000013C

2E 44 41 54 2C 01 00 00 10 00 00 00 3C 01 00 00 

[jonnin]

I avoided a struct due to assuming the images are different sizes, making an easy 'write this' approach more difficult (I guess that means forcing a fixed length string is less useful than I said also). I also assumed C meant C.

cgm2k7, did you get what you needed? Is this C or C++?

[cgm2k7]

http://www.mediafire.com/file/u3gb1fx0fxn4rn1/data.rar/file

Many thanks to everyone for the attention ..
I picked up some pieces of my other work and created this code.

but have a two problem I'm not got weight right from where is the error. If anyone can help me...

First: I am not able to put the information of the other images in the header of the file.
2nd: Something is modified the contents of the other images only the first one stays with the original size
[code]
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <stdint.h>
#include <unistd.h>

typedef unsigned char u8;
typedef unsigned short u16;
typedef unsigned int u32;

u32 LE32(u32 b) {
u32 t = 0x12345678;
if (*((unsigned char*)(&t)) == 0x78) {
return b;
} else {
return ((b & 0xff000000) >> 24) |
((b & 0x00ff0000) >> 8 ) |
((b & 0x0000ff00) << 8 ) |
((b & 0x000000ff) << 24);
}
}

unsigned char dat_header[] = {
0x7f, 'D', 'A', 'T', 0x00, 0x00, 0x00, 0x00,
};

void usage();
void create_dat(FILE * dest, const unsigned char * source, u32 size, int three_argc);

int main(int argc, char** argv)
{
u32 fd_size, start = 0, end = 0xffffffff, size = 0xffffffff;
unsigned char * buffer;
FILE * source, * dest;
char * f_source = 0, * f_dest = 0;

f_source = argv[1];
f_dest = argv[2];
int quantity_of_items = atoi(argv[3]); //quantity of items

if (!f_source || !f_dest) {
usage();
printf("Not enough arguments.\n");
return 1;
}

if (!(source = fopen(f_source, "rb"))) {
printf("Error opening %s for reading.\n", f_source);
return 1;
}

fseek(source, 0, SEEK_END);
fd_size = ftell(source);
fseek(source, start, SEEK_SET);

if (fd_size < end)
end = fd_size;

if (end < (size - start))
size = end - start;

buffer = malloc(size);
if (buffer == NULL) {
printf("Failed to allocate memory.\n");
return 1;
}

if (fread(buffer, 1, size, source) != size) {
printf("Failed to read file.\n");
return 1;
}
fclose(source);

if (access(f_dest, F_OK) != 0)
{
if (!(dest = fopen(f_dest, "wb+"))) {
printf("Failed to open/create %s.\n", f_dest);
return 1;
}
}else{
if (!(dest = fopen(f_dest, "a+"))) {
printf("Failed to open/create %s.\n", f_dest);
return 1;
}
}

create_dat(dest, buffer, size, quantity_of_items);

fclose(dest);
free(buffer);

return 0;
}

void create_dat(FILE * dest, const unsigned char * source, u32 size, int three_argc)
{
u32 data_size[4];
int i;

if(three_argc > 0){ //three argument = 1
for (i = 0; i < sizeof(dat_header); i++) {
dat_header[4] = three_argc; //quantity of items
fputc(dat_header[i], dest);
}
}

data_size[0] = LE32(size);
data_size[1] = 0;
data_size[2] = 0;
data_size[3] = 0;
fwrite(data_size, 4, 4, dest);

fwrite(source, 1, size, dest);
}

void usage() {
printf("Usage: create_dat.exe infile outfile.dat\n--------------------------\n");
}
[\code]
I add the images through a .bat file, like this:

create_dat.exe file1.png myfile.dat 5
create_dat.exe file2.png myfile.dat
create_dat.exe file3.png myfile.dat
create_dat.exe file4.png myfile.dat
create_dat.exe file5.png myfile.dat

only the first file has argv [3];

[jonnin]

translate failed, can you say the problem again using simple words, or in your own language?

second issue I understand. The first one, no.

[cgm2k7]

Muito obrigado a todos pela atenção ..
Eu peguei algumas partes do meu outro trabalho e criei este código.

mas tem dois problema não acho onde está o erro.
Se alguém puder me ajudar ...

1ª: Não consigo colocar as informações das outras imagens no cabeçalho do arquivo.dat
2ª: Somente a primeira imagem fica com tamanho original, as outras imagens alguma coisa esta modificando o seu tamanho original.

[jonnin]

I need some time to look at this as I cannot do so much at work. I will look into it in a few hours.

[jonnin]

you have a problem with the args. it is trying to use argv[3] which may not exist. This short version shows what you should do here. Still looking at it.

what is your goal? replacing a changed file is difficult. adding files to the archive is not too bad. To replace changed files, if you had a max image size and used fixed sizes, it would be easier. Otherwise I think we have to rebuild the file entirely every time that happens.

int main(int argc, char** argv)
{ 
u32 fd_size, start = 0, end = 0xffffffff, size = 0xffffffff;
unsigned char * buffer;
FILE * source, * dest;
char * f_source = 0, * f_dest = 0;

if(argc < 3) {
  usage();
   return 0;	
}
f_source = argv[1];
f_dest = argv[2]; 

int quantity_of_items = -1;

if(argc == 4)
 quantity_of_items = atoi(argv[3]); //quantity of items
printf("%i\n", quantity_of_items);
   return 0;
}

with these changes, it is working for me for adding multiple files to the archive (fixed first issue).

[jonnin]

if this is not dictated from on high as to how to do it, what I would do is probably this:

allocate N fixed sized strings in the front of the file to hold the file names and start locations of the actual data.

file1.png 0
file2.png 1000
file3.png 2963

and then drop the data in the back of the file after that reserved space.

if a file is already in the archive when you want to add (really, update) it, you will have to move all the data around and rewrite the file. Probably the most efficient thing to do is delete the file from the archive and add it back in as if appending a new file, shifting the file data only once as a block and updating the start points for all the other items.

another clean way is if you know the max size a file can be, you can just waste a little space and drop in fixed sized records. This lets you swap the old record with the updated one with no problems.

your approach makes it difficult to detect that you have a duplicate file and difficult to fix the file when you do. Its not impossible, its just extra trouble. Is your file format able to be changed?

if you just need results, and this is not a learning program or a task to get it into a specific format for some other program, the unix tar command comes to mind.

[cgm2k7]

muito obrigado irmão pela a atenção.
Realmente meu argv estava errado.
o 2ª problema consegui arruma, o problema era que eu esta abrindo o arquivo de destino erradamente ("a+") mudei para ("ab") resolveu o problema.
1ª problema também resolvi: criei um segundo arquivo chamado "romlist.dat" nele coloquei o tamanho de cada item ficou assim:

int main(int argc, char** argv)
{
u32 fd_size, start = 0, end = 0xffffffff, size = 0xffffffff;
unsigned char * buffer;
FILE * source, * dest;
char * f_source = 0, * f_dest = 0;

if(argc < 3) {
usage();
return 0;
}
f_source = argv[1];
f_dest = argv[2];

int quantity_of_items = -1;

if(argc == 4)
quantity_of_items = atoi(argv[3]); //quantity of items
printf("%i\n", quantity_of_items);

if (!(source = fopen(f_source, "rb"))) {
printf("Error opening %s for reading.\n", f_source);
return 1;
}

fseek(source, 0, SEEK_END);
fd_size = ftell(source);
fseek(source, start, SEEK_SET);

if (fd_size < end)
end = fd_size;

if (end < (size - start))
size = end - start;

buffer = malloc(size);
if (buffer == NULL) {
printf("Failed to allocate memory.\n");
return 1;
}

if (fread(buffer, 1, size, source) != size) {
printf("Failed to read file.\n");
return 1;
}
fclose(source);

if (!(dest = fopen(f_dest, "ab"))) {
printf("Failed to open/create %s.\n", f_dest);
return 1;
}


create_dat(dest, buffer, size, quantity_of_items);

fclose(dest);
free(buffer);
return 0;
}

void create_dat(FILE * dest, const unsigned char * source, u32 size, int three_argc)
{
u32 data_size[4];
int i;

if(three_argc > 0){ //three argument = 1
for (i = 0; i < sizeof(dat_header); i++) {
dat_header[4] = three_argc; //quantity of items
fputc(dat_header[i], dest);
}


data_size[0] = LE32(size);
data_size[1] = 0;
data_size[2] = 0;
data_size[3] = 0;
fwrite(data_size, 4, 2, dest);
}else
{
data_size[0] = LE32(size);
data_size[1] = 0;
data_size[2] = 0;
data_size[3] = 0;
fwrite(data_size, 4, 4, dest);
}
fwrite(source, 1, size, dest);

}

int main(int argc, char** argv){
int i = 1;
int fl_size;
FILE* in, *out;
// char *buffer;

if(!(out = fopen("romlist.dat", "wb")));

fprintf(stdout, "{ ");
while(i < argc)
{

in = fopen(argv[i],"rb");
fseek(in, 0, SEEK_END);
fl_size = ftell(in);
fseek(in, 0, SEEK_SET);
// buffer = malloc(fl_size);
fwrite(&fl_size, 1, 4, out);
printf("%d ,", fl_size);
i++;
}

fprintf(stdout, " }");
fclose(in);
return 0;
}

coloquei em uma array[] o tamanho de cada item..
mas ai tive de mudar a forma de ler o arquivo que era com quantidade de item limitado
agora a leitura ficou assim:

int HeadDATA()
{
FILE * headFile;
int i = 0;
u32 fl_size = 0;
char * buffer, * tmp;

u32 array[] = {116903, 1076, 152334};

if (!(headFile = fopen("mass:\\myfile2.dat", "rb"))){
printf("Error opening for reading.\n");
return 1;
}


for(i = 0; i < 3; i++)
{

fseek(headFile, array[i], SEEK_CUR);
fl_size = ftell(headFile);
fseek(headFile, fl_size-array[i]+8, SEEK_SET);

// printf("fl_size: %d\n", fl_size);

buffer = malloc(fl_size);
fread(buffer, 1, fl_size, headFile);

printf("fl_size: %d\n", fl_size);


graphicsLoadPNG(gstextureStruct[i].texture, buffer, fl_size, 0);
free(buffer);
}

free(buffer);
fclose(headFile);
return 0;
}

Estou tentando criando um jogo de PlayStation 2, não é muito fácil kkkkkkkk.


Agora uma duvida com posso fazer seu precisasse criar um 3ª arquivo contendo nome do item tamanho de inicio e fim de cada item em hex....

OBS:
lembrando que já resolvei meus problema esta pergunta é só para fins de conhecimento, o tópico já pode ser fechado..

valeu a todos pela ajuda, cada dica que vocês deram talvez não tenha servido diretamente mas me ajudou no entendimento.. VALEU

[jonnin]

Of course you can make a third file with whatever you want in it. Sounds a bit like you want an index file or something with just the offsets? That is doable.

[cgm2k7]

Olá o eu aqui de novo... Estou com um probleminha na leitura do meu arquivo.dat.

Como disse acima consegui criar o arquivo .dat do jeito que eu queria, o problema é na leitura, enquanto eu tava lendo poucos arquivos 20 no máximo estava tudo oks depois que precisei ler mais arquivos 73 agora está dando erro de memória cheia OBS: o sistema que estou rodando este aplicativo só tem 2mega bytes "iop do (ps2) só tem 2mb". Veja abaixo meu código.

void DatLoader(){
	FILE *in;
	int b = 0, a = 8, argc = 0, k;
	int f_size, start = 0, end = 0, item_offset =0, item_size = 0;
 	unsigned char *buffer;
 	unsigned long my_size;

	if(!(in = fopen("mass:\\kabal.bin", "rb"))){
		printf("ERRO-0: Loaded file[kabal.bin]\n");
	 goto err_erros1;	
	}
	fseek(in, 4, SEEK_SET);
	fread(&argc, 1, 4, in);

    while(b < argc){
    	
		fseek(in, a, SEEK_SET);
		if(!(fread(&my_size, 1, 4, in)))

		fseek(in, a+4, SEEK_SET);
		fread(&start, 1, 4, in);
		
		fseek(in, a+8, SEEK_SET);
		fread(&end, 1, 4, in);	

		fseek(in, end, SEEK_CUR);
		fseek(in, start, SEEK_SET); 
				
		buffer = (unsigned char *)malloc((unsigned long)my_size);
		if(buffer == NULL){
			fclose(in);
			printf("ERRO: Memory alloc\n");
		 goto err_erros1;	   	
		} 
		if(!fread(buffer, 1, (unsigned long)my_size, in)){
			printf("ERRO: Read file\n");
		 goto err_erros1;	
		}
		 		
		printf("Size:  %d\n", my_size);
		printf("Start: %d\n", start);
		printf("end:   %d\n", end);
		 
		printf("---------------------\n");
	 
	    graphicsLoadPNG(gstextureStruct[b].texture, buffer, (unsigned long)my_size, 0); 
	b++;
	a+=20;	
	free(buffer);	
	}
	printf("Total File(s) %d\n", argc);
err_erros1:
	fclose(in);
	free(buffer);		
}

a função free() não esta devolvendo a memória de volta para o sistema após cada ciclo do loop.
Se alguém poder me dar uma ideia, uma forma de ler para "graphicsLoadPNG" sem ter armazenar na memória ante ou outra fora de liberar a memória após de cada ciclo do loop. Desde já agradeço.

https://ibb.co/i94uf9

[jonnin]

do you know a max size for your image? I would do this..

malloc()…

while
{
…
}


free()


and let the while loop re-use the memory. This will solve your problem and the program will run faster. If this is not possible, we can look deeper.

[cgm2k7]

O tamanho de cada imagem são diferentes, são 73 imagens de tamanho diferentes por isso eu coloquei o malloc dentro do ciclo loop.. Já tentei de varias formas colocar o malloc único fora do loop mas não consigo achar uma forma de fazer isso kkkk.

[cgm2k7]

help-me