Welcome to Data Crystal's new home! Data Crystal is now part of the TCRF family (sort of).
The wiki has recently moved; please report any issues in Discord. Pardon the dust.

Strike Witches: Silver Wing/Notes: Difference between revisions

From Data Crystal
Jump to navigation Jump to search
(Fix for quickbms reinsert2 bloating archives. Testing on if this fixes freezing from reinsert2 TBD)
(Updated research on reinsert2 failing. Update research on GGXArchiver format)
 
Line 12: Line 12:
| Magic || "GGXArchiver1.00" + 0x00
| Magic || "GGXArchiver1.00" + 0x00
|-
|-
| Unknown (number of files?  more info?) || 16-bytes, ex: '''0x02 0x00 0x00 0x00 0x02 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00''' (/media/data/sc_logo.bin)
| Archive Header || 16-bytes, ex: '''0x02 0x00 0x00 0x00 0x02 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00''' (/media/data/sc_logo.bin)
First four bytes are number of entries in the archive's file table<sup>(LE)</sup>
First four bytes are number of entries in the archive's file table<sup>(LE)</sup>


Line 21: Line 21:
| File List || 32-bytes per file, relative path to file, directories supported
| File List || 32-bytes per file, relative path to file, directories supported
|-
|-
| File Headers Table || Size seems to scale with number of files in archive
| File Metadata Table || Size seems to scale with number of files in archive
0x00-0x03 NID<sup>(LE)</sup>
0x00-0x03 File ID<sup>(LE)</sup>


0x04-0x07 static 1<sup>(LE)</sup>
0x04-0x07 static 1<sup>(LE)</sup>
Line 28: Line 28:
0x08-0x0B uncompressed file size<sup>(LE)</sup>
0x08-0x0B uncompressed file size<sup>(LE)</sup>


0x0C-0x0F Size of compressed file data in archive, probably includes compression commands<sup>(LE)</sup>
0x0C-0x0F Size of compressed file data in archive<sup>(LE)</sup>


0x10-0x13 Boolean? (could also represent compression type), is data compressed<sup>(LE)</sup>
0x10-0x13 Boolean? (could also represent compression type), is data compressed<sup>(LE)</sup>.  Does not seem to support uncompressed data?


0x14-0x17 Offset from end of file headers table to start of file<sup>(LE)</sup>
0x14-0x17 Offset from end of file metadata table to start of file<sup>(LE)</sup>


Repeated for each file in archive
Repeated for each file in archive
|-
|-
| Compression related data || See below
| File Data || The actual data for each of the archived files
|-
| File Data || The actual data for each of the archived files.  Appears to be have some endian weirdness, reversed every 32-bits?  (Xenon is 32-bit BE, makes sense?)
|}
|}


===Compression Information===
====Compression Information====


Appears to be compressed with BPE ([https://en.wikipedia.org/wiki/Byte_pair_encoding Byte-Pair Encoding]) compression (quickbms compression method 25/55).  quickbms sometimes fails decompression, may require new utility or manual decompression.
Appears to be compressed with BPE ([https://en.wikipedia.org/wiki/Byte_pair_encoding Byte-Pair Encoding]) compression (quickbms compression method 25/55).  quickbms sometimes fails decompression, may require new utility or manual decompression.


quickbms's reinsert2 is bugged, and will overwrite large swathes (if not the entire original file data) of the file data with zeroes, and then point GGXArchiver down to the new data.  While this sometimes works, it could be the cause of freezing and bloats files regardless.  Right now the solution is to manually hex edit the files to remove the zero padding and fix the offset pointers for impacted files.  This has been successfully tested in Xenia on sc_title.bin.
quickbms's reinsert2 is bugged, and will not adjust archives appropriately if new zsize < old zsize.  Archives with files that are smaller than originally will cause the game to softlock when unpacking them.  The relatively simple fix is to ensure all repacked files are larger than their originals.  reinsert2 will then overwrite the entire original file with zeroes and adjust offsets appropriately.  This will create a small increase in loading times unless the files are manually (TBD on utility to automate this) stripped of padding.
 
====Strategies to avoid unpack softlock====
 
* '''Enabling mipmaps'''
** Pros
*** Simple
*** Significant file size increase will almost certainly outweigh improved packing
** Cons
*** Only works on textures, and only those with mipmaps disabled originally
*** Creates unnecessary bloat
*** Significant VRAM usage increase
 
 
* '''Packer-unfriendly data'''
** Pros
*** Works on all file types
*** Can make files exactly the desired size to avoid softlock and reduce bloat/load times/VRAM usage
** Cons
*** Time consuming to do manually, unreliable via utility
 
 
* '''Custom utility'''
** Pros
*** Solves all issues elegantly
** Cons
*** High up-front work to write and test utility




{{Internal Data}}
{{Internal Data}}

Latest revision as of 22:07, 20 March 2024

File Formats

GGXArchive Packed Files

Most of the game's files appear to be archives created with "GGXArchiver v1.00". These can contain various types of files, including but not limited to models, textures and text. Some entries are stored as little endian

GGXArchiver Format
Entry Format
Magic "GGXArchiver1.00" + 0x00
Archive Header 16-bytes, ex: 0x02 0x00 0x00 0x00 0x02 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 (/media/data/sc_logo.bin)

First four bytes are number of entries in the archive's file table(LE)

Second four bytes are number of "valid" entries in the file table (blank entries of 32 0x00 are "invalid")(LE)

Remaining 8 bytes are probably padding

File List 32-bytes per file, relative path to file, directories supported
File Metadata Table Size seems to scale with number of files in archive

0x00-0x03 File ID(LE)

0x04-0x07 static 1(LE)

0x08-0x0B uncompressed file size(LE)

0x0C-0x0F Size of compressed file data in archive(LE)

0x10-0x13 Boolean? (could also represent compression type), is data compressed(LE). Does not seem to support uncompressed data?

0x14-0x17 Offset from end of file metadata table to start of file(LE)

Repeated for each file in archive

File Data The actual data for each of the archived files

Compression Information

Appears to be compressed with BPE (Byte-Pair Encoding) compression (quickbms compression method 25/55). quickbms sometimes fails decompression, may require new utility or manual decompression.

quickbms's reinsert2 is bugged, and will not adjust archives appropriately if new zsize < old zsize. Archives with files that are smaller than originally will cause the game to softlock when unpacking them. The relatively simple fix is to ensure all repacked files are larger than their originals. reinsert2 will then overwrite the entire original file with zeroes and adjust offsets appropriately. This will create a small increase in loading times unless the files are manually (TBD on utility to automate this) stripped of padding.

Strategies to avoid unpack softlock

  • Enabling mipmaps
    • Pros
      • Simple
      • Significant file size increase will almost certainly outweigh improved packing
    • Cons
      • Only works on textures, and only those with mipmaps disabled originally
      • Creates unnecessary bloat
      • Significant VRAM usage increase


  • Packer-unfriendly data
    • Pros
      • Works on all file types
      • Can make files exactly the desired size to avoid softlock and reduce bloat/load times/VRAM usage
    • Cons
      • Time consuming to do manually, unreliable via utility


  • Custom utility
    • Pros
      • Solves all issues elegantly
    • Cons
      • High up-front work to write and test utility


Internal Data for Strike Witches: Silver Wing
Retrieved from "https://datacrystal.tcrf.net/w/index.php?title=Strike_Witches:_Silver_Wing/Notes&oldid=62180"