345 lines
19 KiB
Markdown
345 lines
19 KiB
Markdown
# Firefly Static Parser Architectural Specification & Cryptographic Deep Dive
|
|
|
|
This technical specification provides an exhaustive, code-level analysis of **`Firefly-Static-Parser`** (by AzenKain), an offline, static reverse-engineering framework and IL2CPP metadata extractor implemented in Rust. Unlike in-process dynamic dumpers (such as `honkai-dumper`), `Firefly-Static-Parser` operates **completely offline** without launching, injecting, or hooking into the game client process.
|
|
|
|
It achieves this by completely reverse-engineering the proprietary metadata container format (`MHY\0`), the embedded layout tables in `GameAssembly.dll`, dynamic semantic XOR key derivation, and non-linear mathematical scrambling algorithms used in *Honkai: Star Rail*.
|
|
|
|
---
|
|
|
|
## 1. Executive Summary & Architectural Paradigm
|
|
|
|
### 1.1 The Challenge of Custom IL2CPP Obfuscation
|
|
Standard Unity IL2CPP uses known, public data structures:
|
|
- `global-metadata.dat` begins with magic `0xFAB11BAF` (version 29+).
|
|
- Metadata definitions (TypeDef, MethodDef, FieldDef) are stored sequentially in unencrypted flat tables.
|
|
|
|
In *Honkai: Star Rail* and similar protected titles:
|
|
1. **Container Replaced**: `global-metadata.dat` begins with custom magic `MHY\0` (`0x0059484D`).
|
|
2. **Split Metadata**: Metadata ranges and image mappings are split between `global-metadata.dat` and `startup-metadata.dat`.
|
|
3. **Embedded PE Headers**: The true table offsets, sizes, and field layout descriptors are embedded inside `GameAssembly.dll` and encrypted with arithmetic displacements and bitwise XOR masks.
|
|
4. **Scrambled Field/Method Entries**: Entries are not sequential; attributes, indices, and names are encrypted with non-linear hash/PRNG functions unique to each table.
|
|
5. **Seeded Stream Cipher on Strings**: Type names and string literals are packed into bitfields and encrypted with a QWORD running stream cipher.
|
|
|
|
### 1.2 The Two Dumping Paradigms
|
|
|
|
```
|
|
+----------------------------------------------------------------------------------------------------+
|
|
| IN-PROCESS vs. OFFLINE PARSER |
|
|
+----------------------------------------------------------------------------------------------------+
|
|
| |
|
|
| [ In-Process Dumper: honkai-dumper ] |
|
|
| Injected DLL -> Waits 10s -> Lets game unpacker decrypt metadata -> Scans UnityPlayer.dll |
|
|
| -> Calls runtime C-API pointers in RAM -> Dumps memory objects |
|
|
| * Advantage: Simple implementation, relies on game's own decryption routines. |
|
|
| * Disadvantage: Requires running game process, triggers anti-cheat/integrity monitors. |
|
|
| |
|
|
| [ Offline Static Parser: Firefly-Static-Parser ] |
|
|
| Direct disk analysis: GameAssembly.dll + global-metadata.dat + startup-metadata.dat |
|
|
| 1. Parses PE64 headers -> Scans byte patterns for metadata initializers |
|
|
| 2. Solves arithmetic/XOR masks -> Resolves table offsets & payload boundaries |
|
|
| 3. Dynamic Semantic Key Discovery -> Derives XOR keys via CLR invariants (<Module>, ValueType) |
|
|
| 4. Decrypts QWORD string cipher & PRNG-scrambled method/field definitions |
|
|
| 5. Reconstructs TypeDef AST -> Emits dump.cs & methods.json |
|
|
| * Advantage: Zero process injection, zero anti-cheat risk, pure deterministic static analysis. |
|
|
| * Disadvantage: High mathematical and reverse-engineering complexity. |
|
|
| |
|
|
+----------------------------------------------------------------------------------------------------+
|
|
```
|
|
|
|
---
|
|
|
|
## 2. Codebase Architecture & File Organization
|
|
|
|
The project is structured in Rust (2024 Edition) with a modular pipeline separating PE analysis, metadata loading, mathematical key derivation, and AST emission.
|
|
|
|
```
|
|
Firefly-Static-Parser/
|
|
├── Cargo.toml # anyhow, serde, serde_json, windows-sys, iced-x86, il2cpp_dumper
|
|
├── src/
|
|
│ ├── main.rs # CLI entry point -> app::run()
|
|
│ ├── app.rs # Input routing (Standard vs MHY custom metadata branch)
|
|
│ ├── args.rs # CLI arguments & default path resolvers
|
|
│ ├── config.rs # Dumper configuration flags
|
|
│ ├── pe_image.rs # Zero-dependency PE64 section parser, RVA mapper, pattern scanner
|
|
│ ├── static_layout.rs # Signature scanner, PE layout solver, arithmetic/XOR mask decoders
|
|
│ ├── metadata.rs # Magic validator (0xFAB11BAF vs b"MHY\0")
|
|
│ ├── obfuscated.rs # Fallback in-process loader via GameAssembly.dll custom loader RVA
|
|
│ ├── il2cpp_pe.rs # Standard IL2CPP symbol/code registration loader
|
|
│ ├── output.rs # Standard dumper output generator
|
|
│ └── layout_model/ # The custom MHY layout reconstruction engine
|
|
│ ├── mod.rs # LayoutImage, LayoutTypeDef, LayoutMethod, LayoutField AST models
|
|
│ ├── output.rs # Static dump.cs & methods.json serialization
|
|
│ └── parser/
|
|
│ ├── mod.rs # Dynamic Semantic Key Discovery & PRNG scramble formulas
|
|
│ ├── loader.rs # Dynamic scanner for il2cpp_type_table & method_pointer_table
|
|
│ └── reader.rs # QWORD string stream cipher, method/field decoders, generic solver
|
|
```
|
|
|
|
---
|
|
|
|
## 3. PE Layout & Global Discovery Engine (`static_layout.rs`)
|
|
|
|
To parse the custom metadata container, the parser must first extract layout descriptors embedded within `GameAssembly.dll`.
|
|
|
|
### 3.1 Pattern Scanning for Initializers
|
|
`Firefly-Static-Parser` scans the `.text` section of `GameAssembly.dll` for two critical initialization sequences:
|
|
|
|
1. **`static_metadata_initializer_pattern`**:
|
|
The routine that initializes the IL2CPP static metadata globals. From this function, RIP-relative `lea` instructions are decoded to extract:
|
|
- `metadata_registration_rva`
|
|
- `descriptor_rva`
|
|
- `metadata_cache_rva`
|
|
- `embedded_header_rva`
|
|
|
|
2. **`metadata_payload_initializer_pattern`**:
|
|
The routine responsible for loading the external data files. It validates string references:
|
|
```rust
|
|
rva_has_c_string(image, global_name_rva, b"global-metadata.dat")
|
|
rva_has_c_string(image, startup_name_rva, b"startup-metadata.dat")
|
|
```
|
|
Extracts `payload_offset` (typically `0x00` or file header length).
|
|
|
|
### 3.2 Decryption of Embedded Header Offsets
|
|
The table offsets in the embedded header are obfuscated using combinations of 32-bit XOR keys, bitwise arithmetic shifts (`sar_u32`), and wrapping modular additions.
|
|
|
|
```rust
|
|
// Discovered from embedded_header_rva in GameAssembly.dll
|
|
let descriptor_count_80 = image.read_u32_rva(descriptor_rva + 0x80)? ^ 0x69BD_97F1;
|
|
let header_size_1a8 = image.read_u32_rva(header + 0x1A8)? ^ 0x729B_1A9E;
|
|
let header_size_1f8 = image.read_u32_rva(header + 0x1F8)? ^ 0x1608_C2C8;
|
|
let header_size_74 = image.read_u32_rva(header + 0x74)? ^ 0x080E_F250;
|
|
let header_count_134 = (image.read_u32_rva(header + 0x134)? ^ 0x1021_0728) / 40;
|
|
let header_count_178 = sar_u32(image.read_u32_rva(header + 0x178)?, 4) ^ 0x05CC_1AE1;
|
|
let header_count_bc = sar_u32(image.read_u32_rva(header + 0xBC)?, 3) ^ 0x079F_C2EC;
|
|
```
|
|
|
|
#### Complete Table Offsets Mapping:
|
|
|
|
| Offset In Header | Operation | Key / Operand | Output Field in `StaticLayout` |
|
|
| :---: | :---: | :---: | :--- |
|
|
| `+0x164` | `XOR` | `0x7F5C_5934` | `startup_section_164_offset` |
|
|
| `+0x150` | `wrapping_add` | `0xD882_615E` | `startup_image_table_offset` |
|
|
| `+0x158` | `wrapping_add` | `0xE03E_EAC1` | `startup_assembly_table_offset` |
|
|
| `+0xD0` | `wrapping_add` | `0xDFCF_C6B0` | `startup_assembly_name_table_offset` |
|
|
| `+0x84` | `XOR` | `0x6853_1D3F` | `global_type_table_offset` |
|
|
| `+0x20` | `wrapping_add` | `0xB2FC_E189` | `global_field_table_offset` |
|
|
| `+0x40` | `XOR` | `0x3C68_5CDB` | `global_property_table_offset` |
|
|
| `+0x180` | `wrapping_add` | `0xE4DB_E763` | `global_type_field_offset_map_offset` |
|
|
| `+0x9C` | `wrapping_add` | `0xBC7E_C9D7` | `global_field_offset_group_table_offset` |
|
|
| `+0x148` | `XOR` | `0x329E_1172` | `global_field_offset_table_offset` |
|
|
| `+0x14C` | `wrapping_add` | `0xF3A0_4294` | `global_method_table_offset` |
|
|
| `+0x30` | `wrapping_add` | `0xDCF5_FDBE` | `global_parameter_table_offset` |
|
|
| `+0x12C` | `XOR` | `0x26F0_FC20` | `global_nested_type_table_offset` |
|
|
| `+0x1C8` | `XOR` | `0x042F_9275` | `global_interface_type_table_offset` |
|
|
| `+0xF0` | `wrapping_add` | `0x9D48_920F` | `global_generic_container_table_offset` |
|
|
| `+0x140` | `wrapping_add` | `0xDAAC_7309` | `global_generic_parameter_table_offset` |
|
|
| `+0x38` | `wrapping_add` | `0xF778_F1AB` | `global_metadata_usage_type_table_offset` |
|
|
| `+0x1D0` | `wrapping_add` | `0xC9A6_64A5` | `global_metadata_usage_range_offset` |
|
|
| `+0x190` | `wrapping_add` | `0xED79_1073` | `global_metadata_usage_pair_offset` |
|
|
| `+0x1F0` | `XOR` | `0x56C7_D20D` | `global_string_literal_table_offset` |
|
|
| `+0x08` | `wrapping_add` | `0xF711_4E59` | `global_string_literal_data_offset` |
|
|
| `+0x1FC` | `XOR` | `0x6238_CDB0` | `global_field_default_value_table_offset` |
|
|
| `+0x3C` | `wrapping_add` | `0x978B_E7A5` | `global_field_default_value_data_offset` |
|
|
| `+0x1DC` | `XOR / 12` | `0x3E0C_72F0` | `field_default_value_count` |
|
|
| `+0x1B4` | `wrapping_add` | `0x8D43_A4EE` | `global_string_data_offset` |
|
|
|
|
---
|
|
|
|
## 4. Dynamic Semantic Key Derivation (`layout_model/parser/mod.rs`)
|
|
|
|
A major breakthrough in `Firefly-Static-Parser` is how it discovers runtime XOR keys and type sentinels **without hardcoding version-specific constants**.
|
|
|
|
### 4.1 The Semantic Invariant Technique
|
|
In every valid .NET/IL2CPP metadata image, standard CLI core types always exhibit known semantic invariants:
|
|
1. **TypeDef 0 is `<Module>`**:
|
|
- Has no base type $\rightarrow$ raw base value gives `type_base_none`.
|
|
- Has known attribute flags $\rightarrow$ `raw_flags ^ 0x700` reveals `type_attribute_key`.
|
|
2. **`System.ValueType`**:
|
|
- Inherits directly from `System.Object`. Therefore, the raw base type of `System.ValueType` reveals `type_base_object`.
|
|
3. **`System.Enum`**:
|
|
- Inherits directly from `System.ValueType`. Its raw base type reveals `type_base_value_type`.
|
|
4. **`System.AttributeTargets`**:
|
|
- Is an Enum. Its raw base type reveals `type_base_enum`.
|
|
5. **Parent Type Index Formula**:
|
|
$$\text{type\_parent\_key} = \text{type\_base\_object} - 1$$
|
|
$$\text{ParentIndex} = \text{raw\_base\_type} - \text{type\_parent\_key}$$
|
|
|
|
### 4.2 Code Implementation
|
|
```rust
|
|
fn discover_type_layout_keys(layout: &mut StaticLayout, global_data: &[u8]) -> Result<()> {
|
|
// 1. Module Sentinel & Attribute Key
|
|
let type_table_offset = layout.payload_offset as usize + layout.global_type_table_offset as usize;
|
|
let module_name = decode_type_name_at(global_data, layout, type_table_offset)?.1;
|
|
assert_eq!(module_name, "<Module>");
|
|
|
|
layout.keys.type_base_none = read_u32(global_data, type_table_offset + 0x04)?;
|
|
layout.keys.type_attribute_key = read_u32(global_data, type_table_offset + 0x14)? ^ 0x700;
|
|
|
|
// 2. Scan Type Definitions for Standard BCL Anchors
|
|
for type_index in 0..scan_count {
|
|
let entry_offset = type_entry_offset_raw(layout, type_index).unwrap();
|
|
let raw_base = read_u32(global_data, entry_offset + 0x04)?;
|
|
let (namespace, name) = decode_type_name_at(global_data, layout, entry_offset)?;
|
|
|
|
match (namespace.as_str(), name.as_str()) {
|
|
("System", "ValueType") => layout.keys.type_base_object = raw_base,
|
|
("System", "Enum") => layout.keys.type_base_value_type = raw_base,
|
|
("System", "AttributeTargets") => layout.keys.type_base_enum = raw_base,
|
|
_ => {}
|
|
}
|
|
}
|
|
|
|
layout.keys.type_parent_key = layout.keys.type_base_object.wrapping_sub(1);
|
|
Ok(())
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 5. Decryption Algorithms & Mathematical Models
|
|
|
|
### 5.1 The QWORD String Stream Cipher (`decode_string_raw`)
|
|
In the `MHY\0` container, strings are not null-terminated C-strings. String pointers are 32-bit composite integers encoding both length and offset.
|
|
|
|
#### 1. Composite Index Bitfield Decoding:
|
|
```rust
|
|
let signed_index = index as i32;
|
|
let (length, string_offset_mask) = if signed_index < 0 {
|
|
// Long String Format (up to 255 bytes, 23-bit offset)
|
|
(((index >> 23) & 0xFF) as usize, 0x007F_FFFF)
|
|
} else {
|
|
// Short String Format (up to 63 bytes, 25-bit offset)
|
|
(((index >> 25) & 0x3F) as usize, 0x01FF_FFFF)
|
|
};
|
|
let string_offset = (index & string_offset_mask) as usize;
|
|
```
|
|
|
|
#### 2. Seeded Stream Decryption:
|
|
Strings are encrypted in 8-byte QWORD blocks. The key is dynamically computed per string offset:
|
|
|
|
$$K_0 = \text{seed\_add} + (\text{seed\_mul} \times \text{string\_offset})$$
|
|
|
|
For each 8-byte chunk $i \in [0, \lceil \text{length} / 8 \rceil)$:
|
|
|
|
$$P_i = C_i \oplus K_i$$
|
|
|
|
$$K_{i+1} = K_i + \text{increment}$$
|
|
|
|
```rust
|
|
let mut key = string_payload_key(layout, string_offset);
|
|
let mut out = Vec::with_capacity(qword_count * 8);
|
|
for chunk_index in 0..qword_count {
|
|
let encrypted = read_u64(global_data, data_offset + chunk_index * 8)?;
|
|
out.extend_from_slice(&(encrypted ^ key).to_le_bytes());
|
|
key = key.wrapping_add(string_payload_increment(layout));
|
|
}
|
|
out.truncate(length);
|
|
```
|
|
|
|
### 5.2 Non-Linear PRNG Scrambling Functions
|
|
|
|
Every table entry in the metadata layout is obfuscated by non-linear integer hashing functions:
|
|
|
|
#### 1. Image Key (`image_key`):
|
|
$$K_{\text{img}}(i) = ((((i \times \mathtt{0xE07C}) \oplus \mathtt{0x7538159E}) \times \mathtt{0x120D0703}) \oplus \mathtt{0x6032C9D3}) + \mathtt{0x2EBB0085}$$
|
|
|
|
#### 2. Method Key (`method_key`):
|
|
$$v_1 = (i \times \mathtt{0x31E1}) \oplus \mathtt{0x33914937}$$
|
|
$$v_2 = ((v_1 \times \mathtt{0x2C03F17D}) \gg 23) \times \mathtt{0x540CC9F4} \gg 21$$
|
|
$$K_{\text{method}}(i) = (v_2 \text{ as u32}) + \mathtt{0x71BC7861}$$
|
|
|
|
#### 3. Parameter Key (`parameter_key`):
|
|
$$v = ((i \times \mathtt{0x072E1D74B12B} + \mathtt{0x01911D05AFF5}) \gg 11) \text{ as u32}$$
|
|
$$K_{\text{param}}(i) = (v \times \mathtt{0x58B870A2}) + \mathtt{0x83CF7B44}$$
|
|
|
|
#### 4. Field Key (`field_key`):
|
|
$$K_{\text{field}}(\text{start}, j) = \mathtt{0xAD416BB9} - (\text{start} \times \mathtt{0x2C5DCB00}) + (j \times \mathtt{0xD3A23500})$$
|
|
|
|
#### 5. Generic Container Key (`generic_container_key`):
|
|
$$v_1 = (i \times \mathtt{0x3D6913E0AF40} + \mathtt{0x0A64CAD60FA052C0}) \gg 23$$
|
|
$$v_2 = ((v_1 \times \mathtt{0x770E3FE8}) \gg 11) \times \mathtt{0x2C9A0EA3} \gg 23$$
|
|
$$K_{\text{gen\_container}}(i) = v_2 \text{ as u32}$$
|
|
|
|
---
|
|
|
|
## 6. Type System Reconstruction & Output Generation
|
|
|
|
### 6.1 Dynamic Table Resolution in `GameAssembly.dll`
|
|
In addition to metadata parsing, method RVAs require locating the compiled function pointers. In `src/layout_model/parser/loader.rs`:
|
|
- **`il2cpp_type_table`**: Scans `GameAssembly.dll` for continuous 16-byte descriptors where `kind` $\in (0, 0x50)$ and `byref` $\in \{0, 1, 0x80, 0x81\}$.
|
|
- **`method_pointer_table`**: Scans for continuous runs of 64-bit pointers residing within the `.text` segment ($ImageBase + 0x1000 \le VA < ImageBase + Size$).
|
|
- **`generic_inst_table`**: Scans for runs of `argc` ($0 < argc < 30$) with 32-bit zero padding and valid pointer prefixes (`0x01`, `0x00`, `0x7F`).
|
|
|
|
### 6.2 Output Formats
|
|
|
|
#### 1. Reconstructed C# Source (`dump.cs`)
|
|
Outputs complete class declarations with namespaces, type index comments, attributes, and field offsets:
|
|
|
|
```csharp
|
|
// FirelfyShelter static asm parse
|
|
|
|
// Image 0: Assembly-CSharp.dll
|
|
|
|
// Namespace: RPG.Client
|
|
// TypeDefIndex: 1420
|
|
[Serializable]
|
|
public class AutoScrollRect : ScrollRect
|
|
{
|
|
// Fields
|
|
private Vector3[] KOIAJFFGPGF; // 0x180
|
|
|
|
// Methods
|
|
// RVA: 0x1febdf0 VA: 0x181febdf0
|
|
public void .ctor() { }
|
|
|
|
// RVA: 0x1feb700 VA: 0x181feb700
|
|
private void MNOGNALMGPL(AnimatorButton target) { }
|
|
|
|
// RVA: 0x1feb9c0 VA: 0x181feb9c0
|
|
public void SnapTo(GameObject target) { }
|
|
}
|
|
```
|
|
|
|
#### 2. JSON Method Map (`methods.json`)
|
|
Outputs canonical method signatures mapped to hexadecimal RVAs:
|
|
|
|
```json
|
|
{
|
|
"RPG.Client.AutoScrollRect::.ctor()": "0x1febdf0",
|
|
"RPG.Client.AutoScrollRect::MNOGNALMGPL(RPG.Client.AnimatorButton)": "0x1feb700",
|
|
"RPG.Client.AutoScrollRect::POFLDHKMLAF()": "0x1feb7a0",
|
|
"RPG.Client.AutoScrollRect::SetItemSelectCallback()": "0x1feb7f0",
|
|
"RPG.Client.AutoScrollRect::SnapTo(UnityEngine.GameObject)": "0x1feb9c0"
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 7. Comparative Analysis Matrix
|
|
|
|
| Feature | `Firefly-Static-Parser` | `honkai-dumper` | Standard `Il2CppDumper` |
|
|
| :--- | :--- | :--- | :--- |
|
|
| **Execution Mode** | **Offline CLI** | **In-Process DLL** | **Offline CLI** |
|
|
| **Target Files** | `GameAssembly.dll`<br>`global-metadata.dat`<br>`startup-metadata.dat` | Running `StarRail.exe` process | `GameAssembly.dll`<br>`global-metadata.dat` |
|
|
| **`MHY\0` Container Decryption** | **Native Built-in** (Algebraic & QWORD stream cipher) | **Delegated** to game process | **Unsupported** (Fails on magic mismatch) |
|
|
| **Anti-Cheat Exposure** | **Zero** | High (Process injection) | **Zero** |
|
|
| **Key Extraction** | **Dynamic Semantic Discovery** (via `<Module>`, `ValueType`) | Memory query | Static pattern scanning |
|
|
| **Method RVA Output** | Canonical typed signatures in `methods.json` | Flattened overloaded names in `methods.json` | Script files for IDA/Ghidra |
|
|
| **Portability** | Multi-platform (Linux/Windows/macOS) | Windows-only (x86_64) | Multi-platform |
|
|
|
|
---
|
|
|
|
## 8. Integration & Protocol Engineering Synergy
|
|
|
|
### 8.1 Feedstock for `FireflyProtoc`
|
|
In the Star Rail reverse-engineering toolchain:
|
|
1. `Firefly-Static-Parser` extracts the exact class structures and obfuscated field names (e.g., `KOIAJFFGPGF` $\rightarrow$ Wire tag `1` at offset `0x180`).
|
|
2. Engineers cross-reference these offsets with network serialization routines.
|
|
3. The resulting proto definitions with wire masks (`[get: (id ^ 1) + 2]`) and metadata (`meta uint32 cmd_id = 143;`) are compiled using **`FireflyProtoc`**.
|
|
4. The generated high-performance C# or Go models are directly consumed by **`March7thHoney`** or **`GaC-hkrpg-go`**.
|
|
|
|
### 8.2 Compliance with Repository Rules (`AGENTS.md`)
|
|
- `Firefly-Static-Parser` operates purely as an offline analysis tool.
|
|
- It contains no hardcoded user paths, machine IP addresses, or environment-dependent paths.
|
|
- It eliminates the need for prohibited live heartbeat scanning scripts in game client directories (`scripts/`), providing a 100% compliant and safe research workflow.
|