GaCy-GO/docs/scratch/static_cmdid_dumper_deep_dive.md

# Static CmdId Dumper (`StaticCmdIdDumper`) Architectural Specification & Machine-Code Deep Dive

This technical specification provides an exhaustive, code-level analysis of **`StaticCmdIdDumper`** (by Hiro420), an ultra-minimalist, zero-dependency C# (.NET 8.0) static binary analysis utility. The tool is engineered to extract network packet Command IDs (`CmdId` / `Opcode`) directly from compiled IL2CPP game executables (such as `YuanShen.exe`, `StarRail.exe`, or `GameAssembly.dll`) on disk without launching the game process or relying on third-party disassemblers (such as Capstone or Iced).

---

## 1. Executive Summary & Architectural Purpose

### 1.1 The Protocol Command ID Problem
In modern anime titles developed on Unity IL2CPP (Genshin Impact, Honkai: Star Rail, Zenless Zone Zero), client-server communication relies on an enumerated command opcode (`CmdId`) identifying each Protobuf packet.

Inside the game client assemblies, each packet class contains a getter method returning its unique opcode:
```csharp
public class GetAvatarDataCsReq : IMessage {
    public override ushort GetCmdId() {
        return 384; // 0x0180
    }
}
```

In compiled releases:
1. Method names are stripped or obfuscated (e.g., `GetCmdId()` $\rightarrow$ `KOOKJNLBAMH()`).
2. Tools like `Firefly-Static-Parser` or `honkai-dumper` can identify the **Relative Virtual Address (RVA)** of these getter methods.
3. However, obtaining the actual integer value of the opcode traditionally required:
   - Attaching a live debugger / Frida script to invoke the method at runtime, or
   - Opening a 100MB+ binary in IDA Pro / Ghidra and manually decompiling thousands of tiny stub functions.

### 1.2 The `StaticCmdIdDumper` Solution
`StaticCmdIdDumper` demonstrates an elegant, lightweight alternative:
- It exploits the deterministic machine-code pattern emitted by C++ compilers when compiling a constant 16-bit integer getter.
- It parses PE (Portable Executable) headers using pure, hand-written C# structs.
- It maps the getter method's RVA directly to disk file offset, reads **exactly 4 bytes**, and extracts the embedded immediate integer via a micro-disassembler.

```
+----------------------------------------------------------------------------------------------------+
|                                    StaticCmdIdDumper Architecture                                  |
+----------------------------------------------------------------------------------------------------+
|                                                                                                    |
|  [ Inputs ]                                                                                        |
|  1. Binary File (e.g. YuanShen.exe / GameAssembly.dll)                                             |
|  2. list.txt: Mapping of <ObfuscatedName> <RVA>                                                    |
|     Example: "KOOKJNLBAMH 0x0CCA3780"                                                              |
|                                                                                                    |
|  [ Pipeline: Zero External Dependencies ]                                                           |
|                                                                                                    |
|  +----------------------------------------------------------------------------------------------+  |
|  | 1. PEHeader.cs & Misc.cs                                                                     |  |
|  |    - Parse DOS Header ('MZ') -> e_lfanew -> PE Signature ('PE\0\0')                         |  |
|  |    - Parse OptionalHeader64 -> Extract Section Table                                         |  |
|  |    - Locate Target Section ('il2cpp' or '.text')                                             |  |
|  +----------------------------------------------+-----------------------------------------------+  |
|                                                 |                                                  |
|                                                 v                                                  |
|  +----------------------------------------------------------------------------------------------+  |
|  | 2. File Offset Translation                                                                   |  |
|  |    FileOffset = Section.PointerToRawData + (MethodRVA - Section.VirtualAddress)              |  |
|  +----------------------------------------------+-----------------------------------------------+  |
|                                                 |                                                  |
|                                                 v                                                  |
|  +----------------------------------------------------------------------------------------------+  |
|  | 3. Targeted 4-Byte Seek & Read                                                               |  |
|  |    fs.Seek(FileOffset) -> fs.Read(buffer, 0, 4)                                              |  |
|  +----------------------------------------------+-----------------------------------------------+  |
|                                                 |                                                  |
|                                                 v                                                  |
|  +----------------------------------------------------------------------------------------------+  |
|  | 4. Micro-Disassembler (Disassembler.cs)                                                      |  |
|  |    Matches x86-64 pattern: 66 B8 [imm16]  --> "mov ax, imm16"                                |  |
|  |    Extracts Little-Endian uint16 CmdId                                                       |  |
|  +----------------------------------------------+-----------------------------------------------+  |
|                                                 |                                                  |
|                                                 v                                                  |
|  [ Output ]                                                                                        |
|  PacketIds.json:                                                                                   |
|  {                                                                                                 |
|    "KOOKJNLBAMH": 1234,                                                                            |
|    "MGDFLHGODLH": 4567                                                                             |
|  }                                                                                                 |
|                                                                                                    |
+----------------------------------------------------------------------------------------------------+
```

---

## 2. Codebase Organization & Engineering Philosophy

The repository consists of a single C# project targeting `.NET 8.0`:

```
StaticCmdIdDumper/
├── StaticCmdIdDumper.sln
└── StaticCmdIdDumper/
    ├── StaticCmdIdDumper.csproj   # Zero external package references (<PackageReference>)
    ├── PEHeader.cs                # Complete Win32/PE64 data structure definitions & binary readers
    ├── Misc.cs                    # Section search, header dumpers, and file parsing utilities
    ├── Disassembler.cs            # 50-line custom x86-64 machine-code instruction decoder
    └── Program.cs                 # CLI coordinator & JSON serializer
```

### 2.1 The Zero-Dependency Design
Modern .NET reverse-engineering tools typically pull in large third-party dependencies:
- `AsmResolver.PE` or `PeNet` for PE parsing.
- `Iced` or `Gee.External.Capstone` for disassembly.

`StaticCmdIdDumper` explicitly rejects external packages, implementing:
1. Hand-coded binary reader routines (`Read16LE`, `Read32LE`, `Read64LE`).
2. Hand-crafted PE structures mirroring Windows `winnt.h` (`IMAGE_DOS_HEADER`, `IMAGE_NT_HEADERS64`, `IMAGE_SECTION_HEADER`).
3. Direct byte pattern recognition for the specific instruction opcode emitted by MSVC / Clang.

---

## 3. Machine Code Mechanics: IL2CPP Opcode Getters

### 3.1 C++ to Native x86-64 Compilation
In Unity's IL2CPP output, a constant getter function in C++ is written as:
```cpp
uint16_t GetCmdId() {
    return 1234; // 0x04D2
}
```

When compiled by MSVC or Clang for x86-64 with standard optimizations (`/O2` or `-O3`), the compiler emits:
```nasm
mov ax, 0x04D2
ret
```

### 3.2 Binary Instruction Encoding
The x86-64 instruction encoding for `mov ax, imm16` is:

$$\begin{array}{|c|c|c|c|}
\hline
\mathbf{0x66} & \mathbf{0xB8} & \mathbf{\text{Byte}_0} & \mathbf{\text{Byte}_1} \\
\hline
\text{Prefix (Operand Size)} & \text{Opcode (mov ax)} & \text{Imm16 (Low)} & \text{Imm16 (High)} \\
\hline
\end{array}$$

- **`0x66` (Operand-Size Override Prefix)**: The default operand size in 64-bit mode is 32-bit. The `0x66` prefix instructs the processor to decode the instruction for the 16-bit register `ax` instead of 32-bit `eax`.
- **`0xB8`**: Opcode for `mov reg16/32, imm16/32` (specifically targeting accumulator register `ax`).
- **`Byte 0` & `Byte 1`**: 16-bit little-endian immediate value representing the `CmdId`.
- **`0xC3`**: The subsequent byte (at offset +4) is the `ret` opcode.

### 3.3 The Custom Micro-Disassembler (`Disassembler.cs`)
Because the target functions follow this invariant structure, `Disassembler.cs` decodes the opcode in just a few lines:

```csharp
public static int Disassemble(byte[] code)
{
    int ip = 0;

    while (ip < code.Length)
    {
        byte opcode = code[ip];

        switch (opcode)
        {
            case 0x66: // 16-bit operand override
            {
                if (ip + 1 < code.Length && code[ip + 1] == 0xB8)
                {
                    if (ip + 3 < code.Length)
                    {
                        // Read 16-bit unsigned little-endian integer
                        ushort imm16 = BitConverter.ToUInt16(code, ip + 2);
                        return imm16;
                    }
                }
                ip += 1;
                break;
            }
            default:
                ip += 1;
                break;
        }
    }
    return 0;
}
```

---

## 4. PE Header Parsing & RVA-to-Offset Translation

### 4.1 PE64 Structural Traversal
In `PEHeader.cs` and `Misc.cs`, the program traverses the binary structure:

1. **DOS Header**:
   - Reads `dosHeader.magic` (must equal `0x5A4D` / `"MZ"`).
   - Reads `dosHeader.e_lfanew` at offset `0x3C`, which points to the NT PE header.
2. **PE Header & Optional Header**:
   - Validates `dosHeader.pe.signature == 0x00004550` (`"PE\0\0"`).
   - Determines architecture from `optionalHeader.magic`:
     - `0x10B`: 32-bit PE (`x86`)
     - `0x20B`: 64-bit PE (`x86_64`)
   - Reads `numberOfSections` and `optionalHeaderSize`.
3. **Section Table Traversal (`ReadSections`)**:
   - Iterates through `IMAGE_SECTION_HEADER` entries (40 bytes each):
     ```csharp
     dosHeader.section_table[idx].name = ReadStr(reader, 8);
     dosHeader.section_table[idx].virtualSize = Read32LE(reader);
     dosHeader.section_table[idx].virtualAddr = Read32LE(reader);
     dosHeader.section_table[idx].sizeOfRawData = Read32LE(reader);
     dosHeader.section_table[idx].ptrToRawData = Read32LE(reader);
     ```

### 4.2 Locating the IL2CPP Section (`Misc.GetBaseAdd`)
In certain compiled anime games (notably Genshin Impact's `YuanShen.exe`), the linker places compiled game code into a custom named section:
```csharp
for (int idx = 0; idx < dosHeader.pe.numberOfSections; idx++)
{
    if (sections![idx].name != "il2cpp")
    {
        continue;
    }
    return sections[idx];
}
```

### 4.3 RVA to Disk File Offset Translation
Memory-mapped virtual addresses (VAs and RVAs) differ from raw disk offsets due to section alignment. The conversion formula is:

$$\text{FileOffset} = \text{Section.PointerToRawData} + (\text{RVA}_{\text{target}} - \text{Section.VirtualAddress})$$

In `Program.cs`:
```csharp
ulong CmdIDTestAddr = Convert.ToUInt64(pair.Value.Substring(2), 16);
var fileoffset = il2cpp_section.ptrToRawData + (CmdIDTestAddr - il2cpp_section.virtualAddr);

byte[] buffer = new byte[4];
using (FileStream fs = new FileStream(args[0], FileMode.Open, FileAccess.Read))
{
    fs.Seek((long)fileoffset, SeekOrigin.Begin);
    fs.Read(buffer, 0, 4);
}

int cmdid = Disassembler.Disassemble(buffer);
packetIds[pair.Key] = cmdid;
```

---

## 5. Execution Workflow & Serialization

### 5.1 Input Format (`list.txt`)
The tool accepts a plain whitespace-delimited text file containing pairs of function identifiers and hexadecimal RVAs:
```text
KOOKJNLBAMH 0x0CCA3780
MGDFLHGODLH 0x0BE5AA60
CGKBCOJMLPC 0x082AB170
OOMIIJFHPFN 0x0D16A280
```
> [!NOTE] Origin of the Input List
> This list is directly obtained from the output of `Firefly-Static-Parser` or `honkai-dumper`. In `methods.json`, getter methods for packet IDs appear as:
> `"KOOKJNLBAMH": "0xCCA3780"`

### 5.2 Output Format (`PacketIds.json`)
The resulting dictionary is serialized to formatted JSON:
```json
{
  "KOOKJNLBAMH": 101,
  "MGDFLHGODLH": 102,
  "CGKBCOJMLPC": 143,
  "OOMIIJFHPFN": 172
}
```

---

## 6. The Complete 5-Stage Protocol Reverse-Engineering Toolchain

By studying `LunarCore`, `C# NativeAOT Reference Server`, `FireflyProtoc`, `honkai-dumper`, `Firefly-Static-Parser`, and `StaticCmdIdDumper`, we can now construct the entire open-source reverse-engineering pipeline:

```
+----------------------------------------------------------------------------------------------------+
|                                THE UNIFIED PROTOCOL EXTRACTION PIPELINE                            |
+----------------------------------------------------------------------------------------------------+
|                                                                                                    |
|  [ Stage 1: Metadata Extraction ]                                                                  |
|  Tool: Firefly-Static-Parser (Offline) OR honkai-dumper (In-Process)                                |
|  Input: GameAssembly.dll + global-metadata.dat (b"MHY\0")                                          |
|  Output: dump.cs (class schemas) + methods.json (method RVAs)                                      |
|                                                                                                    |
|  [ Stage 2: Static Opcode Extraction ]                                                             |
|  Tool: StaticCmdIdDumper                                                                           |
|  Input: Executable binary + method RVAs of getter functions                                        |
|  Process: Reads 4-byte machine code (66 B8 imm16) -> extracts uint16 CmdIds                         |
|  Output: PacketIds.json (Map of ObfuscatedPacketName -> Integer CmdId)                              |
|                                                                                                    |
|  [ Stage 3: Protobuf Schema Annotation ]                                                           |
|  Tool: Schema Annotator / String Literal Correlator                                                |
|  Action: Injects metadata annotations into .proto files:                                           |
|          meta uint32 cmd_id = 143;                                                                 |
|          uint32 id = 1 [get: (id ^ 1) + 2];                                                        |
|                                                                                                    |
|  [ Stage 4: Specialized Protobuf Compilation ]                                                     |
|  Tool: FireflyProtoc                                                                               |
|  Action: Compiles .proto via AST pipeline (HIR -> MIR -> LIR)                                      |
|          - ExpressionInverter: Automatically derives setter logic for wire masks                   |
|          - MessageFactory: Emits O(1) compile-time jump table switch(cmdId)                        |
|  Output: High-performance C# / Go protobuf stubs                                                   |
|                                                                                                    |
|  [ Stage 5: Server Protocol Routing ]                                                              |
|  Tool: C# NativeAOT Reference Server (.NET 10) / LunarCore (Java) / GaC-hkrpg-go (Go)                              |
|  Action: Instant zero-reflection packet dispatching on KCP connections                             |
|                                                                                                    |
+----------------------------------------------------------------------------------------------------+
```

---

## 7. Technical Critique & Improvements for `GaC-hkrpg-go`

While `StaticCmdIdDumper` is compact and effective, an industrial-grade implementation (e.g. ported to Go for `GaC-hkrpg-go`) requires resolving several architectural limitations:

### 7.1 Limitation 1: Hardcoded Section Name (`"il2cpp"`)
In `Misc.cs`:
```csharp
if (sections![idx].name != "il2cpp") continue;
```
- **Issue**: In *Honkai: Star Rail* and standard Unity builds, code is placed in the standard `.text` section, **not** a section named `il2cpp`.
- **Solution**: Implement dynamic RVA-to-Section resolution across **all** sections:
  $$\text{Section} = \{ s \in \text{Sections} \mid s.\text{VirtualAddress} \le \text{RVA} < s.\text{VirtualAddress} + s.\text{VirtualSize} \}$$

### 7.2 Limitation 2: 32-Bit Register Emptiness (`mov eax, imm32`)
- **Issue**: If the method return type is `int` (32-bit signed integer) rather than `ushort`, compilers emit `mov eax, imm32` without the `0x66` prefix:
  `B8 xx xx 00 00` (5 bytes: `0xB8` followed by 4-byte integer).
- **Solution**: Support both instruction patterns:
  - `66 B8 [imm16]` $\rightarrow$ 16-bit opcode getter.
  - `B8 [imm32]` $\rightarrow$ 32-bit opcode getter.

### 7.3 Limitation 3: Stream Seeking Overhead
- **Issue**: Opening and seeking `FileStream` individually for every single RVA creates heavy I/O overhead for thousands of packets.
- **Solution**: Memory-map the section or load the section into a contiguous `[]byte` slice in memory for instant $O(1)$ random access.

### 7.4 Go Port Blueprint for `GaC-hkrpg-go`
A native, high-performance Go implementation for extracting opcodes directly within the `GaC-hkrpg-go` toolset:

```go
package pe_dumper

import (
	"debug/pe"
	"encoding/binary"
	"fmt"
	"os"
)

func ExtractCmdIdFromRVA(peFile *pe.File, rva uint32, f *os.File) (uint16, error) {
	// 1. Locate the containing section dynamically
	for _, sec := range peFile.Sections {
		if rva >= sec.VirtualAddress && rva < sec.VirtualAddress+sec.VirtualSize {
			fileOffset := int64(sec.Offset + (rva - sec.VirtualAddress))
			buf := make([]byte, 4)
			if _, err := f.ReadAt(buf, fileOffset); err != nil {
				return 0, err
			}
			// 2. Pattern: 66 B8 [imm16]
			if buf[0] == 0x66 && buf[1] == 0xB8 {
				return binary.LittleEndian.Uint16(buf[2:4]), nil
			}
			// 3. Pattern: B8 [imm32] (32-bit fallback)
			if buf[0] == 0xB8 {
				return binary.LittleEndian.Uint16(buf[1:3]), nil
			}
			return 0, fmt.Errorf("unknown instruction pattern: % X", buf)
		}
	}
	return 0, fmt.Errorf("RVA 0x%X not found in any section", rva)
}
```

---

## 8. Compliance with Repository Rules (`AGENTS.md`)

- **Documentation & Protocol Integrity**: `StaticCmdIdDumper` extracts Command IDs purely from the machine code representation of the target client binary. It adheres strictly to the rule requiring handlers to resolve `CmdID` from verified client protocol tables without guessing or hardcoding multi-version aliases.
- **No Machine-Specific Hardcoding**: All file paths and offsets are read dynamically from command-line arguments and PE headers.
- **Zero Runtime Interference**: Operates entirely offline on the executable file, with zero memory injection, hooks, or heartbeat overhead.