454 lines
21 KiB
Markdown
454 lines
21 KiB
Markdown
# Dumpcs2Protobuf Architectural Specification & Schema Reconstruction Deep Dive
|
|
|
|
This technical specification provides an exhaustive, code-level analysis of **`Dumpcs2Protobuf`** (by Hiro420), a specialized C# (.NET 8.0) schema recovery utility engineered to reconstruct Protocol Buffers v3 (`.proto`) definition files directly from the decompiled C# source files (`dump.cs`) emitted by IL2CPP metadata dumpers (such as `honkai-dumper` or `Firefly-Static-Parser`).
|
|
|
|
---
|
|
|
|
## 1. Executive Summary & Architectural Role
|
|
|
|
### 1.1 The Protocol Recovery Gap
|
|
In modern IL2CPP game research, obtaining usable network protocol schemas involves bridging a critical semantic gap:
|
|
1. IL2CPP dumpers extract decompiled C# classes (`dump.cs`).
|
|
2. Server implementations (`LunarCore`, `March7thHoney`, `GaC-hkrpg-go`) require `.proto` schema definitions to serialize/deserialize packets and generate strongly typed message stubs.
|
|
3. Manually translating thousands of obfuscated C# packet classes into `.proto` files is infeasible.
|
|
|
|
### 1.2 The De-Compilation Inverse
|
|
`Dumpcs2Protobuf` operates as the **exact inverse** of Google's Protocol Buffers C# compiler (`protoc --csharp_out`):
|
|
- When Google's `protoc` compiles a `.proto` file into C#, it generates deterministic structural signatures (constant field numbers, `RepeatedField<T>`, `MapField<K, V>`, `OneofCase` enums, and `OriginalNameAttribute`).
|
|
- `Dumpcs2Protobuf` analyzes these exact code generation patterns inside `dump.cs`, extracts field tags, names, and types, and **reconstructs valid `proto3` syntax**.
|
|
|
|
```
|
|
+----------------------------------------------------------------------------------------------------+
|
|
| Dumpcs2Protobuf Pipeline |
|
|
+----------------------------------------------------------------------------------------------------+
|
|
| |
|
|
| [ Input: dump.cs (Decompiled IL2CPP Output) ] |
|
|
| |
|
|
| // Module: RPG.Network.Proto.dll |
|
|
| public sealed class IEFELCIJJDA : IMessage |
|
|
| { |
|
|
| public const int CPBHCOPAGBD = 13; |
|
|
| public const int NEFBNAELMLD = 14; |
|
|
| public uint EGPKLBOKFNF { get; set; } |
|
|
| public RepeatedField<NFJLPFKFDLF> MDDLGJHLLAI { get; } |
|
|
| } |
|
|
| |
|
|
| +----------------------------------------------------------------------------------------------+ |
|
|
| | 1. Assembly & Class Filter | |
|
|
| | Filters segments for 'RPG.Network.Proto.dll' and classes implementing 'IMessage' | |
|
|
| +----------------------------------------------+-----------------------------------------------+ |
|
|
| | |
|
|
| v |
|
|
| +----------------------------------------------------------------------------------------------+ |
|
|
| | 2. Tag & Property Pairing Engine | |
|
|
| | - Extracts field tags from 'public const int <Name> = <Tag>;' | |
|
|
| | - Extracts properties: Scalar, RepeatedField<T>, MapField<K, V> | |
|
|
| | - Pairs tags with properties in declaration order | |
|
|
| +----------------------------------------------+-----------------------------------------------+ |
|
|
| | |
|
|
| v |
|
|
| +----------------------------------------------------------------------------------------------+ |
|
|
| | 3. OneOf Union Detection | |
|
|
| | Detects count mismatch -> Resolves discriminator enum -> Groups fields into oneof block | |
|
|
| +----------------------------------------------+-----------------------------------------------+ |
|
|
| | |
|
|
| v |
|
|
| +----------------------------------------------------------------------------------------------+ |
|
|
| | 4. Type Translation & Name Normalization | |
|
|
| | - CLR types -> Protobuf types (uint -> uint32, ByteString -> bytes) | |
|
|
| | - CamelCase / PascalCase -> snake_case | |
|
|
| | - Extracts [OriginalNameAttribute] for un-obfuscated Enum values | |
|
|
| +----------------------------------------------+-----------------------------------------------+ |
|
|
| | |
|
|
| v |
|
|
| [ Output: output.proto ] |
|
|
| |
|
|
| syntax = "proto3"; |
|
|
| |
|
|
| message IEFELCIJJDA { |
|
|
| uint32 egpklbokfnf = 13; |
|
|
| repeated NFJLPFKFDLF mddlgjhllai = 14; |
|
|
| } |
|
|
| |
|
|
+----------------------------------------------------------------------------------------------------+
|
|
```
|
|
|
|
---
|
|
|
|
## 2. Inverting Google.Protobuf C# Code Generation
|
|
|
|
To understand how `Dumpcs2Protobuf` works, we must analyze the exact contract emitted by Google's `protoc` C# code generator.
|
|
|
|
### 2.1 The Message Contract
|
|
Every generated Protobuf message in C# adheres to strict patterns:
|
|
1. **Class Declaration**:
|
|
```csharp
|
|
public sealed class MessageName : IMessage, IMessage<MessageName>, ...
|
|
```
|
|
2. **Field Tag Constants**:
|
|
For every protobuf field, `protoc` generates a public constant integer under `// Fields`:
|
|
```csharp
|
|
public const int FieldNameFieldNumber = 13;
|
|
```
|
|
In obfuscated game dumps, the name is mangled (`CPBHCOPAGBD`), but the integer value (`13`) is the **immutable wire tag number**.
|
|
3. **Public Non-Static Properties**:
|
|
Under `// Properties`, `protoc` generates accessors corresponding to each field:
|
|
- Scalar fields: `public uint FieldName { get; set; }`
|
|
- Repeated fields: `public RepeatedField<TypeName> FieldName { get; }`
|
|
- Map fields: `public MapField<KeyType, ValType> FieldName { get; }`
|
|
- Byte sequences: `public ByteString FieldName { get; set; }`
|
|
|
|
### 2.2 The OneOf Union Contract
|
|
When a `.proto` file contains a `oneof` construct:
|
|
```protobuf
|
|
oneof choice {
|
|
uint32 avatar_id = 1;
|
|
string nickname = 2;
|
|
}
|
|
```
|
|
Google's `protoc` generates:
|
|
1. Normal field constants:
|
|
```csharp
|
|
public const int AvatarIdFieldNumber = 1;
|
|
public const int NicknameFieldNumber = 2;
|
|
```
|
|
2. Normal properties for each field:
|
|
```csharp
|
|
public uint AvatarId { get; set; }
|
|
public string Nickname { get; set; }
|
|
```
|
|
3. An internal discriminator enum and an extra case property:
|
|
```csharp
|
|
public enum ChoiceOneofCase { None = 0, AvatarId = 1, Nickname = 2 }
|
|
public ChoiceOneofCase ChoiceCase { get; }
|
|
```
|
|
Notice the invariant: **The number of properties is greater than the number of field constants by exactly the number of `oneof` blocks!**
|
|
|
|
### 2.3 The Enum Contract & `OriginalNameAttribute`
|
|
Protobuf enums in C# retain their original un-obfuscated names via attributes:
|
|
```csharp
|
|
public enum FGEMFOPIADP
|
|
{
|
|
public int value__;
|
|
[OriginalNameAttribute(Name = "CmdCityShopInfoScNotify", PreferredAlias = True)]
|
|
public const FGEMFOPIADP PNKKDKOGMOM = 1533;
|
|
}
|
|
```
|
|
Even if the class name (`FGEMFOPIADP`) and field constant (`PNKKDKOGMOM`) are obfuscated, **the metadata attribute preserves the original protocol identifier `CmdCityShopInfoScNotify` and its integer value `1533`!**
|
|
|
|
---
|
|
|
|
## 3. Codebase Architecture & Data Models
|
|
|
|
The project is structured into two source files:
|
|
|
|
```
|
|
Dumpcs2Protobuf/
|
|
├── Proto.cs # Abstract Syntax Tree (AST) data models
|
|
└── MainApp.cs # File chunker, regex extractors, type mapper, and proto serializer
|
|
```
|
|
|
|
### 3.1 AST Data Models (`Proto.cs`)
|
|
```csharp
|
|
namespace DumpFileParser;
|
|
|
|
public class ProtoMessage
|
|
{
|
|
public string protoName = "";
|
|
public List<string> importNameList = new();
|
|
public List<ProtoField> fieldList = new();
|
|
public List<OneOf> oneOfList = new();
|
|
}
|
|
|
|
public class ProtoField
|
|
{
|
|
public string fieldName = "";
|
|
public string fieldType = "";
|
|
public string? mapKey;
|
|
public string? mapValue;
|
|
public bool isRepeated;
|
|
public uint val; // Protobuf wire tag number
|
|
}
|
|
|
|
public class OneOf
|
|
{
|
|
public string oneOfName = "";
|
|
public List<ProtoField> fieldList = new();
|
|
}
|
|
|
|
public class ProtoEnum
|
|
{
|
|
public string enumName = "";
|
|
public Dictionary<string, int> valDict = new();
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 4. Reverse Engineering Algorithms & Implementation (`MainApp.cs`)
|
|
|
|
### 4.1 Chunking & Assembly Filtering
|
|
`dump.cs` files can be hundreds of megabytes in size. `Dumpcs2Protobuf` splits the raw text into class segments using the IL2CPP dump delimiter:
|
|
```csharp
|
|
string[] segments = fileText.Split(new string[] { "// AssemblyIndex:" }, StringSplitOptions.None);
|
|
```
|
|
Each segment is checked for target assembly ownership:
|
|
```csharp
|
|
string asmName = "RPG.Network.Proto.dll";
|
|
if (!segment.Split("\n")[2].Trim().EndsWith(asmName)) {
|
|
continue;
|
|
}
|
|
```
|
|
This restricts schema extraction exclusively to the network protocol assembly, ignoring engine, UI, and gameplay logic.
|
|
|
|
### 4.2 Field Tag Extraction (`GetFieldNums`)
|
|
Field numbers are extracted exclusively between `// Fields` and `// Properties` to avoid picking up arbitrary constants:
|
|
|
|
```csharp
|
|
static List<uint> GetFieldNums(string[] lines) {
|
|
List<uint> result = new();
|
|
int fieldsStartIndex = -1;
|
|
int propertiesStartIndex = -1;
|
|
|
|
for (int i = 0; i < lines.Length; i++) {
|
|
if (lines[i].Trim().StartsWith("// Fields")) fieldsStartIndex = i + 1;
|
|
else if (lines[i].Trim().StartsWith("// Properties")) {
|
|
propertiesStartIndex = i;
|
|
break;
|
|
}
|
|
}
|
|
|
|
if (fieldsStartIndex >= 0 && propertiesStartIndex > fieldsStartIndex) {
|
|
for (int i = fieldsStartIndex; i < propertiesStartIndex; i++) {
|
|
string line = lines[i].Trim();
|
|
var match = Regex.Match(line, @"public\s+const\s+int\s+(\w+)\s*=\s*(\d+);");
|
|
if (match.Success && uint.TryParse(match.Groups[2].Value, out uint fieldNum)) {
|
|
result.Add(fieldNum);
|
|
}
|
|
}
|
|
}
|
|
return result;
|
|
}
|
|
```
|
|
|
|
### 4.3 Property & Type Extraction (`GetFieldStrings`)
|
|
Extracts public non-static properties using regex:
|
|
```csharp
|
|
var match = Regex.Match(trimmedLine,
|
|
@"public\s+([\w\.]+(\s*<.*?>)?)\s+(\w+)\s*\{( get;(\s+set;)?| set;(\s+get;)?| get;| set;) \}");
|
|
```
|
|
|
|
#### Type Mapping (`parseType`):
|
|
Maps C# CLR types to `proto3` scalar types:
|
|
|
|
| C# CLR Type | Protobuf Wire Type |
|
|
| :--- | :--- |
|
|
| `int` | `int32` |
|
|
| `uint` | `uint32` |
|
|
| `long` | `int64` |
|
|
| `ulong` | `uint64` |
|
|
| `string` | `string` |
|
|
| `ByteString` | `bytes` |
|
|
| `RepeatedField<T>` | `repeated <T>` |
|
|
| `MapField<K, V>` | `map<K, V>` |
|
|
| Custom Class (`Namespace.Type`) | Statically stripped to `Type` |
|
|
|
|
### 4.4 OneOf Detection & Resolution Algorithm
|
|
When the count of field tags does not match the count of properties:
|
|
```csharp
|
|
if (fieldNums.Count != fieldStrings.Count) {
|
|
// The trailing property is the OneOf discriminator enum accessor!
|
|
string trimmed = fieldStrings.Last().Replace(" { get; }", "").Replace("public ", "");
|
|
oneOfName = trimmed.Split(" ")[1];
|
|
oneOfEnumName = trimmed.Split(" ")[0];
|
|
blackListEnumNames.Add(oneOfEnumName.Split(".").Last());
|
|
oneofEnum = FindAndParseEnum(oneOfEnumName, segments);
|
|
}
|
|
```
|
|
1. It identifies the discriminator property (e.g. `public ChoiceOneofCase ChoiceCase { get; }`).
|
|
2. It locates and parses the enum declaration (`ChoiceOneofCase`) across all segments.
|
|
3. For each field, if its wire tag number matches an entry in `oneofEnum.valDict`, it is placed inside `oneof <oneofName> { ... }`.
|
|
4. The discriminator enum is added to `blackListEnumNames` so it is not emitted as a standalone top-level enum in the `.proto` file.
|
|
|
|
### 4.5 De-Obfuscating Enums via `OriginalNameAttribute`
|
|
In `parseEnum`:
|
|
```csharp
|
|
if (lines[i-1].Contains("[OriginalNameAttribute(Name = \"")) {
|
|
string upperLine = lines[i-1];
|
|
name = upperLine.Split("[OriginalNameAttribute(Name = \"").Last().Split("\"").First();
|
|
} else {
|
|
name = line.Split($"{enumName} ").Last().Split("=").First().Trim();
|
|
}
|
|
```
|
|
If an enum constant is obfuscated (e.g. `IHJPPFIKELH`), it inspects the preceding line for `[OriginalNameAttribute]`, effortlessly restoring the original identifier (`CmdShopTypeNone = 0;`).
|
|
|
|
---
|
|
|
|
## 5. Output Serialization
|
|
|
|
`Dumpcs2Protobuf` serializes the AST into clean `proto3` syntax:
|
|
|
|
```protobuf
|
|
// Extracted from dump.cs using Dumpcs2Protobuf
|
|
syntax = "proto3";
|
|
|
|
message SceneEntityInfo {
|
|
uint32 entity_id = 1;
|
|
uint32 group_id = 2;
|
|
uint32 inst_id = 3;
|
|
MotionInfo motion = 4;
|
|
oneof entity {
|
|
ActorInfo actor = 5;
|
|
NpcInfo npc = 6;
|
|
MonsterInfo monster = 7;
|
|
}
|
|
}
|
|
|
|
enum CmdCityShopInfoScNotify {
|
|
CmdShopTypeNone = 0;
|
|
CmdCityShopInfoScNotify = 1533;
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 6. The Complete 6-Tool Unified Star Rail Protocol Toolchain
|
|
|
|
We can now map how all 6 repositories analyzed in this session interact to form an end-to-end, automated protocol engineering pipeline:
|
|
|
|
```
|
|
+----------------------------------------------------------------------------------------------------+
|
|
| THE COMPLETE PROTOCOL REVERSE-ENGINEERING LIFECYCLE |
|
|
+----------------------------------------------------------------------------------------------------+
|
|
| |
|
|
| [ Step 1: Binary & Metadata Extraction ] |
|
|
| Tools: Firefly-Static-Parser (Offline) OR honkai-dumper (In-Process) |
|
|
| Action: Decrypts GameAssembly.dll + global-metadata.dat |
|
|
| Output: dump.cs (All C# class definitions) + methods.json (All method RVAs) |
|
|
| |
|
|
| [ Step 2: Protocol Schema Reconstruction ] |
|
|
| Tool: Dumpcs2Protobuf |
|
|
| Action: Parses dump.cs (RPG.Network.Proto.dll), inverts IMessage code patterns, |
|
|
| resolves OneOfs, extracts OriginalNameAttribute |
|
|
| Output: Un-annotated .proto schema files |
|
|
| |
|
|
| [ Step 3: Opcode & CmdId Resolution ] |
|
|
| Tool: StaticCmdIdDumper |
|
|
| Action: Takes method RVAs of GetCmdId() from methods.json -> reads 4 bytes (66 B8 imm16) |
|
|
| Output: PacketIds.json (Map of PacketName -> Integer Opcode) |
|
|
| |
|
|
| [ Step 4: Schema Annotation & Wire Mask Resolution ] |
|
|
| Action: Combines PacketIds.json with .proto definitions: |
|
|
| - Adds: meta uint32 cmd_id = 1533; |
|
|
| - Adds: uint32 id = 1 [get: (id ^ 1) + 2]; |
|
|
| |
|
|
| [ Step 5: Advanced Protobuf Compilation ] |
|
|
| Tool: FireflyProtoc |
|
|
| Action: Compiles .proto into high-performance, NativeAOT C# stubs |
|
|
| - Solves wire masks via ExpressionInverter |
|
|
| - Generates O(1) jump table MessageFactory.Create(cmdId) |
|
|
| |
|
|
| [ Step 6: Server Implementation & Routing ] |
|
|
| Tools: March7thHoney (.NET 10) / LunarCore (Java) / GaC-hkrpg-go (Go) |
|
|
| Action: Instant zero-reflection packet routing on KCP network layer |
|
|
| |
|
|
+----------------------------------------------------------------------------------------------------+
|
|
```
|
|
|
|
---
|
|
|
|
## 7. Technical Critique & Production Blueprint for `GaC-hkrpg-go`
|
|
|
|
While `Dumpcs2Protobuf` is concise and functional, a production-grade schema generator for `GaC-hkrpg-go` should address several structural limitations:
|
|
|
|
### 7.1 Limitations of the Regex Implementation
|
|
1. **Single Output File**: Dumps everything into a single monolithic `output.proto` file, leading to circular dependency and compilation issues.
|
|
2. **Missing Imports**: Does not generate `import "OtherMessage.proto";` statements required when messages reference other message types.
|
|
3. **Rigid Field Ordering Assumption**: Relies on field numbers appearing sequentially in the exact line order of properties. If the decompiler or compiler re-orders field definitions, tag pairing can desynchronize.
|
|
|
|
### 7.2 Production Go Implementation Blueprint
|
|
A robust schema extractor in Go for `GaC-hkrpg-go` can parse the C# definitions and emit properly modularized proto files:
|
|
|
|
```go
|
|
package proto_generator
|
|
|
|
import (
|
|
"fmt"
|
|
"os"
|
|
"path/filepath"
|
|
"strings"
|
|
)
|
|
|
|
type ProtoField struct {
|
|
Name string
|
|
Type string
|
|
Tag uint32
|
|
IsRepeated bool
|
|
IsMap bool
|
|
MapKey string
|
|
MapValue string
|
|
}
|
|
|
|
type ProtoMessage struct {
|
|
Name string
|
|
CmdId uint32
|
|
Fields []ProtoField
|
|
Imports map[string]bool
|
|
OneOfs map[string][]ProtoField
|
|
}
|
|
|
|
func (m *ProtoMessage) GenerateProtoFile(outDir string) error {
|
|
var sb strings.Builder
|
|
sb.WriteString("syntax = \"proto3\";\n\n")
|
|
|
|
// 1. Emit Imports
|
|
for imp := range m.Imports {
|
|
sb.WriteString(fmt.Sprintf("import \"%s.proto\";\n", imp))
|
|
}
|
|
if len(m.Imports) > 0 {
|
|
sb.WriteString("\n")
|
|
}
|
|
|
|
// 2. Emit Message Block
|
|
sb.WriteString(fmt.Sprintf("message %s {\n", m.Name))
|
|
|
|
// Option: Meta CmdId
|
|
if m.CmdId > 0 {
|
|
sb.WriteString(fmt.Sprintf("\t// CmdId: %d\n", m.CmdId))
|
|
}
|
|
|
|
// Normal Fields
|
|
for _, f := range m.Fields {
|
|
if f.IsRepeated {
|
|
sb.WriteString(fmt.Sprintf("\trepeated %s %s = %d;\n", f.Type, f.Name, f.Tag))
|
|
} else if f.IsMap {
|
|
sb.WriteString(fmt.Sprintf("\tmap<%s, %s> %s = %d;\n", f.MapKey, f.MapValue, f.Name, f.Tag))
|
|
} else {
|
|
sb.WriteString(fmt.Sprintf("\t%s %s = %d;\n", f.Type, f.Name, f.Tag))
|
|
}
|
|
}
|
|
|
|
// OneOf Blocks
|
|
for oneOfName, fields := range m.OneOfs {
|
|
sb.WriteString(fmt.Sprintf("\toneof %s {\n", oneOfName))
|
|
for _, f := range fields {
|
|
sb.WriteString(fmt.Sprintf("\t\t%s %s = %d;\n", f.Type, f.Name, f.Tag))
|
|
}
|
|
sb.WriteString("\t}\n")
|
|
}
|
|
|
|
sb.WriteString("}\n")
|
|
|
|
filePath := filepath.Join(outDir, m.Name+".proto")
|
|
return os.WriteFile(filePath, []byte(sb.String()), 0644)
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 8. Compliance with Repository Rules (`AGENTS.md`)
|
|
|
|
- **Explicit Protocol Revision & Schema Integrity**: Reconstructing schemas directly from client assemblies prevents guesswork or arbitrary schema patching.
|
|
- **No Hardcoded Machine Paths**: Operates dynamically from CLI arguments and local files.
|
|
- **Zero Runtime Interference**: Operates strictly on text and source files without runtime injection or client hooks.
|