Reverse Engineering a Serial Validation Challenge

Overview

Reverse engineering analysis of a stripped x86-64 ELF binary (keygenme3) that validates email/serial pairs. Using Ghidra decompilation, I extracted the serial validation algorithm, manually verified the logic, and implemented a working keygen in C. Full decompiled pseudocode, manual calculations, keygen source code, and testing results available on GitHub.

Binary tested: keygenme3 (ELF 64-bit LSB executable, x86-64, stripped). The serial is derived from two mathematical operations on email characters: sum of (char × 8) and sum of (char² - 188), formatted as 4-digit hex pairs.

Why It Matters (Software Protection)

Software license validation is a common anti-piracy mechanism. Understanding how these systems work is crucial for:

• Malware analysis: Many malware samples use similar validation techniques to verify command-and-control credentials
• Security research: Identifying weak cryptographic implementations in commercial software
• Vulnerability assessment: Testing if license validation can be bypassed or predicted
• Learning reverse engineering: Practical exercise in decompilation, algorithm recovery, and reimplementation

Workflow

1. Initial Binary Inspection

First, I examined the binary metadata:

The "stripped" keyword meant no function names or debugging symbols—everything would have generic labels like FUN_00401234.

2. Ghidra Decompilation

• Loaded the binary into Ghidra and ran auto-analysis
• Located the entry point function (generic name entry)
• Followed calls to identify the main function and validation logic
• Renamed key functions for clarity:
  • FUN_00401234 → main
  • FUN_00401156 → serial_check

3. Algorithm Extraction

The decompiled serial_check function revealed the validation logic:

bool serial_check(char *email, char *serial)
{
  int iVar1;
  size_t sVar2;
  uint local_44;  // Accumulator 1
  uint local_40;  // Accumulator 2
  int local_3c;   // Loop index
  char local_38[24]; // Generated serial buffer
  
  local_44 = 0;
  local_40 = 0;
  local_3c = 0;
  
  while (true) {
    sVar2 = strlen(email);
    if (sVar2 <= (ulong)(long)local_3c) break;
    
    local_44 = local_44 + (int)email[local_3c] * 8;
    local_40 = (local_40 + (int)email[local_3c] * (int)email[local_3c]) - 0xbc;
    local_3c = local_3c + 1;
  }
  
  sprintf(local_38, "%04x-%04x", local_44, local_40);
  iVar1 = strcmp(serial, local_38);
  return iVar1 == 0;
}

Key Variables:

• local_44: Sum of (character × 8) for each email character
• local_40: Sum of (character² - 188) for each email character
• local_38: Buffer holding the generated serial in format "%04x-%04x"

4. Keygen Implementation

I reimplemented the algorithm in C to generate valid serials for any email:

#include <stdio.h>
#include <string.h>

int main(int argc, char *argv[]) {
    if (argc != 2) {
        printf("Usage: %s <email>\n", argv[0]);
        return 1;
    }
    
    char *email = argv[1];
    unsigned int local_44 = 0;
    unsigned int local_40 = 0;
    
    for (int i = 0; i < strlen(email); i++) {
        local_44 += email[i] * 8;
        local_40 += (email[i] * email[i]) - 0xbc;
    }
    
    printf("%04x-%04x\n", local_44, local_40);
    return 0;
}

5. Testing & Verification

Tools Used

Tools & Technologies

Result

Reverse engineering analysis of stripped x86-64 ELF binary (keygenme3): successfully decompiled with Ghidra, extracted serial validation algorithm revealing two mathematical operations (sum of char × 8 and sum of char² - 188), manually verified calculations for email "pr0cracker" (serial: 1f68-190d9), and implemented working keygen in C. Algorithm uses no randomness or server validation—serials are deterministically derived from email alone. Complete decompiled code, manual calculations, keygen implementation, and testing results available on GitHub.

Additional Resources