10 May 2026/BY Robert Vähhi

Building a Wasm-in-Wasm Virtualizer (with JIT decrypted paged memory)

Build a Wasm-in-Wasm VM that turns readable code into a hardened binary using JIT page encryption to stop memory scrapers.

14 min read

Background

This one took a while. Building the engine was the shorter half of the work; writing it up took longer, because most of these ideas only make sense once you can watch them move. So the heavier concepts here come with custom interactive visuals, and the post is worth reading in detailed mode rather than skimming.

Click any underlined term as you go. Nothing redirects and nothing opens a new tab: the term expands in place, with a definition and a longer explanation under it.

Intro

WebAssembly was built for portability and raw speed, not for keeping secrets, a property we lean into when reverse-engineering Wasm payloads in TrustSig Lab. If you want the logic protected, the route is to compile it down to a custom, undocumented bytecode and ship a small interpreter inside the binary to run it.

When you compile C, C++, or Rust to native machine code, the compiler strips away a lot of the code's structure. The final assembly is tied to the physical hardware, which is what makes it tough to read. Compiling to WebAssembly does the opposite: the binary keeps a highly structured, strictly typed .

Anyone with a normal browser debugger can grab a .wasm file, run it through a standard decompiler like WABT, and read an almost perfect version of the original code. If your app sends sensitive crypto logic, DRM, game anti-cheat, or validation checks to the client, shipping plain WebAssembly is basically the same as shipping open-source code, the same threat model that drives our Web3 bot protection guide.

What We Will Be Doing

We're going to build a WebAssembly-in-WebAssembly virtualization engine from scratch. Rather than dropping a wall of code on you, we'll map out the concepts first, sketch the execution flow, trace how memory works, then write the code piece by piece.

The End Result

By the end you'll have a working code virtualization pipeline. Your compiler takes a normal binary, pulls out a target function, translates it into a custom instruction set, encrypts it, and injects a tiny virtual machine back into the file to run the function, without the browser knowing.

Prerequisites

A few tools and some background, to follow along with the build phases.

A recent Rust toolchain, to compile the interpreter.
The wasm32-unknown-unknown compilation target, installed via rustup.
The walrus crate in your project dependencies, for syntax tree manipulation.
A working grasp of systems programming, bitwise math, and raw memory manipulation.

Designing the Virtual Instruction Set Architecture

Before any compiler code, we need to define the language our virtual machine reads: an (ISA).

Conceptualizing the Execution Model

Our virtual machine runs as a .

Standard Wasm is also a stack machine. An instruction like 'add' never says which variables to add. It assumes the two numbers it needs are sitting on top of the virtual stack, pops them off, adds them, and pushes the result back.

Adding two numbers in our setup looks like this.

Interactive Stack Engine

Initial State

Stack contains numbers waiting to be processed.

Stack: [ ..., 5, 10 ]

Stack

Matching Wasm's model keeps the translation into our bytecode mechanical and easy to reason about. The numbers are where we diverge: the opcodes get randomised, so anyone reading the payload finds an arbitrary byte sequence that only our interpreter knows how to decode, not standard Wasm bytes.

Defining the Virtual Opcodes

We define the allowed machine instructions as a Rust enum. Everything else in the virtual processor builds on it.

// src/isa.rs

#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
#[repr(u8)]
pub enum Opcode {
    Push = 0x01,
    Pop = 0x02,
    Dup = 0x03,
    Swap = 0x04,
    
    // Arithmetic
    Add = 0x10,
    Sub = 0x11,
    Mul = 0x12,
    DivS = 0x13,
    
    // Bitwise Logic
    BitAnd = 0x20,
    BitOr = 0x21,
    BitXor = 0x22,
    
    // Memory and Registers
    LoadReg = 0x30,
    StoreReg = 0x31,
    
    // Control Flow
    Jmp = 0x50,
    Jz = 0x51,
    Exit = 0xFF,
}

Notice the register instructions, LoadReg and StoreReg. Arithmetic is fine on the stack, but real functions need local variables to hold state, loop counters and the like, and those need to live somewhere other than the stack we are constantly pushing and popping. A static array of virtual registers covers it.

Jmp and Jz (Jump if Zero) are there for control flow. Without them there are no if-statements and no while-loops inside the protected code.

The Dispatch Mapping Layer

We need a reliable way to turn a raw u8 byte back into our typed Opcode enum at runtime.

impl Opcode {
    pub fn from_u8(v: u8) -> Option<Self> {
        match v {
            0x01 => Some(Opcode::Push),
            0x10 => Some(Opcode::Add),
            0x20 => Some(Opcode::BitAnd),
            0x30 => Some(Opcode::LoadReg),
            0x50 => Some(Opcode::Jmp),
            0xFF => Some(Opcode::Exit),
            // repetitive match arms omitted for brevity
            _ => None,
        }
    }
}

std::mem::transmute would cast a u8 straight into the enum and save a lot of typing. It is also the wrong tool inside a VM. Tamper with a single byte of the bytecode and transmute hands you undefined behavior, crashing the module. The explicit match layer means the execution loop only ever handles opcodes it understands.

Chapter 1 Outro

The instruction set is mapped out. None of it lines up with the WebAssembly spec any more, so a standard decompiler has nothing to work from. In a production obfuscation engine you would generate the byte assignments inside from_u8 randomly on every build, which takes signature-based analysis off the table. Next, the memory model that runs this instruction set.

Architecting the VM State and Memory Model

Our interpreter has to be self-contained, with no dynamic memory allocators at all.

Bypassing the Host Allocator

The VM allocates all of its memory statically at compile time, inside the Wasm binary's .data segment.

Dynamic Memory Overhead

0 Bytes

Required runtime heap allocations for the Virtual Machine

Dropping a virtual machine into an already-compiled Wasm module comes with a hard memory constraint. The host module has its own memory allocator (wee_alloc or malloc) managing the heap.

Bring a second dynamic allocator and you risk overlapping and trashing the host's live memory. A full allocator would also bloat the final binary and make it stand out.

Provisioning Static Execution Arrays

With the stack and registers set up as raw, contiguous static arrays, booting the interpreter costs zero clock cycles at runtime.

// interpreter/src/lib.rs

#[inline(always)]
fn abort() -> ! {
    core::arch::wasm32::unreachable();
}

static mut STACK:[i64; 2048] =[0; 2048];
static mut REGS:[i64; 512] =[0; 512];
static mut SP: usize = 0;

The physical memory layout inside the final Wasm module will look something like this:

Linear Memory Layout

Unrelated Host App Data

Variable

STACK [2048 x i64]

VM Exclusive16 KB

REGS [512 x i64]

VM Exclusive4 KB

GLOBAL_SESSION_KEY

VM Exclusive1 Byte

Bytecode Array

Encrypted PayloadPayload Size

I used 64-bit integers (i64) everywhere, so the same slot holds a 32-bit int, a 64-bit int or a float by its raw bits.

static mut in Rust normally demands unsafe blocks, because mutating a global across threads is a data race. Wasm runs single-threaded by default and our loop is synchronous, so the hazard the rule exists for cannot occur here.

Interfacing with Host Memory APIs

We expose a couple of foreign function interfaces (FFIs) so the host module can pass execution arguments and the session key into our static arrays.

static mut GLOBAL_SESSION_KEY: u8 = 0;

#[no_mangle]
pub unsafe extern "C" fn vm_set_session_key(key: u32) {
    GLOBAL_SESSION_KEY = key as u8;
}

#[no_mangle]
pub unsafe extern "C" fn vm_set_reg(idx: u32, val: i64) {
    if (idx as usize) < 512 {
        *core::ptr::addr_of_mut!(REGS)
            .cast::<i64>()
            .add(idx as usize) = val;
    }
}

The bytecode is encrypted, so the VM needs the decryption key before it runs anything. Hardcoding the key into the interpreter is out, because it changes on every build to defeat fingerprinting. That leaves a global for the key, plus an exported function (vm_set_session_key) for the host to pass it in.

A protected function that cannot take arguments is not much use. When the host module intercepts a call, it has to pull the parameters out of the browser's call and push them into our VM's registers.

Exporting vm_set_reg does that, letting the host populate our register array before the main loop starts. We use core::ptr::addr_of_mut! because modern Rust prefers explicit raw pointers over references to mutable statics.

Writing to the register array by raw pointer keeps the boundary narrow. The VM sees nothing of the host and works only on data we hand it.

Chapter 2 Outro

The memory model is done and it allocates nothing. Leaning on static Wasm memory keeps the engine fast and the footprint small. Registers, stack and APIs are in place, so next comes the processor loop.

The Fetch-Decode-Execute Loop and Lazy Decryption

State Tracking and Sliding Windows

The execution state object carries a sliding window and an , decrypting code lazily as execution moves through it.

Exposed Plaintext

256 Bytes

Maximum decrypted instruction window exposed in memory at any time

// interpreter/src/lib.rs

const PAGE_SIZE: usize = 256;

struct VmState {
    ptr: u32,
    len: usize,
    pc: usize,
    session_key: u8,
    current_page_id: i32,
    ves: [u8; PAGE_SIZE],
}

Leaving our bytecode unencrypted in memory defeats the point of the VM. Cheat Engine and automated memory dumpers both scrape the Wasm heap, and plaintext bytecode sitting there is the end of the obfuscation.

So the plaintext instructions have to exist in memory only while the processor needs them, and disappear after.

We slice the encrypted bytecode into 256-byte pages. When the program counter reaches a new page, the VM pauses, decrypts those 256 bytes with the session key, and loads them into a buffer. When it leaves the page, that plaintext is overwritten by the next one.

JIT Sliding Window

Volatile Plaintext Buffer

[Page 0 wiped from memory]
[Decoded instructions for Page 1 active]

The struct tracks where the encrypted code lives in the host's memory (ptr), how far along we are in our virtual loop (pc), and it holds the decrypted bytes (ves, for Virtual Execution Segment).

The JIT Decryption Routine

When we cross into a new page, the VM decrypts that chunk of bytes and copies it into the volatile virtual execution segment.

impl VmState {
    unsafe fn decrypt_page(&mut self, page_id: u32) {
        let start_addr = page_id as usize * PAGE_SIZE;
        
        for i in 0..PAGE_SIZE {
            if start_addr + i < self.len {
                // Calculate physical memory address inside the host Wasm
                let addr = self.ptr + (start_addr + i) as u32;
                let encrypted_byte = *(addr as *const u8);
                
                // Decryption occurs entirely inside local volatile state
                self.ves[i] = encrypted_byte ^ self.session_key;
            } else {
                // Pad unused memory to obscure the exact file length footprint
                self.ves[i] = 0;
            }
        }
        
        self.current_page_id = page_id as i32;
    }
}

Freeze the browser and scrape memory with this in place, and whichever moment you picked, there is at most 256 bytes of plaintext to take. Bulk-analysing the payload stops being practical.

Execution and Opcode Dispatch

This is the core of the interpreter: moving the program counter forward, fetching bytes through the JIT boundary check, and running the math on our stack.

    unsafe fn fetch_u8(&mut self) -> u8 {
        if self.pc >= self.len { return 0xFF; }
        
        let page_id = (self.pc / PAGE_SIZE) as u32;
        let offset = self.pc % PAGE_SIZE;
        
        if self.current_page_id != page_id as i32 {
            self.decrypt_page(page_id);
        }
        
        let val = self.ves[offset];
        self.pc += 1;
        val
    }

    unsafe fn fetch_i64(&mut self) -> i64 {
        let mut b =[0u8; 8];
        for item in &mut b { *item = self.fetch_u8(); }
        i64::from_le_bytes(b)
    }

#[no_mangle]
pub unsafe extern "C" fn vm_exec(ptr: u32, len: usize) -> i64 {
    let base_sp = SP; 
    let mut state = VmState {
        ptr, len, pc: 0, session_key: GLOBAL_SESSION_KEY,
        current_page_id: -1, ves: [0u8; PAGE_SIZE],
    };

    loop {
        let op = state.fetch_u8();
        match op {
            0x01 => { // Push logic
                let val = state.fetch_i64();
                if SP < 2048 {
                    *core::ptr::addr_of_mut!(STACK).cast::<i64>().add(SP) = val;
                    SP += 1;
                }
            }
            0x10 => { // Addition logic
                if SP >= 2 {
                    SP -= 1;
                    let b = *core::ptr::addr_of!(STACK).cast::<i64>().add(SP);
                    SP -= 1;
                    let a = *core::ptr::addr_of!(STACK).cast::<i64>().add(SP);
                    *core::ptr::addr_of_mut!(STACK).cast::<i64>().add(SP) = a.wrapping_add(b);
                    SP += 1;
                }
            }
            0x30 => { // Load Register logic
                let idx = state.fetch_i64() as usize;
                if idx < 512 {
                    let val = *core::ptr::addr_of!(REGS).cast::<i64>().add(idx);
                    *core::ptr::addr_of_mut!(STACK).cast::<i64>().add(SP) = val;
                    SP += 1;
                }
            }
            0x50 => { // Unconditional Jump logic
                let target_pc = state.fetch_i64() as usize;
                if target_pc < state.len { state.pc = target_pc; } else { abort(); }
            }
            0xFF => { // Exit logic
                let ret = if SP > base_sp {
                    SP -= 1;
                    *core::ptr::addr_of!(STACK).cast::<i64>().add(SP)
                } else { 0 };
                SP = base_sp;
                return ret; 
            }
            _ => abort(), 
        }
    }
}

Some instructions need larger parameters. Push has to know the 64-bit number it puts on the stack, so we add a helper that fetches eight consecutive bytes and rebuilds them into an integer. Because it goes through fetch_u8, the multi-byte fetcher inherits the page-boundary checks and the lazy decryption without us writing either again.

The logic inside the match follows the same a + b stack sequence from earlier. It keeps its own context, handles the jumps that if statements compile into, and returns control to the host only on the Exit opcode.

Chapter 3 Outro

The virtual machine works. It runs isolated from the host, allocates nothing, does its math on a stack, and keeps a 256-byte rolling JIT window between a memory dumper and the payload. What it has no way to get yet is code to run, which is the compiler layer that generates those encrypted payloads.

Compiling Intermediate Representation to Custom Bytecode

Parsing the WebAssembly Module with Walrus

We'll use the Walrus crate to parse the target binary and set up a dynamic buffer for compilation.

// src/compiler.rs

use walrus::ir::{Instr, Value};
use walrus::LocalFunction;
use byteorder::{ByteOrder, LittleEndian};
use rand::Rng;

pub struct Compiler {
    pub bytecode: Vec<u8>,
}

The compiler layer reads standard Wasm logic out of an existing file and translates it into our undocumented format.

This is a backend compiler. There is no human-readable Rust or C++ source to parse: we take a pre-compiled Wasm binary, extract the instructions, and translate them down into our own bytes.

Walrus converts the raw Wasm file into an (IR) we can iterate over in Rust. We need a struct to hold the output bytes as we generate them. As we walk the original app's logic, we push our bytes into this vector.

Translating Standard Instructions to Custom Bytecode

Our compiler walks the function's IR recursively, flattening the typed Wasm logic into our randomized byte format.

impl Compiler {
    pub fn compile_paged(&mut self, func: &LocalFunction) -> (Vec<u8>, u8) {
        let entry_id = func.entry_block();
        let block = func.block(entry_id);
        
        for (instr, _) in block.instrs.iter() {
            self.compile_instr(instr);
        }
        
        self.bytecode.push(0xFF); // Forcefully append the exit opcode manually
        self.encrypt_and_finalize() // Execute cryptographic pass
    }
}

    fn compile_instr(&mut self, instr: &Instr) {
        match instr {
            // Translating constant variables
            Instr::Const(c) => {
                if let Value::I32(v) = c.value {
                    self.bytecode.push(0x01); // Custom Push Opcode
                    let mut buf =[0u8; 8];
                    LittleEndian::write_i64(&mut buf, v as i64);
                    self.bytecode.extend_from_slice(&buf);
                }
            }
            
            // Translating arithmetic binary operations
            Instr::Binop(op) => {
                match op.op {
                    walrus::ir::BinaryOp::I32Add => self.bytecode.push(0x10), // Custom Add Opcode
                    walrus::ir::BinaryOp::I32Sub => self.bytecode.push(0x11), // Custom Sub Opcode
                    _ => unimplemented!("Operator not natively mapped"),
                }
            }
            
            // Translating Variable reading
            Instr::LocalGet(l) => {
                self.bytecode.push(0x30); // Custom LoadReg Opcode
                let mut buf = [0u8; 8];
                LittleEndian::write_i64(&mut buf, l.local.index() as i64);
                self.bytecode.extend_from_slice(&buf);
            }
            
            // Translating Variable writing
            Instr::LocalSet(l) => {
                self.bytecode.push(0x31); // Custom StoreReg Opcode
                let mut buf = [0u8; 8];
                LittleEndian::write_i64(&mut buf, l.local.index() as i64);
                self.bytecode.extend_from_slice(&buf);
            }
            
            _ => unimplemented!("Instruction architecture not mapped"),
        }
    }

We need a method that takes a Walrus function (LocalFunction), walks every instruction in its block, and has the translation layer emit the matching virtual bytecode. With the instructions translated, we append our Exit opcode so the VM knows where to stop.

Taking the entry block gives us the root of the function's execution path, so none of the logic the original developer wrote gets left behind.

The translation is a match on the Walrus Instr object against the patterns we know, emitting the opcodes we defined back in Chapter 1.

If we hit a hardcoded number, we emit a push opcode followed by those 8 bytes of data. If we hit a local variable access, we emit a register load opcode followed by the variable's virtual register index.

The process is destructive, and the original context does not survive it. Wasm's rich typing system and readable operations come out the other side as a flat array of bytes.

Cryptographic Obfuscation Pass

Finally, we run the bytecode array through a bitwise XOR cipher using a uniquely generated session key to hide its logic.

    fn encrypt_and_finalize(&mut self) -> (Vec<u8>, u8) {
        let mut rng = rand::thread_rng();
        let session_key = rng.gen::<u8>(); // Generate isolated session key
        
        // Encrypt everything iteratively 
        for byte in &mut self.bytecode {
            *byte ^= session_key;
        }
        
        (self.bytecode.clone(), session_key)
    }

The plaintext bytecode has to be encrypted before it ships. Left in the clear inside the final Wasm file, a patient reverse engineer can eventually map out our math, custom ISA or not.

We generate a random key with Rust's rand crate, loop over the bytecode vector, and apply our bitwise XOR cipher. The function returns both the encrypted payload and the key, so the injector can deploy them into the host module.

One pass is enough to take the structured logic down to high-entropy noise, for a reverse engineer and a malware scanner alike.

Chapter 4 Outro

The compiler bridge is finished. A standard Wasm instruction like LocalGet can now go through the IR, come out as our virtual LoadReg opcode with its arguments appended, and get encrypted on the way. The last hurdle is wiring that ciphertext back into the original .wasm file.

Abstract Syntax Tree Rewriting and VM Injection

Erasing the Original AST

The injector clears the original execution instructions out of the host binary's function.

// src/injector.rs

use walrus::{Module, FunctionId, DataId, MemoryId};

pub fn replace_body_with_thunk(
    module: &mut Module,
    func_id: FunctionId,
    vm_exec_import: FunctionId,
    vm_set_reg_import: FunctionId,
    vm_set_session_key_import: FunctionId,
    bytecode_ptr: i32,
    bytecode_len: i32,
    data_id: DataId,
    mem_id: MemoryId,
    session_key: u8,
) {
    let func = module.funcs.get_mut(func_id);
    let local_func = func.kind.unwrap_local_mut();
    
    let args = local_func.args.clone();
    let entry = local_func.entry_block();
    
    // Destroy the original application logic entirely.
    local_func.block_mut(entry).instrs.clear();

If a JavaScript frontend expects to call an exported Wasm function named calculate_secure_hash, deleting it from the file is not an option. The integration breaks and the web app goes down with it.

So we overwrite the guts of calculate_secure_hash instead. Its original syntax tree goes, the proprietary algorithm goes with it, and in their place sits a small adapter called a .

The injected Thunk's flow is short:

Execution Flow

Host App Callscalculate_secure_hash(param1, param2)

Thunk InterceptsWrites arguments into Virtual REGS

Thunk Sets KeySession Key provided for runtime decryption

Interpreter InvokedExecutes ciphertext logic safely

Interpreter ReturnsYields mathematical result to Thunk

Thunk ReturnsRoutes result back to Host App

At this point in the compile process, calling the function from JS does nothing and returns empty. The original logic is gone from the standard execution path, so there is nothing left there to decompile.

Allocating the Bytecode to Linear Memory

The injector writes new bulk-memory operations into that empty block to deploy the encrypted payload.

    let mut builder = local_func.builder_mut().func_body();
    
    // Deploy the encrypted bytecode from the passive data segment 
    // into active Wasm linear memory dynamically
    builder
        .i32_const(bytecode_ptr)
        .i32_const(0)
        .i32_const(bytecode_len)
        .memory_init(mem_id, data_id);
        
    // Securely set the isolated decryption session key for the VM
    builder
        .i32_const(session_key as i32)
        .call(vm_set_session_key_import);

WebAssembly lets us embed raw bytes inside passive .data segments. A passive segment sits there like a zip file until something unpacks it, so we have to tell the host's memory manager to move our encrypted bytecode out of the file and into active linear memory, where the interpreter expects to find it.

We initialize a function builder in Walrus to start writing the replacement Thunk, deploy the payload with the standard Wasm memory.init instruction, and trigger the VM's key initialization hook straight after.

Constructing the Execution Thunk

The adapter loop translates the host arguments, starts the virtual context, and returns the result.

    for (idx, &arg_local) in args.iter().enumerate() {
        builder.i32_const(idx as i32);         // Argument index -> Virtual Register ID
        builder.local_get(arg_local);          // Actual Argument Value
        
        // Promote 32-bit parameters to 64-bit to match our VM architecture
        builder.unop(walrus::ir::UnaryOp::I64ExtendI32S); 
        
        // Pass the data safely through the isolation barrier
        builder.call(vm_set_reg_import);
    }

    // Trigger the virtual execution loop
    builder
        .i32_const(bytecode_ptr)
        .i32_const(bytecode_len)
        .call(vm_exec_import);
        
    // Format the VM output back into the expected Host application format
    builder.unop(walrus::ir::UnaryOp::I32WrapI64); 
}

The original function took its parameters (ints, pointers) from the external JS context. Those have to reach our virtual machine, or the logic inside it has nothing to work on.

We loop over the arguments the function originally expected. For each one, we push its sequential index (which maps to a Virtual Register ID) and its raw value onto the host stack, cast it up to a 64-bit integer using I64ExtendI32S, and invoke the interpreter's register API.

The VM now holds the incoming call's data. Last step is to call the main execution loop with the memory address and length of the encrypted bytecode.

When the VM finishes working through the JIT sliding window, it returns a 64-bit result, which we cast back down to match the original 32-bit function signature.

Chapter 5 Outro

The final binary exports the same functions it did before, so nothing on the JavaScript side has to change or even know that anything did. Call one, and the engine goes into our thunk, populates the virtual context, and runs the original logic through the interpreter.

Final Conclusion

We've built a WebAssembly-in-WebAssembly virtualization engine, starting from how much a standard Wasm binary gives away. Ship logic in a plain .wasm file and you have shipped something close to source.

So we designed our own over a randomized numerical layout, then built the machine for it in Rust with no allocator anywhere in it, static arrays and raw pointers only.

The JIT sliding decryption window is what keeps the payload out of a memory dump. Whoever takes one gets a 256-byte slice of the program, and that is not enough for the automated reverse-engineering tools to work on.

Walrus gave us the backend compiler that rips the original out and flattens typed Wasm instructions into encrypted bytes. The injection pipeline then puts a where the original function was, translating inputs and outputs between the standard environment and the VM.

Where to go from here?

This VM raises the cost of reverse-engineering sharply, but software protection is a never-ending arms race. If you want to take the architecture into real-world projects, these are the techniques to add to your compiler next.

Control Flow Flattening

Before converting to bytecode, rewrite the Walrus IR so all if statements and loops collapse into one large switch, which hides the real execution paths.

Opaque Predicates

Inject arithmetic into the bytecode that looks real but always evaluates to zero, so an attacker spends their time reversing thousands of instructions that do nothing.

Dynamic Key Provisioning

Instead of leaving the session key inside the Wasm data segment, have the host server stream the decryption key over a WebSocket at runtime. The Wasm file then can't be decompiled locally without a live, authenticated connection.

Stack these on top of the VM architecture we built today and your WebAssembly logic stays protected wherever the binary ends up. The same memory-isolation discipline shows up across every layer of our edge bot protection platform.