Reading assembly

The CPU doesn't read Python or C. It reads instructions encoded as bytes, and assembly is the readable form of those instructions, one instruction per line. So when you analyze a program whose source code you don't have, you read it as assembly, and the first skill you need is reading a small function and working out what it computes.

We write assembly here in Intel syntax, which is what analysis tools like Ghidra, IDA and x64dbg show. An instruction is written as its name, then the destination, then the source, and the result is stored in the destination. So the line add eax, edi means eax = eax + edi. Registers are small, fast slots inside the CPU. A function receives its first argument in edi and puts its result in eax before it returns. The names edi and eax are the lower halves of the full 64-bit registers rdi and rax, that is, their low 32 bits.

There's another syntax called AT&T, which Linux tools like objdump and gdb show by default. It reverses the order, so the source comes before the destination, and it puts % before registers and $ before numbers. Here's the same instruction in both:

Intel   add eax, edi      eax = eax + edi
AT&T    add %edi, %eax    eax = eax + edi

So if you open a file with objdump, add -M intel to see it in the syntax we read here.

• mov a, b copies, meaning a = b.
• add a, b adds, meaning a = a + b.
• sub a, b subtracts, meaning a = a - b.
• ret returns from the function, and the result is whatever is in eax.

Here's a small function as a compiler emits it. The comment after each semicolon tracks the value of eax:

f:
  mov  eax, edi   ; eax = x
  add  eax, eax   ; eax = x + x
  add  eax, edi   ; eax = (x + x) + x
  ret

The method is to walk through it line by line as if you were the CPU, writing down the value of each register after every instruction. Don't judge a function by a single instruction: compilers sometimes write multiplication as addition because it's faster, and the meaning lies in the combined effect of all the instructions. A compiler may even emit the same function in a surprisingly short form, computing the multiplication by three with a single addressing instruction:

f:
  lea  eax, [rdi + rdi*2]   ; eax = x + x*2 = 3*x
  ret

The lea instruction computes an address, but compilers use it for fast arithmetic, and that's a common surprise in analysis. The eax register wraps around when a value exceeds 32 bits, but the values in this lesson are small, so they won't reach that limit.