Why I Worked On This
I started this because I was disappointed that IDA did not handle AVX decompilation automatically. I found the original MicroAVX GitHub project, but it was still very incomplete for the binaries I was looking at, so I forked it to contribute back and make it more useful in practice.
A big part of the work was not just adding instructions, but figuring out how Hex-Rays even wants them to be modeled. The Python IDA API has very little usable documentation, so a lot of this project turned into reverse engineering the API itself, learning how to inspect and emit microcode, and then stitching new AVX instructions into that pipeline.
I also try to avoid adding handlers that only wrap an intrinsic that already exists cleanly in C or C++. For cases like that, the lift often does not buy much and does not really improve the decompiler's understanding of the microcode. The more interesting work is in instructions that meaningfully improve the IR and final pseudocode.
Two resources helped a lot while figuring this out: the microcode dumper extension genmc, which I first used to dump Hex-Rays microcode, and Ilfak Guilfanov's Black Hat paper Decompiler Internals: Microcode, which made the structure of the IR much easier to reason about. I am still adding more instructions as I run into them.
Instructions Added In My Fork
These are the instruction families I added while extending the original project. Each link points to an instruction reference rather than to a commit.
-
vpxor
Packed bitwise XOR on vector lanes. Useful for zeroing registers and building integer-vector logic directly in microcode.
-
vcvtsi2ss
Converts a signed integer from a GPR or memory into the low scalar float lane of an XMM destination.
-
vzeroupper
Clears the upper 128 bits of YMM registers to avoid AVX to SSE transition penalties and to model the register state correctly.
-
vpbroadcastq
Takes one 64-bit value and replicates it across every 64-bit lane of the destination vector.
-
vpextrx
Extracts a selected byte, dword, or qword from an XMM register into a scalar register or memory.
-
minss
Returns the smaller of two scalar single-precision values in the low lane.
-
maxss
Returns the larger of two scalar single-precision values in the low lane.
-
vfmadd*
Fused multiply-add instructions that multiply inputs and accumulate the result in one instruction, preserving the packed floating-point semantics.
Example: vpbroadcastq ymm0, xmm0
This instruction takes the low 64-bit value from xmm0 and broadcasts it into all four 64-bit lanes of ymm0.
Lift Code
def vpbroadcastq(self, cdg, insn):
"""
VPBROADCASTQ xmm1, xmm2/m64
VPBROADCASTQ ymm1, xmm2/m64
"""
return self._vpbroadcastx(cdg, insn, QWORD_SIZE)
def _vpbroadcastx(self, cdg, insn, source_size):
destination_size = XMM_SIZE if is_xmm_reg(insn.Op1) else YMM_SIZE
if is_mem_op(insn.Op2):
s_reg = cdg.load_operand(1)
else:
assert is_avx_reg(insn.Op2)
s_reg = ida_hexrays.reg2mreg(insn.Op2.reg)
d_reg = ida_hexrays.reg2mreg(insn.Op1.reg)
for i in range(destination_size // source_size):
cdg.emit(ida_hexrays.m_mov, source_size, s_reg, 0, d_reg + i * source_size, 0)
return ida_hexrays.MERR_OK
Generated Microcode
The corresponding microcode at address comment 7FF7F889190C in microcode.asm shows the same source qword being copied into each 64-bit slice of the destination YMM register:
10. 5 mov xmm0_8.8, ymm0_8.8 ; 7FF7F889190C u=xmm0_8.8 d=ymm0_8.8
10. 6 mov xmm0_8.8, ymm0_8^8.8 ; 7FF7F889190C u=xmm0_8.8 d=ymm0_8^8.8
10. 7 mov xmm0_8.8, ymm0_8^16.8 ; 7FF7F889190C u=xmm0_8.8 d=ymm0_8^16.8
10. 8 mov xmm0_8.8, ymm0_8^24.8 ; 7FF7F889190C u=xmm0_8.8 d=ymm0_8^24.8
Final Decompiled Output
_RCX += 128;
v8.m256i_i64[0] = _RDX;
v8.m256i_i64[1] = _RDX;
v8.m256i_i64[2] = _RDX;
v8.m256i_i64[3] = _RDX;
*(__m256i *)(_RCX - 128) = v8;
*(__m256i *)(_RCX - 96) = v8;
*(__m256i *)(_RCX - 64) = v8;
*(__m256i *)(_RCX - 32) = v8;
