:[diStorm}: REP+REX prefix ordering

[Sanjay Patel]

Sorry - I made some mistakes with the output disassembly. This is what
GCC/GDB show:

push rbp
mov rbp, rsp
mov [rbp-0x8], rdi
cvtsi2sd xmm1, [rbp-0x8]

I'm getting this output from diStorm currently:

push rbp
mov rbp, rsp
mov [rbp-0x8], rdi
db 0x48
db 0x48
cvtpi2ps xmm1, [rbp-0x8]

--Sanjay


--- Sanjay Patel <sanjay3000 at yahoo.com> wrote:

> I just found a bit of a problem decoding a multi-prefix SSE instruction
> from
> GCC. It produced this byte sequence:
> 
> 0x55, 
> 0x48, 0x89, 0xe5,
> 0x48, 0x89, 0x7d, 0xf8,
> 0xf2, 0x48, 0x0f, 0x2a, 0x4d, 0xf8,
> 
> Which should be:
> push rbbp
> mov rbp, rsp
> cvtsi2sd xmm1, -8(rbp)
> 
> DiStorm got confused by the 0xf2, 0x48 prefix sequence, so it decoded the
> last instruction as a 'cvtpi2ps'.
> 
> Note that the REX prefix and the SSE prefix are swapped from the normal way
> you would expect...but it appears this is legal codegen.
> 
> --Sanjay
> 
> 
> 
> 
>  
>
____________________________________________________________________________________
> TV dinner still cooling? 
> Check out "Tonight's Picks" on Yahoo! TV.
> http://tv.yahoo.com/
> 



 
____________________________________________________________________________________
Cheap talk?
Check out Yahoo! Messenger's low PC-to-Phone call rates.
http://voice.yahoo.com