Common PE-Parsing Pitfalls

June 3rd, 2007

PE, or Portable-Executable is Windows’ executable format. Looking only at the PE , as opposed to the code inside, can teach you alot about the application. But sometimes the tools you use to parse the file don’t do their work well. I, hereby, want to show a few problems about this kind of tools. As a matter of fact, .DLL and .SYS are also PE files under Windows, so I consider them the same when talking about PE files.

  1. If the Export-Directory offset points to a garbage pointer, Windows will still load the PE and run it successfully. It will get crazy and probably crash only when you try to GetProcAddress on this file. You can use this to render some tools unuseful but the file is still valid and runnable. Don’t confuse with Import-Directory which is necessary for loading the PE.
  2. Another annoying thing, whether the so-called “PE-Standard” states it or not, is the way of RVA (relative-virtual -address) offsets. RVA offset is the distance from the beginning of the file in memory to some specific field or section. Most people treat these RVA’s as if they must point into a section. But in reality RVA’s can point anywhere in the memory mapped file. They can even be negative numbers, (at least, as long as they still point to valid memory of the file). The problem is, most tools try to find the pointee field/section by scanning the sections list, but alas, it might be a valid RVA, which points to no section, but another part in the memory mapped file, for example, the MZ header… While Windows load these files well, some tools cannot parse them.
  3.  The most interesting problem that I found myself, not sure if anyone used it before, was changing the section’s File-Offset to be mis-aligned. The File-Offset is actually rounded down to a sector size (that’s 512 bytes) no matter what. So adding some number to the original valid File-Offset of the code section will fool some tools to read the code from that offset, instead of the rounded offset. Guess what happens? You disassemble the code from a wrong address and everything messes up. Even the mighty IDA had this bug. I introduced this technique in my Tiny PE Challenge. It seems most anti-virus software couldn’t chew up this file back then when I released it…Not sure about nowadays.
  4.  While researching for Tiny PE, Matthew Murphy hinted out that you can load files from the Internet with feeding it with a raw URL of the PE file. Later on it was extended such that Windows’ loader will use WebDAV to load an imported .DLL from the Internet! Think of an imported library with the following name \\127.0.0.1\my.dll inside the PE file itself. This one seemed to be a real blow to the AV industry. It means you can write an empty PE file which will have this special import thingy and gets it off the Internet. For samples you can check it out here, which covers Tiny PE (not my ones) very nicely.

The bottom line is that the Windows’ loader is kinda robust and yet very permissive. It seems as virii can exploit many features the PE format has to offer while AV’s still don’t support some. I guess some of the tools (some are really popular) will get better with time. As for now, my PE parser library for Python, diSlib64 seems to do the job quite well.

Recursive Data-Structures

June 2nd, 2007

For the sake of simplicity a linked list structure is a recursive one, it points to itself.

struct Node {
 int data;


 Node* next;
};

This case is easy to follow because no matter what, you always end up with accessing a Node structure.

The problem arises when you have recursive data-structures and terminals. I treat a terminal as a data structure which stops the recursion of the recursive data-structure. Unlike the above linked list structure, which stops when the ‘next’ pointer is NULL, the ‘next’ in the complex data-structure might point to either itself or a terminal.

 struct TerminalNode {

 int data;

};

 struct Node {

 int data;

 Node* next;

};

Now you might find yourselves asking why would one make the ‘next’ point to a TerminalNode, but that’s a common data-structure layout. For example, an expression tree can represent arithmetic operations in a recursive way, but the values are the terminals – Add(5, Sub(3, 2)).

In a structured layout, it looks something like this (where children is an array of pointers to ‘next’ nodes):

0: ExpressionNode: (type) Add, (children) 1, 2

1: ExpressionNode: (type) Value 5, (children) NULL

2: ExpressionNode: (type) Sub, (children) 3, 4

3: ExpressionNode: (type) Value 3, (children) NULL

4: ExpressionNode: (type) Value 2, (children) NULL

So you can see that in this case an ExpressionNode of ‘Value’ type is a TerminalNode whereas the ExpressionNode is a recursive one.

This type of data-structure is not rare: To be honest, I have this situation in both diStorm64 and diStorm3. In diStorm64 the TerminalNode is actually the Instruction-Info structure which describes the instruction, its mnemonic, its operand types, etc… and in diStorm3, it’s the above ExpressionNode thingy, but much more complex, of course…

Everything is good and nice, but finally I can get to my point in this topic – How do you distinguish between a recursive node and a terminal node? That is, how do you know where the “tree” terminals?

So there are many solutions to this one, I chose to cover two of them. The simplest one is to change your structures by adding another flag which states, “I’m a terminal” or “I’m a recursive node”. But then, it means you have to pay at least another extra byte for both structures. And then examining the flag only after you accessed its structure. Not mentioning, it takes more memory space.

The other solution, which seems to be more common, is to set the LSB (least-significant-bit) in the ‘next’ pointer, thus, if the bit is set, you know you reached a terminal node, even before you accessed it. And seeing the LSB is cleared, you know you continue normally with a recursive node. This is also good for caching, because the structures are compact.

But what happens, if you have 4 types of terminal structures? Well, in a 32 bits environment all structures are (usually/99%) aligned to a 4 bytes boundary, so it means you can easily use the two LSBs, which represent 4 different terminals…

So accessing a single terminal structure is as simple as:

if (pNode->next & 1) {

 pTerminal = (TerminalNode*)(pNode & -1);

 // Use pTerminal here…

} else {

 pNode &= -1;

 // Continue with another node…

}

You should have noticed by now that a terminal structure doesn’t have a ‘next’ pointer. So a NULL is out of question for terminals, but it’s still OK for recursive nodes, which means “dead-end”, rather than a terminal to mark something bad happened…

? Hello World

May 31st, 2007

As a programmer that had to be the title of the first post. In this blog I’m going to write about lots of tech stuff, mostly low-level or anything cool I found interesting.

I’m not sure whether it will be on a daily basis or here and there… Being a developer in a startup company will give me more ideas and stuff to write here. The idea is to write about points that I will encounter during my time that is spent on computers.

The name of the blog was taken from a joke (on my behalf) from some people at my company, that I really like everything that is low-level. Hey, that’s true – Every second word I said made someone else shouting “and do it insanely low-level”…

See you soon