The kernel could refuse to load a dll unless
there is sufficient
free address space to allow enough
randomness.
I was
thinking that too, but then considering it further,
I thought to myself why is
Windows allowing a DLL to load in
the last bit of remaining space anyhow? Some
memory is
always going to be needed by the processes that are
currently
running from time to time as they request memory
from the heap and stack. (I'm
not clear on if stack space is
allocated ahead of time when a process loads or
as functions
are called, but for sure there will be random calls for heap
space.) The new DLL wanting to load in the last bit of space
will also be
wanting to request heap space of its own, so it
simply makes no sense to give
away the last of the memory.
You need to hang on to some just to maintain
operations,
have some swap space so you can advise the user there is an
out of
memory condition and allow for graceful termination
of running processes or
shutdown of the system.
Then there is an unrelated issue: memory
fragmentation.
As memory gets used and used up, it tends to get more and
more
fragmented. Of course Windows has algorithms to defrag
the memory as it goes,
but no defragging algorithm can be
perfect and is always going to leave holes
around. So as
someone is trying to fill up the memory then back off until
there is just enough space to load his hack, he can't be
sure that hack will
load where he expects, as it may fit in
one of the holes left behind from the
defragger.
So this attack sounds good on the surface, but has it
been
actually run and proven to work? [ Reply to This | Parent | # ]
| 
