More from Scott Hansleman's list of what great developers should know: What is the difference between an EXE and a DLL?
DLLs contain resources referred to at compile time but called at run time. EXEs are standalone units. (On a side note, static libraries hold resources which are pulled into the target at compile time.) DLL resources can be shared by multiple calling software units. More importantly (to me), DLLs are abstracted out from the calling code. Code using a DLL doesn't have to be recompiled if a new version of a DLL comes out with changes to the non-public implementation, e.g. bugfixes, performance or stability enhancements, etc.
One problem with DLLs, particularly before .NET: DLL Hell.