Provided that the dialog layout is stored as a resource in the resource section of the PE file, you might be able to do something with
ResourceHacker:
http://www.angusj.com/resourcehacker/
However, I think that "file selection" dialogs are usually
standard dialogs. For this kind of dialogs you won't find any resources in the PE file. Showing a
standard file open dialog is pretty much just a single API call and all the details (layout, etc) are hidden in the system libraries, e.g. Comctl32.dll. So, in order to "upgrade" to a newer file open dialog, you would probably have to hack the code to call a different API function, or pass different flags, or link against a different library.
Anyhow, which version of Common Controls (Comctl32.dll) an application will use is controlled by the applications
manifest - which is also stored in the resource section of the PE file. Thus, by "implanting" the
right Manifest, you may be able to trick the program into using a newer version of Comctl32.dll. This is also required to enable the "visual styles" of Windows XP and later. But, of course, it can have some unexpected side-effects...