FreeSpace 2 directory structure
FS2 and FS2_Open work in slightly different ways when it comes to finding the files used to run the games. Although you can get away with simply following the instructions in the installation guides and mod installation readmes it's very easy to make a small mistake and end up with one mod interfering with another.
This page gives you a detailed guide to the places FreeSpace 2 looks in when loading. Once you understand where it is looking you can often solve problems you are having with mods.
FreeSpace uses a fairly interesting directory structure. FS2 has a data directory with subdirectories for ship models, ship textures and missions etc. If you look in here you'll probably find the drawers mostly empty. So if the drawers are empty where does FS2 keep the files it needs to run the game? Well all the files to run the game are contained within a series of files with the extension .vp. The .vp files work similarly to a .zip file in that each .vp contains many smaller files within it.
When FS2 needs a file it looks first for the file. Then it looks through the .vp files (in alphabetical order) to see if the file has been packaged up in one of them. This means you can replace any file within a .vp by simply placing a file with the same name within the data directory.
This makes modifying the way FS2 works much easier and also means that you can't accidentally wipe an important file when installing a mod (well you can but only if you don't follow the advice I give on this page). If things go horribly wrong you can simply delete all the files in you data directory (with the exception of the players subdirectory which contains your save games) to return to an unmodified version of FS2.
FreeSpace 2 looks in the following folders in this order.
It then looks through the vp files in your FreeSpace 2 directory.
FreeSpace 2 Open
FS2_Open uses the same file order as FS2 by default. However one of the most useful abilities of FS2_Open is the ability to change mods from the launcher. By using the -mod option you can add files to the path FS2_Open uses. Be warned that FS2_Open still uses the old directory path too. This means that you can still screw up an FS2_Open mod by putting something in the \FreeSpace2\Data folder that you shouldn't have.
When using the option -mod DirName, FreeSpace 2 Open looks in the following folders in this order.
It then goes and looks through any .vp file it finds in FreeSpace 2\DirName\ before finally looking through the vp files in your FreeSpace 2 directory.