Testing

Keywords: patience, time, circumspection, purposefulness

General hints

• Never be afraid of testing. Always be straightforward and logical: If you think that part is buggy and you are afraid that you discover that you did something wrong, other people, who played that mission will warn you and that's most inconvenient.
  
• Do not be afraid of time: the more time you spend to look after your mission the better it will become, the more ideas you will get how to make it better.
  
• Never, never, never get annoyed by a bug you can fix. If a debriefing shows up at a bad time every time you test your mission, keep cool. Open up the debriefing window, look at the SEXP tree and think slowly, logically and first of all, calmly. The more time you spend with a bug the more chance you have to fix it. If you manage to fix a problem that you had been fighting with for hours, you will feel more deliberated.
  
• Never be satisfied: Always want to add new things. If needed, sit ten, twenty or even thirty minutes looking onto the boring grid of FRED while thinking into the possible ways of adding new items. Look into the Events window and zoom in/out at the ships and an idea will soon come what to add.
  
• Use the Designer's notes! Note down bugs you do not want to fix in the moment you discovered it, note down anything you will want to add. Keep erasing your notes as soon as they become obsolete. Do not let others see your mission until there are any notes left. This window can also be used to communicate with other mission designers, if not only you have been assigned to that mission.
  
• Concentrate on fixing only one bug: Do not try to fix multiple bugs unless you are sure it won't need any testing after you have fixed it (typing mistakes, no ships in Escort List, etc).
  
• Never release a mission without testing its last version twice or three times. A seemingly minor change like changing wing delays can result in unexpected difficulties.
• If you have the connections, give it out to others, preferably from any skill level, to test for difficulty, pacing, etc. What works for you, might not work for others.

The testing process

There are numerous and diverse ways to test a mission. It's a matter of personal preference, so this article can't be all-encompassing or applicable to all types of mission. Some of the most common mistakes (or oversights) can be found here. For a more comprehensive list of possible mistakes, see this article.

The first few test runs will likely turn out to be overwhelming. You'll notice several items that you missed: Events or mission goals that don't work, or ships that don't behave as intended. Make your mission operable first, remedying only fatal mistakes that prevent the player from completing the mission. Mission critical ships not jumping out belong here. Note down anything you can find, but don't want to fix all that at once. Some easy-to-fix oversights that require only a minimal effort may be fixed now. Others that require some further testing and SEXPing may be delayed a bit.

Now that you know how your mission is played, you must finalize some parts of the mission's gameplay. Decide which ships the player will have to protect or destroy and stick to it. Adding some more mission critical craft will require far more additional testing and refinement than you may think. The sooner you make such drastic changes to your mission the less time you'll have to spend with testing altogether. This doesn't mean that your mission's balance must be perfected in this stage. Minor alterations like adding one more wing of bombers or demoting the player's wingmen from Major to Captain can be made much later. What's important here is that you won't have to create a dozen new SEXPs because you've decided that the player must defend two Argos instead of one.

Test every imaginable outcome that a player might produce. Even though there aren't enough bombers to destroy a capship that you must defend, you must nevertheless take into account what would happen if the bombers did destroy your capship. You may not have a "Return to base" directive show up and there may not be a debriefing stage that deals with such an outcome. Test only "imaginable outcomes," however. In Their Finest Hour, there is really nothing the player can do to save the Colossus. No thought is given to that possibility because no player can be fast and destructive enough to take down all four of the Sathanas's forward beam cannons, so it's not an "imaginable" outcome. The Colossus can only be saved using cheats. The mission designer in such cases cannot be made responsible.

On a few occasions, it helps not to test your mission, but play it. To clarify, you must not be specifically looking for mistakes, but try to see your mission through the eyes of a player who hasn't seen the mission yet and just wants to play. You may notice some inconsistencies or logical flaws between the briefing and how the mission actually plays out.

In the last testing stage, you should concentrate on minor details like adding cargo to cargo containers, prettifying your background, or adding some more mission chatter. This stage will take a lot more time than the previous ones, because you won't be hunting for something particularly obvious or observable. As a general rule, note down anything you find and after a few more test runs, once you have accumulated a long list of observations, decide which ones are worth it. You'll also have to finalize your mission's difficulty. An adequately balanced mission requires several additional test runs which are solely devoted to finding the "right" difficulty. For more information and tips for balancing your mission, see Mission balance .

Testing someone else's mission

The testing process

Remember that by the time you receive the mission file, it's supposed to be more or less complete. For the first test run, play as if you were any casual player who is playing a user-made mission for fun. Don't specifically look for mistakes. If you find some mistakes even so, those mistakes must be ones that any player will come across. These mistakes will likely come up every time you play the mission, so they are to be reported and fixed as soon as possible. You'll now have a clue about the mission. After a few more casual plays, dig yourself deep into some real testing.

Now start testing a mission as if it were your own, except you can afford more test runs because you don't have to fix the mistakes. Use notes to keep track of your findings. Pay attention to details and test every possible mission outcome. You may win the mission totally by blowing up all the three hostile cruisers while keeping your corvette alive, but try to lose the corvette once and see what the debriefing will say.

Lastly, open the mission in FRED and look around. You may discover some unused Events, a forever-invalid mission objective. You'll be point out some mistakes that would otherwise require a dozen re-runs to be found.

Submitting feedback

Submitting feedback to the mission designer is the last step in your testing of someone else's mission. Whatever way of information echange you choose (email, forum post, instant messaging) remember to be objective. Don't use attributives like "bad" or "amateurish," even if the mission designer made a very blatant mistake. Even experienced mission designers tend to forget about putting mission critical craft on the Escort List. A factual "There is no Escort List." sentence will do it. If the mission designer is a beginner, explain why the Escort List is needed. This must indeed be an explanation.

Don't report all your findings. A disproportionately long list of mistakes is discouraging. Concentrate on the really important mistakes first and put forward some propositions on how to fix them.

While it has previously been said to be as objective as possible, some praise—which is inherently subjective—helps maintain a steady level of encouragement. Be specific about your compliments, also. A very general, "The mission was excellent, but I have a long list of mistakes here" sentence sounds dishonest. A dishonest praise is no praise. It is okay to praise the mission for its intensity and point out some minor briefing icon-related mistakes.

Speeding up your testing

FREDders, especially when making a campaign, are faced with time constraints due to real life reasons. This section will help you minimize your testing time without sacrificing effectiveness.

• Most importantly, minimize your loading times. A very straightforward way of doing this is to disable the MediaVPs and all graphical enhancements (FreeSpace Open users only). Don't be too minimalistic, however. Keep it to your tolerable standard, otherwise it will distract your attention from the testing part.
• Using windowed mode will prevent your game from crashing if you change between the game and FRED using ALT-Tab. (FreeSpace Open only).
  • Note: Retail FreeSpace users will not encounter the above problem.
• Rename your mission to make it the first on the list in your Mission Simulator. Renaming your mission to a_<mission name>.fs2 or 1_<mission name>.fs2 will be sufficient. Don't forget to rename it again to its finalized name when you start thoroughly testing another mission.
• There are no moral standards for testing. Feel free to use cheats to easily and quickly reach the part of the mission you want to test. Making use of "developer Events" like ones that make friendly mission critical craft invulnerable is also possible. Don't delete such Events when you are done, because that would change the order of Events, and break the mission. Use false instead to disable them.
• If your mod makes use of a custom weapons.tbl, you can add a very powerful weapon that you use solely for testing runs. Having such a powerful laser will help you finish the mission far more quickly than with weapons that will be otherwise available to your players—this is especially useful for testing debriefing stages. To make sure that your players won't use this weapon, just don't set it as an initial weapon in the campaign file. (Use "normal" weapons otherwise for testing mission balance). If you don't want this weapon to be available at all, remove it from the tables (and the mission's weapon loadout tab) entirely.
• If it's not about testing the mission's difficulty, simply lower the difficulty level. It's less stressful and allows you to concentrate on what you want to test.
• You can set up a one-mission campaign to easily access your mission by clicking on the Ready Room, instead of using the Tech Room method. This method may compromise your pilot file when not done right, so either make a backup copy or a new pilot. Less experienced mission designers may not want to follow this method at all.
• Decide which changes actually need testing. You may want to test if your briefing icons are set up right, but you may not have to test spelling corrections. The more experienced you are, the better you'll be able to judge which changes require testing.
• To test if certain Events work or not, use directives. Therefore, you will be able to tell when and if the problematic Event turns true or false.

Testing your campaign

By the time you had set up your campaign file, you'll ideally have a series or more-or-less finalized missions. This section will give you advice on how to test if your campaign file is set up properly. Remember that changes applied to your campaign file will take effect only when you reset your campaign in the Campaign Room.

• First and foremost, your Error Checker must not indicate a problem. If it does, it's very likely something won't work right.
• Enter the Campaign Room, select your campaign, and read your campaign's description at least twice to discover if you have done some typos or made some spelling mistakes. If you discover something you want to change, don't do that yet!
• Run the first mission and discover if the player has access to all the intended ships and weapons. First-time campaign makers tend to forget that they must allow the player to use certain ships and weapons in the campaign file. If the player has weapons and flies the intended fighter by default, it's okay. If you have found anything you want to change while you were in the Campaign Room window, do that change now. If the player doesn't have access to all the intended weapons or ships and you found a spelling mistake in your campaign's description, you can change all that in one run.
• To test if your Repeat Mission SEXPs work, jump out at the beginning of each mission and see if the game tells you to replay the mission in the debriefing room. Ideally, it will. If not, revise.
• Test what happens if the player is minimalistic. Complete only those objectives that will allow the player to progress in the campaign. You can be a maximalist for the second or third re-run.
• If you complete your campaign, see what happens if you click on the Ready Room. If it says you have completed the campaign, you're done.
• If you're using custom tables and/or mods, set up a mod folder that contains only the files you're about to release. This will objectively reflect what your players will have when they download your campaign. If there's a mod you've forgotten to include in your download, this method will bring that to light.
• You needn't test your campaign as a whole in the initial stages of creating the campaign file. You can test it in small chunks of three or four missions to see if the campaign file is set up correctly. Some missions might require a complex set of conditions to allow the player to progress; this method helps a bit in testing these kind of missions.

See also

• General hints
• Mission balance
• Typical FREDding mistakes