Bruce Eckel says of computer source code, "If it's not tested, it's broken." It's no different with self-defense skills. An outgrowth of this philosophy is the notion of Test-Driven Development (TDD), in which the automated tests are specified before the code that'll pass the tests is even written. In self-defense, let's examine the philosophy of TDD -- that's Test-Driven Defense.
- solo training - understanding your own body mechanics through e.g., kata, shadow boxing, shrimping on the mat
- partner training with low resistance - understanding the principle(s) of the technique applied against another human body through e.g., striking mitt work, drills against a fairly passive partner
- partner training with higher resistance - understanding how to apply the technique against an unwilling opponent through e.g., more dynamic mitt work, or drills with moderate resistance
- contested situational sparring with partner - understanding how to through e.g., boxing with only the jab, hand-fighting for a wrist tie, positional Jiu-Jitsu sparring
- partner sparring with many different techniques - understanding the technique in the context of a fully resisting opponent with wide freedom
Analogs of these steps can be also be applied to self-defense training using a handgun. I'll use the concealed-carry draw-stroke as an example, because it's a prerequisite for everything else.
- solo training - instruction and slow, careful execution of the sequence of movements, tying in to the student's understanding of good grip and stance
- partner training with low resistance - untimed execution of the draw from concealment with an external start stimulus (a beep or instructor who says, "Go!")
- partner training with higher resistance - timed execution of the draw with an external start and stop stimulus, using progressively decreasing par times, approaching a goal that's "good enough" (e.g., 2 sec to first shot)
- contested situational sparring with partner - one vs one competition against reactive targets with an external start stimulus; situational "sparring" with Airsoft experimenting with draw speed, running to cover, shooting on the move, etc.
- partner sparring with many different techniques - force-on-force scenario training using Airsoft, Simunitions, or other simulated handguns where the "script" and outcome is unknown
Notice that, as you progress toward the "free sparring" end of the spectrum, the focus of the training widens, and the importance of the technique shrinks relative to the overall outcome. The greater your mastery of the physical skill, the more time and attention you can devote to your tactics, which are far more likely to decide the outcome than raw physical skill. Continuing to use the draw-from-concealment example, a fouled drawstroke in a force-on-force scenario could lead to bad outcome. (I've been there!) A 1.0 second drawstroke might bail you out of a situation where a 1.5 second drawstroke won't -- but that's rare. Positioning yourself in advance for a slower, surreptitious draw so the gun is already in your hands is vastly superior.
Your self-defense education -- unarmed or armed -- needs to include all of these phases. You need sufficient mastery of technical skills to employ them within a "live" scenario, against fully resisting opponents. If you don't have that yet, assume your technical skills are tenuous until you subject them to more realistic tests.