Magnetic confinement devices for nuclear fusion can be large and expensive. Compact stellarators are promising candidates for costreduction, but introduce new difficulties: confinement in smaller volumes requires higher magnetic field, which calls for higher coil-currents and ultimately causes higher Laplace forces on the coils-if everything else remains the same. This motivates the inclusion of force reduction in stellarator coil optimization. In the present paper we consider a coil winding surface, we prove that there is a natural and rigorous way to define the Laplace force (despite the magnetic field discontinuity across the current-sheet), we provide examples of cost associated (peak force, surface-integral of the force squared) and discuss easy generalizations to parallel and normal force-components, as these will be subject to different engineering constraints. Such costs can then be easily added to the figure of merit in any multi-objective stellarator coil optimization code. We demonstrate this for a generalization of the REGCOIL code [1], which we rewrote in python, and provide numerical examples for the NCSX (now QUASAR) design. We present results for various definitions of the cost function, including peak force reductions by up to 40 %, and outline future work for further reduction.