Designing and optimizing novel computing systems became intolerably complex, ad-hoc, costly and error prone due to an unprecedented number of available tuning choices, and complex interactions between all software and hardware components. I present a novel holistic methodology, extensible infrastructure and public repository (cTuning.org and Collective Mind) to overcome the rising complexity of computer systems by distributing their characterization and optimization among multiple users. This technology effectively combines online auto-tuning, run-time adaptation, data mining and predictive modeling to collaboratively analyze thousands of codelets and datasets, explore large optimization spaces and detect abnormal behavior. It then extrapolates collected knowledge to suggest program optimizations, run-time adaptation scenarios or architecture designs to balance performance, power consumption and other characteristics. This technology has been recently successfully validated and extended in several academic and industrial projects with NCAR, Intel Exascale Lab, IBM and CAPS Entreprise, and we believe that it will be vital for developing future Exascale systems.