Designing, analyzing and optimizing applications for rapidly evolving computer systems is often a tedious, ad-hoc, costly and error prone process due to an enormous number of available design and optimization choices combined with complex interactions between all components.Auto-tuning, run-time adaptation and machine learning based techniques have been investigated for more than a decade to address some of these challenges but are still far from the widespread production use. This is not only due to large optimization spaces, but also due to a lack of a common methodology to discover, preserve and share knowledge about behavior of existing computer systems with ever changing interfaces of analysis and optimization tools. In this talk I will present a new version of the modular, open source Collective Mind Framework and Repository (cTuning.org) for collaborative and statistical analysis and optimization of program and architecture behavior. Motivated by physics, biology and AI sciences, this framework helps researchers to gradually expose tuning choices,properties and characteristics at multiple granularity levels in existing systems through multiple plugins. These plugins can be easily combined like LEGO to build customized collaborative or private in-house repositories of shared data (applications, data sets, codelets, micro-benchmarks and architecture descriptions), modules (classification, predictive modeling, run-time adaptation) and statistics from multiple program executions. Collected data is continuously analyzed and extrapolated using online learning to predict better optimizations or hardware configurations to effectively balance performance, power consumption and other characteristics. This approach was initially validated in the MILEPOST project to remove the training phase of a machine learning based self-tuning compiler, and later extended in the Intel Exascale Lab to connect various tuning tools with an in-house customized repository. During this talk, I will demonstrate the auto-tuning using the new version of this framework and off-the-shelf mobile phones while describing encountered challenges and possible solutions.