Despite significant advances in oncological research, cancer nowadays remains one of the main causes of mortality and morbidity worldwide. New treatment techniques, as a rule, have limited efficacy, target only a narrow range of oncological diseases, and have limited availability to the general public due their high cost. An important goal in oncology is thus the modification of the types of antitumor therapy and their combinations, that are already introduced into clinical practice, with the goal of increasing the overall treatment efficacy. One option to achieve this goal is optimization of the schedules of drugs administration or performing other medical actions. Several factors complicate such tasks: the adverse effects of treatments on healthy cell populations, which must be kept tolerable; the emergence of drug resistance due to the intrinsic plasticity of heterogeneous cancer cell populations; the interplay between different types of therapies administered simultaneously. Mathematical modeling, in which a tumor and its microenvironment are considered as a single complex system, can address this complexity and can indicate potentially effective protocols, that would require experimental verification. In this review, we consider classical methods, current trends and future prospects in the field of mathematical modeling of tumor growth and treatment. In particular, methods of treatment optimization are discussed with several examples of specific problems related to different types of treatment.