Challenges the Internet of Things (IoT) is facing are directly inherited from today's Internet. However, they are amplified by the anticipated large scale deployments of devices and ser- vices, information flow and involved users in the IoT. Chal- lenges are many and we focus on addressing those related to scalability, heterogeneity of IoT components, and the highly dynamic and unknown nature of the network topology. In this paper, we give an overview of a service-oriented middle- ware solution that addresses those challenges using semantic technologies to provide interoperability and flexibility. We especially focus on modeling a set of ontologies that describe devices and their functionalities and thoroughly model the domain of physics. The physics domain is indeed at the core of the IoT, as it allows the approximation and estimation of functionalities usually provided by things. Those function- alities will be deployed as services on appropriate devices through our middleware.