PerformLTE: A Testbed for LTE Testing in the Future Internet

. The ﬂat architecture adopted in LTE increases the scalability of the network in order to accommodate large volumes of user traf-ﬁc, reduces packet latency and the cost per byte. At the same time the enhanced Node B (eNB) has increased its complexity which have implied the appearance of new challenges in the ﬁeld of experimental performance tests [1]. To cope with these challenges, access to a real and controlled experimentation environment is needed. Nevertheless, the high cost of laboratory equipment makes it diﬃcult to carry out realistic experiments for most research teams, whose work usually rely on simulations. A combination of highly conﬁgurable equipment and software tools accessed re-motely seems to be the best solution to improve research activities around LTE technologies and beyond. PerformLTE testbed provides a controlled environment where LTE end-to-end IP communication, including radio impairments and network perturbations, and complex network setups can be reproduced.


Introduction
According to Cisco [5], "the global mobile data traffic grew 69 percent in 2014, reaching nearly thirty times the size of the entire global Internet in 2000". It is also expected that "by the end of 2014, the number of mobile-connected devices will exceed the number of people on Earth and by 2019 there will be nearly 1.5 mobile devices per capita". Meanwhile mobile users continue to demand a high quality of service every day and at all locations. To ensure that users are satisfied when accessing services, very expensive tools and hardware are required. In this global economic context, it is difficult for researchers, application developers and service providers to obtain these tools. Network operators usually have access to different levels of information regarding the performance of their networks, which are known as Key Performance Indicators (KPI), but this information may not be enough to evaluate how the network's internal variables are related to the actual user experience. To overcome these limitations, we have developed an experimentation platform that enables advanced research studies over LTE. This platform provides access to very complex configurations of the radio access, basic channel emulations based on 3GPP reference channel profiles and measurements which cover most of the layers of the radio access and user application protocols.
FIRE is an European initiative which aims to create an open research environment to facilitate strategic research and development on new Internet concepts providing an instrument to carry out large-scale experimentation on new paradigms. PerformLTE is one of the facilities of the FIRE ecosystem. It is part of the Fed4Fire [2] initiative, which aims to federate open, reliable and accessible platforms for the research communities and FLEX, an initiative towards the provision of 4G and 5G experimental facilities.
The paper is organized as follows. In Section 2 we provide a detailed description of the functionalities provided by the testbed and the different scenarios offered. Section 3 introduces our approach to position the testbeds towards 5G. Finally section 4 summarize the contributions of the testbed in the field of LTE experimentation.
2 PerformLTE: a testbed for mobile experimentation in the Future Internet

PerformLTE methodology
PerformLTE methodology follows an holistic approach, combining different type of equipments, LTE radio access equipments, Evolved Nodes B (eNBs), User Equipments (UEs), both commercial and engineered to provide measurements, and an Evolved Packet Core (EPC) emulation system. All these elements can be combined and experimentation can be performed in all the components of an LTE network. In general terms, LTE connectivity is provided through three different solutions, each one with a focus on a different research aspect, moving between emulation and real-world environments. Three main scenarios can be differentiated based on the radio access type: -Scenarios based on conformance testing equipment that provides a full LTE end-to-end emulation, including channel emulation with different fading profiles and operation in all the standardized LTE bands, both FDD and TDD. This equipment allows the configuration of multiple levels of the LTE Radio Access Network (RAN) stack, so researches can study the effect of different parameters as well as the motorization of the full network. -Scenarios based on commercial off-the-shelf eNBs. These scenarios provide functionality close to the one provided by operators. Configuration is based on OAM interfaces, some proprietary, some other based on standards like TR.069. Researchers can test the policies that operators are able to setup and can provided very complex configurations of the EPC network. -Scenarios based on commercial LTE networks, in which the testbed offers applications and tools to extract information from the state of the network and correlate it with performance indicators from IP and application levels.
Furthermore, the testbed integrates some standard experimentation tools, like OMF and OML [3]. These experimentation interfaces enables the orchestration and configuration of the elements of an experiment using a standard description (OEDL), that provides a flexible and programmatic way of executing experiments and collecting results, and is under integration to provide resource reservation via the Fed4FIRE portal.
All these scenarios can be used with instrumented Android phones that provide information of the network extracted from the UE side, as well as capture incoming and outgoing IP traffic, that can be later used with post-processing tools to correlate KPIs from different layers of the stack, including QoE measurements based on subjective perception estimation. Additionally, a power analyzer can be connected to the UEs in order to obtain also information about energy consumption which remains one of the most important challenges of future mobile communication. The different radio access types can be used incrementally in an experiment, moving from laboratory to real life. Figure 1 depicts some of the possible usages of the testbed.  The configurations offered by the testbed cover a broad spectrum of experiments, from applications and services to innovative network solutions. For example, developers could take advantage of the eNB emulator to test their applications under controlled radio propagation scenarios. Researchers interested in radio resource management could access the information retrieved from real mobile devices to see how different radio access configurations affect the performance of mobile applications running on mobile devices.

University of Málaga Campus
Currently, the performance metrics collected by the PerformLTE testbed are oriented to radio, IP communications, QoS and QoE, EPC, and battery consumption. Additionally, the testbed is capable of providing traces from most of the involved signaling procedures and statistics of network procedures. All these measurements are post-processed and correlated in order to obtain a fully detailed picture to characterize the performance of mobile applications and services in a specific context.

Instrumented mobile devices
Since 2004, the UMA has worked on the development of monitoring tools for smartphones [7][8] [9]. We started our proprietary development with SymPA tool for the Symbian platform and have continued with TestelDroid, an Androidbased tool for monitoring communications performance and radio issues [10]. These tools have been used to analyze the performance of video streaming [11], VoIP services and also energy consumption [12]. These tools collect not only simple metrics such as throughput, but also radio parameters such as received signal strength, radio access technology in use, current IP traffic and much more to obtain a fully detailed picture to characterize the scenario where the results have been obtained. Table 1 provides a summary of the main functionalities provided by TestelDroid. Collected data can be logged using highly analyzable plain text files (except for traffic capture, stored in PCAP format). The objective of these tools is to take advantage of the engineering features provided by current commercial smartphones for the development of software tools which enable us to capture the QoS (Quality of Service) perceived by final users of mobile services and its correlation with the internal performance and configuration of networks and devices. At this point, we would like to remark on the importance of capturing, in a standard format, the traffic in a mobile phone. This capture can be used by experimenters to analyze the traffic in more detail and use their own tools and methodologies. So as a key element of our testbed, we provide a pool of devices running our passive monitoring software tool called TestelDroid. Passive monitoring tools represent a scalable alternative for general purpose analysis of mobile applications' performance. Different aspects may be studied together using the passive approach including communication performance, memory usage and battery consumption among others. These monitoring tools complement and enhance network self-optimization procedures extending the observation points to additional protocol layers and to the user device and applications perspective. The instrumented mobile devices can be used by service providers and mobile developers to deploy their latest services, applications, protocols or any other aspect susceptible to modification in Android smartphones, and to test their performance over LTE mobile networks.

LTE radio access
For LTE radio access we provide three experimentation scenarios, each one with a different profile moving between emulation and the real-world. These scenarios are described in the following subsections.
Emulated radio access The team at UMA has experience in the development of LTE signaling software for commercial eNB emulators. We have collaborated with companies such as AT4 Wireless (one of the major mobile certification laboratories), Keysight Technologies (formerly Agilent, one of the most relevant providers of testing equipments) and Alcatel Lucent. In this context, the UMA team has modified an eNB emulator, the T2010A from Agilent Technologies (formerly E2010 from AT4Wireless), to enable its use as an experimentation platform [14] to which commercial smartphones can be connected. The result is an experimental environment where radio access conditions can be fully configured and the effect of these configurations at the application level on the UE can be measured using our monitoring software, TestelDroid. Some of the parameters which are configurable through the emulator are described in Table 2 and lists some of the measurements that can be obtained. The T2010A, is a generic platform used not only in conformance RF and signaling testing but also for design verification. In addition to these features, it also integrates channel emulation and digital generation of impairments such as AWGN, which is a critical feature for achieving high accuracy when setting SNR conditions. Standard multipath fading profiles defined by 3GPP are supported to emulate reference propagation conditions. MIMO is a key feature in LTE, as it is one of the foundations of the technology's high rates and spectral efficiency. The T2010A provides up to 4x2 integrated MIMO features, thus increasing the range of test possibilities with interesting network configurations.
This radio access scenario provides an experimental environment where experiment conditions and results can be repeated and reproduced. The scenario is limited to one device connected to one emulator. For large scale experimentation, we recommend scenario 2 described in the following section.  Table 3. Measurements in the emulation system As an example of the type of results we can obtain in this scenario, 3 shows the relation between the PDCCH (Physical Downlink Control Channel) & PDSCH (Physical Downlink Shared Channel) BLER (Block Error Rate) and the IP packet losses. It can be appreciated that the relation is not strictly linear because of the randomness associated with AWGN and fading impairments, resulting in bursty behaviors. This relationship is also governed by the operation of HARQ, as the incremental redundancy increases the probability of correct reception with each retransmission. An RLC SDU containing an IP packet is lost only after the maximum number of unsuccessful retransmissions has been reached; in this case there were three retransmission configured.

A proprietary LTE deployment
The testbed is also composed by two commercial LTE pico-cells and a commercial EPC. This deployment with real antennas provides a more realistic environment still under the control of the experimenter. The eNBs can be configured via proprietary interfaces, and there are ongoing efforts on the automation the configuration of new nodes based on OAM standards. The following table summarizes some of the key features that can be configured in the pico-cell. Moreover, pico-cell logging files will be accessible. Resource Management Radio admission control, DL and UL minimum bit rate, VoIP consumption, etc., Transmit power and power control, Downlink and uplink scheduler, maximum number of transmissions, maximum HARQ timer, etc., Semi-static scheduling parameters Connection Management Access Barring parameters, Cell Parameters, cell id, plmn, etc, RRC timers, reestablishment timer, connection timer, etc Mobility Maximum transmission power that the UE might use in uplink, Minimum required reception level, Cell reselection parameters, S1 and X2 handover parameters, Measurements configuration Transport QoS configuration Table 4. Summary of configurable parameters in a commercial pico-cell This nodes are integrated with an EPC emulator that can be used to define multiple network setups. For instance using the emulators, roaming scenarios can be executed by the definition of two different PLMNs each one with its corresponding LTE deployment. Furthermore these emulators provides carrier grade performance, we have measured 60Mbits/sg using a band 7 base station with 10MHz and supports the configuration of negative testing behaviors. Other interesting features are the provision of interfaces to establish dedicated bearer creation properties and in the latest version the inclusion of an IMS deployment which will be used to test VoLTE scenarios.

Commercial LTE deployments
In addition to the emulated eNBs and the commercial pico-cells, PerformLTE provides a pool of instrumented smartphones connected to national LTE deployments available in Málaga (Spain). In this scenario it is not possible to configure LTE radio access, only mobile devices are accessible to install new applications or modify the software. However results obtained in previous scenarios can be matched with results obtained in commercial LTE networks.

Power consumption measurements in mobile devices
To carry out the analysis on battery life characterization, a key factor in mobile devices, we have a power analyzer provided by Keysight Technologies. This device can act as a 2-quadrant DC voltage source capable of generating arbitrary waveforms and/or an oscilloscope with data capturing capabilities. As a source, it can provide up to 20 V, up to ± 3 A. Power rating is 20 W. As a measurement tool, it is capable of measuring values down to nA and µV at a rate of 5.12 µs/sample for one parameter. Our equipment features advanced source modes, including a battery emulator mode and a battery charger mode. The former imitates the effects of charging or discharging the battery, including a programmable output resistance from -40 mΩ to +1Ω, while the latter emulates the behavior of a battery charger, which cannot sink current. In addition, the source mode also provides an arbitrary waveform generator to create custom waveforms. Table 6 shows the adjustable parameters in source mode. It is possible to store measurement data for up to 99,999 hours using the data logger function. Data can be exported to a file, which can be analyzed and graphed later.

Post-processing and reporting tools
In addition, the testbed incorporates post-processing tools to further analyze and correlate the data collected. The toolchain allows the inspection of the IP traffic both at source and destination, to obtain typical communication statistics such as packet loss rate, throughput, inter packet arrival, end to end delay and jitter. Figure 2 is an example of the type of result that can be reported in this scenario. The figure has been obtained using the post-processing tool chain.  Figure 2 is a vector graphic generated using the files containing the temporal evolution of the IP traffic and radio parameters collected by TestelDroid. In the legend below it, we find the color code and on both sides a number of vertical axis are used for different magnitudes. In the background, different colors are used to illustrate which Radio Access Technology (RAT) is being used at a certain point in time. Cell changes would have been indicated by a vertical dashed line. The instantaneous bandwidth is represented with a green dotted line, with each dot representing the throughput calculated when receiving a new data packet averaging over the last second. Although the line is shown as continuous for the sake of clarity, only the dots indicate actual reception of packets. The inter-packet delay represented with orange dots, which represent the time elapsed between the receptions of consecutive data packets. This magnitude is extremely valuable for identifying service interruptions and delay quantification effects in the network and codecs. An inter-packet delay of 0 indicates that two packets have been received together at the IP level, and the concentration of points suggests the technology is the codec frame timing. The jitter (in red) is an indication of the delay variation. Packets received at a constant rate would have a low jitter, whereas connectivity gaps or highly varying delays would result in a high jitter. A burst of lost packets is marked with a black arrow and a number, indicating the amount of packets that have not been received. Packets received out of sequence would be marked with yellow points. A blue line represents the Received Signal Strength indicator (RSSI), which is a measurement of the total signal power received by the mobile phone in its reception bandwidth.

5G Challenges and trends
The 5G Infrastructure Public Private Partnership has defined some of the challenges that will have to face future 5G communication: 1000 times higher wireless capability, 90% of energy savings per service provided, reducing service creation time, facilitating dense deployments, reducing downtime to a "zero perceived" and so on. These high level requirements are produced to enable new incoming services and products, that will push to the limits the performance and capacity of future networks and will be provided by improvements in all the elements of the network, from the appearance of new radio access technologies to new designs of the core and transport networks. NGMN [6] provides an excellent overview on future applications related with their expected user experience KPIs, as well as some of the current trends to support these new scenarios. 5G will provide heterogeneous access to the radio access technologies, combining them to improve reliability, performance and availability of connections. There are research groups working on the provision of new waveforms and it is clear the mmWave spectrum will be exploited [13]. Massive MIMO is another important trend being explored.
Other trends on the radio access which are already coming in future 3GPP specifications are control/data plane separation, also referred as dual connectivity. In these scenarios, the control plane is transported over a different carrier, even from a different technology that the data plane. This is particularly interesting, as it allows to combine macro cells to provide the control plane with small cells that will provide better performance figures on the data plane. This type of architecture will improve also handover time. Along these lines, there is the so called network densification, which is essential to guarantee availability and to improve performance on the network. Carrier aggregation, already standardized in LTE Release 10, is very interesting to increase throughput figures. The concept is now being extended to provide aggregation using unlicensed bands where more spectrum might be available.
The core network will also suffer major changes. There are some trends to simplify its architecture and reduce the signaling overhead. Some trends also cover the deployment of core network functionality inside the nodes to drastically reduce latencies and even the forwarding of traffic from one node to another when communication peers are connected to geographically close nodes. NFV and SDN are also very important concepts as they will provide the required network flexibility, in order to integrate and deploy new services or radio access nodes. The functions of the networks will be distributed and accessed from the best location to reduce delays. Operation and management will also has to be improved. The flexibility and densification of the network will demand the improvement of SON techniques as well as of the simplification and standardization of OAM interfaces. To reduce CAPEX and OPEX infrastructure sharing techniques are being developed, tools to integrate third parties deployment are also on the way. Virtualization of the full is a key concept to target this objectives.

Future work: PerformLTE Evolution
PerformLTE is evolving to enable 5G experimentation for the future Internet. Figure 4 depicts the target architecture of PerformLTE. As represented, UMA expect to operate a very dynamic testbed where different types of solutions could be deployed and combined easily.
The research roadmap for PerformLTE covers improvements in all the aspects of the testbed, our goal being to support experimentation and research on future wireless technologies and to contribute to the materialization of future 5G standards. In the next cycle of functionality, PerformLTE will strengthen its current research methodology, broadening the spectrum of scenarios to enable advanced applied research in real-life conditions.

Fig. 4. PerformLTE Evolution
PerformLTE will evolve in the field of SDR (Software Defined Radio) and open source protocol stacks that could provide flexible use of spectrum, established as requirement for 5G networks, and enabling researchers to modify to the communication stack, that will potentially boost reliability, quality and interoperability at the same time speeds up 5G development. One of the platforms that is being explored is OpenAirInterface, an open source solution that implements the stack of user equipment, base stations and core network.
Extensions of the core network to evolve toward Release 12 are also foreseen. The testbed will include support for non 3GPP radio access technologies, in particular Wi-Fi handover is envisioned, and support for broadcast services based on the eMBMS architecture. The core network is currently being modified to enable remote creation of network elements to support the definition of complex network architectures in an automated fashion. The team in UMA has also an implementation of a small core network that will be extended to support NFV and SDN. The transport network will integrate OpenFlow equipments and the IMS architecture is currently being deployed. APIs to the core network will be provided to enable on demand QoS reservation by OTT applications.
In the radio access network, modifications of the conformance testing equipment are also envisioned to support communications with standard core networks. This communication will provide very complex system configurations, where researchers could test core network improvements in any licensed band even considering the effects of the wireless channel. Furthermore the testbed has reached collaboration agreements with more suppliers in order integrate more emulation and conformance testing equipments. New commercial off-the-shelf (COTS) solutions will also be integrated both in the radio access and in the user side, this new elements will be in charge of introducing standardized commercial functionality, and can be combined with wireless channel emulators to have controlled propagation conditions. Automation of the Operacion And Manteinance (OAM) interfaces will also be very useful, as permits the fast integration of heterogeneous network equipment.
Agreements with commercial operators are also being negotiated, and UMA expects to be able to extract more information from commercial deployments and even to test new network solutions in real-life urban radio access networks, boosting the realism of PerformLTE to a new level.

Conclusion
This paper introduces PerformLTE, an LTE tesbed which enables the deployment and execution of complex LTE experiments in three different ways. As a first option, the experimenter can use the most controlled and configurable platform, a complete proprietary LTE network build on top of the LTE testing equipment employed by certification laboratories. Once the configurations have been evaluated in this scenario, the same experiment or new experiments can be validated with real eNBs deployed in a proprietary LTE network integrated by commercial pico-cells and an EPC. In the last scenario, PerformLTE allows the remote evaluation of the experiments providing access to off-the-shelf devices connected to LTE commercial networks deployed in Malaga. The mobile devices used in the three scenarios are Android phones which are instrumented to monitor radio and IP traffic parameters. The usage of commercial devices is a key factor in testing real applications, monitoring radio parameters and measuring application's performance and QoS as perceived by final users. In addition, we provide post-processing tools to further analysis and correlate the data collected. The tool chain allows the inspection of the IP traffic both at source and destination to obtain typical communication statistics such as packet loss rate, throughput, inter packet arrival, end-to-end delay and jitter. Combining this information with information provided by the eNB emulator, pico-cells and EPC we can obtain very useful cross layer information and correlations. PerformLTE also incorporates a DC Power Analyzer to cope with the challenge of optimizing battery life. The development of better batteries still presents significant challenges and the deployment of energy saving mechanisms across the range of devices, technologies and services becomes highly relevant when trying to increase mobile devices' autonomy and their intensive usage.