Analysis of CQI Traces from LTE MIMO Deployments and Impact on Classical Schedulers

. This paper deals with the classical problem of radio resource management in LTE MIMO, with special focus to the speciﬁc radio channel characteristics in commercial LTE networks. First, we provide some conclusions about the Channel Quality Information traces obtained through ﬁeld testing. From this analysis, we perform a series of experiments by means of an LTE emulator in order to evaluate the accuracy of diﬀerent fading models used in the 3GPP. As a ﬁrst outcome of the paper, we infer several ﬁne-grain channel characteristics useful to feed further research work in the area of LTE MIMO. As a second contribution of the paper, we analyze the performance of classical schedulers (i.e., Round Robin, Best CQI and Proportional Fair) under the obtained LTE MIMO channel conditions. Speciﬁcally, we analyze the impact of the channel feedback reporting rate in scenarios with multiple greedy sources.


Introduction
Nowadays, Long Term Evolution (LTE) technology has widespread support through all the continents. According to the Global mobile Suppliers Association (GSA), as for January 2015 there are 360 commercial LTE networks deployed in 124 countries [1]. From the five different LTE device categories defined in 3GPP TS 36.306 (until Release 9) [2], actual deployments only support up to Category 4. This feature implies a peak downlink bitrate in LTE networks of 75Mbps when single-input single-output (SISO) is used, and 150Mbps in multiple-input and multiple-output (MIMO) transmission mode. Additionally, 49 of these networks support LTE-Advanced (LTE-A) (3GPP Release 10 and on) with Carrier Aggregation (CA) and peak data rates of 300 Mbps for MIMO-enabled Category 6 devices.
Jointly with these worldwide deployments, there is an increasing interest from the research community to study the performance of different proposals (e.g., channel-aware schedulers, channel-aware application servers, etc.) in realworld conditions (e.g., [3]). Although well-known LTE simulators [4][5][6] have been widely used in the past years, it is important to analyze and characterize the real-world radio performance in different conditions and mobility patterns.
The quality of the radio channel may be estimated through a series of radio parameters, i.e.: Received Signal Strength Indicator (RSSI), Reference Signal Received Power (RSRP), Reference Signal Receive Quality (RSRQ) and Signalto-Noise Ratio (SNR). Most of these radio parameters can be monitored in regular LTE smartphones though dedicated software (e,g, [7]). Beyond all these power-related measurements, the key parameter to determine the performance of User Equipments (UE) in LTE is the Channel Quality Information (CQI). This parameter determines the coding efficiency [8] supported to meet the target Block Error Rate (BLER) of 10%. Under good radio channel conditions, the CQI value reported by the UE to the E-UTRAN Node B (eNodeB) would be high, resulting in a high Modulation and Coding Scheme (MCS) index and better radio efficiency.
UEs may be requested to report CQI values in a periodic basis and to send aperiodic CQI reports upon request by the eNodeB. Reported CQI values can be configured to a unique value for the whole bandwidth (wideband CQI) or different CQI values for determined frequencies (sub-band CQI). In addition to CQIs, UEs are requested to report the Rank Indicator (RI) parameter in MIMO transmissions. This parameter is critical to understand if UEs estimate that two spatially multiplexed layers shall be used (RI=2) or if it is better to receive the same information from different antennas (RI=1). When reported periodically, the CQI Reporting Rate (CRR) determines the frequency for sending quality feedbacks in the uplink [9]. The frequency for reporting RI values is usually configured as a multiple of CRR value.
Unfortunately, CQI values cannot be generally accessed though commercial LTE devices and UEs specifically enabled for drive testing are required. Few studies provide detailed information concerning CQI values. In [10], authors illustrate the temporal evolution of CQI values in stationary and mobility conditions in one LTE network with 10MHz of bandwidth. However, no information is provided about the RI value and consequently the specific MIMO mode at each moment is unknown. In [11], an almost linear relationship between CQI and RSRP values is depicted. However, these experiments are performed in an experimental testbed and not in real-world commercial LTE networks. Additionally, the experiments are based on SISO transmissions and the applied fading channels are not detailed.
Most of LTE performance studies are based on other radio parameters. Generally speaking, measured SNR is used to estimate the expected BLER in different coding schemes, and thus to generate the most accurate CQI value for the current conditions. However, this relationship may be complex in MIMO scenarios [12] and its actual implementation is up to device vendors. Thus, SNR values are not always an accurate estimator of the reported channel quality and CQI traces are generally preferred to analyze the experienced performance. As shown in [10], the relationship between SNR and CQI at different antennas depends on the experienced channel variability at each antenna. Additionally, SNR values are not always accessible from LTE devices. In [13], authors propose a method to estimate SNR values from RSRP and RSSI measurements. Finally, some other authors try to analyze the experienced performance and its root causes by profiling application-level data [14][15][16].
In summary, it is evidenced that analyzing and characterizing the behavior of CQI values in modern LTE networks is a very challenging but appealing problem. This paper is intended to provide some evidence about the expected radio channel characteristics, focusing on the downlink of LTE MIMO transmissions. In Section 2, beyond providing raw CQI traces, we identify several specific effects and limitations that we found when performing drive tests, and we discuss the main causes of these issues. Additionally, in Section 3 we show a basic performance study concerning classical radio resource schedulers under the identified propagation conditions. Finally, Section 4 provides the conclusions to the paper.

Characterizing LTE MIMO channels
This section deals with the first objective of the paper, namely the characterization of downlink radio channels in deployed LTE MIMO networks. As explained before, instead of using radio parameters such as RSSI, RSRP, RSRQ or SNR, we analyze the behavior of CQI traces obtained from real-world measurements in commercial networks. Regular LTE devices do not provide these CQI values to drive test tools, and thus specialized equipment must be used to gather CQI traces from outside the operator. As we will discuss in this section, mapping lower level radio parameters (even SNR values) to CQI is not always a direct relationship. First, specific mapping schemes are open to vendors implementations. Second, the values for radio parameters that can be gathered by different drive test tools only provide limited information in LTE MIMO scenarios.
As a result, we make use of a specialized drive test toolset that allows us to gather CQI values and analyze their relationship to other parameters. However, even this kind of specialized equipment imposes a series of limitations to the collected CQI traces that need to be studied and addressed.
As a step forward, we use an LTE emulator to run a series of experiments taking into account different LTE Test Points used by the 3GPP in TS 37.901 [17]. This way, we are able to compare the coarse-grain characteristics of the drive test traces with the experimental traces, and to detect some behavioral patterns. Finally, we infer a series of fine-grain characteristics that could be used to derive CQI models useful for realistic LTE MIMO simulations.

Coarse-grain CQI traces through field testing
At an initial step, drive tests were performed to gather the performance experienced by UEs in different live LTE networks. Test data were gathered in different mobility patterns and different cities in the North of Spain, using commercial cells deployed in the 2.6 GHz band (LTE Band 7) with 20 MHz of bandwidth.
The test equipment consisted of a laptop connected to LTE with a Samsung GT-B3730 USB dongle and running ASCOM TEMS Investigation as drive test tool. This USB dongle is an LTE Category 3 device, which imposes a series of constraints in the measurements.
First, Category 3 devices are capable of operating in SISO and MIMO with a maximum of two layers for spatial multiplexing in the downlink [2]. Since October 2014, several deployments in Spain support LTE-A with Carrier Aggregation of 40 MHz, by mixing downlink transmissions from 1.8 GHz and 2.6 GHz bands. However, Category 6 or higher devices are required to support CA and therefore the downlink bandwidth is limited to 20 MHz with Category 3 devices.
Secondly, Category 3 devices are limited to support up to 75 Mbps in the downlink with one layer, and 100 Mbps with two spatial layers in MIMO. This limitation imposes a restriction in the MCS value that can be assigned to an UE in MIMO (MCS of 23 for 20 MHz cells). Category 4 devices do not impose this limitation, and therefore 150 Mbps could be achieved in downlink. However, this effect does not impact generation and reporting of CQI values since the limitation is applied in the eNodeB.
In addition to UE capacities, the configuration of the deployed LTE networks has an impact on the experienced results. As an example, Figure 1 illustrates an excerpt of 260 s from a drive test trace obtained from TEMS Investigation. In the first subplot, we show RSRP and SNR values reported by the device. Ideally, SNR is used to calculate CQI values in the two available antennas, although the actual mapping is open to device manufacturers. As can be observed, UEs mobility implies high variability and wide range of SNR values.
In the second subplot, we show the collected CQI values reported by the two antennas. First, it must be noted that reported SNR values are aggregated for the two antennas, being difficult to infer the contribution of each antenna and the expected individual CQI values. Second, it can be observed that the second CQI value is either zero or the same as the first CQI value.
This effect must be understood by taking into account the Transmission Mode of the LTE cell [8]. The monitored networks are mainly based on Transmission Mode 3, which allows one of the following transmission schemes at each moment: -Transmit Diversity or Space Frequency Block Codes (SFBC). In this MIMO scheme, the eNodeB transmits the same information through two antennas. CQI values reported by UEs are associated to the best channel condition from the two antennas and RI values are set to one. -Large delay CDD or Open Loop Spatial Multiplexing (OLSM). In this MIMO scheme, the eNodeB transmits different information through two antennas and higher bitrates can be achieved. CQI values reported by UEs are associated to the best condition achievable at the two antennas simultaneously, which is the worst of the two channel conditions. Reported RI values are set to two in this case.
As a result, based on the radio performance experienced by two different reception antennas, the MIMO scheme varies along time as illustrated in Figure 1. UEs are in charge of generating both CQI values and estimating the most beneficial transmission scheme that maximizes the downlink bitrate. From that estimation, UEs report a single CQI value together with the preferred RI.
The main implication of this effect is the complexity to characterize the behavior of CQI values directly from CQI traces. Since different fading conditions may affect to different antennas, similar SNR values may result on higher or lower CQI-1 values if SFBC or OLSM is preferred. Thus, it must be noted that CQI-1 and CQI-2 values reported by the drive test tool are not directly those estimated for the two antennas. Instead, those CQI traces would include some periods of individual CQI values and some periods of the maximum CQI achievable at both antennas at the same time.
As a step forward, we processed the captured traces isolating the different transmission schemes. Figure 2 illustrates these results for a series of traces in similar mobility conditions at 20 km/h. The left plot in Figure 2 illustrates the Empirical Cumulative Distribution Function (ECDF) of CQI values. As can be observed, OLSM periods provide lower CQI values than SFBC periods. Although this fact is observed in most of the performed drive tests, several scenarios provide high CQI values also for OLSM, when both antennas experience good channel conditions at the same time.
In order to provide some further evidence of the different characteristics experienced by CQI in both transmission schemes, the right plot in Figure 2 shows the ECDF of monitored CQI burstlengths. In this paper, CQI burstlengths are considered as periods of time with non-variable CQI value. At this time scale, CQI values seem to be less variable for OLSM periods than SFBC periods although for both of them maximum burstlengths are around 15 s. Figure 2 also provides evidence of another limitation of the drive test toolset concerning the granularity of the samples. As can be observed, CQI burstlengths start with a lower value of 1 s and this fact is not caused by either the network configuration or the drive test software.
Monitored LTE networks are configured to operate with Transmission Mode 3, requesting UEs to send wideband CQI reports every 5 ms and RI feedbacks every 80 ms. The former value determines the granularity of CQI measurements at the eNodeB, while the latter value limits the frequency for switching between transmission schemes. Likewise, the drive test software is configured to gather test data every 200 ms from the UE. Therefore, the limitation seems to be related to the granularity used by the LTE device to report radio parameters to the drive test software. In this sense, it is not clear how these values are generated from the whole set of values in the last period.
In summary, we state that specialized drive test tools, as used in this paper, provide significant information to understand the experienced performance of radio channels in LTE MIMO through the analysis of CQI values. However, in order to characterize its temporal evolution, we must identify the transmission scheme at each moment. Additionally, although useful for coarse-grain estimations, the granularity of CQI traces is too low and invalidates the results to generate fine-grain CQI traces aimed to feed further LTE simulation studies.

Fine-grain CQI traces through LTE emulation
Taking into account the previous conclusions, we performed a series of experiments in order to understand the possible behavior of CQI traces at sub-second scope. The most accurate methodology would involve accessing real-world traces from deployed commercial eNodeB equipments, which is problematic due to accessibility / privacy and performance issues. Even in that case, the trace granularity would be limited to the CRR value configured at the mobile network operator, i.e. 5 ms in the monitored networks.
Therefore, this paper addresses the problem from an alternative methodology, which is illustrated in Figure 3. We used the Aeroflex 7100 LTE emulator to gather finer-grain radio measurements, which provides CQI samples up to the practical limit of 2ms in CRR. This emulator performs the role of the eNodeB, since it creates the LTE radio signal and all the necessary LTE protocol events to support the attachment and registration of the Samsung GT-B3730 dongle through a radiofrequency cable. In order to emulate the overall radio conditions of the field tests, the LTE emulator is properly configured with the monitored LTE radio characteristics in terms of frequency, bandwidth, power levels, etc. Additionally, we use the Aeroflex 7100 Test System Application Programming Interface (API) in order to dynamically modify the target SNR according to the values collected by TEMS Investigation in a per second basis. The LTE emulator is configured to request CQI reports from the UE every 2 ms, and the received CQI values are logged and saved to a file for further offline processing. At the same time, TEMS Investigation is used to collect coarse-grain statistics in different experimental conditions in order to check reported values against real-world data. -EVA5: EVA occurs frequently in deployments.
-EVA70: Adds EVA70, high SNR coverage which is common in low frequency (< 1 GHz) band networks. -EVA200: Covers high Doppler, high SNR scenario which is common in high frequency (≥1 GHz) band networks. Figure 4 illustrates obtained experimental coarse-grain results in terms of CQI ECDF (upper plot) and CQI burstlength ECDF (lower plot) as reported by TEMS Investigation. This figure also includes the statistical properties of the selected field test CQI traces, separated between OLSM and SFBC periods. As expected, none of the propagation conditions perfectly matches any of the real-world traces. However, we can observe specific trends as follows: -CQI traces in MIMO Transmit Diversity periods are closer to EVA5 fading condition in both CQI and CQI burstlength first order statistics. -CQI traces in MIMO OLSM periods are closer to EVA70 and EVA200 fading conditions in both statistics.
We may interpret these results considering the LTE Transmission Mode 3 and possible switching between the two alternative transmission schemes. As explained before, SFBC is used when one of the antennas experiences a significant better channel condition compared to the other antenna. In this case, it seems that the best antenna follows a less severe fading condition such as EVA5. OLSM is used when the estimated bitrate using two layers in the worst radio condition achieves a higher bitrate than a unique layer through the best antenna. Therefore, OLSM CQI traces are associated to the more severe fading condition of the two antennas. Although high CQI values have been monitored in OLSM, most of the field test experiments provide lower CQI values and are therefore associated to more severe fading conditions. In order to infer the possible behavior of CQI at sub-second level, we use the experimental CQI traces reported by the Samsung GT-B3730 and captured at the LTE emulator. Figure 5 illustrates the results of the actual CQI traces at 2 ms of reporting period. In the upper plot, we confirm that aggregated CQI values reported by TEMS and actual CQI values reported by UE to the eNodeB follow similar statistics. However, CQI values over time are much more variable than coarse-grain reports. In the lower plot, we illustrate the ECDF of CQI burstlengths and we confirm that burstlengths beyond 200 ms are hardly probable in any scenario.
Additionally, we observe that EVA5 fading channels exhibit higher burstlengths than the other two fading conditions. This feature may be benefitial to avoid low-level retransmissions due to fast channel variability, and can be exploited by channel-aware scheduling strategies to optimize the performance of the cell.

Impact on classical scheduling strategies
In order to evaluate the impact of the obtained channel characteristics, this section presents the results of a simple exercise concerning the introduction of experimental CQI traces into the analysis of LTE eNodeB scheduling strategies.
We analyze the performance of well-known scheduling algorithms (Round Robin, Best CQI and Proportional Fair) under the obtained LTE MIMO channel conditions. Specifically, we focus on the impact of CRR in a controlled simulation-driven LTE scenario due to the significance of CQI feedback granularity in variable radio channels.

Simulation methodology
In order to introduce the required experimental CQI values in the simulations, we implemented a basic LTE simulation environment in MATLAB, where obtained LTE MIMO CQI traces are configured as input vectors for the event-driven simulation. The main features considered in the simulation code are the effect of different multi-user scheduling strategies and the effect of retransmissions due to inaccurate channel estimations associated to CRR and CQI delay values.  Once the scheduler decides the number of assigned RBs per user, the eNodeB determines the MCS and the Transport Block Size (TBS) for each user according to [8] ( Aimed at analyzing the impact of the CRR in classical schedulers, we consider CRR values employed in commercial LTE networks [17]: 2, 5, 10, 20, 40, 80, 160 (in ms). Moreover, we assume a CQI delay of 3 ms caused by the CQI measurement at the UE side, the CQI feedback transmission and the CQI processing at the eNodeB [18]. This way, the resource allocation algorithms use partial channel quality information. If the available CQI estimation in the scheduler is higher than the actual CQI at the UE, we force a retransmission in the following Transmission Time Interval.
We have carried out simulations of 1000 s length. We consider 20 greedy traffic sources in the cell, each traffic source corresponding to a mobile user. We consider two classes of users in reference to the channel characteristics: 10 users follow the obtained EVA5 CQI model and 10 users follow EVA200. Concerning these two channel models, we shall note that EVA5 provides lower CQI variability and higher CQI values.

Analysis of results
We focus on the study of the achievable aggregated cell throughput as a measure to evaluate the performance of the different scheduling strategies under the described LTE channel characteristics and CRR configurations.
First, we analyze the results for the aggregate values taking into account the two considered classes of users.
Left-graph in Figure 6 shows the results for the effective throughput, which refers to correctly transmitted data (without taking into account retransmissions). Right-graph of Figure 6 illustrates the total amount of traffic used in the cell for retransmissions due to inaccurate channel estimations at the eNodeB. As can be observed, channel-aware disciplines exhibit severe degradations as CRR increases, while RR shows a more stable performance. Furthermore, it is clear that channel-aware policies cause more retransmissions than RR; with a notable increase with the CRR value.
This effect is mainly associated to the inherent behavior of the analyzed schedulers. Channel-aware schedulers prioritize the transmission of traffic associated to users with high CQI values. However, those channel states exhibit more variability and the probability to transit to channel states with lower CQI is high. As a result, the eNodeB would overestimate the transmission capacity for those users leading to higher error probability in the transmissions. In the case of RR, users are selected regardless their channel condition. As cited before, users with lower CQI values are associated to less variable channel states, thus leading to lower transmission capacity but lower error probability.
As a result, the overall performance of BC provides higher bitrates in the lower CRR values (up to CRR=20 ms) and exhibits more severe degradations for higher CRR values where the probability of inaccurate CQI estimations increase.
Beyond aggregate cell performance values, Figure 7 illustrates the throughput results of the two classes of users; the left-graph belongs to users in better channel conditions (EVA5) and right-graph belongs to users in worse channel conditions (EVA200). EVA5 results in higher CQI values and less CQI variability compared to the EVA200 channel model. In the left-graph, we can see the prominent improvement of BC for the class of users in good channel conditions, while users in bad channel conditions are penalized. In these conditions, RR behaves as the fairest scheduler and the most reliable against high CRR values.

Conclusions
The main contribution of this work is the characterization of downlink radio channels in deployed LTE MIMO by means of CQI traces obtained from realworld measurements in commercial networks. We state that, although specialized drive test tools provide useful information to understand the achieved performance of LTE MIMO channels, the granularity of CQI traces is not enough for characterize its temporal evolution. For that purpose, we achieve to obtain CQI samples each 2 ms by means of Aeroflex 7100 LTE emulator for different fading conditions. We conclude that CQI traces in MIMO Transmit Diversity are closer to less severe fading conditions (such as EVA5), and that OLSM CQI traces are associated to more severe fading conditions (such as EVA70 and EVA200).
Although this is not an intrinsic feature to the different MIMO transmission modes, it has to do with the fact that MIMO OLSM uses the lower of the different CQI values. In the scenarios analyzed in this paper, it is hardly probable that all the antennas are in good channel conditions. Anyway, the results obtained in this paper do not preclude the possibility of experiencing high CQI values in MIMO OLSM in other scenarios.
Furthermore, we briefly depict the performance of classical schedulers under the obtained LTE MIMO CQI traces for different channel quality reporting periods. We consider a simulation scenario with two classes of users that differ in channel characteristics, EVA5 (good conditions) and EVA200 (bad conditions), driven in both cases by the coarse-grain SNR values. As main conclusion, we state that for CRR values lower than 20 ms BC exhibits the best performance in terms of throughput, even though it causes a notable number of retransmissions.
As future work, we identify two main areas for study. First, obtained CQI traces could be used to derive a parametrized LTE channel model that provides accurate statistical/temporal LTE channel information. Second, in order to avoid the excessive wasted throughput due to retransmissions, channel-aware scheduling policies that cope with partially observable channels should be studied/proposed. Therefore, a future CQI-based channel model will be useful to feed up channel-aware schedulers employed in LTE networks.