3.1 Data sources

It is important to emphasize that the UNODC recognizes the quality of the Mexican homicide data as good^[3]. These data consist of 2,673,887 observations from 2014 to 2017. Additionally, it was necessary to make the following assumptions to make the subsequent estimates:

1.Sociodemographic information on the marginalization index is the same for the years under the scope of the study, and there are no significant differences between the actual number of homicides and those reported in the INEGI statistics by the entities. Although a recognizable difference exists, it is believed to be systematic and similar throughout the entities during the period under study.

2.Gender alert and the other variables are constantly maintained for all entities during the period; the quality of reported homicides is the same in all states in the country.

New variables were created to include information regarding the current state of violence and insecurity. Particularly, a variable was created to reflect information related to what is called Gender alert. This mechanism has emerged due to the violence against women. This variable was defined as 1 for every state that has a Gender alert, and 0 otherwise, based on the National Women's Institute. The states that have at least one municipality declared with Gender alert by year are, according to the ^{National Women's Institute, 2020}: a) State of Mexico and Morelos in 2015; b) Michoacán, Nuevo Leon, Veracruz and Chiapas in 2016; c) Sinaloa, Colima, San Luis Potosí, Guerrero, Quintana Roo and Nayarit in 2017.

According to the ^{Instituto Mexicano de la Competitividad (IMCO) (2018),} the Rule of Law Index (RLI) is a global instrument that looks at the Rule of Law comprehensively. It consists of eight factors further disaggregated into 44 sub-factors; the eight factors followed by their number of subfactors (in parenthesis) are: constraints on government powers (6), absence of corruption (4), open government (4), fundamental rights (8), order and security (3), regulatory enforcement (5), civil justice (7) and criminal justice (7). Among the sub-factors that are obviously related with this paper are: effective control of crime, the right to life and security of the person is effectively guaranteed, people can access and afford civil justice, criminal investigation system is effective and criminal adjudication system is timely and effective. The RLI is built from results taken from applied surveys to citizens, lawyers and experts.

In accordance with ^{IMCO (2018a)}, the Corruption Index (CI) is a perception taken from experts and businesspeople about public sector corruption. In fact, Mexico's CI is at the same level of those of countries such as Guinea, Iran and Russia. In the report, it is argued that the decline of basic political rights in México, is one of the principal causes why the corruption in Mexico is limited. Among some suggestions made for Mexico in order to improve its score, is to eradicate impunity by closing the gap between anti-corruption legislation and its application.

^{México Evalúa (2018)}, stablishes that the Criminal Justice System Maturity measures whether the states are able to implement integral public policies focused on strategic planning processes, as well as follow-up mechanisms and an efficient resource management system. Thereby, it indicates if the Criminal Justice System has the necessary elements required in order to have the institutional capabilities and the ability provide suitable results in the prosecution and administration of justice.

The sociodemographic marginalization index (^{CONAPO, 2016}), is a linear combination of the following factors: total population in 2015; % of illiterate population 15 years or older; % of the population 15 years or older with incomplete elementary education; % of occupants of households with neither drainage nor toilets; % of occupants of households without electricity; % of occupants of households without tap water; % of households with some level of overcrowding; % of occupants of households with dirt floor; % of the population in places with less than 5,000 inhabitants; % of the population employed with incomes up to two minimum wages. It is worth mentioning that minimum wage changed in the analysis period (ranging from approximately 124 to about 154 US dollars per month). The distribution of municipalities according to this index is (from more to less marginalization): 283 (11.5%), 817 (33.3%), 514 (20.9%), 498 (20.3%) and 345 (14%). Also, some variables from the Consejo Nacional de Evaluación (CONEVAL) (2018) and the ^{US Department of State-Bureau of Consular Affairs (2018)} were taken into account. In summary, the variables considered are listed in Table 1.

3.2 Data limitations

It is important to recognize the limitations of the data employed in this work and point out that there are other sources not considered here. Among these, there are official sources such as The Executive Secretariat of the National Public Security System (SESNSP) (^{2015a, 2015b}, ²⁰¹⁶, ²⁰¹⁷), and non-official, like those coming from the Mexican newspaper Milenio, or a map of homicides elaborated by an activist. However, there is not enough information about the victims; in general, only the total number of homicides is available. It should be noticed that the VMS does not report homicide classifications such as intentional homicide, killings in self-defense, killing in legal interventions or non-intentional homicide (for details see ^{UNODC, 2019}).

We should also note that we could not consider other factors that contribute to homicides in Mexico, since they are either non-public or unquantifiable, or both (at state or municipal or individual level). For instance: (a) an index that reflects the flux of guns from the United States to Mexico through the so-called "Fast and Furious" agreement from 2006; (b) empirical evidence of the possible association between politicians and criminals, (c) quantifiable features about criminal groups and (d) drug trafficking and drug consumption. We recognize that our paper just tries to explore the phenomenon with public and quantifiable information at hand using a statistical perspective.

Overall, there is no relevant information available about some of the victims' conditions like income, heritage, bank account, use of credit cards, number of children, workplace, leisure activities and so on. Information is also limited concerning the murderer such as family conditions, parental violence, parental drug or alcohol abuse, absence of parents, economic situation of his/her family, his/her criminal or psychological record, educational level, etcetera. It is also important to say that we do not pretend to show any formal theory about homicides using the Mexican case, but we just want to present results obtained from a statistical approach.

3.3 Methods

Our approach relies on ML and DOE methods. Since for each observation in our data set, we have an associated response, which is either homicide or non-homicide, we are dealing with supervised learning in the case where the response is categorical and we are trying to ''learn'' a function that relates the input factors with the response.

Given that cause of death is recorded for each of the deaths reported in VMS, we approached the identification of key factors influencing death by homicide from a classification perspective, as a supervised learning problem, having two possible outcomes, either the cause of death was homicide, or it was not. From the classification perspective, ''homicide”, is the class of interest or positive class, meanwhile ''not homicide'' is the negative class. It should be noted that this homicide identification in the dataset is equivalent to that of the WHO mentioned above (WHO 2015). Thereby, the target variable was recoded as 1 if the case corresponds to homicide, and 0 otherwise. Then, a three-step approach followed in this work is: 1.- Select the best classification model, 2.- Model tuning and variability assessment and 3.- Apply a DOE considering key factors influencing homicide incidence. Model fitting was done in the statistical software R version 4.1.0 (^{R Core Team, 2021}). The caret package written by ^{Kuhn (2020)}, short for classification and regression training, was used for model selection and tuning. Prior to modelling, data was pre-processed to eliminate highly correlated and near zero variance predictors.

Given the significant unbalance in the data since less than 3.62% of the deaths in the dataset were homicides, the accuracy is not appropriate as a criterion to select a model to predict homicides. That is why, we use the area under the Receiving Operating Characteristic (ROC) curve as the selection criterion (^{Fawcett, 2006}). This criterion considers the sensitivity, that is, the model's ability to correctly identify actual homicides (true positive rate), and the specificity, which is the ability to correctly identify the cases that were not homicides (true negative rate). The area under the ROC curve is denoted by AUC, although in the figures, it is identified as ROC.

We split the data in a training set (70%) and a test set (30%), ensuring that the proportion of homicides was the same in both sets as in the original full data set, using then only the training set for model selection. Five-fold cross-validation repeated five times was used to estimate the quality measure, associated with a given ML method to evaluate its performance (model selection), and to select the appropriate level of complexity (model tuning). The folds and repetition subsets were set before model training, thus, ensuring that each algorithm uses the same data partitions and repeats to achieve a fair comparison.

In general, K-fold cross-validation (CV) involves randomly dividing the set of observations in K groups of approximately the same size. One of the folds is treated as a validation set (test set) and the other (K-1) folds are used to train the model; the procedure is repeated for each of the K folds, resulting in K estimates of the AUC, similar to ^{Fujita et al. (2016)}, to evaluate the validity of our model. The process is repeated R times, hence obtaining KR estimates of the area under the ROC curve for each of the models included in the analysis. Five-fold cross-validation is illustrated in Figure 1. It is important to mention that ^{Molinaro (2005)} and ^{Kim (2009)} showed that repeating K-fold cross-validation increases the precision of the estimates, thereby maintaining a small bias.

Figure 1 Schematic of 5-fold cross-validation 

Furthermore, given the high level of unbalance present in the data, prior to training each model for each fold and repetition, the training subset was balanced. There are two commonly used methods to balance the data, the first being oversampling the minority class so that in the training subset, there is almost equal representation of the minority class (homicide) and the majority class (not homicide). The second method, which is the one applied in this work, was to under-sample the majority class in the training set, so that there is almost equal representation of the majority and minority classes in the training subset. Whether under-sampling or over-sampling is used, this is only done on the training subset and not in the validation subset.

We compare eight different statistical classification methods for our analysis, for details we suggest see the cited references. Two linear models: Logistic Regression (^{Agresti, 2002}) and Elastic Net (^{Friedman et al., 2010}). Three non-linear models: Neural Networks (^{Bishop, 2009}), Support Vector Machines (^{Cortes and Vapnik, 1995}; ^{Vapnik, 1996}) and Naïve Bayes (^{Rish, 2001}). And finally, three classification algorithms: CART (^{Shannon, 1948}, ^{Breiman et al., 1984}, Therneau and Atkinson, 2019), C5.0 (^{Quinlan, 1992}; ^{Kuhn and Johnson, 2013}, ^{Kuhn and Quinlan 2018}) and Random Forest (^{Breiman, 2001}).

4.1 Model selection

As a first step, we compared the models pointed above. Our prioritization is based on empirical instead of statistical criteria such as the one proposed by ^{Bengio and Grandvalet (2004)} and ^{Bouckaert (2004)}. Considering only the ROC criteria, the best model was the rule model based on the C5.0 algorithm, followed by the Random Forest as can be seen in Figure 2.

Figure 2 Boxplots comparing performance of eight models 

4.2 Model tuning

As a second step, once the C5.0 algorithm was selected, instead of using a 70/30 mix for training and testing, data was split into training (80 %) and test (20 %) sets to carry out fine tuning of the model (^{Alberg, 2015}, June 14). Given the significant imbalance present in the data, the 80/20 percent split was done utilizing stratified sampling to ensure that in both, the training set and test set, the proportion of homicides was approximately 3.62 %. Given this class unbalance, as done during the model selection stage, training for each fold and repetition, was done down-balancing the data in the subset used for training, so that the number of observations belonging to both classes were approximately the same. The tuning parameters were the number of boosting iterations (trials), with or without variable selection (winnow) and tree or rules. The objective of the tuning process is to maximize the area under the ROC curve.

The tuning grid consisted of 25, 50, 75 and 100 trials, variable selection (yes / no) and type of model (tree/rules) yielding a total of 16 combinations of the hyperparameters. The ten-fold cross-validation procedure is repeated 3 times, resulting in 30 estimates of the area under the ROC curve for each of the 16 combinations of the tuning parameters. This process led us to a best model with 100 boosting iterations (trials), without variable selection and rules generation. Since this final model provided 30 estimates of the area under the curve, these were used to assess its variability, yielding a mean AUC of 93.14% (sd 0.16 %); a sensitivity of 85.00% (sd 0.37 %); and specificity of 88.32% (sd 0.52 %). The overall results are shown in Figure 3, while the top ten results are shown in Table 2.

Figure 3 C5.0 tuning parameters selection. 

The final model was then used to predict homicides in the test data set (20 % of the complete data set). The results are presented in the Confusion Matrix in Table 3, and the ROC curve is presented in Figure 4. The following statistics were obtained: AUC 93.23 %, accuracy 85.00 %, sensitivity 88.73 % and specificity 84.86 %.

Figure 4 ROC curve for the final model with the test sample with hyperparameters: trials=100, rules and without variable selection. 

4.3 DOE to Estimate the Effects of Key Factors Influencing Homicides

The third step in our approach is to estimate the effect of key factors influencing homicides. The factors in Figure 5 were found to be the most important factors in the final model.

Figure 5 Factors marked with an arrow are those that can be influenced directly by changes in public policies and regulations. 

Among the 26 most important factors, we selected those that can be influenced directly, by changes in public policy, laws and/or regulations, and to make recommendations on where efforts must be made to be able to significantly reduce homicide rates in Mexico. The selected factors (predictors), marked with arrows in Figure 5, together with their lowest and highest levels are presented in Table 4, where low and high levels correspond to the extremes of each indicator found in some states of the country.

To assess how these factors can impact homicide rates through the predictive model, we designed a resolution V two level fractional factorial design in 128 runs denoted by 2_V^(11-4). Resolution V designs have the property that main effects are confounded with four factor interactions and two factor interactions are confounded with three factor interactions. The design was generated and its results analyzed with the FrF2 R package (^{Grömping, 2014}). For a complete discussion of fractional factorial designs at two levels, the reader is referred to the classical book by ^{Box et al. (1978)}.

For each factor, the low and high levels were set, respectively to the lowest and highest values for that factor in the country. We ran the 128 combinations with the optimized model, replacing the corresponding factors in all the observations by the levels indicated in the design, obtaining for each one of these 128 executions of the model, a predicted homicide rate at the country level. The values of the other variables remained unchanged from actuals in each of the runs.

After executing the 128 runs, the normal probability plot of the effects (Figure 6 (left)), showed that the only significant effects are: RLI, CCJSI, PP65WASS, GINI, PPLE, EPoverty and CI, as well as the interactions PPCLHS*CCJSI, and RLI*MPoverty and RLI*GINI.

After analyzing the effects and interaction plots, we concluded that, although they were statistically significant, the only effect with practical importance is the RLI. This is not only due to its large impact in magnitude, but also because this is an index that, with adequate public policies, can be directly impacted by its components linked with order and security as well as criminal justice. In other words, from our perspective, given a poor RLI, impunity emerges as an incentive for the continuous consummation of homicides in Mexico.

Figure 6 Normal probability plot of the effects (left) and Effect of the RLI on homicide incidence (right). 

As shown in Figure 6 (right), if all states in Mexico had an RLI = 0.45 corresponding to the state of Yucatán, which is the best in the country, given that all other conditions remained the same, homicide rates could be reduced by up to 46.5 %. On the other hand, if all states in Mexico had an RLI as bad as the one for Guerrero, where RLI = 0.29, given all other conditions remained unchanged, homicide rates could increase by 20.8 %. Certainly not all the states in Mexico have an RLI as bad as that of Guerrero, so, if we assumed that all states had the median RLI of 0.37, moving them to an RLI such as the one for Yucatan, would reduce homicide rates by approximately 23.3 %, provided all other conditions remained without change. Therefore, a very important lever to reduce homicide rates in the country is to implement policies and procedures in all states to achieve a suitable RLI.