I’m Not That Great a Forecaster: Looking back on my past predictions and learning how to improve

In early January of 2013, I wrote two posts that outlined six conflicts to watch for civilian protection advocates in the coming year.  Without any concrete methodology, I picked out Sudan, Afghanistan, Mali, Kenya, Syria (specifically violence in a post-Assad Syria), and Central Asia.  Arguably, I was wrong in five of the six cases if the criteria is that the violence had to get significantly worse from 2012 to 2013 in the manner that I predicted  (it gets even worse when you think about all 2013 conflicts I omitted; Nigeria, Egypt, CAR, and Iraq all experienced episodes of mass killing that have intensified since 2012).  I’ll briefly outline how I did country by country, address what I did wrong, and because it’s that time of year again, propose predictions for 2014.  If you don’t want to read the whole thing, feel free to skip to the bulleted predictions.

Sudan had a turbulent year, but it’s nothing really out of the ordinary for the troubled country.  #SudanRevolts returned in September and October and prompted a fairly superficial cabinet reshuffle, but not much else.  Violence continued to rage in Darfur, South Kordofan, and Blue Nile.  Although violence increased in Darfur and perhaps South Kordofan, it was not a radical departure for 2012.  In my last sentence of my Sudan section, I briefly mentioned South Sudan.  While the violence in Jonglei between the Lou Nuer and Murle ebbed slightly in 2013, any progress made in the country was obliterated by the crisis that began on December 15th and has enveloped the country.  It’s unclear how many people have died, but it probably constitutes a mass killing. Mali has certainly experienced some violence in 2013, but there is no way that it was worse than 2012.  The French-African intervention was fairly successful at expelling the jihadist-Tuareg alliance from Northern Mali.  Fairly successful elections went ahead and the leader of the initial coup, Amadou Haya Sanogo, has been arrested and will be put on trial.

Afghanistan is probably the only case I got right.  Civilian casualties increased in the first half (and possibly the second) of 2013, marking a reversal in 2012’s trend.  For every success there’s a failure, and Kenya is that failure.  The March elections went off smoothly, and so I was really wrong.  I was right to predict that 2013 would be worse than 2012 for Syrians, but it didn’t happen in the way I thought.  At the time, it seemed very likely Assad would fall, initiating a mass killing of Alawites in and around Latakia.  That didn’t happen.  Instead, massive violence continued against civilian populations throughout Syria mostly with the exception of Latakia.  Finally, like Kenya, I really got Central Asia wrong.  There was not significant violence in any Central Asian country (excluding Afghanistan) this calendar year.  Regime change could have potentially caused conflict, but those pesky autocratic regimes just won’t go away.

So how can I improve?  First, it would have helped if I had had a concrete measurement for what constituted conflict.  Second, it would have made sense to have had a constant metric for assessing why I thought conflict would occur in certain places.  Figuring out what my predicted conflict zones had in common and why they were different from other potential conflict zones would have improved my methodology, even if creating a model from a hunch isn’t the best social science (if someone wants to pay me to blog I promise my methodology will be more robust).  My analysis also would have improved if I had laid out specifically what changes I was looking for and how they fit into a larger historical narrative.  For example, while there was both a history of and a potential for political instability in Central Asia, my only data points were the 2005 massacre in Andijan, Uzbekistan and the 2010 violence in Osh, Kyrgyzstan.

So moving into my predictions for 2014, rather than creating a complete methodology of my own, I’m going to borrow two of Jay Ulfelder’s crowd-sourced forecasting projects as points of reference.  The first is the Center for Genocide Prevention’s online opinion pool (password needed).  The opinion pool aggregates the opinions of currently fifty individuals interested in atrocity prevention to create averaged probabilities about the likelihood of a particular event.  The questions are generally phrased like this, “Before X date, will an episode of state-led mass killing occur in Y country.”  The second is a wiki survey also originating from the Center for Genocide Prevention.  The results demonstrate how much more likely any particular country is to experience an episode of state-led mass killing than other countries during 2014.

In order and with percentages, these are the countries that I think are most likely to experience a mass killing (defined as more than 1,000 civilian deaths) in 2014:

• Syria (95%)
• South Sudan (85%)
• Iraq (85%)
• CAR (75%)
• Sudan (60%)
• Afghanistan (50%)
• North Korea (50%)
• Mexico (35%)
• Nigeria (30%)
• Burma (20%)
• DRC (20%)
• Egypt (10%)
• Mali (5%)
• Venezuela (5%)

My predictions are largely based on the crowd-sourced forecasts from Jay’s two projects, aren’t identical.  I’ll explain why, country by country, below.

Obviously, the chances that a mass killing will happen in Syria are very, very high (the wiki survey is definitely wrong in placing Syria 6th).  However, the opinion pool’s average probability that there will be a mass killing of Alawite civilians in Latakia province stands at 43%.  I think this is too high, and the real probability stands nearer 15%.  For a mass killing of Alawites to happen, the Assad regime would have to collapse or severely weaken.  Demonstrated by my false prediction of his doom in early 2013, Assad has proven surprisingly durable.  Civil wars tend to last a long time, so especially with the increasing fragmentation of the opposition, it’s doubtful Assad will be defeated anytime soon.

South Sudan, Iraq, and CAR all have ongoing conflict that will almost certainly include a case of mass killing, committed either by government or rebel forces, in 2014.  Iraq and CAR definitely experienced a mass killing episode in 2013, and South Sudan probably did, but the concrete numbers to confirm it don’t exist.  I pegged the chances of a future mass killing as slightly lower in CAR only because of the combination of the peacekeeping force and the higher potential for resolution than in South Sudan.

Jay Ulfelder, in his review of mass killing in 2013, wrote of Sudan, “…where the uncertainty is not whether the regime is engaging in mass killing but in how many parts of the country at once and targeting how many different groups.”  He’s right, and unfortunately civilians in Darfur, South Kordofan, and Blue Nile will likely continue to suffer in 2014.  In the opinion pool, a question asks the probability that Sudan will carry out a mass killing of anti-government activists will occur in 2015, and places the average at 31%.  I think this is far, far too high.  Despite significant anti-government protests, the body count has always remained low.  The Sudanese regime is intelligent in this respect, instead opting for mass arrests of protest leaders.  The scarcity of bloody street confrontations decreases the potential of igniting the paradox of repression.  It is also telling that the bloodiest anti-government protest this year happened in Nyala, South Darfur.  Khartoum is happy to take the fight to enemies in the periphery, but remains wary of the threat from the center.  If the government were to significantly weaken, there’s an increased chance it would unleash lethal violence against anti-government activists.  However, despite increasing organization from the political opposition and calls that the regime was about to fall, the NCP remains in power.

Afghanistan probably experienced a mass killing with the Tailban as the perpetrators in 2013, and there aren’t a lot of positive signs for the future.  In North Korea, it’s basically impossible to tell how many people are dying each year in giant concentration camps.  50% is simply a wild guess.

Drug violence in Mexico is out of control, but I’m hesitant to peg a high probability of a mass killing because it’s unclear what share of civilians vs. cartel members are killed in drug violence (in a tweet, Jay confirmed that cartel members count as combatants).  Bottom line: a lot of people will die in Mexico in 2014, but it may or may not constitute a mass killing.

Nigeria experienced a classic case of a counterinsurgent mass atrocity last year, and despite some international attention on the issue, there is still a decent chance it happens again.  Nigeria is 14th on the wiki survey for the chances of state-led killing (too low in my opinion) for 2014 but doesn’t appear as a question in the opinion pool.

Burma is a country that is very much in the news for people interested in atrocity prevention, but I’m more optimistic than other analysts about the prospects for 2014.  Burma’s counterinsurgencies against ethnic-minority armies are long-running, but have recently had quite low body counts.  I see no reason for that trend to stop.  The opinion pool predicts there is a 35% chance of a mass killing of Rohingya in 2014.  No single rioting incident has yet escalated to widespread killing (the Meiktila incident killed dozens, not hundreds).  Despite the massive persecution the Rohingya face, the levels of lethal violence have remained relatively low.  Without an obvious trigger, I think more slow-motion ethnic cleaning is far more likely than a full-blown mass killing in 2014.

DRC placed fourth in the wiki survey.  Perhaps this is a bit excessive, but not completely unwarranted.  The combination of a country in transition from autocracy to democracy, the prevalence of armed groups in the Kivus, and intrusive neighbors means the risk of a mass killing in the DRC remains relatively high.

Egypt also suffers from an unfortunate history.  Last year, the government undertook a mass killing in response to pro-Morsi demonstrations.  A similar scenario could repeat itself, violence in the Sinai could rapidly increase, or even less likely, a civil war that involves mass killing could erupt.  All of this is plausible, if not incredibly likely (Egypt is 15th in the wiki survey).

The situation is not absolutely analogous to the DRC’s, but Mali also suffers from a somewhat unstable post-major conflict environment.  The opinion pool average for a state-led mass killing rests at 13%.  I think this is too high (probably suffering from the bias that comes from forecasting rare events) because of the progress Mali has made since 2012, but not terribly so.  In the wiki survey, Mali is 3rd, which strikes me as overly pessimistic.

Finally, Venezuela is a bit of a stretch, but I decided to include it anyways.  Few atrocity prevention advocates are paying attention despite the high political instability and absolutist rhetoric coming out of the Maduro administration.  Though Venezuela appears 82nd on the wiki survey and isn’t in a region of the world that has been prone to mass killing recently, I think a political crisis resulting in a government mass killing is plausible if still very unlikely.

Correction: Jay Ulfelder wrote this in a comment, “One point of clarification about Syria and some of the other cases you discuss: in both the statistical modeling and the wiki survey, we’re looking at the risk that a *new episode* of mass killing will start, not the risk that the one(s) we’re seeing now will continue. So Syria could hypothetically get a very low predicted probability or rank if the models or crowd deemed it unlikely that the state would begin deliberately killing large numbers of civilians from a discrete group it isn’t already targeting now. Hence the question in the Syrian case about Alawites but not one about the groups the regime is killing in large numbers now.”  I didn’t realize that when I wrote the post.

Conflicts to Watch in 2013 (Part I)

The Council on Foreign Relations and Foreign Policy have both come out with lists of potential conflicts to watch in 2013.  Both provide good summaries of potential global hotspots, but instead of concentrating on potential geopolitical struggles, I’d like to take a brief look at the conflicts that will likely be important for civilian protection advocates.  While the conflicts in the DRC and Burma, for example, are always at the top of the civilian protection list, I’d like to focus on six conflicts that have the potential to 1) impact civilian populations and 2) take a very different form in 2013 than they did in 2012.  Here are the first three.

Sudan

The insurgencies in Blue Nile and South Kordofan are likely to continue, and the government’s heavy handed response is also likely to stay the same.  While these two issues are currently the country’s two biggest humanitarian crises, they might not even be the biggest problems in 2013.  Growing divisions within the NCP caused what appears to have been a coup attempt in November, and combined with the return of #SudanRevolts, Bashir now looks far weaker than he has in years. Jay Ulfelder’s 2013 coup forecasts puts the chances of another coup in Sudan at number two worldwide, an event which is likely to not only initiate major power struggles within the Khartoum elite, but also involve various factions fighting it out on the ground.  There is also a real danger of a low-intensity war between North and South Sudan along the border, as the North continues to bomb within Southern territory.  In South Sudan, cattle raids between the Dinka, Lou Nuer, and Murle tribes are accruing huge casualties, and there are no signs that the South Sudanese governments will seriously address this crisis.

Afghanistan

Afghanistan has been in the news for years as one of the most violent places in the world, but in 2013, it’s only going to get worse.  In short, U.S. policy in Afghanistan has been a total failure, and the Afghan government does not look ready to govern on its own once U.S./NATO forces begin their draw-down, and eventually leave in 2014.  Afghan security forces are ill-trained and unable to function independently, the government is impressively corrupt, the Taliban remains strong, and Pakistan continues to meddle.  All of these factors point to an uptick in violence in the coming year.  There are few positive signs for 2013.

Mali

Designating Mali as the new Afghanistan is simplistic, but like Afghanistan, Mali is a hot mess.  In March, junior officers angry at the government’s inability to properly supply soldiers fighting a Taureg rebellion in the north mutinied, and then, perhaps accidentally, seized the state.  A few weeks later, Tuareg rebels succeeded in pushing Malian forces at of northern Mali and declared the independence or a new state, Azawad.  Since then, there has been another coup against a prime minister who pulled too hard on the leash held by the original coup plotters.  Despite facades of democracy, the real power in Mali rests in Kati, an army town outside of Bamako.

In the north, things are even worse.  Following the defeat of the the Malian army, the situation in the north collapsed into yet another civil war, as the MNLA, a secular Tuareg group, battled Ansar Dine, an Islamist faction.  The Islamists eventually gained the upper hand.  The struggle for Azawad is a complex mix of ethnic and political affiliations, and this deadly, multifaceted conflict has had a disastrous affect on the civilian population.  The conflict has caused a full-fledged humanitarian crisis, with hundreds of thousands becoming either refugees or internally displaced.  On December 20th, the UN Security Council voted unanimously to back the creation of the international force to retake northern Mali.  The plan, according to both Susan Rice (who called it “crap) and Daniel Drezner, has some problems, “…the Security Council has pledged to send peacekeepers on a timetable that makes academic publishing seem speedy, without any idea of how it will be funded, staffed, or operate with indigenous forces, married to vague calls for political action to lay the groundwork for said peacekeepers.”  With or without an intervention force, the lack of any real progress toward a political solution will mean a long, deadly year for northern Mali.

‘Til the Revolution Comes

*This is the first post of a two part series in reaction to my participation in the Women and Genocide in the 21st Century Conference organized by the Darfur Women Action Group.  This one will focus on realism in advocacy, and the second will focus on the dehumanization of perpetrators of mass atrocities.

The conference last weekend was full of many insightful perspectives and personal stories, but there was one assertion, repeated a few times, that really irked me.  The charge was that Omar al-Bashir is the root of all the problems in Sudan, and that until he is removed by the international community, Sudan will not improve.  During these moments, my mind wandered back to my dad telling me as sixteen-year-old about the dangers of manichean theories, or as he called them, “‘Til the revolution comes” arguments.  These lines of thoughts are not productive, as they downplay the agency of everyone involved and glaze over existing political scenarios.

Firstly, this  approach is simply not realistic.  The ICC has no mechanism to forcibly arrest Bashir, and rather relies on the cooperation of signatories to provide enforcement.  Even if countries refuse to arrest Bashir, the ICC indictment puts long-term pressure on Bashir, limiting his ability to interact diplomatically and decreasing his legitimacy.  The ICC is one way to bring about Bashir’s removal, or at least his restraint, but it is not the only way.   Placing all of our eggs in the ICC basket makes us forget all of the other policy and advocacy options that are available to us.

If the only way for Sudan to improve is for the ICC to apprehend and try Bashir, then us Sudan activists should probably stop reading the Sudan Tribune, forget where Abyei is, and play more golf.  This attitude gives advocates almost nothing to do.  We can ask Kenya and Egypt to take a tougher stance on Bashir all we want, but our chances of success are about as good as Obama claiming an endorsement from the NRA.  The belief that Sudan cannot improve with Bashir in power uses moralizing rhetoric to cover-up political realities.  Omar al-Bashir, is, without of doubt, one of the worst heads of state of the 21st Century (he might even be up there with legends), but he is also beholden to outside interests and powers.  Bashir needs certain things, like a steady supply of arms, cash, and oil.  In order to maintain access to these necessities, he has to be willing to negotiate.  He’s also clearly pretty good at it: it’s not easy to stay in power for twenty-three years.  Therefore, it is possible to use leverage against Bashir to improve conditions in Sudan.  Also, improvement in Sudan does not necessarily have to go through Bashir.  The government of Sudan, while repressive, is not all-powerful, and has a limited capacity to snuff out positive change (also, it as violence against civilians is always in the interests of Bashir).  Sudan can improve, even with Bashir at the helm, and advocates shouldn’t turn to fatalist language to rally support for the cause.

Relying on the international community to bring Bashir to justice implicitly overlooks the ability of Sudanese to make organic political change.  This summer, Sudan was rocked by #SudanRevolts, Bashir is running out of money, and his governing coalition is starting to fall apart.  Movements like Girifna have organized activists and put real pressure on the NCP.  In this scenario, the main actors are the Sudanese, and relying on the ICC as an agent of change is disrespectful to the many dedicated activists risking their lives to topple Bashir’s administration.  The international community can certainly hasten Bashir’s downfall through sanctions and diplomatic pressure, but ultimately it is up to the Sudanese to make political change.  Enforced political change from the outside won’t last, and so Sudan advocates must cease relying on the ICC as an all-powerful, semi-magical tool to bring about regime change.

I’m betting that Bashir will have lost power by the time I graduate from college.  This won’t happen because the ICC finally figures out an ingenious way to arrest him, but because the internal dynamics of Sudan’s politics and economics force him out.  Therefore, Sudan advocates need to plan for the day after Bashir leaves, but also remember that there is a lot of work to be done in the interim.