Gartner GRC Magic Quadrant Rant, Part 3
Uh Oh, It’s Magic, Gartner’s Got a Hold On You . . .
Tossing and turning, anxiety is stirring me. I am trapped in a labyrinth of quadrants with flying dots that do not make any sense coming at me from all directions. One appears in front of me, I am startled. I remark, “you do not belong here, that does not make any sense, you should really be over in that quadrant.” All around me I eerily here the 80’s group The Cars singing “Uh Oh, It’s Magic, Gartner’s Got a Hold On You . . . “. I tremble. I am overwhelmed . . . I wake up screaming, covered in sweat. My wife once again, as she has done so many times this past month, looks over at me and offers me a Xanax, yet again.
OK, it is not quite that extreme – but it is bad. I have lay awake in bed until two in the morning many nights over the past four weeks pondering the black magical depth of the Gartner GRC Magic Quadrant. Perhaps depth is not the right word – more like the mysterious shallows. Actually, I cannot tell you how deep or shallow it is as Gartner gives me no indication of the depth of their analysis. We are left to assume Gartner has depth and objective criteria and detail to their analysis. Where is it? I am unable to reconcile how Gartner came to this place yet again. It is like Gartner is playing mind games with me – intentional infliction of emotional distress.
GRC. I take it seriously. The GRC market is something I have been tending and caring for since February of 2002 in my early days at Forrester. I have watched the market for GRC solutions, services, and content grow and mature. I watched it grow in GRC 1.0 (2002-2006) as it grappled with SOX and internal controls but yet I knew it was going to do much more than that. The breadth was apparent in the Forrester GRC Wave that I wrote and and it grew rapidly into GRC 2.0 (2007-2012). In the second Forrester Wave it had advanced so much there were four separate Wave graphics as it could not be contained and represented in just one two-dimensional graphic any longer.
Then it happened – the separation. Forrester and I parted ways six years back. The GRC market (which is technology, services, and content that supports GRC strategy and processes) became a joint custody arrangement between Forrester, Gartner, and myself. I continued to see that GRC is a broad market with a lot of segments and sectors within those segments. The proper way to understand the GRC market is as an ecosystem of offerings and as a GRC architecture within a specific organization and not as a single platform. However, the other custodians – they kept GRC back into one two-dimensional graphic. Where I used four graphics before leaving Forrester, Forrester went back to a single graphic. Gartner did the same, but worse. While Forrester objectively tries to model GRC in a way that is transparent and publishes the criteria and scores used, Gartner simply states here is the grade I think you should have and gives us no transparency into how GRC solutions are objectively measured. There is a lot of truth to the Magic Quadrant being Magic – it is beyond our comprehension.
This is my third rant against Gartner on GRC Magic Quadrant. For the past four weeks I have been pursued by many to respond to the new version released in September 2013. I guess I have a loyal following of GRC groupies that are crying foul, down with injustice to GRC! I struggled with responding yet again. I do not want a reputation as an aggressor – it does not interest me. However, I am an idealist to the core and have a soft heart for the mistreated and maligned . . . so I lay awake late into the night fretting over Gartner and their 2013 Magic Quadrant for Enterprise Governance, Risk and Compliance Platforms.
For those interested in the historical back and forth, my previous rants are:
- Round 1 (2009): Gartner’s EGRC “Arcane” Magic Quadrant
- Round 2 (2012): Rethinking GRC: Analyst Rant, Gartner’s 2012 EGRC Magic Quadrant and a recap of the back and forth between French Caldwell and myself Concluding the GRC Analyst Rant
In all fairness, I do really like French Caldwell. He is a very gracious nemesis and we have some great discussions. While we debate, and at times collaborate, he is always very engaging and polite. I tell myself it is not French it is Gartner and their confounded approach and process to the Magic Quadrant. That allows me to continue to be cordial and attempt to be half as gracious as French is toward me when my hackles are raised and I am screaming at the injustice done to the GRC market.
There is a lot I would like to say about vendor positioning in the Magic Quadrant, but most of it I will not. Perhaps if you take me out for pint in a nice British Pub (going to London next week) you will get the depth of my thoughts with the dirt and praise on specific vendors. I hold back particularly because I accuse Gartner of not showing objective criteria and scores that map vendors on their graphic and would be doing the same if I tell you where vendors should be positioned and do not give you specific criteria and scores. While I provide my commentary below, I will be agnostic when it comes to specific vendor names.
My grievances with the 2013 Magic Quadrant for Enterprise Governance, Risk and Compliance Platforms are:
- Consistency. When you read the strengths and cautions on the vendors in the MQ and know these products personally as an analyst you see issues. For example, when one (actually a few) is beaten up because a few clients have referenced implementations greater than six months yet several in the Leaders quadrant have implementations on a regular basis greater than a year and some for over two years – we are not comparing apples to apples. One RFP I assisted with selected a prominent Leader against my recommendation. I specifically told them the Leader does some good things but they will come in well over budget and well beyond their six month implementation plan. Two years later . . . guess what, still rolling out and way over budget. Or consider when I have to tell attendees (from three different organizations) at my GRC workshops (recent) to stop complaining about their GRC solution (again in the Leaders) because they keep turning the workshop into a gripe session about the vendor’s missed expectations, length of implementations, being over budget, and the amount of staff and services needed to maintain what they were told was so simple and easy to configure. It irritates me as this gets referenced as a caution for some, with an implication that it lowered their score, but for the greatest offenders it does not appear to be an issue. And some get dinged for just over a six month implementation as opposed to years for others. I do not get it. I want transparency in the MQ.
- Where’s the Beef? One would assume that Gartner assesses solutions against a defined set of required functionality (that is the assumption and very words of my friend Norman Marks in his rant with Gartner). It would be nice to believe – but I am not sure it is true. Honestly Gartner, give us details. Yes, this goes back to the transparency point. This is a huge market with billions being spent. Organizations are making huge financial commitments to solutions based on this two-dimen
sional diagram. How do they stack up? The MQ states solutions were evaluated around risk management, audit management, compliance and policy management, regulatory change management, and incident or case management. That isa great; they are in my taxonomy of the GRC market along with more. Gartner, tell us who is better at each of these and why? I cannot find any detail on how one vendor is better at risk against another. I cannot find any real detail on how one vendor is better at a range of GRC areas against another. So what does your MQ really prove? This is wrong. I can tell you who is better in risk management, audit, or any of these areas whether you were looking for just that solution area or or a GRC implementation that combines these areas. Gartner it is your bloody report; you give us a misleading graphic and no details to back it up. Forrester gives you a spreadsheet with all of the criteria and scores so you can see how vendors score in different areas. This alone makes the MQ not only useless but also absolutely dangerous. Gartner, show us the criteria you measured, the grading scale used , and the scores for each criteria given to each vendor! Forrester does it. The MQ is rubbish without this. I challenge you to be transparent. Good grief, the price organizations pay for your research you would think the depth of criteria and scoring would be made available.
- Depth. I challenge you, my reader, look at the breadth of areas that Gartner states it covers in the MQ: risk management, audit management, compliance and policy management, regulatory change management, and incident or case management. The Gartner MQ for GRC gives vendors a few hours to demo their solution to cover all these use case areas. Gartner, you cannot be serious? I myself could not do justice to the market presenting a comparative ranking of vendors with just a few hours to demo all these areas together. Two hours in just one of these areas would not be acceptable – particularly when it impacts a market that is over a billion dollars and this is the go to report for decisions on who to engage. How does Gartner do it? It must be all the time Gartner analysts spend up on those ivory towers where they are endowed with unnatural wisdom from on high and gives them amazing ninja like perception abilities to distinguish solutions in a short demo covering the range of use cases. That must be why they call it Magic as Gartner analysts are really omniscient beings from another dimension.
- Fairness. In fact, I challenged French in person at a vendor conference in Las Vegas last spring on the issue of expecting vendors to cover all of these areas in a short demo and basing a MQ that is the key report by which organizations make significant spending decisions. He said that is the way it works and that GRC vendors have all year to engage him through strategy days to show the depth of their offerings in these areas. That is a serious issue of fairness. There is an unfair advantage toward those willing to fork out the $10,000 to $15,000 a day to educate Gartner on their offerings that others in the Magic Quadrant do not do and some do not have the means to do. Some of this cannot be prevented as vendors seek to gain Gartner’s insight. However, the playing field can become much more fair by allowing vendors a half-day to a full-day to go through their GRC solution. For what Gartner makes from reprints vendors pay to distribute the MQ you think they would invest more time with each solution to go deep into it. Perhaps Gartner would uncover that some in the Leaders quadrant have issues with normalization and aggregation of risk in an enterprise perspective. That some may have issues with the complexity of their platform and how much time it takes to configure. Or how weak one of the existing Leaders is in risk analytics and modeling. Perhaps they may even discovered what they were told was functionality in the system and the demo they saw was smoke and mirrors and not reality in functionality.
- Breadth. Vendors with the broadest use cases covering things like product quality, environmental monitoring, health and safety, legal matter management, 3rd party GRC (vendor/supplier), global trade compliance, automated controls, corporate social responsibility did not seem to have the breadth of these GRC offerings considered. Some of the Leaders do not have as much breadth of GRC coverage as solutions in other quadrants. Even in the Leaders quadrant solutions with broader use cases and functionality seem to have not faired as well. There appears to be a biased toward a field of dreams approach in which solutions that promise to be all things to all organizations and anything can be built and configured on the platform get rated higher than vendors that have working real-world solutions with domain expertise and industry depth for addressing a variety of challenges that do not have to be built or configured (but are still highly adaptable). How is Gartner handling diverse GRC scenarios? Success in a few functional areas is great, but there any consideration for breadth of use across a range of functional areas? And depth of use getting into content and industry specific needs? This is critically important as organizations are headed towards an integrated GRC architecture. Some Leaders seem to have a narrow focus in specific solution areas, yet they appear to be the strongest “broad” GRC platform in the MQ, which they are not. I also do not see proper evaluation of content integration as a factor of consideration in GRC offerings, particularly depth of content across compliance and risk areas.
- Requirements to play. Another sore point I have is Gartner’s requirements to be in the MQ. There are a lot of very capable GRC solutions that would love to be in the Magic Quadrant but will never get in because they do not fit Gartner’s specific mold of GRC or they do not meet the every increasing ceiling of requirements. To get in you a vendor has to have a solution that delivers across compliance, risk, and regulatory change management as a minimum (interesting, I see regulatory change management as part of compliance). They need to have at least $12 million in revenue, one-hundred or more customers with live implementations, reference customers for corporate governance activities (seriously, I would like to know how many board members or corporate secretaries Gartner actually talked to though Gartner in the MQ relates ERM and financial reporting compliance as governance), be in multiple industries with a worldwide presence. That simply means only large GRC players will be represented in the MQ. And very capable GRC solutions that are new and innovative, operate in just one geography or industry, and have good traction and are growing but have not hit the right level of customers or revenue will not be considered. This cuts out some really great solutions that end up not getting to the decision table because Gartner did not include them. This ends up with very frustrated organizations that come to me and ask about solutions to meet their specific industry challenges. I had a tier 1 bank tell me that they did not think Gartner could spell FCPA because every time they asked about it they were sent the Gartner GRC MQ and Gartner could not interact with them on solutions to address FCPA specifically (which every solution in the MQ would tell you they do).
Honestly, the Gartner GRC Magic Quadrant really does not provide what is needed to make business decisions on GRC solutions. It is not complete, is not consistent, and has issues. The best use for it I have found is to start a fire in my fireplace on this cool autumn day. Sorry French, I know it is a lot of work. The whole process seems like a reality show for GRC . . The Gartner Bachelor with a bunch of GRC solution providers in a beauty contest trying to pull off t
he slickest short demo (remember just a few hours) to woo the Gartner Bachelor. I say roll up the sleeves and get involved in the solutions, build relationships, be easy to approach and engage, interact on a detailed basis. Go deep.
Let’s now see if I can get some sleep tonight . . .