Mercury's Blog - Latest Comments

Re: A review of idea and innovation software

jedc_mercury — Wed, 27 May 2009 15:21:40 -0000

Thanks, Jeffrey. Will definitely take a look at it and include it in an updated post.

Re: A review of idea and innovation software

Jeffrey Baumgartner — Mon, 25 May 2009 12:01:17 -0000

Have you looked at Jenni innovation process management? Details at http://www.jpb.com/jenni/index.php

We'd be happy to arrange a demo for you...

Jeffrey Baumgartner

Re: Measuring the prediction market industry - a proposal

jedc_mercury — Thu, 23 Apr 2009 12:14:45 -0000

Actually, I disagree a bit. While each vendor sells prediction market software, each has a different mix of software and services, so even I wouldn't be able to really figure out how well each of them are doing. Even if client engagements are flat year-over-year, if each engagement is of significantly more value (ie, a lot of consulting services), the overall value has increased.

Thanks for the comments; I hope to get this going soon!

Re: Measuring the prediction market industry - a proposal

TorontoBentley — Thu, 23 Apr 2009 09:41:38 -0000

True enough for the general public, but the "aggregator" would know which vendors were doing better than the others. It may affect the reporting. I'd be surprised if any are doing particularly well, unfortunately.

Still, it will be interesting to see the results. All the best in this endeavor!

Paul

Re: Measuring the prediction market industry - a proposal

jedc_mercury — Thu, 23 Apr 2009 08:45:40 -0000

Thanks, Paul.

I'm hopeful that they will release the data. I would hope that it's more important to show that the industry is growing/thriving (my intuition) than holding the info back.

However, only the *aggregated* data would be released, so there's no way to compare vendors against each other. I will also provide each vendor their own data compared to the aggregated data for their own benchmarking. (In other words, you won't be able to tell from this which vendors are growing, just how the industry as a whole is growing.)

Part of this is also due to the nature of clients that each vendor takes on. Inkling cater more (in general) to the self-service customers, while others (such as Newsfutures) have generally fewer clients but with a lot more strategic consulting around their engagements.

Re: Measuring the prediction market industry - a proposal

TorontoBentley — Thu, 23 Apr 2009 08:03:53 -0000

Hi Jed...

A very good idea for a start. It is great that you have some "solid interest" from some of the software vendors. Assuming you do get the data, it will also show whether the industry is growing (net new clients over time). I'm a bit skeptical that they will release actual data on how many paid clients they have, because I don't think the numbers are there, so far. It would also show which vendors are growing. I may be wrong (hope I am). If you look at Inkling's web site, they list a few of their clients, of which most appear to be continuing their use of prediction markets.

Paul

Re: A review of idea and innovation software

jedc_mercury — Sun, 19 Apr 2009 13:22:27 -0000

Thanks, Hutch. I appreciate the additional info.

Re: A review of idea and innovation software

bhc3 — Fri, 17 Apr 2009 15:52:11 -0000

Hi Jed -

Thanks for including Spigit in your write-up. You're right - we are seeing great uptake by enterprise customers. Managing the multiple sources of ideas, and filtering for those that are most useful, is an *idea* that is taking root in the corporate world.

You raise some good questions about us, and I'll answer them here. First, in terms of complexity vs simplicity. The platform is actually quite easy for employees, customers and partners to use. It leverages some of the best practices one finds for social software: votes, tags, discussion forums, clickable reviews, wikis, blogs. We also provide a market for users to buy and sell ideas. Any or all of these are available for use when deployed, internally or externally.

We put a premium on a superior user experience.

The analytics engine and enterprise workflow of the platform is why companies are signing up for Spigit. Simple popularity contests for ideas do give a single metric - number of votes. Certainly that is a form of simplicity. But it's doesn't come close to mapping to the way innovation really works inside corporations. Spigit has designed-in functionality that reflects the different influence users have in advancing an idea forward.

Spigit's platform has enterprise workflow built-in. Role-based reviews and approvals, quantifiable criteria to advance through stages, and high configurability to adapt to each company's specific processes.

Happy to tell you more about what Spigit is up to. Please drop us a note at info@spigit.com.

Hutch Carpenter
Director of Marketing
Spigit, Inc.
http://spigit.com

Re: Follow-up to “Approaching business problems differently”

TorontoBentley — Sun, 05 Apr 2009 17:43:50 -0000

Here is a link to a new blog on practical aspects of enterprise prediction markets, where I also discuss some of the issues holding back acceptance. I'd appreciate your comments when you have time.

http://torontopm.wordpress.com/2009/04/05/pract...

Re: Follow-up to “Approaching business problems differently”

Paul Hewitt — Sun, 05 Apr 2009 11:09:17 -0000

Hi Jed...

I agree with your summary. In addition, I think prediction markets offer several additional benefits, including: faster predictions, continuous updating, a measure of uncertainty surrounding the prediction, and reasonable cost. I'm sure there are a few more, but for now, this should do.

I am strongly in favour of using prediction markets to complement existing forecasting methods, especially where they are able to quantify uncertainty. Your example of project milestones is another excellent use for prediction markets.

Although your research indicates that as few as 15 participants can achieve calibrated results, I am skeptical. All of the market maker mechanisms will ensure that there is market liquidity, but I believe they influence trading behaviour (generally making traders quite risk-seeking), which undermines the accuracy of the predictions. Such market maker mechanisms were designed to allow markets to operate with smaller numbers of participants, but doesn't this degrade the "crowd" precondition for successful market predictions? You aren't likely to have a very diverse group with a small number of participants. The basic theory behind prediction markets is that each trader has a piece of information combined with an error factor, and the aggregation method adds the pieces of information together, with the error factors cancelling out (more or less), resulting in more, accurate information. Having a smaller number of traders not only means having fewer pieces of information to aggregate, but also, the error factors will not "cancel out" properly.

As I see it, one of the major stumbling blocks is getting enough people to be interested in each market (and staying interested). The greatest benefit of prediction markets comes from forecasting the outcome as far in advance as possible, but this runs counter to the traders' need to know the outcome "immediately" (or at least soon). You can see this in most marketplaces where very few people trade in long-term markets. Most trade in the markets that will close in the next few hours, days or maybe a week. These very short term market predictions have little value to a decisionmaker.

If prediction markets are to gain more widespread acceptance in the business community, they will need to focus on ways to ensure their predictions are useful. That is, by providing predictions well in advance of the outcome, accurately (diverse crowd, properly motivated) and with a very reasonable cost. Much more work needs to be done in the area of motivating traders.

Just my thoughts for now.

Re: Approaching business problems differently

jedc_mercury — Fri, 03 Apr 2009 06:47:38 -0000

HI, John.

"Slow burn" isn't a diffusion method or metric, just a description of growth. Not dying, but also not "burning like wildfire." It's continued, steady growth.

I agree that PMs aren't an academic backwater; far from it! Yet they've also not been adopted widely. I have no idea where you see order-of-magnitude growth per year, and would love to see evidence of it. That leads to your last point...

I would love to set up an PM on diffusion and adoption, but am a bit stumped on what exactly to trade! Ideally it would be on some metric of number of clients and/or revenues of the PM software companies, broken out between software and consulting fees. But since everyone is private none of that information is available.

Re: Approaching business problems differently

jheuristic — Thu, 02 Apr 2009 11:26:27 -0000

Jed --

Thanks for the reply. You may wish to look at the 'bible' of technology adoption, "Diffusion of Innovations," by Everett Rodgers --

http://www.amazon.com/Diffusion-Innovations-5th...

I'm not sure there is a diffusion method or metric called a 'slow burn.' By most measures, slow burn would mean failure, at least from the perspective of diffusion science.

PMs are no longer a curious academic backwater. The factor growth you need to see is in play.

Anyway, one view that informs the community is Malcom Gladwell's observation, "The success of any kind of social epidemic is heavily dependent on the involvement of people with a particular and rare set of social skills." More than ever those skills are in place and operational. That's not easy and takes a while.It is a critical success factor.

BTW, is there a market for PM diffusion and adoption someplace? Why not spin one up? It would be popular.

Thanks again, good discourse.

Cordially,

John

Re: Approaching business problems differently

jedc_mercury — Wed, 01 Apr 2009 18:46:36 -0000

Hi, John.

While I certainly see growth, and even strong growth, I haven't seen evidence of order-of-magnitude growth, and don't believe it will happen this year. Last year many people said that 2008 would be the big year because of the US election and all of the press on prediction markets there... hasn't happened. You're saying 2009 will be the big year; I'm just saying that I think it's still going to take some time.

Each year for the past ten years companies have been saying that *that* year was going to be the big year for innovative uses on mobile phones; yet it was only once Apple came out with the iPhone (arguably, when they came out with the 3G iPhone) that the industry finally started to radically change for the better. I'm just wary that if I start saying that *this* is the year for prediction markets, that I'll look like a fool. I'm not going to say it until and unless I have really good reasons to believe it.

If you had more concrete evidence (sales growth, etc.) I would certainly be open to changing my mind, but I haven't seen anything like it.

Again, as I wrote in my post, prediction markets have the capability to explode at any given time; what's limiting their adoption is that they approach the problem completely differently from current solutions. It takes time for people (let alone organisations) to wrap their heads around new ideas.

Re: Approaching business problems differently

jheuristic — Wed, 01 Apr 2009 18:33:15 -0000

Hi - I wish it was a simple as my opinion. Again, I am only sharing the discourse and evidence that's out there in the clusters. Cluster lead technology adoption... all the cylinders are firing for PMs. Same as search, social networks, social media, and others the cluster action/research networks forecast and led. There is a inflection point fast approaching, trust me. John

Re: Approaching business problems differently

jedc_mercury — Wed, 01 Apr 2009 18:24:55 -0000

Hi, John.

While I respect your opinion, I disagree. I think prediction markets will see continued growth for quite some time, but don't believe that 2009 will see "sharp growth." (This depends on what you mean by sharp growth; if we had a market on this question it would need to define this. I would use a metric of an order-of-magnitude growth in the industry in 12-18 months.)

As I wrote in the post, I think prediction markets are a kind of technology that will see a "slow burn" in growth for quite some time; it will be many years before we see any truly dramatic spike. Again, this doesn't mean that they're not useful, just that they're a different enough tool that it's tough for companies to adapt to use them.

Re: Approaching business problems differently

jheuristic — Wed, 01 Apr 2009 15:57:25 -0000

Hi --

The global prediction market community and cluster are more bullish on the ascension of PMs in 2009. Judging from the popularity of your PM Clusters, stunning enterprise case studies, renewed investment and talent, it is safe to conclude sharp growth in PMs in 2009 and beyond. This growth is also consistent with Gartner Research and PMs exiting the "Peak of Inflated Expectations." See:

http://www.pmcluster.com/NYC09.htm

Cheers,

John

Re: Approaching business problems differently

jedc_mercury — Wed, 01 Apr 2009 06:25:46 -0000

Hi, Jacob.

This is partly what I was getting to in my post. Traditional forecasting (such as FMCG for hundreds/thousands of SKUs) will probably be best served by traditional techniques. If the product demand is relatively stable, I don't see how prediction markets would add a lot of value.

But I have seen some impressive case studies for prediction markets where a company is trying to forecast something that's not been forecasted before. Mat Fogarty (of Crowdcast) used prediction markets at EA to forecast quality scores for new games. The prediction market forecasts reduced the error rate in half!

There have also been some great examples of prediction markets being used in project management, specifically to assess the probability that a project will meet specific milestones. (I wrote a bit about that here: http://blog.mercury-rac.com/2008/07/14/rag-stat... )

The general trend here is that prediction markets can be used to ask more strategic questions that are difficult to assess analytically.

In general I agree that the industry needs specific, successful examples of great prediction markets. Inkling has started posting some here (http://inklingmarkets.com/homes/howtouse) but even those don't have real specifics.

I'd be happy to help talk you through some ideas; you can e-mail me at jed (dot) christiansen (at) mercury-rac.com.

Re: Approaching business problems differently

Jacob Rigby — Wed, 01 Apr 2009 00:47:41 -0000

My company is wrestling with this issue right now. We're big fans of the theory of bottom up forecasting via PM, but haven't found many examples of their use outside of Google, Best Buy, and elections. Without compelling case studies most people can't be swayed from their top down models (that seem to work alright)

One example of skepticism comes from a client in the FMCG industry who already has a strong demand forecasting model based on sales data. In a sense aren't their consumers participating a giant prediction market all the time, voting for outcomes by simply buying goods? What information could an employee based PM take advantage of that weekly sales data hasn't already factored in?

Re: Prediction Markets and March Madness

jedc_mercury — Tue, 17 Mar 2009 20:37:06 -0000

I'm not so sure about either your "locked up" or "best academic school"! And the second one's 'fightin' words'... :)

I know it wasn't easy for Michigan this year, but like I mentioned, they almost had to get in after the football teams' collapse.

Re: Prediction Markets and March Madness

vide0star — Tue, 17 Mar 2009 20:29:10 -0000

I'm in it to win it! Might as well stay home, I think I've got it locked up.

I'm gutted Northwestern didn't pull off their first tournament bid. They finally had a decent team and some good shooters. If they had beaten OSU and Minn at the end of the season, I think they would have been in. I'll guess I'll just have to settle for going to the best academic school in the big ten :-)

Re: Evaluating prediction market success in the 2008 election (…or, why wrong is right)

Fryzury — Thu, 12 Mar 2009 11:46:12 -0000

There are also hypotheses that economic conditions are an important determinant of election results, and that market prices reflect economic changes before polls do.

Re: The value of prediction markets

jedc_mercury — Mon, 09 Mar 2009 15:25:35 -0000

Very interesting thoughts here.

I like that you distinguish between the PM software, the market itself, and how the market is used in decision making. I completely agree. Personally I believe you have to work backwards from how the market will be used to eventually get to the software you need to use.

I think public prediction markets can certainly almost always provide informational value to people unfamiliar with a topic; the value to people familiar with an industry/contract may be substantially less, though seeing immediate trends may still be useful.

Finally, I definitely agree with your last point on the corporate world missing opportunities. Fundamentally I think it comes down to the organisational tension: connecting the lower-level employees to the upper management threatens middle management, who like to be the conduit of information. I hope that more successful stories are published in the future about how PM's can in fact put numbers behind key areas of uncertainty in corporations.

Thanks again for your comment!

Re: The value of prediction markets

TorontoBentley — Mon, 09 Mar 2009 10:09:04 -0000

I agree, public markets are very different from internal ones. Most of the public markets suffer from having too small of a "crowd". Though several cite thousands of users, few of the individual markets have more than a few actual traders. As a result, they have to use a MSR or some other form of automated market maker in order to ensure enough liquidity for the market to function at all. In my opinion, this is simply a workaround to the problem of not having enough traders (let alone their having sufficient diversity). While a MSR does allow for trading among a small number of traders, the fact that it automatically allows traders to sell out of positions affects the traders' risk preferences, skewing their "investment" behavior. Internal markets should (and must) avoid this problem, in order to be successful.

The software vendors make it sound very easy to set up internal prediction markets. We should keep in mind that the software is but a tool to be used by the prediction market, which is a tool to be used in corporate decision making. The software solution is relatively easy (apart from the choice of mechanism). The hard part is satisfying the necessary conditions (crowd, diversity, independence) for accurate predictions.

Generally, public markets stay open until just before the outcome is revealed. They provide very little useful predictive value, if any. Internal markets must reveal their predictions much earlier, if they are to be useful and become more widely used.

Finally, I believe the corporate world has missed the opportunity to use prediction markets to measure the uncertainty surrounding predictions of key events and conditions (so far, at least). There is very little literature on the use of prediction markets to measure uncertainty. I believe this would be a very valuable use of prediction markets in the corporate world.

Re: A review of idea and innovation software

jedc_mercury — Wed, 04 Mar 2009 20:25:01 -0000

Hi, Jenny. Thanks for commenting.

I have to be honest; based on my understanding of Qualcomm's use of prediction markets, I'm really skeptical of how they're used. I understand that they're using PM's to rank new product/service ideas, which I think is an inappropriate use of PM's. What tends to happen is that people trade/buy those ideas that they think _other_ people will buy, and not necessarily the best ideas. I believe that doing so may find good ideas, but they'll be conventional ideas. The ground-breaking ideas won't get noticed.

That said, if they feel it works for them, that's fine. But I think there are other tools that are better suited for innovation. I'm doing a fair bit of work on this now, and hope to be able to talk about it more widely later this year.

Re: A review of idea and innovation software

Jenny Ambrozek — Wed, 25 Feb 2009 15:06:26 -0000

Jed,. Good to see you continuing to push thinking about predction markets, collective intelligence and innovation forward. You might be interested in a piece a colleague and I wrote last year for Inside Knowledge Magazine: http://snurl.com/cmzd5 For me, Carlos dos Santos" use of prediction markets at Qualcomm remains the most evolved and intelligent PM application. His integration of the technology into a thoughtful organizational structure promising the opportuity to present a new product business plan to the C-suite is inspired.

Separately there are some folks to whom I would like to introduce you. Can we please connect via email? I look forward to catching up. If you are on Twitter please share your user name. I'm "sagenet". Thanks