Mercury's Blog - Latest Comments

Re: Prediction Market wrap-up for 2009

Brad Inggs — Fri, 01 Jan 2010 18:20:35 -0000

Hi Jed, just a quick one to say keep up the great posts, fantastic reading your views and notes. Congrats on google too btw.
Kind regards
Brad Inggs

Re: Anatomy of a blog hack

Mark — Tue, 24 Nov 2009 07:56:21 -0000

This allows the person who has the password embedded in this file to upload any file they want.

Re: Anatomy of a blog hack

garethdp — Tue, 24 Nov 2009 06:05:18 -0000

The PHP code shown is used to handle a file upload to the server through an html form.

The attacking server mimics a form submission to the PHP code shown, and it tests to see if the file is successfully uploaded. If the test is successful the attacking server can then send down more files

Re: Anatomy of a blog hack

Errant — Tue, 24 Nov 2009 05:04:55 -0000

The code does 2 things (just do you know).

It requires the password the hacker set up to be posted; along with a file - one assumes PHP or Html script.

Then it will either (depending on the options chosen) save the file to the server. Or execute code in the file (using eval()).

The it prints a success / fail.

From the looks of things it looks tailored for use of some sort of bot - because the save_ok/fail/ok text that gets printed is MD5 hashed.

Re: A new competitor in prediction markets, and their brilliant case study

dmreinke — Thu, 01 Oct 2009 13:42:36 -0000

Jed - great comment. Really appreciate your talking about how prediction market gets around problem of algorithms not responding quickly enough to dramatic market changes.

That's the reason we have high hopes at StyleHop that crowdsourced forecasting could make a big difference in fashion - the algorithms have never worked because of the changing trends. However, we don't use PM methodology - just too complicated for our fashionista panel.

Re: A new competitor in prediction markets, and their brilliant case study

jedc_mercury — Tue, 29 Sep 2009 12:36:55 -0000

I just made a substantial comment over on Midas Oracle here: http://www.midasoracle.org/200...

I'm copying it here since it seems relevant:

Hi, Paul.

I agree that CrowdClarity’s slides don’t get into the detail necessary to understand why they were successful. But that’s largely because those slides were published as a sales tool… a top-line attention grabber for potential sales leads.

The things I would want to understand to truly evaluate success would include: demographics of traders, the types of forecasts that already exist, what error rates are when sales volatility is more “normal”, what trader incentives were used, etc.

Regarding your point on number of traders, my own research showed that as long as you have more than 15 traders, the market generates a calibrated result. (The markets I ran were probabilistic.) So that’s actually quite a realistic number, even though it might intuitively seem low. But I would point out that because of the nature of the markets I ran, the likelihood that traders knew each other was low, so there was a natural diversity in the 15+ that I studied.

I would certainly consider the prediction market / case study CrowdClarity published a “success.” The key for me is turning that into a *business* success. You’re very right that if this was one of the first months that a PM was run, no one would have likely believed the results, even if they were the closest to the eventual truth.

But let’s say that a PM has been showing reasonable accuracy for several months, which I would define as similar or better accuracy than other forecasting methods. Then, like the case study, the prediction market shows a drastically different result than any other forecast. While a manager probably won’t “bet the farm” on a prediction market alone, that certainly would warrant re-thinking the forecast.

The reason is simply that the market aggregates the information more quickly than other methods. I would argue that GM and Edmunds.com used forecasting models that are quite good most of the time, but completely wrong when some of the basic assumptions collapse. Since the PM doesn’t rely on algorithms, collapsing assumptions won’t affect accuracy.

In many ways, I think this last point is the most important with prediction markets. 80-90% of the time a prediction market might generate a forecast with accuracy that’s on par with other methods… maybe a little better, maybe a little worse. But the other 10-20% of the time, when the forecasts diverge significantly, is where prediction markets can be *very* useful. When assumptions behind traditional models weaken or collapse, a prediction market can be the early warning signal, since it uses a different methodology to generate a forecast.

Maybe a company spends $3-4k a year on a prediction market that generally confirms or better brackets existing forecasts 11 months out of the year. But if the intelligence it generates that 12th month of the year helps the company save $20k, it strikes me as a wise investment.

Again, that’s not to say that a prediction market is going to solve a company’s problems. It needs to address problems where the cost of the error is worth the investment and where a prediction market can effectively address it. And that’s a sensitive balance.

Re: A new competitor in prediction markets, and their brilliant case study

JT Maloney — Mon, 28 Sep 2009 17:03:50 -0000

Hi Jed -

Thanks, good post. Here is a another new (to me at least) player.

http://wiscom.co.il/

-j

Re: A new competitor in prediction markets, and their brilliant case study

dmreinke — Mon, 28 Sep 2009 11:40:49 -0000

Thanks for sharing, Jed. Great Slides by the CrowdClarity team.

Re: A new competitor in prediction markets, and their brilliant case study

johndelaney — Mon, 28 Sep 2009 08:39:08 -0000

Jed and Alex,

I echo both your sentiments.

Very good fortunes to all at CrowdClarity.

John Delaney
CEO
Intrade

Re: A new competitor in prediction markets, and their brilliant case study

Alex Kirtland — Mon, 28 Sep 2009 07:48:29 -0000

Jed, good find. Nice to hear about a new and worthy prediction market startup, and a positive corporate PM case study.

~alex

Re: A review of idea and innovation software

jedc_mercury — Wed, 27 May 2009 15:21:40 -0000

Thanks, Jeffrey. Will definitely take a look at it and include it in an updated post.

Re: A review of idea and innovation software

Jeffrey Baumgartner — Mon, 25 May 2009 12:01:17 -0000

Have you looked at Jenni innovation process management? Details at http://www.jpb.com/jenni/index...

We'd be happy to arrange a demo for you...

Jeffrey Baumgartner

Re: Measuring the prediction market industry – a proposal

jedc_mercury — Thu, 23 Apr 2009 12:14:45 -0000

Actually, I disagree a bit. While each vendor sells prediction market software, each has a different mix of software and services, so even I wouldn't be able to really figure out how well each of them are doing. Even if client engagements are flat year-over-year, if each engagement is of significantly more value (ie, a lot of consulting services), the overall value has increased.

Thanks for the comments; I hope to get this going soon!

Re: Measuring the prediction market industry – a proposal

Paul Hewitt — Thu, 23 Apr 2009 09:41:38 -0000

True enough for the general public, but the "aggregator" would know which vendors were doing better than the others. It may affect the reporting. I'd be surprised if any are doing particularly well, unfortunately.

Still, it will be interesting to see the results. All the best in this endeavor!

Paul

Re: Measuring the prediction market industry – a proposal

jedc_mercury — Thu, 23 Apr 2009 08:45:40 -0000

Thanks, Paul.

I'm hopeful that they will release the data. I would hope that it's more important to show that the industry is growing/thriving (my intuition) than holding the info back.

However, only the *aggregated* data would be released, so there's no way to compare vendors against each other. I will also provide each vendor their own data compared to the aggregated data for their own benchmarking. (In other words, you won't be able to tell from this which vendors are growing, just how the industry as a whole is growing.)

Part of this is also due to the nature of clients that each vendor takes on. Inkling cater more (in general) to the self-service customers, while others (such as Newsfutures) have generally fewer clients but with a lot more strategic consulting around their engagements.

Re: Measuring the prediction market industry – a proposal

Paul Hewitt — Thu, 23 Apr 2009 08:03:53 -0000

Hi Jed...

A very good idea for a start. It is great that you have some "solid interest" from some of the software vendors. Assuming you do get the data, it will also show whether the industry is growing (net new clients over time). I'm a bit skeptical that they will release actual data on how many paid clients they have, because I don't think the numbers are there, so far. It would also show which vendors are growing. I may be wrong (hope I am). If you look at Inkling's web site, they list a few of their clients, of which most appear to be continuing their use of prediction markets.

Paul

Re: A review of idea and innovation software

jedc_mercury — Sun, 19 Apr 2009 13:22:27 -0000

Thanks, Hutch. I appreciate the additional info.

Re: A review of idea and innovation software

Hutch Carpenter — Fri, 17 Apr 2009 15:52:11 -0000

Hi Jed -

Thanks for including Spigit in your write-up. You're right - we are seeing great uptake by enterprise customers. Managing the multiple sources of ideas, and filtering for those that are most useful, is an *idea* that is taking root in the corporate world.

You raise some good questions about us, and I'll answer them here. First, in terms of complexity vs simplicity. The platform is actually quite easy for employees, customers and partners to use. It leverages some of the best practices one finds for social software: votes, tags, discussion forums, clickable reviews, wikis, blogs. We also provide a market for users to buy and sell ideas. Any or all of these are available for use when deployed, internally or externally.

We put a premium on a superior user experience.

The analytics engine and enterprise workflow of the platform is why companies are signing up for Spigit. Simple popularity contests for ideas do give a single metric - number of votes. Certainly that is a form of simplicity. But it's doesn't come close to mapping to the way innovation really works inside corporations. Spigit has designed-in functionality that reflects the different influence users have in advancing an idea forward.

Spigit's platform has enterprise workflow built-in. Role-based reviews and approvals, quantifiable criteria to advance through stages, and high configurability to adapt to each company's specific processes.

Happy to tell you more about what Spigit is up to. Please drop us a note at info@spigit.com.

Hutch Carpenter
Director of Marketing
Spigit, Inc.
http://spigit.com

Re: Follow-up to “Approaching business problems differently”

Paul Hewitt — Sun, 05 Apr 2009 17:43:50 -0000

Here is a link to a new blog on practical aspects of enterprise prediction markets, where I also discuss some of the issues holding back acceptance. I'd appreciate your comments when you have time.

http://torontopm.wordpress.com...

Re: Follow-up to “Approaching business problems differently”

Paul Hewitt — Sun, 05 Apr 2009 11:09:17 -0000

Hi Jed...

I agree with your summary. In addition, I think prediction markets offer several additional benefits, including: faster predictions, continuous updating, a measure of uncertainty surrounding the prediction, and reasonable cost. I'm sure there are a few more, but for now, this should do.

I am strongly in favour of using prediction markets to complement existing forecasting methods, especially where they are able to quantify uncertainty. Your example of project milestones is another excellent use for prediction markets.

Although your research indicates that as few as 15 participants can achieve calibrated results, I am skeptical. All of the market maker mechanisms will ensure that there is market liquidity, but I believe they influence trading behaviour (generally making traders quite risk-seeking), which undermines the accuracy of the predictions. Such market maker mechanisms were designed to allow markets to operate with smaller numbers of participants, but doesn't this degrade the "crowd" precondition for successful market predictions? You aren't likely to have a very diverse group with a small number of participants. The basic theory behind prediction markets is that each trader has a piece of information combined with an error factor, and the aggregation method adds the pieces of information together, with the error factors cancelling out (more or less), resulting in more, accurate information. Having a smaller number of traders not only means having fewer pieces of information to aggregate, but also, the error factors will not "cancel out" properly.

As I see it, one of the major stumbling blocks is getting enough people to be interested in each market (and staying interested). The greatest benefit of prediction markets comes from forecasting the outcome as far in advance as possible, but this runs counter to the traders' need to know the outcome "immediately" (or at least soon). You can see this in most marketplaces where very few people trade in long-term markets. Most trade in the markets that will close in the next few hours, days or maybe a week. These very short term market predictions have little value to a decisionmaker.

If prediction markets are to gain more widespread acceptance in the business community, they will need to focus on ways to ensure their predictions are useful. That is, by providing predictions well in advance of the outcome, accurately (diverse crowd, properly motivated) and with a very reasonable cost. Much more work needs to be done in the area of motivating traders.

Just my thoughts for now.

Re: Approaching business problems differently

jedc_mercury — Fri, 03 Apr 2009 06:47:38 -0000

HI, John.

"Slow burn" isn't a diffusion method or metric, just a description of growth. Not dying, but also not "burning like wildfire." It's continued, steady growth.

I agree that PMs aren't an academic backwater; far from it! Yet they've also not been adopted widely. I have no idea where you see order-of-magnitude growth per year, and would love to see evidence of it. That leads to your last point...

I would love to set up an PM on diffusion and adoption, but am a bit stumped on what exactly to trade! Ideally it would be on some metric of number of clients and/or revenues of the PM software companies, broken out between software and consulting fees. But since everyone is private none of that information is available.

Re: Approaching business problems differently

jheuristic — Thu, 02 Apr 2009 11:26:27 -0000

Jed --

Thanks for the reply. You may wish to look at the 'bible' of technology adoption, "Diffusion of Innovations," by Everett Rodgers --

http://www.amazon.com/Diffusio...

I'm not sure there is a diffusion method or metric called a 'slow burn.' By most measures, slow burn would mean failure, at least from the perspective of diffusion science.

PMs are no longer a curious academic backwater. The factor growth you need to see is in play.

Anyway, one view that informs the community is Malcom Gladwell's observation, "The success of any kind of social epidemic is heavily dependent on the involvement of people with a particular and rare set of social skills." More than ever those skills are in place and operational. That's not easy and takes a while.It is a critical success factor.

BTW, is there a market for PM diffusion and adoption someplace? Why not spin one up? It would be popular.

Thanks again, good discourse.

Cordially,

John

Re: Approaching business problems differently

jedc_mercury — Wed, 01 Apr 2009 18:46:36 -0000

Hi, John.

While I certainly see growth, and even strong growth, I haven't seen evidence of order-of-magnitude growth, and don't believe it will happen this year. Last year many people said that 2008 would be the big year because of the US election and all of the press on prediction markets there... hasn't happened. You're saying 2009 will be the big year; I'm just saying that I think it's still going to take some time.

Each year for the past ten years companies have been saying that *that* year was going to be the big year for innovative uses on mobile phones; yet it was only once Apple came out with the iPhone (arguably, when they came out with the 3G iPhone) that the industry finally started to radically change for the better. I'm just wary that if I start saying that *this* is the year for prediction markets, that I'll look like a fool. I'm not going to say it until and unless I have really good reasons to believe it.

If you had more concrete evidence (sales growth, etc.) I would certainly be open to changing my mind, but I haven't seen anything like it.

Again, as I wrote in my post, prediction markets have the capability to explode at any given time; what's limiting their adoption is that they approach the problem completely differently from current solutions. It takes time for people (let alone organisations) to wrap their heads around new ideas.

Re: Approaching business problems differently

jheuristic — Wed, 01 Apr 2009 18:33:15 -0000

Hi - I wish it was a simple as my opinion. Again, I am only sharing the discourse and evidence that's out there in the clusters. Cluster lead technology adoption... all the cylinders are firing for PMs. Same as search, social networks, social media, and others the cluster action/research networks forecast and led. There is a inflection point fast approaching, trust me. John

Re: Approaching business problems differently

jedc_mercury — Wed, 01 Apr 2009 18:24:55 -0000

Hi, John.

While I respect your opinion, I disagree. I think prediction markets will see continued growth for quite some time, but don't believe that 2009 will see "sharp growth." (This depends on what you mean by sharp growth; if we had a market on this question it would need to define this. I would use a metric of an order-of-magnitude growth in the industry in 12-18 months.)

As I wrote in the post, I think prediction markets are a kind of technology that will see a "slow burn" in growth for quite some time; it will be many years before we see any truly dramatic spike. Again, this doesn't mean that they're not useful, just that they're a different enough tool that it's tough for companies to adapt to use them.