This is Brent Frei's Typepad Profile.
Join Typepad and start following Brent Frei's activity
Join Now!
Already a member? Sign In
Brent Frei
Recent Activity
I like the term Chris Matthews at Specialized Bicycle uses for improved accountability through visibility... "Having a collaborative tool like Smartsheet adds what I call Clear Pressure," says Matthews. "Since all of the activities are transparent, no team member wants to be the one holding up the project. People are more responsive and perform in a more reliable fashion. They actually begin to cooperate in moving the whole project forward."
Dennis, good point. To be clear, I said their API is impressive, not that it's been evolving. For companies like Smartsheet that are writing a re-usable interface on top of the MTurk platform, the command line interface is sufficient and solid. For end users of MTurk, it's pretty spartan. My point is that they "could expand their API" not that they are or will.
Brent Frei is now following The Typepad Team
Mar 15, 2010
John, Good question. Our internal intern performed the task on enough companies to give us a confident estimate of how long it would take to do the full list should she actually have to do it. (Bookmark that work for later use in quality testing the Smartsourced results accuracy.) Our confidence in the quality of the crowdsourced answers comes from the parts of the process not specified in the post because they would have added extra complexity to understanding the basic point of the article - cost and earnings comparisons. The statistics of the 4 tasks per company are a roll-up summary of the actual task process. Given the nature of the data being asked for, some parts were easier than other parts. "Does this company offer a GL module?" was dead simple and fast, whereas "Give an average price paid for the product" had varying degrees of difficulty in coming to an answer. So, in some of the task categories, we asked more than one worker to answer the same question and averaged the answers. The price paid per task ranged from $0.01 to $0.05 depending on the difficulty. Therefore, the overall average time to complete and cost per task are a rollup of what was actually varied task prices and more than 600 tasks. We compared the early results with those of the intern to gauge quality, and then at the end inspected it all in the same fashion you might review an employee's work. Spot checks and eyeballing.