|
|
| Groklaw Review of "The Wisdom of Crowds" |
 |
|
Sunday, November 07 2004 @ 09:04 PM EST
|
On October 15th, Mike Perry of Inkling Books in a comment recommended a book to Groklaw readers called "The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations". The book is by James Surowiecki, who writes The Financial Page for The New Yorker magazine. Perry asked if any Groklaw readers would write a review, and one reader, Nick, accepted the challenge. Here, then, is a his review of "The Wisdom of Crowds":
**********************************
We all know that crowds are dumb, right? That's where riots come from. Lowest common denominator, and all that. So why did Mr. Surowiecki call his book "The Wisdom of Crowds" (and yes, it is a direct allusion to Charles Mackay's "Extraordinary Popular Delusions & The Madness of Crowds")? Simply put, while not denying the madness of crowd model in the right circumstances, it turns out that under other circumstances the crowd is wise. Very wise. In fact, wiser, more insightful, more accurate, more intelligent than any individual genius can hope to be. That's a strong statement, but Surowiecki backs it up with example after example to show what he means and why.
For example, note this quote:
"A classic demonstration of group intelligence is the jelly-beans-in-the-jar experiment, in which invariably the group's estimate is superior to the vast majority of the individual guesses. When finance professor Jack Treynor ran the experiment in his class with a jar that held 850 beans, the group estimate was 871. Only one of the fifty-six people in the class made a better guess."
This is only one example of many in the book. No matter how knowledgeable the individual observer, a group estimate, even a group composed on non-experts, routinely trumps the individual in insight. Whether it involves counting jelly beans, estimating the weight of an ox, or assigning blame in the stock market to the correct company responsible for the Challenger accident in 1986, the crowd gets it right faster and more accurately than the individual expert. Note this other fascinating example:
"In May 1968, the U.S. submarine Scorpion disappeared on its way back to Newport News after a tour of duty in the North Atlantic. Although the navy knew the sub's last reported location, it had no idea what had happened to the Scorpion, and only the vaguest sense of how far it might have traveled after it had last made radio contact. As a result, the areas where the navy began searching for the Scorpion was a circle twenty miles wide and many thousands of feet deep. You could not imagine a more hopeless task. The only possible solution, one might have thought, was to track down three or four top experts on submarines and ocean currents, ask them where they thought the Scorpion was, and search there. But...a naval officer named John Craven had a different plan.
"First, Craven concocted a series of scenarios -- alternative explanations for what might have happened to the Scorpion. Then he assembled a team of men with a wide range of knowledge, including mathematicians, submarine specialists, and salvage men. Instead of asking them to consult with each other to come up with an answer, he asked each of them to offer his best guess about how likely each of the scenarios was..Craven believed that if he put all the answers together, building a composite picture of how the Scorpion died, he'd end up with a pretty good idea of where it was...He took all the guesses, and used a formula called Bayes's theorem to estimate the Scorpion's final location..When he was done, Craven had what was, roughly speaking, the group's collective estimate of where the submarine was.
"The location that Craven came up with was not a spot that any individual member of the group had picked. In other words, not one of the members of the group had a picture in his head that matched the one Craven had constructed using the information gathered from all of them. The final estimate was a genuinely collective judgment that the group as a whole had made, as opposed to representing the individual judgment of the smartest people in it. It was also a genuinely brilliant judgment. Five months after the Scorpion disappeared, a navy ship found it. It was 220 yards from where Craven's group said it would be."
Remarkable, right? Is it just in humans that we see this sort of behavior? No. Consider how bees find good sources of nectar:
"They don't sit around and have a collective discussion about where foragers should go. Instead, the hives sends out a host of scout bees to search the surrounding area. When a scout bee has found a nectar source that seems strong, he comes back and does a waggle dance, the intensity of which is shaped, in some way, by the excellence of the nectar supply at the site. The waggle dance attracts other forager bees, which follow the first forager, while foragers who have found less-good sites attract fewer followers and, in some cases, eventually abandon their sites entirely. The result is that bee foragers end up distributing themselves across different nectar sources in an almost perfect fashion, meaning that they get as much food as possible relative to the time and energy they put into searching. It is a collectively brilliant solution to the colony's food problem.
"What's important, though, is the way the colony gets to that collectively intelligent solution. It does not get there by first rationally considering all the alternatives, and then determining an ideal foraging pattern. It can't do this, because it doesn't have any idea what the possible alternatives -- that is, where the different flower patches -- are. So instead, it sends out scouts in many different directions and trusts that at least one of them will find the best patch, return, and do a good dance so that the hive will know where the food source is."
Now we begin to see the secret to this group wisdom effect. The more people involved (or the more bees), the greater the input from the group as a whole and the more likely it is that the correct solution is reached. That makes intuitive sense, for we all know that "two heads are better than one." So that means instead of relying on one expert, get a group of experts together, right? Wrong:
". . . [A] group made up of some smart agents and some not-so-smart agents almost always did better than a group made up just of smart agents. Diversity is, on its own, valuable, so that the simple fact of making a group diverse make it better at problem solving. That doesn't mean intelligence is irrelevant.. but it does mean that, on the group level, intelligence alone is not enough, because intelligence alone cannot guarantee you different perspectives on a problem.. Adding in a few people who know less, but have different skills, actually improves the group's performance."
OK, now we're getting radical. A group of experts and non-experts is better than just a group of experts, even if the group size is the same? Surowiecki knows what you are thinking at this point and addresses it:
"Again, this doesn't mean that well-informed, sophisticated analysts are of no use in making good decisions. (And it certainly doesn't mean you want crowds of amateurs trying to collectively perform surgery or fly planes.) It does mean that however well-informed and sophisticated an expert is, his advice and predictions should be pooled with those of others to get the most out of him. (The larger the group, the more reliable its judgment will be.) And it means that attempting to 'chase the expert,' looking for the one man who will have the answers to an organization's problem, is a waste of time."
So don't worry, he's not deprecating intelligence or expertise, and he acknowledges there are obvious times when you do want the lone expert working on your problem, especially if "your problem" is you need brain surgery. And Surowiecki absolutely acknowledges the problems that can come from relying on the crowd to achieve wisdom. But the principle upon which this book rests is expressed simply thus:
"The idea of the wisdom of crowds is not that a group will always give you the right answer but that on average it will consistently come up with a better answer than any individual could provide."
It's that group experience that makes the difference. Expertise is needed, but relying on expertise alone will leave you worse off than if you couple expertise with diversity. Does that sound familiar? It should. It's the Linux model for developing software, and it's the Groklaw model for gathering legal news and insight. The more diverse the crowd, the greater the chance that one of those waggling bees will stumble upon the right answer, or the best answer. It works when looking for nectar, and it works when submitting bug fixes and new features for Linux. Notice what Surowiecki says about Linux:
"In the way it operates, in fact, Linux is not all that different from a market.. Like a bee colony, it sends out lots of foragers and assumes that one of them will find the best route to the flower fields. This is, without a doubt, less efficient than simply trying to define the best route to the field or even picking the smartest forager and letting him go. After all, if hundreds or thousands of programmers are spending their time trying to come up with a solution that only a few of them are going to find, that's many hours wasted that could be spent doing something else. And yet, just as the free market's ability to generate lots of alternatives and then winnow them down is central to its continued growth, Linux's seeming wastefulness is a kind of strength (a kind of strength that for-profit companies cannot, fortunately or unfortunately, rely on). You can let a thousand flowers bloom and then pick the one that smells the sweetest.
"So who picks the sweetest-smelling one? Ideally the crowd would. But here's where striking a balance between the local and the global is essential: a decentralized system can only produce genuinely intelligent results if there's a means of aggregating the information of everyone in the system. Without such a means, there's no reason to think that decentralization will produce a smart result. In the case of Linux, it is the small number of coders, including Torvalds himself, who vet every potential change to the operating-system source code. There are would-be Linux programmers all over the world, but eventually all roads lead to Linus."
So we see that wisdom from crowds comes as a result of certain conditions. There are principles by which wisdom can come from the crowd (as opposed to madness):
". . . the four conditions that characterize wise crowds: diversity of opinion (each person should have some private information, even if it's just an eccentric interpretation of the known fact), independence (people's opinions are not determined by the opinions of those around them), decentralization (people are able to specialize and draw on local knowledge), and aggregation (some mechanism exists for turning private judgments into a collective decision). If a group satisfies those conditions, its judgment is likely to be accurate."
What about the opposite result, the one where crowds are not wise and even dumb? Under what circumstances do crowds go wrong and start to riot (or in the case of online communities, start to turn on the community)? Surowiecki discusses this as well, offering this analysis of what causes a crowd to riot:
"The process by which a violent mob actually comes together seems curiously similar to the way a stock-market bubble works. A mob in the middle of a riot appears to be a single organism, acting with one mind. And obviously the mob's behavior has a collective dimension that a group of random people just milling about does not have. But sociologist Mark Granovetter argued that the collective nature of a mob was the product of a complicated process, rather than a sudden descent into madness. In any crowd of people, Granovetter showed, there are some people who will never riot, and some people who are ready to riot at almost any time -- these are the 'instigators.' But most people are somewhere in the middle. Their willingness to riot depends on what other people in the crowd are doing. Specifically, it depends on how many other people in the crowd are rioting. As more people riot, more people decide that they are willing to riot, too...
"This makes it sound as if once one person starts a ruckus, a riot will inevitably result. But according to Granovetter, that's not the case. What determines the outcome is the mix of people in the crowd. If there are a few instigators and lots of people who will act only if a sizable percentage of the crowd acts, then it's likely nothing will happen. For a crowd to explode, you need instigators, 'radicals' -- people with low thresholds for violence -- and a mass of people who can be swayed. The result is that although it's not necessarily easy to start a riot, once a crowd crosses the threshold into violence, its behavior is shaped by its most violent members. If the image of collective wisdom that informs much of this book is the average judgment of the group as a whole, a mob is not wise. Its judgment is extreme...
"There is, in Granobetter's work, a hint as to what markets need to avoid endless bouts with irrational exuberance or irrational despair. In Granovetter's world, if there are enough people in the crowd who will not riot under any conditions -- that is, whose actions are independent of the crowd's behavior as a whole -- then a riot will be far less likely, because the more people who do not riot, the more people there will be who don't want to riot."
So there we have it, both the pattern for useful behavior from crowds, as well as warnings on how to avoid being sucked into negative mob behavior. When utilized properly, the wisdom of crowds is real. Surowiecki writes interestingly too about the financial markets and political thinking. But for Groklaw readers the application is direct and obvious. We've seen the wisdom of the crowds in the growth of Linux, and also in the growth of Groklaw. Both models follow principles that allow wisdom to bubble up from the group, with, as Surowiecki puts it, someone to pick the "sweetest flower". When you let a thousand flowers bloom, it's easy to pick the one that smells the sweetest. "The Wisdom of Crowds" is very interesting, and I highly recommend it.
|
|
|
|
| Authored by: Hiro Protagonist on Sunday, November 07 2004 @ 09:52 PM EST |
Correction Here Please.
---
I Grok... Therefore... I am.[ Reply to This | # ]
|
| |
| Authored by: spuluka on Sunday, November 07 2004 @ 09:53 PM EST |
The current issue of "Embedded Sytems Programing" makes a similar
claim for acurate software design estimates. You us at least three developers
working separately to give individual estimates. These are then combined using
wideband delphi and this results in a more accurate estimate than any of the
three individually.
---
Steve Puluka
Pittsburgh, PA[ Reply to This | # ]
|
- Wideband Delphi - Authored by: Anonymous on Monday, November 08 2004 @ 02:09 AM EST
| |
| Authored by: Hiro Protagonist on Sunday, November 07 2004 @ 09:54 PM EST |
OT here Please.
---
I Grok... Therefore... I am.[ Reply to This | # ]
|
| |
| Authored by: Simon G Best on Sunday, November 07 2004 @ 10:01 PM EST |
And even if it does, wouldn't that be begging the
question?
:-)
---
Open Source - open and honest? Not
while the political denial continues.
[ Reply to This | # ]
|
| |
| Authored by: Carlo Graziani on Sunday, November 07 2004 @ 10:12 PM EST |
| The idea that the "ability" of a crowd to estimate jellybean counts
accurately
(which is really just a statement about regression to the mean)
generalizes to
the "ability" to make complex judgements that are beyond the
reach of
individuals strikes me as naive mysticism.
It is more-or-less the kind of
reaoning that led the DOD to attempt to
establish a "Terrorism Derivatives
Exchange" under the leadership of that
nutjob Poindexter, a few months ago.
Apparently the hope was that a crowd
of investors could outperform professional
intelligence analysts in
providing terrorism warnings. While this may very well
be the case, the
same might be said of tarot card readers, with about as much
evidence. [ Reply to This | # ]
|
- Mysticism - Authored by: Nick on Sunday, November 07 2004 @ 10:18 PM EST
- Mysticism - Authored by: Anonymous on Sunday, November 07 2004 @ 10:21 PM EST
- Mysticism - Authored by: Nick on Sunday, November 07 2004 @ 10:29 PM EST
- Mysticism - Authored by: pdp on Monday, November 08 2004 @ 01:16 AM EST
- Mysticism - Authored by: Anonymous on Sunday, November 07 2004 @ 10:31 PM EST
- No mysticism here - Authored by: Anni on Monday, November 08 2004 @ 02:21 AM EST
- Mysticism - nah, it's just statistics at work - Authored by: Duster on Monday, November 08 2004 @ 03:32 AM EST
- Mysticism - Authored by: Anonymous on Monday, November 08 2004 @ 04:24 AM EST
- Regression to the mean - not - Authored by: Anonymous on Monday, November 08 2004 @ 07:18 AM EST
- Mysticism - Authored by: Basset## on Tuesday, November 09 2004 @ 11:21 PM EST
- Mysticism - Authored by: Anonymous on Wednesday, November 10 2004 @ 09:11 PM EST
| |
| Authored by: jarichte on Sunday, November 07 2004 @ 10:52 PM EST |
Main posts in this thread may only be made by senior managers or attorneys
for
"The SCO Group". Main posts must use the name and position of the
poster at
"The SCO Group". Main posters must post in their official
capacity at "The SCO
Group".
Sub-posts will also be allowed from non-"The SCO Group"
employees or
attorneys. Sub-posts from persons not connected with "The SCO
Group"
must be very polite, address other posters and the main poster with
the
honorific "Mr." or "Mrs." or "Ms.", as
appropriate, use correct surnames,
not call names or suggest or imply unethical
or illegal conduct by "The SCO
Group" or its employees or attorneys.
This thread requires an extremely high
standard of conduct and even slightly
marginal posts will be
deleted.
P.J. says you must be on your very best behavior.
If
you want to comment on this thread, please post under "O/T"[ Reply to This | # ]
|
| |
| Authored by: Anonymous on Sunday, November 07 2004 @ 10:52 PM EST |
For those familiar with Asimov's Foundation series, this sounds like the
forerunner to Seldon's study of psychohistory - the fictional science able to
predict the future actions of large populations of humans.[ Reply to This | # ]
|
| |
| Authored by: Rasyr on Sunday, November 07 2004 @ 10:54 PM EST |
"So who picks the sweetest-smelling one? Ideally the crowd
would. But here's where striking a balance between the local and the global is
essential: a decentralized system can only produce genuinely intelligent results
if there's a means of aggregating the information of everyone in the system.
Without such a means, there's no reason to think that decentralization will
produce a smart result.
I have actually seen this sort of
thing in operation (in a bad way) in another field. I am currently working as an
editor/line developer for a small role-playing game company (pencil & paper
tabletop games, not computer).
A few years ago, one of the largest of our
competitors (Wizards of the Coast) released their game system (D&D) under a
licence known as the OGL (Open Gaming License).
Unfortunately, they did not
include any sort of local control, and thus there was shortly a few hundred
companies putting out tons of products for this game in a relatively short
period of time (as of right now, there are over 2,000 products for this one game
alone). The fact that there were no controls left the players bombarded with
products that tried to make themselves look better and shinier than their
competitors. Products that used one-up-man-ship to do, in what is often termed
"power creep". It also hurt the industry as a whole that a vast majority of
these products were of extremely low quality (personal opinion is that many were
outright garbage), making it even more difficult to find the wheat for all the
chaff.
Instead of using a little bit of control, and picking the best of all
options, it was left on its own, and is currently extremely fragmented when it
comes to different rules for different settings and for any sort of chance of
getting a quality product in the long run.
All my little story goes to show
is that it is the local control that makes a wise group mind successful.
Looking at the example with the sub, the local control was the guy who took all
the answers and correlated them. For Linux, Linus is the local
control.
Note: While I may work for a rival company, I would like
to point out that the d20 System (the name given to what was released under the
OGL) is a pretty decent system at its core, although a little too inflexible for
my tastes. For any fans of the system who might read this sight, please
understand that I am not bashing d20, just pointing out the similarities
of the article to the way that I perceive that the OGL was handled.
[ Reply to This | # ]
|
| |
| Authored by: artp on Sunday, November 07 2004 @ 11:47 PM EST |
"Diversity is, on its own, valuable, so that the simple fact of making a
group diverse make it better at problem solving. That doesn't mean intelligence
is irrelevant.. but it does mean that, on the group level, intelligence alone is
not enough, because intelligence alone cannot guarantee you different
perspectives on a problem.. Adding in a few people who know less, but have
different skills, actually improves the group's performance."
My management mentor at a Fortune 30 company liked to tell the story about a
team that he put together once. He was competing against an experienced, veteran
team that had worked together for years. He took the minorities, plus some
rejects, nonconformists, cultural misfits and other losers and built a team that
outperformed the existing team within 18 months.
Diversity is a secret weapon for those firms that value it. I carry that lesson
with me and try to live it.
I think that the diversity that is Groklaw is a large part of its success. It
isn't just techies or just lawyers or just management, or just computing
pioneers. PJ is our aggregator, and does a fine job of it, IMO.
One place that we could apply this is in areas that are not covered here at
Groklaw. I am thinking specifically of religion, or lack thereof, but it also
applies to other things. Our diversity is valuable even if we don't recognize
it.
In the last book of C. S. Lewis' Narnia Tales, Aslan says that any good deed is
done in his name. Any evil deed is done is Tash's name, the local demon. Or to
put it in modern terms, anything good comes from God. Anything evil is not from
God. Ancient texts on spiritual discernment point out that being in touch with
the divine does not result in evil. Feelings of divine contact that result in
consternation, restlessness, irritability and discontent are from the devil.
Perhaps we could accept each other as we are, and recognize that the good things
that we do are a sign of truth in our lives, and leave it at that. There is no
need to discount someone else's beliefs in order to validate our own. I say this
as a practicing, and struggling, member of a large and ancient religion. What it
is doesn't matter, does it? But I do feel compelled to defend it when attacks
are thrown about at random.
I would like to have PJ's consideration for those who visit her site. I'm still
not sure whether we agree or disagree, but that is the beauty of it. It doesn't
matter, because we are working toward something really awesome.
I don't know how the above translates into atheism, but I'm willing to listen if
someone wants to tell me. Private messages accepted if PJ doesn't want this
discussion here.[ Reply to This | # ]
|
| |
| Authored by: Anonymous on Monday, November 08 2004 @ 12:55 AM EST |
My apologies for probably mangling this quote and its context. I believe that
it was Thomas Paine, remarking on the propriety of embedding public voting in
the constitution who remarked,
"I do not believe in the collective wisdom of individual ignorance."
It is, I think, a most appropriate quote for our time.
My thoughts on the wisdom of crowds are that much of the claims for the crowds
are misdirected. It was the agent whow combined the knowledge of the many in a
useful way that made the crowd "wise." Without his contribution,
there would have been little more than chaos. The sub, recall, was nowhere near
any contributors suggestion.
Similarly, it is Linus's, and his elite team, who turn the multitudes
submissions into an eligant work. I believe that I can state with some
confidence that without them, the many submissions would not even compile, let
alone run as a single, coherent object.
Basically, what the book's premise boils down to is that a group of people led
by an effective individual is more productive than a group that is ineffectively
led, no matter how smart them members. I certainly concur with that premise.
Oh, and thank you Linus -- and PJ! It is your contributions that make the
difference between cacophony and sweet music!
[ Reply to This | # ]
|
| |
| Authored by: inode_buddha on Monday, November 08 2004 @ 02:43 AM EST |
Let's try this -- should there be a "Book Reviews" on Groklaw? My
initial vote is "Yes" because the chances are very good that you have
read something that I may find useful, and vice versa.
---
"When we speak of free software, we are referring to freedom, not price." --
Richard M. Stallman[ Reply to This | # ]
|
| |
| Authored by: Anonymous on Monday, November 08 2004 @ 04:10 AM EST |
What we have is, in the end, the actions, positive or negative of agents.
A company, with a large number of employees, must deal with the different levels
of agents... and control them.
The laws of agency prevails from any point of view.
SCO, sold and distributed LINUX, endorsed the GPL, willingly had agents writing
code for LINUX, and even today seems to be unwilling to attemp stop any actual,
apparent, or ostensible agents from distributing LINUX. And the innocent 3rd
party users (who aquire LINUX, while newSCO stands by and does not attempt to
stop the the distribution, hence the terms it is distributed under...) are not
to be harmed by SCO.
Someday, the larger number rule will have a case that supports the idea that due
to the large amount of IP that exists in any product... that a citizen who buys,
or legally aquires, a product can not KNOW every inch of IP that is in the
product. Heck, programmers have no way of knowing if they are violating
someone's software patent... so how is the innocent 3rd party user ever going to
know. They can't. There needs to be a case that establishes that when someone
buy a car, that has a computer in it, that the any software IP holder who has
some of their "claimed to be theirs" IP that runs as part of the
car... can not demand payment of any fee from the owner of the car, who is
truely innocent.
SCO should have no right to go after any innocent user of any of their claimed
IP. It just is plain wrong. And the laws of Agency support this 100%. Now, we
need a case that applies to SCO, and any other "late to the deal" IP
holder who wishes to act like SCO, where the innocent buyer of anything that
contains IP is protected from abuse by those who think that they are owed for
the use of something that the customer thinks they bought free and clear. If no
case settles this, then laws need to be brought forware that does. Everything
these days has a computer as part of the product. Even if it is a simple
computer, the buyer should be held to be an innocent 3rd party user... that can
not be harmed by any "claimed to be" IP holder AFTER they have legally
aquired such a product for their use.
The larger the number of computer product buyerw we have, and the fact that
there are software patents, means that more of us are impacted by IP. The
logical course of action for society is that society will react to the threats
of IP holders in way that will hold the innocent 3rd party users of IP based
products harmless! The laws of Agency have already done the same for other
products, software based products are next! The logic in PJ's article, and in
the book, say just that! Either the courts will act, or the law makers will
act... it would be only logical that in the interest of commerce, in general,
the consumer will benefit.
[ Reply to This | # ]
|
| |
| Authored by: troll on Monday, November 08 2004 @ 04:45 AM EST |
This could make an interesting experiment to try out
this theory on a very large scale.
I suggest groklaw put up a page where we could
gues the date SCOX shares sink under one dollar,
or the price at the given date.
Everybody would be alowed to "vote". (try to predict)
After a week of voting an average would be calculated
and published.
Please do not ignore me ;-)
Yours truly
T.[ Reply to This | # ]
|
| |
| Authored by: mobrien_12 on Monday, November 08 2004 @ 04:47 AM EST |
| Thank you all for the interesting article and comments. [ Reply to This | # ]
|
| |
| Authored by: sjgibbs on Monday, November 08 2004 @ 05:27 AM EST |
I can't claim to have read many book reveiws but it seems to be that this kinda
misses the point of a review in the sense that it doesn't actually evaluate the
quality of the argument, rather it effectively paraphrases the theme with quotes
in support.
The only qualitative opinion proffered is ""The Wisdom of Crowds"
is very interesting, and I highly recommend it." well thats a useful
pointer. I can see that some people may base a decision on that and be rewarded
if in fact it is a good book, but since the author's thinking has not been
logically criticsed how do I know if the assertion of quality is in fact
accurate? Am I to believe that it is a perfect book?
Furthermore, how does this interesting piece of philosophy help me? Will it make
me any money? Will it help me to find happiness? What in this review addresses
the value proposition of the boook?
Like I said, I'm no book reviewer either but I think there is something missing
from this reveiw. For all I know the same thing may be "missing" from
all book reviews and its me thats off. In any case, are able to supplement the
review in this forum and debate these points?
SJG
---
Unemployed Geek
www.cantorva.com[ Reply to This | # ]
|
| |
| Authored by: Anonymous on Monday, November 08 2004 @ 05:30 AM EST |
Dear all,
To add my 2 cents: this book refers to a phenomenon known as 'emergent
behaviour', an aspect of complex, distributed systems, e.g. see (Holland, 1995)
as an oft-cited reference. For example, the 'swarm' notion is used, in which
individual software programs mimick birds (or bees, ants, ...) have simple
decision rules, yet the whole exhibits complex behaviour.
The trick, of course, is to design a complex system (e.g., a peer-to-pear
file-sharing network, or a job-shop scheduling system, or
highway-traffic-control, etc.) that exhibits 'wanted' emergent behaviour, and
can somehow avoid 'unwanted' emergent behaviour. And this is an interesting and
much needed field of research. The Internet is currently facilitating the
development of complex systems which consist of numerous humans and artificial
systems. How this is to be regulated, to which extent, and by what means, is an
open research question.
Thank you for the book reference, I intend to buy it,
regards,
Niek
Reference:
Holland, J.H. (1995), Hidden Order: How Adaptation Builds Complexity. Perseus
Books, Cambridge, Massachusetts.[ Reply to This | # ]
|
| |
| Authored by: Anonymous on Monday, November 08 2004 @ 05:42 AM EST |
So next time someone snottily tells you through his nose "Ghwat? You don't
do command line?" you can refer him to this here book and tell him how
valuable we simple GUI loving dribs are.
;-)
[ Reply to This | # ]
|
| |
| Authored by: Darkelve on Monday, November 08 2004 @ 05:44 AM EST |
In my very humble opinion, the word 'intelligence' is used too easily without
further reflexion upon what 'being intelligent' really means. If you take it to
the extreme, I believe 'intelligence' does not even really exist, because it is
defined by how we perceive it. Even when we take that as a basis, the question
remains which aspects of 'intelligence' are the most important ones. A question
that has no true answer. The current system of using the IQ, or even EQ, or even
IQ and EQ combined, is short-sighted.
For a brilliant mathematician, it is still quite possible he cannot write
without spelling errors, just as a brilliant linguist may not be able to solve
the simplest of equations. Even when we take the Renaissance Ideal of the 'Homo
Universalis', who is a brilliant mathematician as well as a geologist, an artist
and a philosopher, this extremely 'smart' person can still do dumb things and
make stupid decisions.
So we can see that usually, intelligence is perceived to be strongly related to
being 'learned'. But study, experience, persistance etc. are the factors that
lead to this state of being 'learned'.
A rope dancer is an expert at maintaining his balance. Study (practice),
experience, persistance, ... come into play here as well.
A person can be -in the opinion of society- be as 'dumb as a doorway', yet still
have the ability to seduce whichever person s/he lays eyes on.
People with paranormal abilities, even when they have an IQ of 60, might be able
to read your mind, have visions, ... (yes, I *do* believe people can read your
mind, since I've experienced this personally and it was not some trick).
I believe it paints a more accurate picture when these 'skills' are interpreted
as different 'kinds' of intelligence. EQ in addition to IQ is already an
improvement, but in my opinion it is much broader. Every person has his own
value and no one is 'smart' or 'stupied', just different, unique. A strong sense
of community brings these people together, making a more diverse skillset
available to the community. Which itself, can only really get stronger from
this.
In this sense, it is logical that a group of diverse people can come up with a
better estimate/answer/solution, since they each have their own unique
skillsets. 'A group of experts' in this article means: 'with a similar, limited,
area of expertise' (no one person has an unlimited area of expertise). Throw in
'non-experts' and you have 'with a *diverse*, limited area of expertise'.
It's easy to call someone smart or dumb, because it is convenient. But in the
end, it does not make much sense.[ Reply to This | # ]
|
- About Intelligence - Authored by: Anonymous on Monday, November 08 2004 @ 06:34 AM EST
- Mind-Reading - Authored by: Anonymous on Monday, November 08 2004 @ 08:32 AM EST
| |
| Authored by: Anonymous on Monday, November 08 2004 @ 07:25 AM EST |
Perhaps the best example of the "wisdom of crowds" is the wisdom of a
crowd of some 100 billion neurons....
[ Reply to This | # ]
|
| |
| Authored by: wobbegong on Monday, November 08 2004 @ 07:29 AM EST |
Jack Cohen who writes on science by himself and with Ian Stewart has written
about the idea of 'extelligence', the externalised manifestation of intelligence
by a culture/society. [Try reading 'Collapse of Chaos' or 'Figments of Reality'
both written by Cohen & Stewart for interesting views of science from a less
reductionisy viewpoint]
In many respects it is a way of dealing with one of the final points up for
plebescite in Shockwave Rider (that we as a race know far more than anyone
as an individual) and is a way of considering how this additional knowledge is
stored and used. A library (or the internet) is a manifestation of the
extelligence of humanity. The vast amount of information that is 'out there'
but not nessecarily consciously known or used by so many people could have
an effect on the accuracy of the Delphi polling / random amateurs technique.
[I wonder how many people who did higher math, but don't use it anymore in
normal life, still subconsiously parse some news reports in a way that makes
them not trusty some stastics without really knowing why?]
BTW - what John Brunner really did with many of his novels (Shockwave Rider,
Stand On Zanzibar, The Sheep Look Up, The Jagged Orbit, The Stone That
Never Came Down, and probably more) was do a lot of research, on things
like 'Future Shock' [that is credited in the preface to Shockwave Rider] and
then carefully extrapolated from there. What is sometimes frightening is that
he has a remarkable hit rate, and I think what makes it seem different from
much SF is that he was predominatantly extrapolating social & cultural
patterns wheras most SF writers who research a lot are producing things with
a higher technological basis.
Tim Kirk.
[ Reply to This | # ]
|
| |
| Authored by: Hydra on Monday, November 08 2004 @ 07:33 AM EST |
It's that group experience that makes the difference.
Expertise is needed, but relying on expertise alone will leave you worse off
than if you couple expertise with diversity. Does that sound familiar? It
should.
Isn't this, in a way, simply a plea for more of
the proverbial "Thinking outside of the box." ? Because with expertise,
which is usually limited to a specific narrow area of knowledge and the method
of thinking/reasoning that comes with it, you can miss out on the more original
and/or radical and/or revulutionary ideas.
2 examples of solutions that
were never thought of by mankind, because of practicality and existing
ideas:
Since 1993 Adrian Thompson has been evolving logic circuits. One
of the things he did was to create, by evolving algorithms, a circuit that could
distinguish 2 input signals, 1 kHz and 10 kHz. One 100-cell 'field-programmable
gate array' (actual physical silicon, so no virtual simulation of the logic
circuit!) and some 4100 generations later, he got exactly that.
No human
engineer would have been able to do this with just 100 logic cells. One would
start with conctructing a circuit for a clock, so the input signal can be
referenced against it. But even just building a clock with only 100 logic cells
is impossible. The evolved solution that Thompson found did none of that:
Apparently the input signal got looped back onto itself in a complicated series
of loops, no clock involved. It gets even more spectacular, because further
study showed that only 32 of the 100 logic cells were actually
needed.
There is more to that story than this. One of the things is that
silicon properties depend on temperature, so the temperature range in which the
32-cell logic circuit would work was narrow.
The other example is also a
result of evolving algorithms, in this case dog fight programs for AI pilots.
After repeatedly mutating and recombining the flight programs of the top
performing AI pilots, the researchers ended up with AI pilots that pulled off a
manouvre that no human figher pilot ever considered: Rolling the plane while
turning, in order to keep the plane more stable!.
Of course these are
extreme cases of "thinking". To the point that is is not even thinking anymore,
just massive trial-and-error. So there is no expertise or intelligence involved
directly. But it still proves the point that preconceived ideas, knowledge and
thinking along certain lines can keep you from finding the more radical
revolutionary solutions. Putting people without expertise into a group with
people with expertise will broaden the search to a solution considerably. [ Reply to This | # ]
|
| |
| Authored by: Anonymous on Monday, November 08 2004 @ 08:31 AM EST |
As someone else on this thread stated, this field is a matter of research in
nanotechnology, biology, and other disciplines.
It would be something new if it explained or presented a new model that would
possibly improve upon the current state of art in that sciences. Only it
doesn't.
The famous Cluetrain Manifesto also (less scientifically, more
"wiredish") reported this social phenomena before, perhaps, during the
dawn of the new media and the dot com furor.
The Cluetrain Manifesto reads:
"These markets are conversations. Their members communicate in language
that is natural, open, honest, direct, funny and often shocking. Whether
explaining or complaining, joking or serious, the human voice is unmistakably
genuine. It can't be faked."
I still haven't read the reviewed book, and the "review" doesn't shed
a light of criticsism, even more so, it seems as the reviewer honestly believes
the author of the book has just discovered New Foundland.
Nothing wrong with that, just label it under "recommended reading"
instead of "unbiased, critical review" which is what most of the
Groklaw readers will probably imply.
On a side note... the section that refers to the Asimovian idea of crowd
behaviour prediction plainly sucks.
Asimov and Orwell, among other thinkers and writers have been warning us of the
dangers of crowd control for more 50 years now.
http://www.online-literature.com/orwell/1984/
Interestingly, Orwell said "1984" was written "to alter other
people's idea of the kind of society they should strive after."
Dictatorial regimes have tried and consistently, miserably failed over and over
again.
The quoted section about emergent behaviour elimination is nothing but
fascistoid propaganda.
The subyacent meaning of this is that a mighty international business company
could effectively learn how to eliminate subversion in the ecosystem it
shepherds.
It's depicting emergent behaviour as something negative, when it's exactly the
contrary, emergent behaviour is the only thing that can generate the new
paradigm shift after the current one.
In fact, I believe we can safely say that learning how emergent behaviour works,
and letting it flourish is the only thing that can save companies and societies
from obsolesence.
(also for a good read, another sci-fi writer, Sir Arthur Clarke, in a book of
his series "Rendezvous with Rama", showed us a theoretical
extraterrestrial society, where the government, sociologists and engineers (the
society architects in a technologically advanced society) respected and nurtured
the role of artists, thinkers and scientists, that would provide the raw
materials, the new ideas that would feed the rest of their world)
In this sense, Linux and Groklaw are more revolutionary than conservative, as
they represent a paradigm shift in the current ecosystem they live.
But they won't be the last one, I can sign that.
I can be found at horacio.lopez (at) gmail (dot) com
if you find the topic interesting enough to continue the discussion (that I'm
driving sort-of off-topic)
Thank you guys for the interesting read and hard work.
Best,
vruz
[ Reply to This | # ]
|
| |
| Authored by: qu1j0t3 on Monday, November 08 2004 @ 10:27 AM EST |
| And here I was thinking you were leading up to mention: A JURY. [ Reply to This | # ]
|
| |
| Authored by: Anonymous on Monday, November 08 2004 @ 12:28 PM EST |
Excellent comment.
I couldn't have said it better.
cheers,
vruz
[ Reply to This | # ]
|
| |
| Authored by: Ares_Man on Monday, November 08 2004 @ 12:41 PM EST |
Food for thought: Is this book--"The Wisdom of Crowds"--the product of
the collective wisdom of a crowd, or is it the product of the expert intellect
of an individual? :P
---
The DMCA is a blueprint for turning business models into law![ Reply to This | # ]
|
| |
| Authored by: Anonymous on Monday, November 08 2004 @ 01:04 PM EST |
I believe in "the majority of one", as in one person in a
group
is the only one who is right. I don't necessarily believe in the
wisdom of
crowds, for example those crowds who are giving me George
Bush as our reelected
President. I'd like to believe in the wisdom of
communities, but what are we to
make of the all-white juries in Tulia,
Texas who wrongly convicted their black
neighbors? Or the groupthink
among George Bush's advisors that got us into
Iraq?
Wisdom at the individual level would suggest not
judging a result before
its consequencies come in. Linux wasn't very impressive
when it was
Linus' playtoy. And Reagan's critics in the 1984 election were
claiming
he would get us into a nuclear war. He ended the Cold War quite
handily.
The same may prove true of Bush's 2004 critics. They are, after all,
many
of the same people.
The results of the Iraqi War are still to be
determined. Afghanistan just
held a successful democratic election, something
few would have
believed possible five years ago. Iraq could easily follow early
next year.
Iran now has five protests a day against its brutal rulers, so its
democratization may not be far off. Claiming that the Middle East can't
be
democratized is saying that the people there aren't able to function
wisely as
crowds, that they require your brutal "majority of one" even
when that one is
an evil Saddam.
And a similar 'one leader' rule by corporate CEOs is
probably Microsoft's most powerful argument with the CEOs of
businesses
considering Linux. "Where," Bill Gates can ask, "is someone who can be
held
accountable for Linux and its intellectual property problems. Linus
is a mob of
nobodies working in their pajamas."
"One leader" can certainly make a
difference. With one order, Gates
turned Microsoft
around in the mid-1990s and
forced it to adapt to the Internet.
Microsofties are proud of that. But would a
Microsoft that's
more like open source have already adjusted to the Internet
and not
required a top-down dictate by "one leader?"
Last of all, some of
the wisdom of crowds may lie in selecting the
proper leader. It was the poorly
educated crowd of working
class men, mostly members of the Labor party, whose
opinions forced
Britain's ruling classes to make Churchill their Prime
Minister in WWII. It's popular opinion that gives Linus his authority over
Linux development. Wise crowds don't have to lead. They can wisely
select
their leaders.
--Mike Perry,
Inkling blog , Seattle [ Reply to This | # ]
|
| |
| Authored by: jaydee on Monday, November 08 2004 @ 03:45 PM EST |
| PJ, I hope this doesn't fall into "political" I've tried to answer in terms that
apply to the discussion. The discusion itsself does seem to lean in the
direction of discussing the mechanics of the democratic process.
"I don't
necessarily believe in the wisdom of crowds, for
example those crowds who are
giving me George Bush as our reelected President."There are
several points here: 1 The amalgamation process of the individual decisions is
very crude (one person one vote + electoral college has no averaging). 2 It
looks to me that diversity of opinion was not available. i.e. it was not a
question of the crowd finding the best personto be President, but the crowd
deciding between two options. 3 How do you know John Kerry would be better? I'm
thinking mainly of the Bob Hope quote about the evil of two lessers.
"I'd like to believe in the wisdom of communities, but what
are we to make of the all-white juries in Tulia, Texas who wrongly convicted
their black neighbors?" Where does diversity in the decision
making group come in to this.
"Or the groupthink among George
Bush's advisors that got us into Iraq?"Again diversity is
missing.
I quite like some of your alternative rules, (6) reminds me
of "Nothing is fool proof because fools are too ingeneous".
---
If you
find yourself in a fair fight... you didn't plan you mission properly. [ Reply to This | # ]
|
| |
| Authored by: rsteinmetz70112 on Monday, November 08 2004 @ 06:11 PM EST |
This theory is one way of describing both open democracies and open markets,
where the independent action and interaction of many people integrate a large
number of individuals into the optimal outcome, balancing risk and reward.
There may be short term aberrations caused by external factors or even short
term irrational behavior, but eventually the system moves back into balance. The
very essence of Adam Smith.
The genius of the America system of governance is that the Constitution acts as
a brake preventing volatility. The existence of a two year feed back cycle
between elections helps keep people for opting out of the system. There's always
next year.
In parliamentary systems, especially where there are large disparities between
the elite and the masses, with the winner take all features and high degree of
party identification, the possibility always exists for a radical government to
be swept into power and disrupt social institutions. That has seldom happened in
Europe with large numbers in the middle classes and relative prosperity, but it
can and has happened in other places particularly in Latin America.
---
Rsteinmetz
"I could be wrong now, but I don't think so."[ Reply to This | # ]
|
| |
| Authored by: Anonymous on Tuesday, November 09 2004 @ 12:18 AM EST |
All the examples cited deal with (large) numbers of individuals acting
independently of one another. That's statistics, and well-known to anyone with
the relevant education (though also widely misunderstood, ignored, and abused).
The author is stumbling from a position of ignorance towards some elementary
results, which he then imbues with some pretentious significance.
What distinguishes a crowd from a collection of individuals is that the crowd
involves interaction: its members act in a manner that's very far from
independent. That's when elementary statistics give way to "social
science", and explanations of behaviour cease to be trivial. And that's
precisely what's missing from these examples.
[ pedantic caveat: In introducing statistics above, I was firmly within my own
academic and professional expertise. I'm now about to stray outside that into
areas where I don't qualify as expert ]
Groklaw's strength is the team: we have people contributing expertise in a range
of different fields. Groklaw's weakness is crowd interaction: a tendency to
reinforce whatever beliefs and prejudices we may happen to share. For every
genuine expert post, there's a great deal of uninformed speculation. Except -
it's not quite uninformed; it's self-informed. There's little to distinguish
information from prejudice and misinformation, and both can easily propagate.
Actually it's not as bad as that. Sure, we have misinformation and prejudice -
as does any forum open to the public - but things that matter don't go
unchallenged. Groklaw's crowd isn't going to work itself up into a frenzied
"Crucify", "Sich Heil", or "Four more years".[ Reply to This | # ]
|
| |
| Authored by: Anonymous on Thursday, November 11 2004 @ 10:25 AM EST |
... to the extent that the world seems to have
chosen Windows over Linux
[ Reply to This | # ]
|
| |
| Authored by: Anonymous on Thursday, November 11 2004 @ 10:32 AM EST |
| Many eyeballs spotted that document forgery in the 60 Minutes II report. [ Reply to This | # ]
|
|
|
|
|
