Kim Baine after the session at
I was pleased to attend Forum One’s Online Community Summit again this October. It’s not the wine country setting but the intoxicating ideas that bring me back for another year. For me some of the most exciting ideas on design and group behavior came out of the “Recent Research in Online Community” session.
Paul Resnick of the University of Michigan presented on design and group behaviors. He is starting a project to build an open design handbook on the web, based on actually testing and quantifying the gut design choices we make when designing for interactive groups.
Fundamental findings so far:
- The ease of discarding identities does matter to the quality of discourse. (We all know in our gut that this degrades interactions, he designed a test and confirmed it.)
- A “can’t trust newcomers” attitude grows if sockpuppets, reincarnation with another persistent identity and driveby posting are too easy.
Solutions and useful approaches include:
Ways to reduce disposable identity problems:
- Make barriers to entry or to full entry,
- No voting/ranking allowed until proving oneself to be reliable or at least a real and unique person.
Examples of high barrier to entry include http://www.sermo.com/ — Social networking for doctors: Trusted by physicians because they look up your license to admit you to hang around with other doctors. (That’s way tougher than The WELL!)
- Research on collaboration implies designers should assign challenging specific goals!
This was tested at the research project at the Movie Lens site. The site’s goal was to get lots of user-rated movies.
- Don’t just ask to “please rate as many as you can” but ask people for specific targets: please rate 10 this week was compared to please rate 30; 90; 120. PROVEN: a specific goal is preferable, and a higher goal works even better so long as it is doable. (A drop off in contributions was seen when they asked for 120 movie ratings in a week. Most likely this was seen as impossible. Researchers expected a softer goal to keep the group happier and performing better, but saw the higher but not impossible goal of 90 yielded the best outcome.)
An audience question was asked about aggregate goals for group (Moveon, etc) From research on NPR fundraising, etc. aggregate goals DO help. Tell what others are donating or accomplishing … but tell about the 90th percentile, not top 10% of donors.
- Speculation from the researchers anticipated that people will shirk work that is assigned collectively to a group but the opposite effect was seen. An arbitrary group membership was announced to users. The other group members were unseen, there was no feedback or ability to observe the group’s work independently. Still, in this arbitrary group members showed loyalty to making the group succeed, and did better compared to those not told they were in a group.
- If you’ve got some kind of karma rating, when you show people at the bottom how they compare to median they will move up. (unless it looks impossible)
- If you tell top producers that they are excellent they will do less. (But they may do something different and even more special if challenged to do that.)
(He shared the amusing tidbit that Psych research (where deception is required in almost any test) and Economics research (where there is no deception allowed) were the two study types merged for this study. The solution: Selective truth. No lies used, but no whole truth was told to various sample groups.)
On welcoming newcomers:
- What happens in response to a first post or any initial contribution matters
- Getting a human response makes a 12% difference in retention.
- The tone of the response does not matter! Fights, praise, whatever!
- Accuracy of information in the response does not matter either! Response creates stickiness.
The Hawthorn effect is a problem in studying this stuff, too. (That’s from an old AT&T study in the 20s. More lighting = better productivity; then less lighting also = more productivity! One might cite the Hawthorn effect as a reason for redesigns, actually.)
Neel Sundareson, in-house EBAY scientific researcher – had some good info about reputation and behavior too.
Interestingly enough, classic academic auction theory is not a good fit with Ebay. In addition, people do not behave as they say they do if you ask them about their buying and selling. Hence they needed a serious academic research initiative inhouse to better their reputation systems.
- Ebay has an implicit desirability index: Search relevance has to be adjusted according to supplies of items available, so a market is not pure search. People search a lot for what is not for sale. What do you show them? Fascinating challenge.
- He noted there is a lot of community spirit out in the long tail of transactions with specific groups of collectors, etc.
- Items get sold, bought and then resold up the trusted seller reputation chain on EBay, for better prices.
He’s got gorgeous charts to show these things and more. It was cool seeing that level of research and analysis.
TALK TO ME: Foundations for Successful Individual-Group Interactions in Online Communities
http://www.cs.cmu.edu/~cprose/pubweb/arguello-chi.pdf (includes the welcoming effect)
Paul Resnick’s Using Social Psychology to Motivate Contributions to Online Communities
http://www.si.umich.edu/~presnick/papers/cscw04/ (includes group identity and contribution dynamics, and a link to the ongoing research home for this work, and his upcoming Handbook Project where he will be calling for case studies of group behavior effects seen in response to changing interfaces or services in communities of users.)