First implementation: many problems

I tried a first launch with a sample implementation of the algorithm. So far, with only 15 human users, it’s been a failure. Some problems come from the design of the site and others from algorithm choices, I’ll only describe the last ones.

The algorithm was set up with a pool size of 5 fragments, using multiplicative update (even after positive feedback), and never had to delete links (because of the low user count). In addition to the 15 users, the network included about 20 automatic submitters, all wired to RSS feeds from categories on reddit.com and lemonde.fr.

Interestingly, taste tracking was never criticized in itself and most users clearly felt the response to their negative or positive responses.

The most significant problem was in the algorithm for selecting new links. The chosen algorithm first tries to pick the user with the biggest number of common negative votes, if none match it tries to pick the user with the biggest number of common votes, if none match again, it picks the most active user (the one who submitted the biggest number of fragments).

This strategy was used both when selecting new user after negative feedback and to avoid fragment starvation. Because of the small size of the network, and because of the last picking rule, all human users ended with a source link from the same automatic subscriber, regardless of their taste. Next iteration will remove this rule and allow starvation.

The second problem is in the selection of the fragment among the source pools, if sources have nearly no common fragments, the algorithm will simply pick all fragments from the best source one after another until all available fragments from this source are used, then pick the next best source, etc… Even if all sources are all initially good, this behavior is boring for human users, they prefer diversity. Moreover, this effect is increased when the sources are automatic subscribers from a single source, the users sees all fragments about one topic, then about another… The plan for the next iteration is to try a selection algorithm based on weighted random selection.

The third, and very human, problem is the pool size. When the linked fragment is long to read (or watch, sometimes it’s a link to a video), some users hesitate in their vote and want to read it later. They skip it and examine the next fragment in the pool. This effectively reduces the size of the pool, if they repeat this behavior too often, they end up being locked and forced to vote on at least one fragment, but voting on a single fragment does not “unlock” other undecided fragments on the pool. Because of this, the benefit of the vote seems too small compared to the effort of voting (It’s a lot like having a mailbox full of unread mail, except each time you read one you receive another one…). To prevent users from locking themselves in this frustrating situation, we will simply reduce the pool size to 1.