I saw an interesting post introducing one of a new game designed specifically for iPad, called “Xylem: the code of plants”, or Xylem for short thereon. The blurb for this game is that it gamifies (part of) formal software verification so that a particular task, dubbed as finding the “loop invariants”, can be achieved by the players in a crowdsource manner.
Well, most of the claim is true, but deceitful.
Let me introduce the game a little more and then start shooting guns at it.
The Game
Storyline
You are a student who studies botany and are sent for internship in a place that no one has ever heard of (well, it is called “The Island of Miraflora”). Your task is to “catalog the flowers found on this island”. That would be an exaggeration, in terms of the boredness of the game.
What you in fact would have to do, is to think of the different flowers as variables and establish a mathematical equation, or, “an invariant”, between them that applies to a few pictures given to you, expressing relations of the variables (well, flowers, in the game’s terminology).
Game Design
Frankly speaking, the game designers have put some thoughts in it. You have a name in the game, and constantly mail (yes, mail) your mom to tell her when you can finish your task and spend the rest of your summer with her. Yet this is useless for the purpose of cataloging the flowers (well, finding the invariants).
The graphics of the game is fine. Very low resolution, and the flowers look cheap. Yeah, cheap. They do not look like something beautiful that you may find on an exotic land, but like plastic ones that you can get for two bucks a dozen in your convenience store.
There are five small islands (levels) that you can play. At the first time I played this game, only the tutorial level (Base Station) and the first level (Poppy Cove) is available. After about a month when I get back to this again, the second level (Lambert Mountains) is available too, but others are not.
The Name of the Game
Xylem, according to Merriam-Webster dictionary online, means some stuff related to plants that I do not care. Well, I made that up. The meaning can be found here.
The Deceitful and the Truths
Overall, the game is designed for five-year olds. And I guess he/she will get bored pretty quick. I played for about 30 minutes only to find out what would happen if you play it over and over. Guess what I’ve found out?
Unlimited Boredness
Inside one level, you just play the “find the invariant” stuff over and over, in batches of ten. When you finish a batch, another batch comes to your feet to go on.
The only way to finish this, is that all players of this game collaboratively finish all the puzzles that the developers have prepared (very likely generated automatically).
The only tiny change in the progress is that, the home screen, there is a piece of statistics of how many puzzles are still left in this level, and how many are finished. At the time of this writing, Poppy Cove is finished (4500 out of 4500 puzzles), and Lambert Mountains is in progress (562 out of 2500). (According to the game forum, the initial goal was an ambitious 30,000 for Poppy Cove alone! This must have been changed after some game updates.)
This is what “crowdsourcing” is all about: you do not finish a task by yourself but collectively by a large group of people. So do not be silly and play the game over and over. Just wait for others to do your job for you if you do not enjoy “finding the invariants”. After all, if you are neither getting paid or enjoying the play, what’s the point of wasting the time?
Limitations of Game Design
First, only non-negative numbers can be produced by your equation. This is why I say it is designed for five-year olds. You could say something like 3 + 6 – 5 = 4, but not 3 – 5 + 6 = 4? Then, what is the point of teaching five-year olds loop invariants? Why not teaching them some real mathematics instead?
A second limitation is that, you can never find your solutions to puzzles you finished before. You are just ruthlessly used to produce some statistics to their research project. Period. (On the contrary, bravo for the team that decide to divide the island into five to have created an illusion that you might advance to higher levels for more challenging puzzles, which is not the case: all puzzles are randomly generated and those from higher levels are not necessarily more challenging in general.)
By now you probably realize that the game is just a hoax to trick you to contribute statistics, but a third limitation of the game is just sicker than that. Every time you finish one puzzle, you have to click the Transmit button to advance to the next puzzle. What it does is to transfer your solution back to their secret server for their purpose. If you do not have an Internet connection (who would expect that in the remote Island of Miraflora!?), you are in trouble. You are stuck forever because you will be prompted that you, a useless monkey, could not continue playing their game if you cannot contribute your time and intelligence for free to their statistics.
A fourth issue that limits the fun of the game is that, you could easily outsmart a puzzle by creating a simpler or even meaningless invariant. For example, in a puzzle where the number of red flowers is exactly two more than the number of blue flowers (i.e., red = blue + 2), you can simply outsmart the invariant by saying that the red flowers are more than blue ones (red > blue). This is a weaker invariant, but it still fools the game judge. The fun of the game decreases sharply once you find this out and decide to be lazy. A star system that evaluates your invariant is there (the more exact the invariant, the more stars), but unfortunately it is not smart enough to decide what is “more exact ”.
Lastly, the customer support of this game is interesting. They are just like any customer support on Twitter: “Yes we have heard what you say. Now shut up because we have forwarded your comment to someone you will never know and now just wait.” The only difference is that the team cared enough to build an online forum for that.
Last Words
Overall, the game is designed for five-year olds. I’ve wasted about one hour playing it and maybe two hours bashing it. I do not recommend the game for anyone who is interested in finding a good game to play. However, if you are, as I am, working in the field of software verification, it is suggested that you take a look and think about how to improve player engagement for them to crowdsource enough data for your research. The SRI and USC team have made an attempt. This may be a good thing, but I cannot like it much.
(Disclaimer: I have been a student intern at the CSL lab of SRI. I am not involved in this project in any means, though some of my previous colleagues that I know of work in this field.)