For almost each set, there is one tile that is completely wrong, but the rest is at least in the good category. Overall, I am really surprised how well it performs.
You can try it yourself online with your own images here, and of course find the code on GitHub.
Technically it is built entirely in the browser, there is no server side component except the what’s behind the API of course:
Images are loaded from presets or via the browser’s File API.
Each tile is converted in its own image, and converted to base 64.
All of this is sent at once to the Google Cloud Vision API, asking for label detection results (this is what matters to us here, even if the API can do much more like face detection, OCR, landmark detection…)
Only the label with the highest score is kept from the results and printed back into the main canvas.
I’m glad to present the work of a week-long hackathon I did last November with three other googlers.
I’ve been having for a long time the idea of a real time game using phones as controller and a big screen as the main game screen. You may recall a previous blog post about a first prototype. A few years later, I could pitch the idea at an internal hackathon and gather a team around the concept. While I initially wanted to build a dancing game, I realized that this was already done, so we decided to go towards mini games, an idea very compatible with a hackathon timeframe.
Realizing after a day that we would not be able to have a reliable and scalable real time architecture and a fun game after just five days using regular backend technologies, we decided to focus on the game itself and to use Firebase to handle the real time and backend parts of our system. It allowed us to get familiar with this very interesting technology. I already used Parse in the past, while they both fall into the Backend as a Service (MBaaS) category, I could notice with pleasure that Firebase was able to perform as well as Parse for the regular MBaaS features (and even better to my taste, by providing an Angukar SDK) but that it also provides impressive real time capabilities.
After a single day, we had a functional prototype of our game: Android phones were interacting in real time with an Angular application in a “Shake” game. The next three days were spent creating other mini games and streamlining the game mechanism.
And we won the Fun category!
While the hackathon was internal, it was not a problem at all for Google to let us open source and release our work to the public. You can find the sources on GitHub: Android and Web. Firebase showcased the project on its official blog.