World Draw

Having worked with Google on the interactive opening to I/O for the past three years, our goal with World Draw was to push web technology as far as possible to create a massively engaging, interactive experience. The idea was simple: have users from across the globe come together to create a world by drawing objects that would be placed into a world.

Draw Things

The core interaction involves the user drawing a 2D creation on a canvas which is recognized, turned into a 3D object and placed into the world. We called upon the AI & ML technology behind AutoDraw and QuickDraw to recognize what the user is drawing, and this is translated into a model using WebGL. Once created, the model is placed into the world in a position based on available placement inventories. We used Google Poly as a base to create over 90 customizable models, ensuring users have maximum flexibility when building the world. To streamline engagement, users can download the experience as a Progressive Web App (PWA).

Creating the World

When creating the world, we needed a system that could scale infinitely and maintain structural integrity while allowing for customization from a massive base of contributing users.

In order to lay out the world, we used a tiling system. Based on incoming drawings, a type of ML called Decision Tree would lay out tiles of pre-defined environments to receive objects. To make these tiles, we created a tool for our designers that allowed them to construct tiles with multiple configurations of objects across all the different environments. This interface allowed the designers to individually prepare every part of the world, from the tiles themselves, to the degree of customization on objects. Once the tiles were created, they were saved as tiny 128x128 images with colors representing information about the objects and their placements on the tile.

In order to maintain consistency in the world, our artists created individual color sliders for each object to ensure users could customize their drawing without degrading the look of the world. We used Firebase Realtime Databases to give users drawing suggestions based on vacancies within tiles already existing in the world. A multiplier was also worked into this system, allowing a single user to create multiple objects to ensure the open inventory on tiles was loosely matched with incoming drawings, where possible.

Finally, to ensure the infinitely scalable world rendered efficiently, we grouped all unique objects together allowing them to be rendered in a single draw call and instanced as they were placed in the world.

Bringing The World To Life

We used a variety of technologies in order to facilitate thousands of real time connections between servers and devices. For large scale events, such as Google I/O, we use Socket Network; a massively scalable Node JS application that allows a large number of people to connect and send messages to a single installation point. In World Draw, Socket Network enables drawings to be sent to a server and placed in the world in a synchronous way to ensure no two objects are ever placed in the same spot. Socket Network is also used to trigger the Flying Game event, where a socket is sent out to every connected phone, transitioning everyone into the game experience at the same time.

Taking Flight

Once Socket Network transitions users into the game experience, we use another Node JS application, Game Server, to group users together based on location in order to minimize latency in the multiplayer game experience. This allows people to see planes of other users while flying over the world.

Real Time Data Communication

We used Google Cloud Platform utilizing App Engine, Cloud Functions, Firebase Realtime Database, Firestore and Cloud Storage to collect, store and communicate information in the world.