TileCode: Creation of Video Games on Gaming Handhelds

Thomas Ball, Shannon Kao, Richard Knoll, Daryl ZunigaMicrosoft

Redmond, USA{tball, shakao, riknoll, dazuniga}@microsoft.com

ABSTRACTWe present TileCode, a video game creation environment thatruns on battery-powered microcontroller-based gaming hand-helds. Our work is motivated by the popularity of retro videogames, the availability of low-cost gaming handhelds loadedwith many such games, and the concomitant lack of a meansto create games on the same handhelds. With TileCode, weseek to close the gap between the consumers and creators ofvideo games and to motivate more individuals to participatein the design and creation of their own games. The TileCodeprogramming model is based on tile maps and provides a vi-sual means for specifying the context around a sprite, how asprite should move based on that context, and what shouldhappen upon sprite collisions. We demonstrate that a varietyof popular video games can be programmed with TileCode us-ing 10-15 visual rules and compare/contrast with block-basedversions of the same games implemented using MakeCodeArcade.

Author Keywordsgaming handhelds, video games, visual programming, cellularautomata

CCS Concepts•Software and its engineering → Visual languages;•Human-centered computing → Human computer inter-action (HCI);

INTRODUCTIONMost video game development environments require accessto a tablet, laptop or desktop machine, often with an Internetconnection, putting game development out of reach for thosewho cannot afford such devices or don’t have Internet connec-tivity. On the other hand, a search for “gaming handhelds” onAmazon reveals a huge variety of low-cost gaming handheldsthat come loaded with hundreds of retro video games.

Furthermore, for those wishing to create their own games,the conceptual gap between the idea for a video game and

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than theauthor(s) must be honored. Abstracting with credit is permitted. To copy otherwise, orrepublish, to post on servers or to redistribute to lists, requires prior specific permissionand/or a fee. Request permissions from permissions@acm.org.UIST ’20, October 20–23, 2020, Virtual Event, USA© 2020 Association for Computing Machinery.ACM ISBN 978-1-4503-7514-6/20/10 ...$15.00.http://dx.doi.org/10.1145/3379337.3415839

Figure 1. Screen snapshot of “Boulder Dash” game (written usingTileCode) in the web-based MakeCode Arcade game machine simula-tor; underneath the simulator are the backgrounds (Wall, Dirt, Space)and sprites (Player, Boulder, Diamond, Enemy) in the game.

its realization via a computer is non-trivial. While game me-chanics, the desired interactions among video game elements,can be described informally and succinctly (“a boulder fallsif there is space below it”), encoding a game’s mechanics fora computer to execute may require mastering various con-cepts, ranging from coordinate systems and game sprites tothe syntax/semantics of a programming language. Web-basedprogramming environments such as Scratch [23] and Make-Code Arcade [6] use block-based editors [9] to simplify gameprogramming, but still introduce a large number of concepts.These environments also require a computer with a modernweb browser and Internet connectivity.

Our main goal is to enable the process of game creation totake place on low-cost gaming handhelds themselves, ratherthan tablets/laptops/desktops. A secondary goal is to reducethe gap between game mechanics and game programming somore people can participate in game creation.

(a)

(b)Figure 2. (a) Screen snapshot of the TileCode game creation environ-ment with a “When-Do” rule for a boulder at rest to start moving down;(b) TC running on a MakeCode Arcade-compatible gaming handheld.

Towards this end, we have designed and developed a restrictedand visual game programming model with a supporting gamecreation environment called TileCode (TW). TC games andits game creation environment both are based on tile maps,a foundation of many retro video games, and can run on avariety of low-cost microcontroller-based game handhelds.

A TC game has a set of sprites (such as the player, boulder,and diamond shown in Figure 1); each sprite is centered ona tile and each tile displays a background image. A spritecan move in one of four directions (left, right, up, down) toan adjacent tile. The TC game creation environment itself isimplemented using tile maps as well: all editing takes placevia a cursor moved with the 4-way directional pad (dpad) ofthe device.

A TC program is a set of rules, each of which is associatedwith a sprite kind (such as player or boulder). A rule takes theform of a “When-Do” pair, as illustrated in Figure 2(a). TheWhen section visually describes a pattern/predicate over the3x3 local neighborhood around the central sprite (a boulder,in this case) to be matched against the tile map. The top rowof the figure shows that the tile below the boulder should havethe grey-blob (space) background and not contain any sprite.When this pattern matches on the tile map, then the boulderis sent a move down command, as shown in the Do section.This rule does not match on the tile map from Figure 1, butwill match once the player sprite moves out from under theboulder. Other rules can specify what happens when buttonsof the game handheld are pressed, or when sprites collide.

As in cellular automata [25], TC rules operate in parallel on thesprites on the tile map: the 3x3 predicate of the When section

determines whether or not a rule applies to a sprite (note thatmultiple rules can apply to the same sprite). The result ofa rule execution is to send commands to the sprite/tiles atthe center of the 3x3 neighborhood or adjacent to the center(unlike cellular automata, which only updates the center tile).As multiple commands can be sent to a sprite/tile, conflictingcommands are resolved automatically, which may result innon-deterministic/random behavior (for example, if one rulecommands a sprite to move left and another commands it tomove right).

TC is implemented on top of MakeCode Arcade, a web-basedgame creation environment for retro arcade games (160x120pixel screen with 16 colors) that can run in the web browser oron a variety of microcontroller-based gaming handhelds [26].Figure 2(b) shows such a handheld displaying TC’s loadscreen. MakeCode Arcade supports both block-based andTypeScript-based programming [2] (TC is implemented inTypeScript). The code of TC and all the games described hereare open source (see https://github.com/microsoft/tilecode).

The main contributions of our work are:

• the design and implementation of TC, a game creation envi-ronment that runs on battery-powered gaming handhelds;

• a restricted programming model with a mapping to a visualediting experience that closely matches the domain of tile-based games;

• implementations of a variety of well-known games (BoulderDash, Snake, Bejeweled) in TC, using just 10-15 rules each;

• a comparison with block-based versions of the games im-plemented in MakeCode Arcade.

Our evaluation shows that the ability to pattern match againstmultiple tiles/sprites and to refer to the direction of a sprite’slast movement is key to enabling more games to be written ina succinct fashion and without the explicit use of loops andarrays.

RELATED WORK

Turtles, Automata, and ObjectsThe Kara programming environment [10] introduces studentsto turtle-style programming (ala Papert [18] and Pattis [19])in a tile-based world, one of many examples of the sort. Karais a programmable ladybug that can sense local conditionsand respond by executing primitive actions, like TC. Karaallows the user to structure their program using finite-stateautomata/machines with user-defined states; TC lacks such acapability, so we instead encode needed states in the tile map(see the TC encoding of the Bejeweled game), but has morepowerful pattern matching capability.

Moving beyond finite-state automata, environments such asPlayground [8] and Alice [4, 12] use object-oriented or agent-based paradigms for novice programming of interactive anima-tions. TC’s programming model is not object-oriented, thougha single TC rule can match and act upon multiple sprites, arestrictive form of multiple dispatch [16].

Novice Game ProgrammingOur work is heavily inspired by the Kodu Game Lab [29], anintroductory game making environment situated inside a 3Dworld of programmable objects that feature robot-like sensorsand actuators. As Kodu objects operate in an environment ofcontinuous motion, much like that of robotics, the language isextremely high level and inherently ambiguous. TC shares the“When-Do” programming paradigm of Kodu, but departs in anumber of ways: TC operates in a discrete 2D world of tiles,sprites, four directions, and focuses on retro video games.

There are many systems, in addition to Kodu, to help novicescreate their own games. Minecraft is a popular example where“redstone” blocks in the world can be laid out to create logiccircuits (in fact, complete computers have been built insideMinecraft worlds using redstone). [22] Super Mario Maker isa side-scrolling platform game that allows the user to createtheir own game levels within the application [13], but the gamemechanics are fixed.

The Scratch [23], Stencyl [14], and MakeCode Arcade [6]environments use the Blockly [9] framework to build gamesby programming with structured control-flow; while Blocklyhelps to prevent syntax errors, there still is a large concep-tual/visual gap between the block-based program and the gameworld. TC is based instead on pattern matching against thetile map, rather than control-flow, as the primary programstructuring mechanism.

The “program by demonstration” model of novice simula-tion/game programming was introduced in AgentSheets [20,21], and then championed by Cypher, Smith and colleaguesin their KidSim [5, 27] and StageCraft Creator [28] systems.In these systems, the user demonstrates how a sprite shouldmove/change in a local context. A graphical rewrite rule is theresult of the demonstration, expressed in terms of “before” and“after” pictures. BlockStudio [3] is a newer demonstration en-vironment for children to create games and animations, in themodel of KidSim. TC rules are not graphical rewrite rules, buta form of guarded command [7]; the When predicate (guard)corresponds to the “before” picture of a graphical rewrite rule,while the Do section is a set of explicit commands.

ToonTalk is based on concurrent constraint programming, afully general programming model [11]. While TC does permitrules to fire concurrently, it has a domain-specific way toresolve conflicts arising from concurrency.

Game Modeling LanguagesThe Video Game Description Language (VGDL) [24, 17] pro-vides a high-level way to describe many common video arcadegames, for the purpose of conducting research on learningand planning algorithms. VGDL contains a set of classes forcommon types of sprite motion found in games, based on anunderlying ontology of video games. The user programs agame by choosing among the motion classes and specifyingan action to execute when sprites collide with one another.

In contrast, TC is based on a set of lower-level primitives toencourage exploration of new game mechanics. For example,in TC it is possible to program the logic for a boulder tumblingoff of another boulder (as shown in Figure 4 and discussed

later); this behavior is not found in the VGDL implementationof Boulder Dash. The user would have to extend the VGDLruntime itself, written in Python, to provide a new primitivefor this behavior. The Snake and Bejeweled games, discussedin the evaluation section, are not expressible in VGDL withoutsubstantial coding in Python.

Mek [30] is a language and prototyping tool for 2D turn-and tile-based deterministic games that also uses visual rulesas its primary structuring mechanism. Mek has no conceptof a sprite moving in a direction - rather all game state andcommands are based on changing a tile’s background color.While the system can be used to prototype game mechanics, itcannot produce playable games, as TC can.

Retro Game Creation EnvironmentsPICO-8 [1] is a retro game engine supported by a virtualmachine that supports a 128x128 pixel screen with 16 colors,much like MakeCode Arcade. PICO-8’s goal is to providea “fantasy console”, which is like a gaming handheld but“without the inconvenience of actual hardware”. The PICO-8programming environment is available for MacOS, Windowsand Linux and supports a text-based Lua code editor. The TCprogramming environment is much more restricted than thatof the PICO-8, as it was designed to run on gaming handheldsrather than desktops.

OVERVIEW OF TileCode APPLICATIONBob purchases a gaming handheld for his daughter Alice.To her surprise, in addition to the set of built-in games, thehandheld comes loaded with TC, which allows modificationof the built-in games, as well as creation of new games.

From the load screen of TC (Figure 3(a)), Alice can select agame from one of eight slots. She moves the square-shapedcursor between adjacent tiles using the dpad of the handheld,selects a tile using the A button to perform an action, andpresses the B button to return to the menu of the current screenor to the previous screen.

As shown in Figure 3(a), Alice has selected the game in slot 1(Boulder Dash), which brings up the game’s home screen ofFigure 3(b). The home screen has a menu bar and displays thefour kinds of backgrounds and four kinds of sprites. Selectinga background or sprite opens a gallery of images that Alicemay select from, not shown here. The green play button on themenu bar allows Alice to play the game, as shown in Figure 1.

The first item on the menu bar is the red map icon—Aliceselects it to open the tile map editor, shown in Figure 3(c). Sheselects the player sprite from the menu bar and moves to thetile map (of reduced 8x8 tiles) to place the sprite. Alice returnsto the game home screen and selects the paint icon (to the rightof the map icon) to bring up the bitmap editor (Figure 3(d)),which she uses to color the player sprite’s eyes green. Gameassets are saved automatically in the non-volatile flash of thehandheld on each transition between screens.

From the game home screen, Alice now selects the codingicon (to the right of the paint icon), which brings her to the ruleselector screen (Figure 3(e)). Alice selects the boulder spriteto see what rules already exist for the boulder. The rules are

(a) (b) (c)

(d) (e) (f)Figure 3. TileCode screens: (a) load screen; (b) game home screen; (c) tile map editor; (d) bitmap editor; (e) rule selector; (f) rule editor.

Figure 4. Rule for a boulder to fall off another boulder to the left, pro-vided there is space available. The numbers show the correspondencebetween the rows of the Do section and the tiles of the When section.

summarized in four quadrants: upper-left–there is a changeto a sprite’s 3x3 neighborhood; upper-right–button pressevent (A, dpad-left, dpad-right, dpad-up, dpad-down); lower-left–collision (smash) of sprite into a tile/sprite; lower-right:miscellaneous events. The highlighted squares represent therules with code.

Alice sees that there are rules in the upper-left and lower-leftquadrants of the screen. She decides to investigate the rule fora boulder at rest, and selects that tile (lower-left tile of upper-left quadrant), which brings up the rule editor (Figure 3(f)).The menu bar of the rule editor provides quick access to themap editor and game play. The blue arrow on the far right ofthe menu indicates there are other rules for the boulder.

Alice knows that the original version of Boulder Dash allowsboulders to tumble off of other boulders when there is space tothe side of the boulders, but finds no such rule for this behavior.

Using the rule editor, as shown in Figure 4, she creates a newrule in which the “When” section has a resting boulder underthe central boulder and empty space to the left of both boulders;the central boulder is sent a move left command.

PROGRAMMING MODELThe TC programming model has a small set of concepts andprimitives, from which a host of different behaviors can bedescribed using a visual rule-based paradigm. This sectionfocuses on the core aspects of the programming model and itsvisual representation. The next section describes the editingexperience.

Game State and RoundsThe basic atoms of TC are tiles and sprites that are located ona grid coordinate system. A tile must contain a background im-age and may contain any number of sprites. As we saw before,there are four kinds of background images and four kinds ofsprites in TC (the implementation actually can accommodateup to 16 kinds of backgrounds and 16 kinds of sprites). Thetile map and the set of sprites make up the entirety of the TCgame state. The game screen can only display a 10-by-7 areaof the tile map, so the tile map scrolls during game play tofollow the player’s avatar (by default, the first sprite kind).

A TC game proceeds in rounds: a round executes rules inparallel on the current state to determine if the game shouldproceed/end, the direction that each sprite should next take,which sprites should be spawned/destroyed, and the actionsthat should be taken upon a pending collision. After a round,the movements/actions are executed by the underlying gameengine to update the state (all sprites move synchronously and

at the same speed). A sprite stores the direction (left, right, up,down) it moved in the last round (or if it remained at rest), forinspection by the rules in the next round.

Game Rules and EventsAs already seen, a TC rule is formed by a pair of a Whenpredicate and Do commands. Rules fire in parallel on thecurrent game state and set of events (e.g., such as buttonpresses), which results in commands being sent to tiles andsprites; each tile/sprite stores a local log of the commandssent to it. A resolution step determines which of the (possiblyconflicting) commands in each log will be executed. A newgame state is produced by executing the commands.

Every rule is parameterized by an event. There are three maintypes of events in TC, corresponding to the quadrants seen inthe rule selector screen of Figure 3(e): (1) change fires whenthere is a change to the local neighborhood around a sprite;(2) press fires for a button press of the dpad or the A button(the B button is not available to the game writer as it is usedby TC to exit the game); (3) smash/collide fires when a spriteis about to collide with another sprite. Section “EvaluationI: Three Popular Games” presents examples of all the aboverules.

When, Tile and Direction PredicatesA When predicate is a predicate on the game state that exam-ines the 3x3 local neighborhood around a tile/sprite. This pred-icate can specify the presence or absence of background/spritekinds, as well as the direction that sprites last moved in.Moreprecisely, a When predicate is a conjunction of tile predicates,one for each of the nine tiles in a sprite’s neighborhood, in-cluding the center tile. Most of these predicates will simplybe “true”, corresponding to a black tile, which means that noconstraints are placed on that tile. A tile predicate is definedby three non-intersecting sets (Include, Include’, and Exclude)and a direction predicate:

• Include: the tile must contain at least one background/spritewhose kind is in this set and whose direction (in the case ofa sprite) matches the direction predicate;

• Include’: the tile must contain at least one back-ground/sprite whose kind is in this set;

• Exclude: the tile must not contain any of the back-grounds/sprites from this set.

A black tile’s include and exclude sets all are empty. Thesecond row of Figure 3(f) shows the include and excludesets for the tile one space down from the center boulder: theInclude set is denoted by green check marks; the Exclude setis denoted by red “no-entry” circle-slash signs; membershipin the Include’ set is denoted by a yellow dot. In this case,the space background is included in the tile and the boulderand diamond are excluded from the tile. Figure 5(a) showsthe seven icons corresponding to the seven possible directionpredicates (left, up, right, down, resting, moving, any). Thedirection predicate icon is overlaid on the sprite, as shown inthe figure.

Sprite WitnessesIn all TC rules, the center tile predicate has a special form,which is to have a non-empty Include set that contains onlysprites. This guarantees that if the predicate matches thenthere will be a sprite witness to execute commands against.The four tiles adjacent to the center tile also may have spritewitnesses. This is the main form of variable binding in TC.Sprite witnesses are displayed in the column to the right ofthe Do keyword, as discussed further below. The Include’ setdoes not bind a sprite witness.

Do Commands and Conflicting CommandsIf a rule’s When predicate matches on the tile map, then theassociated commands in the rule’s Do section are sent to theobjects that they address. The commands can be addressed tothe center tile/sprite and to the four tiles/sprites adjacent to thecenter tile. Thus, there are five rows in the Do section. As theuser moves the cursor over these rows, the corresponding tile ishighlighted in the When section. A sprite witness is displayedat the beginning of a row. If there is no sprite witness, an emptycircle is shown instead. Figure 4 identifies two sprite witnesses,the center boulder and the boulder beneath the center boulder.The numbering of the five rows in the Do section and the fivetiles in the When section shows the correspondence.

To reiterate, commands are not immediately executed but sentto the addressed tile/sprite object, which maintains a com-mand log. Conflicting commands are resolved automatically,by removing commands from the log before execution. Thecommands that can be sent to a sprite are: move–conflictswith itself–resolve by choosing a move command at random;destroy–no conflicts; stop–conflicts with move command–overrides all move commands (can only be issued by collisionrules). The commands that can be sent to a tile are: paintbackground–conflicts with itself–resolve by choosing onepaint command at random; spawn sprite–no conflicts; opena portal to a random tile on the tile map that has the givenbackground and contains no sprite–no conflicts. Finally, thereis a command to disable (for one round) all the rules associatedwith a sprite.

CollisionsOnce all rules have fired and commands resolved, each spritehas a unique direction to move in (or no direction at all, if itreceived no move command in this round). This new set ofdirections is used to determine pending sprite collisions. TCgives the end-user the ability to determine what will happenfor each pair of sprites about to collide: destroy, stop motion.The evaluation section presents examples of collision rules.

Progress/Termination ConditionsMany game progress/termination conditions are existencerules, such as “the game ends when a boulder falls on theplayer” and are easily specified using a When predicate. Othersuch conditions require that a predicate holds for every mem-ber of a set: “the player goes to the next level when everydiamond has been collected from the game board”. For thesecases, we can often use the negation of When predicate: “theplayer goes to the next level when no diamond is on the

(a) (b) (c)

(d) (e) (f)Figure 5. (a) Selecting the sprite witness in the Do section brings up the direction predicate menu; (b) A rule that applies to both diamond and bouldersprites and shows the attribute menu for specifying a tile predicate; (c) Screen for generalizing a rule by flip and rotation operations; (d) selecting thetile to the right of the sprite witness brings up the command menu; (e) selecting the move icon from the command menu; (f) selecting the move downcommand. The command menu icons, from left to right, are: move, destroy sprite, paint tile, create sprite, block rule, portal, game win/over.

game board”. The evaluation section presents an exampleof a negated When predicate.

RULE EDITOR DESIGNDesigning the TC game creation environment so that gameprogramming is possible on a 160 × 120 pixel screen withonly A, B and dpad buttons is an interesting challenge. The tilemap editor and paint editor are straightforward adaptations ofstandard interfaces for creating tile maps and pixel art, so thissection focuses mainly on creating and editing of TC rules.

Our design was guided by the following two goals: (1) min-imize the amount of hidden user-modifiable state as muchas possible (though the size and complexity of a programwill fight against this goal, as will be seen); (2) enable visualprogramming via a painting metaphor that is similar to the cre-ation of tile maps and sprites. To meet these goals, we severelyrestricted the expressiveness of the programming model, asdetailed in the previous section. Here we focus on the userinterface to support the programming model.

Creating and Browsing RulesRules are identified by their event type and the kind of spriteat the center tile of the When neighborhood. The rule selectorscreen of Figure 3(e) was designed to give users an overviewof the space of rules. The user can choose among the foursprite kinds on the left; the four quadrants update to display therules available for that sprite kind. Each quadrant summarizesa related group of rules, parameterized by direction. As shownin the figure, the boulder has been selected: we can see in the

lower-left quadrant that there is a collision rule for the case ofthe boulder moving down, as this square is highlighted. If asquare is not highlighted, then there is no rule of that type forthe sprite. Clicking on a blank square will create a new rule ofthat type and enter the rule editor so the user can edit the rule.Clicking on a highlighted square will enter the rule editor onthe first instance of that rule type. Once in the rule editor, theuser can browse all the rules associated with the sprite kindwithout returning to the rule selector screen.

Editing RulesThe rule editor (see Figure 3(f)) has three main active areas be-sides the top menu bar: (1) the 3x3 neighborhood of the Whensection allows the user to define the Include/Include’/Excludesets of the tile predicates; the column immediately to theright of the Do keyword identifies the sprite witnesses andallows the user to set the direction predicates for them; (3) thesquares to the right of that column allow the user to define thecommands associated with the central tile/sprite and its fouradjacent tiles/sprites.

Editing and Visualizing Tile PredicatesSelecting a tile in the 3x3 neighborhood brings up a menu thatallows the user to attribute each background/sprite kind with agreen check mark (Include set), yellow dot (Include’ set), or ared-slash circle (Exclude set). Figure 5(b) shows the attributemenu for the central tile (where both diamond and boulderhave been added to the Include set).

(a) (b) (c)

(d) (e) (f)

Figure 6. Boulder Dash game: rules for (a) moving the player avatar, (b-c) pushing boulders, (d-e) handling collisions, and (f) game win condition.

A main issue is how to summarize the multi-faceted attributionfor a tile predicate in the 16x16 pixels of the tile. Our choiceis to “accentuate the positive”, as seen in Figure 3(f). Theattribution for the tile below the central boulder includes the“space” background and excludes the diamond and boulder.This is summarized by showing just the space backgroundin the tile below the central boulder (that is we prioritizevisualizing non-empty Include sets over non-empty Excludesets). The user can move the cursor over the tile to see theprecise attribution, as shown just above the When section. Ifa tile’s Include sets are empty but Exclude set is non-empty,we will then display the exclusion, adding the red-slash circleto make it clear (see Figure 6(a) for an example).

Direction PredicatesIf a tile predicate has identified a sprite witness, then we maywish to constrain the direction of that sprite. Selecting thesprite in the column to the right of the Do label, brings upthe direction predicate menu, as shown in Figure 5(a). Thecurrently selected direction predicate (resting) is selected andmay be changed by navigating to the desired new predicateand selecting it. Note that the direction predicate is reflectedin the tile predicate in the When section, as the predicate takespart in the pattern matching, as described previously.

CommandsWe finally come to the editing of commands in the five rows ofthe Do section, corresponding to the center tile and its four ad-jacent tiles in the When section. Figure 5(d)-(f) demonstratesthe steps needed to program the move down command (solidblue arrow pointing down). As shown in Figure 5(d), we startby selecting the tile to the right of the boulder (sprite witness)in the Do section, which brings up the command menu at thetop of the screen. We then select the leftmost menu item (theblue 4-way move icon), which brings up a sub-menu of fourmove commands, as shown in Figure 5(e), selecting the moveleft command. We move the cursor to select the move downcommand (Figure 5(f)). Selecting an existing command fromthe Do section allows one to change the command or delete it.

Base Derived TileGame Rules Rules Predicates Cmds.

Boulder Dash 14 15 29 16Snake 9 24 15 15

Bejeweled 11 14 33 38Table 1. Table summarizing the TileCode implementations of the threegames (see main text for details).

Generalizing Rules: Multiple Sprites and DirectionsThere are two main ways to generalize a rule in TC. The first isto have a rule apply to multiple kinds of sprites. For example,in Boulder Dash, diamonds fall just like boulders do (so far,we have shown the falling rules for just boulders). Figure 5(b)shows how we generalize a boulder falling rule to include thediamond: this is done simply by adding the diamond sprite tothe Include set of the center tile. The center tile now showshalf of each sprite (more than two sprites can be added to theInclude set, but the visualization shows at most two).

The second way to generalize a rule is by direction. As wehave seen, sprite directions are stated explicitly by the user,as shown in Figure 4. Often, when a user codes a behaviorfor a sprite to move in one direction (boulder tumbling to theleft), they will also want to code a behavior for the oppositedirection (boulder tumbling to the right).

As shown in Figure 5(c), TC provides a feature for derivingnew rules from existing rules: flip vertically/horizontally androtate clockwise/counter-clockwise (by 90 degrees). In thiscase, we have used the flip horizontal operation to create thedesired derived rule. The derived rules are represented asviews of the original rule, so if the user changes the originalrule, the changes are propagated to the derived rules. Theexistence of derived rules is shown in the rule editor by ayellow dot on the flip icon (to the left of the garbage can). Theparent-child relationships between rules created by this featurecannot be broken, but they can be reverted.

(a) (b) (c)

(d) (e) (f)

Figure 7. (a) Screen snapshot of the Snake game and (b)-(f) some TC rules of the game.

EVALUATION I: THREE POPULAR GAMESIn this section, we show that three popular video games canbe expressed in TC with a modicum of rules. We informallydescribe each game’s mechanics and show how it can be codedin TC. Table 1 summarizes the total number of rules, derivedrules (via flip/rotate operations), tile predicates and commandsfor each game. A number of other games, including Pac-Man,have been coded in TC and can be found at the URL given inthe Introduction.

Boulder DashThe Boulder Dash game mechanics are as follows: the userguides the player’s avatar (red elflike figure) through the map.The avatar (and all other sprites) cannot cross walls (blue tiles)or boulders; the player turns “dirt” (orange tiles) into “space”(dark tiles) upon moving onto them. The goal is to collect allthe diamonds in the map, while avoiding falling rocks (a rockis a boulder or a diamond). Rocks fall when there is spacebelow and can also tumble off rocks (to the left or right), aswe have already seen. The player can push a boulder to theleft or right when there is space available on the other side ofthe boulder.

We have already presented rules for expressing how bouldersfall. Figure 6 shows rules for moving the player avatar and itsinteraction with boulders and diamonds: (a) on a right-dpadbutton press, when there is no wall or boulder to the right ofthe avatar, it is instructed to move right (rule generalized tofour directions by rotation); (b) on a right-dpad button press,when there is a boulder to the right of the avatar, and no spacebelow the boulder, both the avatar and boulder are instructedto move right (rule also flipped horizontally);1 (c) is similarto (b) but has the boulder fall into the empty space below tomake room for the player to move to the right (rule also flipped

1This rule optimistically assumes that there is a space to the rightof the boulder; collision rules are used to determine if this is not thecase and to stop the movement of boulder and player.

horizontally); (d) is a collision rule for a player moving onto adiamond at rest, which results in the diamond being consumed(generalized to four directions by rotation); (e) is a collisionrule for a boulder falling onto a player, which results in thegame ending with a loss; (f) is a negation rule that checks forthe absence of diamonds on the board - in this case, the gameends with a win.

The full implementation of the mechanics described abovetakes a total of 14 rules, as show in Table 1.

SnakeThe mechanics of the Snake game (see Figure 7(a)) are asfollows: the snake is always in motion; the user guides thehead of the snake using the direction pad and each segment ofthe snake’s body follows the one in front of it (its predecessor).If the head of the snake hits a wall or the snake’s body, thenthe game ends. The effect of eating an apple is to increase thelength of the snake by one segment, as well as to spawn a newapple at a random position.

Let’s first describe the motion of the snake. We must start thesnake moving with no input from the user, so we choose tocreate the snake oriented from head at right to tail at left, asshown in Figure 7(a), and start the head and segments movingright from rest (this rule is simple and not shown). The dpadcontrols the direction of the snake’s head (rule not shown).

Making a snake segment follow its predecessor is an inter-esting challenge. Consider a segment that has just movedright into the center tile and the question: which tiles can itspredecessor occupy and which direction must the predecessorhave moved to now be in that tile? As shown in Figure 7(b)-(d), there are three basic cases and the move command forthe center (following) segment must be in the direction thatits predecessor last moved (all these rules are generalized to4-way by rotation).

(a) (b) (c)

(d) (e) (f)

Figure 8. (a) Screen snapshot of Bejeweled game and (b)-(f) some TC rules of the game.

Growing the snake is done using two rules. As shown inFigure 7(e), when the head of the snake collides with the apple,the apple sprite is destroyed (via the pac-man “eat” command)and an apple core sprite is spawned in its place. This rulealso invokes the at-sign “portal” command: this commandrandomly picks a tile location on the map with the givenbackground that has no sprite on it; the command followingthe at-sign command (a spawn of an apple) is executed atthat location.2 As seen in Figure 7(f), when the apple coreimmediately follows the end of the snake, the apple core spriteis destroyed and a new snake segment is spawned, moving inthe same direction as the end of the snake.

BejeweledBejeweled has the following mechanics: the user navigatesa cursor around a field of jewels arranged in a matrix (seeFigure 8(a)); the user selects a jewel (with the A button) andthen moves the cursor to an adjacent tile and presses the Abutton again to swap the previously selected jewel with thecurrent jewel. If the swap operation creates a sequence ofthree or more jewels of the same kind then that sequenceis eliminated, leaving empty space; otherwise, the swap isundone. Jewels fall into empty space. When jewels come torest, they may create new three-jewel sequences to eliminate.Once all jewels stop moving, the user may again select a pairof jewels to swap. We do not consider the mechanic of fillingempty space with new jewels here, due to lack of space.

Figure 8(b) shows the basic rule for removing sprites: if thereare three diamonds at rest in a row, they are all removed andthe backgrounds painted with the empty space (there is a newcommand shown in the top row - more on this later). Theparallel semantics of rule matching means that this singlerule can eliminate a row of diamonds of any length. The

2The portal command has many other uses: when there are just twotiles of a given background B in the tile map, the portal commandcan be used to send a sprite from one of the B tiles to the other B tile.

rule is generalized to a column via rotation. Rules to removesequences of apples and boulders must be coded separately.Making sprites fall into empty space is straightforward andhas been covered already.

The main challenge of Bejeweled is the implementation ofcursor motion, the swap operation, and its potential interactionwith the removal/falling of jewels. We use the orange-dirtbackground (dirt, for short) to mark the tile where the user firstpresses the A button (rule not shown). Pressing the A buttonwhen the cursor is on a dirt tile will mark the tile empty (alsonot shown), to permit the user to abort the swap and movefreely again. The cursor motion rules of Figure 8(c) and (d)will limit the cursor motion to the dirt tile and its four adjacenttiles.

Now, when the user moves the cursor to a tile adjacent tothe dirt tile and presses the A button, another dirt tile willbe set, denoting the pair of sprites to be swapped. The ruleof Figure 8(e) checks for a pair of sprites that both have dirtbackgrounds behind them (note the use of both the Includeand Include2 sets here). The sprites are swapped and theirbackgrounds are changed to blue (so rule (e) will not fireagain). The rule of Figure 8(f) matches on blue backgroundsbehind two resting sprites and swaps the sprites back whileresetting the background to empty space.

Once the sprites have been swapped the first time, the spriteremoval rules will fire. Recall that the removal rules, if suc-cessful, reset the backgrounds of the removed sprites to emptyspace. Thus, if no sprites are removed then the blue back-grounds will remain and the second swap rule will fire. Onthe other hand, if a removal rule does fire then two interest-ing things happen: at least one blue background and perhapsboth will be set to the empty space. We use an extra rule (notpresented here) to deal with the case when exactly one bluebackground is left. The second aspect of the removal rule(Figure 8(b)) is that it disables all the rules associated with the

Game Blocks Globals Arrays LoopsBoulder Dash 107 2 1 2

Snake 52 2 1 1Bejeweled 111 5 1 9

Table 2. Table summarizing MakeCode Arcade implementations.

yellow cursor for the next round. That is, as long as jewels arebeing removed, the cursor will remain stationary. Once thisremoval rule stops firing, then the cursor will be able to moveagain.

SummaryWe see from Table 1 that two to three handfuls of base rulessuffice to encode the basic mechanics of all three games. Howcomplex are these rules and how much reuse of the base rulesdo we get through generalization? The ratio of tile predicatesto base rules and of commands to base rules gives us a mea-sure of rule complexity, while the ratio of derived rules tobase rules gives us a measure of rule reuse: Boulder Dashhas, on average, 2.2 tile predicates, 1.2 commands and 1.1derived rules per base rule; Snake has 1.7 tile predicates, 1.7commands and 2.7 derived rules per base rule; Bejeweled has3 tile predicates, 3.5 commands and 1.3 derived rules per baserule. From this basic analysis, we can see that the Snake gamehas the simplest rules and most reuse of the three games.

EVALUATION II: BLOCKS-BASED VERSIONS OF GAMESThe paper’s last three authors, professional software develop-ers on the MakeCode Arcade team, each wrote a version ofone of the three games (Boulder Dash, Snake, Bejeweled) us-ing the Blocky editor of MakeCode Arcade. The input to thisprocess was the informal game mechanics description createdby the first author, though all games were already well-knownby all; the TC versions of the games also were available toplay. The goal was to implement the same basic mechanics ofthese games using the standard blocks and APIs available inMakeCode Arcade. Each co-author implemented his/her gamein isolation (not difficult in these times). Table 2 containsbasic metrics about the games. For each block-based game,we discuss how it implements the basic game mechanics andcompare to the approach taken in TC. Links to the games’code can be found from the URL given in the Introduction.

Boulder DashUnsurprisingly, the movement of boulders in Boulder Dashis the central mechanic of the game. The blocks-based ver-sion uses a recursive algorithm to compute the next state foreach rock, with user-defined functions to determine if rocksshould fall or tumble. In this game especially, the TC conceptsof a round and internal command conflict resolution allowsthe state to synchronize separately from the game engine,which helps to resolve inherent conflicts between differentrules before rendering the next frame. That is, there is a strictseparation of model-view-controller (MVC) enforced in TC.In MakeCode Arcade, once a sprite is rendered to the screen,it’s visible to the engine and treated as a valid state, which putsthe onus on the programmer to maintain the MVC abstraction.

SnakeAs discussed before, the main complication of the Snake gameis to have the snake’s body segments follow their predecessors.The block-based implementation represents the snake as anarray, stored in a global variable, and loops over the array tochange the direction of each segment to that of its predecessor.In TC, we used the direction each segment last moved toidentify predecessor segments using three rules and patternmatching. The looping over the segments happens implicitlyin TC and there is no need for a separate array.

It is worth noting that without TC’s direction predicate, itwould be impossible to identify the predecessor of each spritebody segment; since the snake can double back on itself, agiven sprite can have more than one adjacent tile containinganother body segment—in such situations, the direction isneeded to distinguish which of the adjacent tiles contains thepredecessor segment. In an imperative language with arrays,this problem does not arise, as the ordering of sprites in thearray establishes the predecessor relation.

BejeweledIn the TC implementation of Bejeweled, there is one rule tosay “if three consecutive items match, eliminate them”, aboutthe most declarative statement of the mechanic possible. Inthe MakeCode Arcade version, this mechanic is implementedby using a sliding three-wide window that traverses every rowand column of sprites and checks each triple of jewels forsameness. This is done using nested loops over a 2D array ofsprites, with three local variables indexing into the array totrack the current triple.

The TC has a runtime optimization that keeps track of whichtiles/sprites changed in the previous round and only checks arule on a sprite when its neighborhood has changed in the lastround. This means that not every sprite has to be examined, asin the MakeCode Arcade version.

SummaryIn all three games, we see that the pattern matching capabilityin TC eliminates the need for explicit use of loops and arrays,compared to the block-based implementations. While TCgenerally has less code, some of the TC rules take quite a bitof thinking to come up with (for example, changing the tilebackgrounds to keep track of swap state in Bejeweled). On theother hand, such state seems to be required to implement themechanics correctly, regardless of the programming paradigmchosen.

LIMITATIONSThe TC programming model was designed so that: (1) gamescould be created on a gaming handheld with a small screenand limited input affordances; (2) a variety of popular gamescould be created. This section describes the major limitationsof the programming model and how this affects the kinds ofgames that can be created with the model.

Local State/Compute/BehaviorBy design, the only state values in TC are the tile backgroundand anonymous sprite objects (with a kind and direction), each

having a visual representation. Furthermore, as in cellularautomata, predicates/commands may only access a finite 3x3neighborhood relative to the center sprite of a rule, so thatinstantaneous state access at an arbitrary distance (as enabledby random access memory) is not possible.

This restriction makes certain mechanics impossible to achieve.For example, in Pac-Man, when the pac-man avatar eats apower-up all the ghosts immediately turn dark blue and runaway from the pac-man, reverting to their original form aftersome period of time. This mechanic is not possible in TC cur-rently. One realization would set a global countdown counterto a non-zero value upon a power-up and count down to zero.When the counter is non-zero, the ghosts would exhibit thefleeing behavior [15]. This would require an extension to TCto support global counters.

It is also difficult to program ghosts to move towards (or awayfrom) pac-man, as there is no mechanism for comparing thelocations of different sprites in TC.

No Hidden ComputationIn TC, every command has a visible effect, whereas in generalprogramming many commands have no visible effect. Forcertain game mechanics, it would be useful to perform an(invisible/hidden) computation over the tile map to decidewhether or not to perform some visible action. A good exam-ple is the mechanic of clearing a complete row/line in Tetris: ifthe insertion of a Tetris piece completes a line then that line isremoved. From the user’s view, the determination of whethera line is complete happens instantaneously (though the actualremoval of a complete line may take some time). Such an(apparently) instananeous and invisible computation currentlyis not possible in TC.

Synchronous Grid-based MotionIn TC, sprites may only move in one of four directions; further-more, all sprites move synchronously and at the same velocity.This severely limits the kinds of mechanics possible, such asmissiles that move faster than the player/enemy sprites andparabolic motion, such as found in Angry Birds.

CONCLUSIONWe have designed and developed a novel game programmingmodel, TileCode, for low-cost gaming handhelds that enablesvisual programming on small screens, yet is expressive enoughto program a number of different popular video games. Itsability to pattern match on tiles, sprites, and sprite directionsallows for a surprising amount of expressivity. TileCode’sprogramming constructs are made available via an editingparadigm that is also tiled-based.

Users studies are clearly needed to evaluate TileCode. Wewill begin by seeing if users can successfully make smallmodifications to existing games, after a brief demonstrationand walkthrough. We will follow-up with a set of interestingchallenge problems and evaluate the solutions that novicesproduce.

ACKNOWLEDGEMENTSThanks to the MakeCode team, Peli de Halleux, StefaniaDruga, Teddy Seyed, and Kimberly Ying for their input.

REFERENCES[1] PICO-8 Fantasy Console. In

https://www.lexaloffle.com/pico-8.php. Accessed July2, 2020.

[2] Thomas Ball, Peli de Halleux, and Michal Moskal. 2019.Static TypeScript: an implementation of a staticcompiler for the TypeScript language. In 16th ACMSIGPLAN International Conference on ManagedProgramming Languages and Runtimes. ACM,105–116.

[3] Rahul Banerjee, Jason Yip, Kung Jin Lee, and ZoranPopovic. 2016. Empowering Children To RapidlyAuthor Games and Animations Without Writing Code.In 15th International Conference on Interaction Designand Children (IDC ’16). 230–237.

[4] Stephen Cooper, Wanda Dann, and Randy Pausch. 2003.Teaching objects-first in introductory computer science.In 34th SIGCSE Technical Symposium on ComputerScience Education. ACM, 191–195.

[5] Allen Cypher and David Canfield Smith. 1995. KidSim:End User Programming of Simulations. In HumanFactors in Computing Systems, CHI.ACM/Addison-Wesley, 27–34.

[6] James Devine, Joe Finney, Peli de Halleux, MichalMoskal, Thomas Ball, and Steve Hodges. 2019.MakeCode and CODAL: Intuitive and efficientembedded systems programming for education. Journalof Systems Architecture 98 (2019), 468–483.

[7] Edsger W. Dijkstra. 1975. Guarded Commands,Nondeterminacy and Formal Derivation of Programs.Commun. ACM 18, 8 (1975), 453–457.

[8] Jay Fenton and Kent L. Beck. 1989. Playground: AnObject-Oriented Simulation System With Agent Rulesfor Children of All Ages. In Conference onObject-Oriented Programming: Systems, Languages,and Applications. ACM, 123–137.

[9] Neil Fraser. 2015. Ten Things We’ve Learned fromBlockly. In 2015 IEEE Blocks and Beyond Workshop(Blocks and Beyond). 49–50.

[10] Werner Hartmann, Jürg Nievergelt, and RaimondReichert. 2001. Kara, finite state machines, and the casefor programming as part of general education. In IEEECS International Symposium on Human-CentricComputing Languages and Environments. IEEEComputer Society, 135–141.

[11] Kenneth M. Kahn. 1995. ToonTalk - ConcurrentConstraint Programming for Kids. In TwelfthInternational Conference on Logic Programming. MITPress.

[12] Caitlin Kelleher, Randy F. Pausch, and Sara B. Kiesler.2007. Storytelling Alice motivates middle school girls tolearn computer programming. In Conference on HumanFactors in Computing Systems, CHI. ACM, 1455–1464.

[13] Isabelle Lefebvre. 2017. Creating with (Un)LimitedPossibilities: Normative Interfaces and Discourses inSuper Mario Maker. The Journal of Canadian GameStudies Association 16 (2017), 196 – 213.

[14] Jiangjiang Liu, Cheng-Hsien Lin, Joshua Wilson, DavidHemmenway, Ethan Philip Hasson, Zebulun DavidBarnett, and Yingbo Xu. 2014. Making games a "snap"with Stencyl: a summer computing workshop for K-12teachers. In The 45th ACM Technical Symposium onComputer Science Education, (SIGCSE). ACM,169–174.

[15] Richard G. McDaniel and Brad A. Myers. 1999. GettingMore Out of Programming-by-Demonstration. InConference on Human Factors in Computing Systems,CHI. ACM, 442–449.

[16] Radu Muschevici, Alex Potanin, Ewan D. Tempero, andJames Noble. 2008. Multiple dispatch in practice. In23rd Annual ACM SIGPLAN Conference onObject-Oriented Programming, Systems, Languages,and Applications. ACM, 563–582.

[17] Thorbjørn S. Nielsen, Gabriella A. B. Barros, JulianTogelius, and Mark J. Nelson. 2015. Towards generatingarcade game rules with VGDL. In IEEE Conference onComputational Intelligence and Games. IEEE, 185–192.

[18] Seymour Papert. 1980. Mindstorms. Children,Computers, and Powerful Ideas. Basic Books, NY.

[19] Richard E. Pattis. 1981. Karel the Robot – A gentleintroduction to the art of programming. Wiley, NewYork.

[20] Alexander Repenning and Tamara Sumner. 1992. UsingAgentsheets to create a voice dialog design environment.In ACM/SIGAPP Symposium on Applied Computing.ACM, 1199–1207.

[21] Alexander Repenning and Tamara Sumner. 1995.Agentsheets: A Medium for Creating Domain-OrientedLanguages. IEEE Computer 3 (1995), 17–25.

[22] Alexander Repenning, David C. Webb, Catharine Brand,Fred Gluck, Ryan Grover, Susan B. Miller, Hilarie

Nickerson, and Muyang Song. 2014. Beyond Minecraft:Facilitating Computational Thinking through Modelingand Programming in 3D. IEEE Computer Graphics andApplications 34, 3 (2014), 68–71.

[23] Mitchel Resnick, John Maloney, AndrésMonroy-Hernández, Natalie Rusk, Evelyn Eastmond,Karen Brennan, Amon Millner, Eric Rosenbaum, Jay S.Silver, Brian Silverman, and Yasmin B. Kafai. 2009.Scratch: programming for all. Commun. ACM 52, 11(2009), 60–67.

[24] Tom Schaul. 2013. A video game description languagefor model-based or interactive learning. In IEEEConference on Computational Inteligence in Games.IEEE, 1–8.

[25] Joel L. Schiff. 2008. Cellular Automata: A DiscreteView of the World. Wiley-Interscience.

[26] Teddy Seyed, Peli de Halleux, Michal Moskal, JamesDevine, Joe Finney, Steve Hodges, and Thomas Ball.2019. MakerArcade: Using Gaming and PhysicalComputing for Playful Making, Learning, and Creativity.In Extended Abstracts of the Conference on HumanFactors in Computing Systems, CHI. ACM.

[27] David Canfield Smith, Allen Cypher, and James C.Spohrer. 1994. KidSim: Programming Agents Without aProgramming Language. Commun. ACM 37, 7 (1994),54–67.

[28] David Canfield Smith, Allen Cypher, and Lawrence G.Tesler. 2000. Novice Programming Comes of Age.Commun. ACM 43, 3 (2000), 75–81.

[29] Kathryn T. Stolee and Teale Fristoe. 2011. Expressingcomputer science concepts through Kodu game lab. In42nd ACM Technical Symposium on Computer scienceEducation (SIGCSE). 99–104.

[30] Rokas Volkovas, Michael Fairbank, John R. Woodward,and Simon M. Lucas. 2019. Mek: MechanicsPrototyping Tool for 2D Tile-Based Turn-BasedDeterministic Games. In IEEE Conference on Games.IEEE, 1–8.