The challenges with prototyping for TV, and how we solved it.

As designers, our task is to facilitate solutions. When designing, prototyping and testing big screen apps, there are unique challenges, benefits, and opportunities to be highlighted and tested. We couldn’t find what we needed in the current prototyping tools… so we made our own.

Digital problems

With web and native apps, one can usually display the prototypes in their natural habitat of a laptop, phone or tablet. The experience of prototyping TV apps is much more disjointed as the screen and the remote are entirely separate entities

The standard method is to display the remote control and the app on the same screen, but this gives a misleading representation.


   A prototype of remote & app side by side © Common Ground AB
A prototype of remote & app side by side © Common Ground AB

During tests, we often saw users trying to click directly on the app interface, rather than using the remote control. It’s sometimes difficult to understand that the journey between point A and point B might appear small, but it requires a sequence of 5 button-presses to get there.


   Image courtesy of Google ©
Image courtesy of Google ©

In addition, user-test subjects should be sat 3–5 meters away from the display to ensure a realistic set of circumstances.


   Image courtesy of Google ©
Image courtesy of Google ©

Physical problems

We were also tasked with designing a remote control to accompany the TV app. Developing and manufacturing a physical remote control represents a whole other set of challenges. It’s a time-consuming and hugely costly process, so it really needs to be perfect the first time around. It, therefore, makes sense to extensively test and prototype the digital interface and the physical peripheral together , allowing you to quickly iterate on both.

So what’s the best way to mock up a remote control which can be easily iterated upon? Tactile feedback is important. Touching and handling a physical object with some weight. Maybe having to point it in the right direction? Getting the feedback of a button-push.

Creating our own remote control from a banana and a makeymakey kit was one of many suggestions floated in the room, and one we’ll probably return to soon.

The solution we chose was simple and we had a fully functional proof of concept in just over two working days.

We made a website

Just like other prototyping solutions, we had the TV app and the remote control rendered side by side on each and every html page. Flows were mapped out in Sketch artboards.


   © Common Ground AB
© Common Ground AB

With some simple CSS responsive breakpoints, we set the big-screen content to be hidden on mobile, and the remote control to be hidden on the big screen.


   © Common Ground AB
© Common Ground AB

The remote section

Inside the mobile breakpoint, a full width & height div uses the remote control as a background image. This serves as our navigation. Pseudo-elements were created and placed absolute over the remote control’s buttons. Each of these pseudo-elements were then assigned an anchor tag.

Some of these buttons went on to become constants . Home , for example, will always go to the home page, profile will go to the profile page etc.

But most of the buttons will be contextual. The directional arrows, ‘ ok ’ & ‘ return’ will all go to different screens relative to where you in the flow.


   © Common Ground AB
© Common Ground AB

On the big screen

There’s not really too much to say other than; this could be anything. Static images, gifs & movie files, modals, pop-ups, overlays, menus… if it’s an existing web technology, we can put it into this prototype.

Sticking point

But, so what, right? How can clicking through a website on a phone affect what’s happening on the TV?

This is how

Our iPhone and mac are on the same wireless network. The mac is screen-mirroring via HDMI to the TV. Nothing new here.


   © Common Ground AB
© Common Ground AB

The magic element that brings it all together is Codekit app . It’s a websocket tool aimed at front-end designers and developers. One of its greatest features, in my opinion, is the live preview in-browser. Changes to the code and styling are updated immediately to all browsers using the unique bonjour url. Furthermore, if you click a link to change the page on one device; all the others will follow.


   © Common Ground AB
© Common Ground AB

Now we can use the remote display on the phone to click through the prototype’s pages on the TV.

© Common Ground AB

The possibilities are almost endless. The only limitation is how much time we want to put into a prototype. It might be tempting to strive for something so advanced, we’re essentially recreating the product itself.

To summarise version 1.0

Good

  • It ticked all the boxes for the problem we were trying to solve with the resources available
  • We made the whole thing in about 20 hours
  • It’s a highly scalable solution

Bad

  • The larger the prototype gets, the harder it is to develop and maintain (as it is with other solutions). It’s easy to lose track of where everything is going, especially with non-linear flows and slap-dash naming discipline
  • A lack of GUI means some basic front-end skills are required
  • It’s dependent on Wi-Fi and the Apple ecosystem
  • The changing of the page causes a flash every time

Next steps

  • Rebuild the platform with a wysiwyg/CMS/GUI
  • Test conductive stickers on the phone to mimic physical buttons
  • Investigate if we can utilise the sketch prototyping Output data
  • Investigate if React/Vue could fix the flashing problem
  • Test hardware integration such as makeymakey