The key to effective multi modal design is?

1. Antennae Sculpture 2. My shadow – a radio frequency 3. Studio environment housing equipment.
From “Frequency and Volume” an installation by Rafael Lozano Hemmer on display at SFMOMA, December 2012

My first project on joining [24]7 Inc was to design a multi mobile consumer application. What made it multi modal was the use of the speech and mobile web browser used together on a smartphone.

Creating this multi modal application provoked rich collaborations across parts of the company that had been recently acquired and were new to one another. The combination of late nights, adrenalin and frank conversations sparked an energy vortex that was hair raising, heart pumping and the very essence of Silicon Valley startup life at its finest.
As I tried to write up my notes about the affordances our design had created to share with our sales team, I was horrified to discover that the passion was not coming through on paper. Was it easier to design than to write about it? Was I too close to reflect on the enormity of what we had just accomplished? My writing fell flat where spoken word inspired confidence.

“The key to effective multi-modal design, I wrote, is to ensure each mode plays to its own strength. When combining modalities of speech and screen together, the screen is best used for displaying visually intense or data rich information, for example, choosing a date from a calendar. The voice channel, when employing continuous interactive speech is good for taking short cuts, for example, speaking a bank account number, than typing it using a mobile keypad… “

Unimpressed with my write up of our design, the team asked for a re-write, but nothing came to me. Seeking inspiration for my writers block, I went to the San Francisco Museum of Modern Art and found myself experiencing an alternative multi modal design experience.

Rafael Lozano Hemmer built a participatory installation that makes radio frequencies visible in the gallery. The installation is made up of a large antenna sculpture outside on the roof; a gallery, where visitors interact with a vast array of private and public radio signals and a studio environment housing technical equipment that works on its own.

In the gallery each persons shadow is detected by a computerized tracking system and their unique position and outline determines which frequency is scanned. Visitors use their bodies as antennae and tune into each radio station.  Moving your body left and right lets you tune into different radio frequencies. With people walking around the gallery together, each affecting a different signal, a cacophony of radios channels is heard. A voice in a sea of voices. Chaotic but not discordant.

Changing the volume involves walking back and forth making the silhouette of your body on the gallery wall change from big to small – loud to soft. The playful engagement, easy discovery and group dynamic put a huge smile on my face.

The installation, “Frequency and Volume” is one of several projects Rafael Lozano Hemmer has worked on since 1997. His participatory installations examine ways “the physical and public environment can be transformed into a data space in which intangible information is mapped onto a physical location.” This piece is a response to the Mexican governments shutting down of unregulated stations in indigenous communities in the states of Chiapas and Guerrero in 2003.

The key to effective multi-modal design, I can now rewrite, extends far beyond describing modalities and what they are good for. The key to effective multi modal design is to make the user smile. A smile born from the joy of playful discovery. A discovery that comes from participating with technology in community where one can be seen and heard and feel safe to do both.

Guidepad: design vision for a mobile travel application

Guidpad mobile application screen design

Mobile screens for travel application Guidepad

Founder Labs is a five week program focused on the first phase of launching start ups in the mobile space. In January 2011 I  participated in the program and collaborated with an engineer and product business lead in the design of a mobile application Guidepad.

Challenge: In only five weeks, the team met, determined a problem space and customer, prototyped a solution, sought customer insights and presented the findings and results to an audience of investors.

Response: Guidepad sought to reduce the stress of traveling for affluent travelers visiting destination cities in the USA. Available from concierges at high end hotels, Guidepad offered travelers a customized mobile phone with essential travel services, internet and phone. Above are screen shots from a use case scenario I designed for supporting a traveler safely back to their hotel in a city they may not be familiar with.

Response: To learn more about the my lean start up mobile work at Founders Lab, please read my blog posts that track the experience week by week.

Role: Designer

Year: 2011

ReQall: design vision for a mobile personal assistant

ReQall images from workshop

Scenes from the Reqall design vision workshops

Engaged by ReQall, a 4 year old mobile startup, to infuse the spirit of human centered design into defining the next generation of mobile personal assistant

Challenge: I set up and led a series of workshops with the entire international mobile team exploring the creation process for the next generation of service, informed by customer and team insights. The vision process included some of the following team activities which I led.

  • Sharing stories from the team and from customer insights, helped the team frame the context for design. The stories were about real people and their lives and served as inspiration opportunities, ideas and solutions.
  • Identifying patterns and themes arising from the stories, revealed overarching truths. The truths let us see the design challenge in a new light
  • Creating opportunity areas meant spending time on each pattern or theme and asking “How might we…” It was a stepping stone to idea generation
  • Brainstorming new solutions took the themes to the next level and were illustrated simply with words and pictures often on bits of card or the whiteboard. To keep the brainstorms real, we kept asking; How might we? Who will benefit from the idea? What is the value? How will payment be collected?


  • Generated opportunities, solutions and storyboards to inform prototypes for testing and feedback
  • United the team around a shared context for design of the next set of services
  • Fostered a generative mindset within the team
  • Led to solutions informed by genuine needs and connections to deep thoughts and feelings

Testimonial: “We brought in Sally to bring cohesion to our design process. In particular, Sally facilitated the team’s participation in the process by infusing human centered and emotional aspects into design – an area in which she excels. Her contributions are much appreciated by one and all on our team.”

Rao Machiraju, CEO, ReQall

Role: Workshop leader and facilitator

Year: 2010

Yahoo! Connected Life Design Vision

Yahoo! Connected Life Design Vision

Yahoo! Concept Diagram for the Connected Life Consumer Experience

Connected Life brought together three digital components of a typical consumers life – their phone, their television, and their computer. As creative director I led the user experience design teams to deliver the connected life customer experience extending Yahoo’s brand reach to become as ubiquitous as possible.

Challenge: Connected life consisted of Yahoo Go Mobile (aimed at mobile phones), Yahoo Go TV (for televisions), and Yahoo Go Desktop (for computers). With a Yahoo ID, users could connect the whole Internet to their mobile phone and Internet-connected TVs, as well as personal computers without using a browser.

Response: As head of user experience design for Connected Life, I built a world-class team of design experts to develop engaging personal and community-aware products in TV, desktop, broadband and mobile platforms.

The broadband and mobile teams had established products and revenue streams and could provide resources to develop the Yahoo! Go experience. The TV and desktop platforms were unproven businesses but needed a design vision which could demonstrate the value of the business.

Results: Using my experience in R&D and waterfall product development practice, I was able to build a multi disciplinary team of user experience designers capable of both pioneering new user experiences for TV and Desktop and evolving the existing businesses in broadband and mobile.

The investment in design paid off. In 2006 Yahoo! Go was unveiled to good reviews at the Consumer Electronics Show (CES)

Today, much of the Yahoo! Go vision has been released from service. However, the legacy of the early ground breaking work lives on through Yahoo Connected TV,  and Yahoo! Mobile.

Role: Senior Director, Product Design

Year: 2006

Yahoo! Go Mobile Design Vision

Yahoo! Go Mobile Design Vision

Yahoo! Go Mobile Design Vision

Yahoo has made its content available on mobile phones since the first wireless enable cellphones came to market in the late 90‘s. In 2005 Yahoo! developed Go Mobile, a Java-based client application, the crown jewel of mobile offerings.

Challenge: Under the design direction of Pamela Mead, several levels of mobile service were developed allowing users with different types of phone to have an experience of Yahoo! on their phone, whatever the make or carrier.

Response: The most distinctive service, Yahoo! Go Mobile featured a “game changing” interaction design with carousel navigation for browsing Yahoo! content.

Result: The service allowed users to get all their Yahoo! information and content services in one place. Yahoo! Go Mobile seen above, was deployed as beta software and performed well until the Go platform underpinning the service was closed in December 2009.

Role: Senior Director, Product Design

Year: 2006

Yahoo! Connected TV Design Vision

Yahoo! Go TV Design Vision

Yahoo! Connected TV Initial Design Concepts

Yahoo! Go TV, or Connected TV as it is called today, explored the opportunity to transport the Yahoo! user experience from the desktop to the living room. It meant building a design team who could develop a 10 foot user experience.

Challenge: The challenge of Yahoo! Go TV was to create a team of designers who could develop a go to market solution and a design vision that could attract new partners to collaborate and invest in an emerging business.

Response: Dr. Mike Mills led the design team who prototyped a series of short and long-range concepts. A short term go to market solution made entertainment services available on any PC-connected TV through a small downloadable application on the PC. The services included local and video search, movie trailers, movie times, TV shows, user ratings and weather, sports scores, stock data and news from My Yahoo.

Long term design visions were created as working prototypes to show how Yahoo! personal content could be integrated with live broadcast. Single point solutions called ‘widgets’ were developed for delivering the latest sport or news highlights. A flexible interface for navigating and filtering personal media content for photos and music with10 levels of zooming was also built.

Results: The go to market solution was demonstrated at the CES electronics show in 2006 to good reviews. The long range design prototypes were shown to business partners and succeeded in creating opportunities for new joint ventures.

Since August 2010, Yahoo! Connected TV has deployed widgets for TV’s providing customers with the latest updates on information from across the Yahoo! network along with Movies and TV shows on demand.

Role: Senior Director, Product Design

Year: 2006

Yahoo! Sports Design Improvement

Yahoo! Sport Website Design Improvement

Yahoo! Sports redesign 2003 before and after

Evolving the popular sports website for Yahoo! was a huge challenge. With a very small team and resources compared to rival ESPN, becoming the leading online Sports editorial service and leader in fantasy sports and broadband programming for both users and advertisers was a tough task.

Challenge: Brian Grey, then head of Yahoo! Sports, requested a redesign that would compete head to head with rivals such as ESPN, Fox and Sportsline. The challenge was tough.

First, Making the site compete on content would be a challenge, because Yahoo! Sport creates less original sport content than media companies such as ESPN or proprietary right owners like the NFL and MLB.

Second, to attract fortune 100 advertisers to invest more heavily, the site needed to feel more sporty and less like a list of sport statistics and news feeds. This meant adding new content sources and new team members including user experience design.

Lastly, a new navigation experience was needed to accommodate more sports, enabling greater content and business to be developed in each vertical. Navigation had to be flexible to accommodate seasonal variations in sports, whilst remaining easy to use so repeat visitors could easily find sport results on a daily basis.

Results: The sports team had very clear goals and objectives, but no dedicated design resources to help achieve them. I argued the business case for investing in two dedicated heads (visual and interaction design disciplines) and negotiated borrowed headcount from user research and web development. With a world class design team hard at work, the site went through the full user centered design process. Within a year the news site was launched.

Results following the launch of the redesign were spectacular. The site moved from #4 to #2 on the internet behind market leader ESPN. Unique monthly visitors increased by 52%.

Three new sponsors, Pepsi, Ford, and KFC, invested heavily in the newly designed sponsorship templates for major league sports. Companies such as Ace Hardware, Nike, Cingular, Planters, and Tums followed suit, bringing new advertising to the service.

Our users were impressed, too. The amount of time spent on each page was greater than on competitors ESPN, Sportsline, and AOL Sports. The look and feel of the service set a new standard for online sport media that has been very hard to beat but often imitated.

Role Design Director

Year 2003

Yahoo! News Website Design Improvement

Yahoo! News Website Design Improvement

Yahoo News Design Improvement before and after images

Yahoo! News requested a redesign of the their popular website. What started out as a dynamically generated set of text based news feeds was transformed into a customer centric, advertising friendly #1 news internet site on the web.

Challenge: Yahoo! News wanted to be the best and most relevant online news service on the Internet. Its goal was to create a compelling and customizable new view for users; build deeper feature and content packaging to enhance contextual advertising opportunities; and drive traffic from the site, deeper into other parts of the Yahoo! network.

Response: Making over the site involved introducing new skills to the News department who were reluctant to let go of the way things worked at first.

With the introduction of a six-month research phase consisting of focus groups, usability studies, segmentation analysis, field research, and advertising studies, I was able to bring the news team to a shared understanding of the scope and scale of what we needed to do from a design perspective to help them achieve their goals. Resources were approved and I built a world class team of interdisciplinary designers to transform the site.

Findings from the research phase informed the design of a customized digital news experience targeted at specific customer segments. Users were offered up-to-the minute, comprehensive, contextual news with features informed by consumer insights. Pages were easier to navigate and were laid out to be clearer to read.

For advertisers, the new site offered integrated and more valuable ad units. The design enabled sponsors to underwrite parts of the site by providing templates that they could “plug and play” their content and brand into.

Results: Yahoo! News moved to #1 news site category on the web according to Neilson Netratings enabling it to reach its goal of becoming the best and most relevant online news service on the Internet of its time.

Unique visitors to the News site increased five times within a year of launch. With new visitors and greater traffic to the site, advertising revenue increased and more people were drawn into the Yahoo! brand experience.

Testimonial from Kourosh Karimkhany, Senior Producer of Yahoo! News

“I met Sally when she was running one of Yahoo’s design groups and I was running the day-to-day operations of Yahoo News. My product had been performing well, but was not well respected and was not growing fast enough in comparison to our competitors. Sally was assigned to my redesign project – which turned out to be one of the most fortuitous turns in my career. She led a team that essentially invented the current form of Yahoo News – the largest news site in the world.

She came up with very clever solutions to devilish problems – like how to display a lot of information on one page without overwhelming the reader, and how to present the same story from several news organizations. These were radical ideas that Google News aped a couple years later. 

She was a consummate pro as well.

My team had several strong personalities (yours truly included), but she managed to keep egos and tempers in check. What’s more, even though different members gave wildly different suggestions, Sally managed to sprinkle just the right pinch of each of our demands into the final product. 

The result: Yahoo News’ traffic exploded. A year after the redesign traffic was five times higher.

I would LOVE to work with her again. While I like to brag about Yahoo News’ explosive growth as one of the highlights on my career, Sally truly deserves more of the credit.

Role: Design Director

Year: 2004

Yahoo! Movies Design Improvement

Yahoo! Movies Website Design Imrpovement

Yahoo! Movies Design Improvement

The web as a medium for promoting film had yet to prove its worth in Hollywood in 2002 at the time of the sites redesign. By making the presentation layer of Yahoo! Movies more familiar to often-conservative, middle-aged studio executives, Yahoo! was able to capture a major investment from the movie industry.

Challenge: Yahoo! movies was a popular site with users on a Thursday evening when people were looking for a film to see on the weekend. It did rather less well during the week. In the eyes of potential advertisers in Hollywood, it did little to entice them to spend advertising dollars. The challenge was two fold, create a site to attract advertisers and keep users coming back every day of the week to the website.

Response: For the site to be appealing to the movie crowd, it had to offer movie sponsorship opportunities and integrate broadband content, editorial reviews and, most important, the promise to sell movie tickets over the web.

Getting the right balance between content aggregation, movie promotion, ads, and editorial content was a new design challenge. By offering fewer larger ad positions at the top of the page, producers were excited to invest. By rotating the content on the home page according to the ebb and flow of the movie industries production and promotion cycle we were able to attract users back on a daily basis.

For example, traffic to the site peaked on Friday, when users wanted to find a movie and show times. Promotions of weekend releases drove users to the theaters for opening day. Monday’s home page featured the box office highs and lows over the weekend. Understanding the movie studios’ timing, established Yahoo! Movies as a part of the entertainment industry.

Results: Yahoo! Movies became the #1 entertainment site on the web following the launch of the redesign. Within a year of the launch, unique monthly visitors increased by 34%.

The Movies team relocated to Santa Monica and quickly became enmeshed in the glamor of Hollywood. The movie industry took this commitment seriously, and their influence on the team’s thinking benefited both industries.

Role: Design Manager.

Year: 2002

Excite Mobile Application Design

Excite Mobile Application Design

Excite Mobile Application design

By 2000, mobile phones with Internet capabilities were coming to market rapidly. As part of its business strategy to extend customer reach and gain market share in the emerging mobile Internet market.

Challenge: As the mobile application designer, my task was to transform existing internet applications like email, news, entertainment and weather onto a limited-screen-space mobile phone experience.

Response: At the time, images on mobile internet phones were black and white and with screen sizes that could be as small as three lines of text. The work was highly creative despite the design constraints imposed by the hardware and software limitations.

Results: Of the many applications I designed, developing the first customizable internet enabled mobile phone home page in the U.S. allowed users to choose the mobile applications they wanted to use. The innovation gave Excite leverage to compete head-to-head with Yahoo! for placement of WAP services on the highly valued mobile home pages of carriers such as Sprint.

For Excite customers, the service provided increased freedom from their PC, now that applications were available anytime, anywhere and on any computer or mobile Internet device. The Excite, the service extended the brand to more customers, some of whom were accessing the Internet for the first time from their cell phones.

Role: Designer

Year: 2000