A few days later I produced a second video -- suggesting a few added features:

1. • The ability to have chapter markers in a story -- I wish podcasts had that today!
   

2. • Content streamed from a content provider (e.g. CNN) based on user specified “interests”
   

3. • Traffic info customized for the car’s location and direction
   

4. • The ability to forward a program to another person
   

5. • The ability to record a note that's tagged with this story for later access
   

Demo #2

Our concept of programs recognized that new episodes would be produced over time, and that some kinds of content, like news, would replace earlier episodes. But other kinds of programs, like Freshair, would just be an additional episode you could listen to. We tried using the Skip Forward and Skip Backward buttons to navigate between different episodes, but decided that jumping between chapters of the episode was more interesting. We never did build a mechanism to jump between episodes that we were happy with.

At this point in the project I was still thinking that all content would be streamed to the car. But one day, one of the other team members (Behfar Razavi) suggested we make use of the hotspots we were building around campus to deliver and cache non-time critical information in large chucks. Content like Fresh Air. Content like breaking news or traffic alerts would still be streamed to the car. That's what we implemented. But, we still thought streaming was sexier, so we didn't emphasize caching when discussing the work.

Why Were We Doing This?

Remember that this was during the dot com boom. No one was sure where the next “killer app” was and so they were pouring money into a lot of different possibilities. The car companies were the same. The internet looked like a good way to get people to trade in their old cars for new ones with fancy new features. The car companies were also interested in becoming service providers and deliver lots of content over the wireless internet to cars -- for a monthly fee. (It’s surprising how little profit most cars generate when sold!)

This would also allow the car companies to maintain a relationship with the owner after the car was sold. And, Sun saw this as a way to deliver more services -- which would need our servers to run on.

I had been working with another Sun Microsystems team that was building an intranet portal server. MyYahoo and MyNetscape had just come out and allowed users to decide what info they wanted to see on their Yahoo or Netscape pages. Corporations were also looking for this capability -- to build MySun, or MyCisco, etc.

Our development team had begun to deliver the iPlanet Portal Server, but they wanted to know what else it could be used for. Someone on the team knew I was good at inventing things, and I was senior enough that they could give me general direction and let me run.

My first investigation was whether we could connect the first internet PDA, the Palm VII, to our portal server. It soon became clear that although it was a great product, it didn't have sufficient power for what we wanted it to do. The portal team continued investigating different technologies to identify the device connected to the server and customize the content to that device -- but they didn’t need me for that.

I then discovered Owen Densmore’s JavaCar team. Unlike the Palm VII, the JavaCar had plenty of power -- and networking. It seemed clear that delivering personalized content to the car would be a good thing -- for drivers and car companies and Sun! But, what kind of content? That was my next task.

Research

One of our first projects was to identify what tasks a user in a car would want to perform while connected to the network. A group of students from Stanford University's CS 247b class worked with me on this phase. In brief, we learned that people spend a lot of time in their cars (Americans average 80 minutes a day), and during that time they spend a lot of time listening to the radio to keep them entertained during their commute and often switched to other stations for traffic or weather updates. We also learned that drivers were frustrated by radio -- they had to know when the traffic reports were broadcast, and were frustrated like Owen and I were about missing programs.

With confirmation that our initial hunch was correct, and with more details about what users want / need, we set about building a solution.

What We Built

Between late 1999 and early 2001 when our prototype was introduced at the ACM CHI Conference in Seattle (April 3-5, 2001) we did a lot of research and protottyping. Here's a high level description of that what we built:

• A Linux based appliance designed to look / operate like a car radio
• A Sun / iPlanet Portal UI that allowed users to choose the content they wanted to listen to on the radio, conceptually like Apple's iTunes Store
• GPS unit that returned the car's location to our backend server for tracking and alert generation
• A mechanism that allowed the backend server to interrupt what the radio was playing so it could play alerts, e.g. traffic in the area

Here's what the radio looked like on our workbench:


Displaying the FreshAir logo to identify which program was being played

The Portal widget controled which syndicated content was delivered to the radio. It looked like:

In the left column you see the car's location (thanks to GPS) and the Playlist Provider listing programs the user has selected. Notice that some content is described as Live (streamed) in contrast to the rest, which we expected to be cached. We also anticipanted, but didn't build, a recommendation mechanism.

Clicking the Edit button in the previous figure would reveal this page:

This interface allowed the user to choose which content they were interested in. We expected service providers would aggregate content from different sources, e.g. BBC, CBC, CNN, NPR, and deliver it to their customers.

The big picture looked like this:

Not shown on this diagram is the hard drive on the In-Car Server where the cached content was stored.

In the Car

When the user got into the car they inserted a special key that the system read and then used to log into the Service Provider server. That authentication step allowed the car to pull the correct user's most recent content from the server. It also identified which user was in the car, something we wanted to show via the Car Location widget on the Portal Server page. We planned to use this information might be used to send traffic and weather alerts that were specific to this user's location / direction of travel -- instead of broadcasting to everyone. (Our prototype didn't really use the car's location when it sent an alert -- too difficult to demo if relying on actual location and traffic info.)

Showing Off Our Work

We presented two short papers on our work (paper 1, paper 2,) at the CHI 2001 conference in Seattle. My presentation went into a lot more detail (my slides). We also took our car with the radio installed and a server to do live demos on the exhibit floor. That demo was one of the most visited exhibits on the floor -- lots of people from Microsoft and other companies came by to see what we were showing. Here's a photo:

CHI Conference Exhibits, Seattle (from http://www.sigchi.org/chi2001/photos.html)
I'm standing on the driver's side (green shirt) talking to Don Norman.
I'm partially obscuring Kelly Kruz and Rob Mori who did the industrial design for the radio.
We also appeared on the front page of the Seattle-Times business section, April 4, 2001 (here).

Here's a better view of our internet radio installed in the Smart car.

A few months later, Rob Mori and I did a longer talk at BayCHI, a gathering of user interface professionals that meet once a month at the Xerox PARC auditorium. The slides from that presentation are available (here). And there's a report on our presentation available (here). Below you'll see the demo we showed that night. Rob is narrating.

Demo #3

What About Patents?

I'm not opposed to patents, I have 6. But I wasn't sure that an internet radio could be patented (and besides, I thought others might have gotten there first -- like Kerbango). That someone could patent the idea / mechanism for delivering a program didn't make sense. MyYahoo and other portals were already doing content syndication / aggregation -- just not for audio content so how could that be patentable? But, it seemed reasonable that we could patent a mechanism for generating / delivering custom content based on the car / user's location and deliver it via our internet radio system. And indeed we were granted two patents in that area (see patents #6,636,801 and #6,694,259).

More Details

A couple of years later (Feb. 2003) I was interviewing for a new job and put together a presentation that explaned more of the architecture than I discussed in teh CHI or BayCHI presentations. I've removed the interview-related material from the front of the deck and made that presentation available (here). Page 27 is where the detailed description begins. A couple of years later someone asked for more details and I did an even more detailed set of slides on the architecture (here).

What's Missing?

I don't know how many times people have asked me when this would be coming to market. An iPhone running the Public Radio, NPR Addict, Pandora, etc. are finally getting close to our vision, but there isn't an integrated solution to managing all of my content like our radio's user interface. And, our prototype included features that haven't make it into products yet. For example:

• An integrated experience for podcast / streamed content -- iPhone isn't there yet
• An implementation for the car -- not everyone wants to try to dock an iPhone
• Chapter Markers
• Stinger, a short clip summarizing or introducing the program
• The ability to stop playing a program in the car, go to another device and resume at the same point

That last point deserves a bit more expanation. When the radio started up it would connect to backend servers as described above. When the radio was turned off it also connected to those servers, but now it was updating a database that remembered the last program being played and the position within that program. When you started another device, like our Home Stereo prototype (shown at JavaOne (June 4-8, 2001), it would retrieve that information and continue playing that program from the same position. This was in response to our recognizing that people often sit in their cars to listen to the end of a program. This was our solution.

Credits

• Dave Curbow (Sun Senior Staff Engineer) team lead
• Owen Densmore (Sun Labs Java Car Principal Investigator) co-invented the concept and generously shared his people so the project could happen
• Kelly Kruse (Industrial Design Intern) did the industrial design work for the radio
• Rob Mori (Sun Industrial Design and Human Factors) help refine the concept and led Kelly Kruse
• Ron Lee (Technical Product Marketing) helped find hardware, money and reason for management to continue supporting the project
• Eric Macintosh (Stanford Intern) helped with prototyping
• Mike Jahr (Stanford Intern) wrote a lot of the prototype's code
• Steve Drach (Sun Labs Senior Staff Engineer) created Linux Drivers
• Mark Koch (Sun Labs mechanical & hw designer), circuit boards and mechanical engineering
• Behfar Rezavi (Sun Labs Java Car Engineer) suggested that cached programs complimented streamed programs
• Vicky Oliver (Sun / iPlanet Staff Engineer) for help with the Portal coding
• Java Media Framework Team for delivering a version of JMF to run on Linux
• Guy Martin (Sun Labs Java Car Engineer)
• Pete St.Pierre (Sun Labs Java Car Engineer)
• Prototypes Plus who built the plastics for our prototype
• Stanford CS247b students who helped research what users wanted / needed (I'll add their names when I find them in my notes)
• Dilshad Uvez Simons (Sun Director), my boss who believed in the project and supported it with my time and her budget
• Terry Gross (Freshair host), who inspired us and made available content to use in our prototype

The End...