I am a Research Staff Member at IBM Research – Almaden located in San Jose, California. I am working in the USER group on crowdsourcing and social media. My main project, called qCrowd, is a system for extracting hidden information from social media users by automatically asking questions. One of the initial prototypes of this work is the TSA Tracker, which crowdsources airport security wait times by asking questions of users who have indicated that they are located at an airport via a status update on Twitter.

Until recently, my research focused on end-user programming and the web. The Highlight system allowed users of any technical proficiency to create mobile versions of existing web sites. I also spent time on a variety of projects related to CoScripter.

I work with a bunch of great people, including Allen Cypher, Clemens Drews, Tessa Lau, Jalal Mahmud, Jeff Pierce, Michelle Zhou, and many others.

I received my Ph.D. in Human-Computer Interaction from the Human-Computer Interaction Institute at Carnegie Mellon University in Pittsburgh, Pennsylvania. My Ph.D. thesis is entitled Automatically Generating High-Quality User Interfaces for Appliances, and was advised by Prof. Brad A. Myers. The focus of my thesis was on the personal universal controller (PUC, pronounced "puck"), a solution for reducing the complexity of the appliance interfaces that we use everyday. See my Projects section for more information.

Before going to Pittsburgh, I completed a B.S. degree in Computer Engineering from the University of Washington. I worked on several projects while at UW: user interfaces for constraint-based systems with Dr. Greg Badros and Prof. Alan Borning, biologically-inspired VLSI and circuit design with Prof. Chris Diorio, and embedded and ubiquitous computing systems with Prof. Gaetano Borriello.

For even more information, see the various sections below or my vitae.

I was born and raised in Portland, Oregon.

Highlight: Mobilizing Existing Web Sites

Highlight allows end users to create mobile applications from existing web sites and deploy those applications to mobile devices. Highlight's design, which incorporates a fully-functional modern web browser engine as part of its server-side infrastructure, allows existing sites that contain substantial client-side JavaScript or AJAX technology to be modified for use on mobile devices. Our end-user authoring environment, which makes use of programming-by-demonstration techniques, was the subject of a paper at IUI 2008. The server-side infrastructure was the subject of a paper at UIST 2008.

For more information on Highlight, watch this hour-long talk that I gave at MIT CSAIL in October.

Information Triggers

At times while browsing the web, users may become aware of tasks or information they wish to attend to in the future. The Information Triggers project is is examining means for users to associate web pages with future events that are indicative of when those pages will be particularly relevant or useful. When an event occurs, the relevant page is automatically opened within the user's browser to provide the appropriate information. So far, we have implemented a Firefox extension and a server-side repository that allow users to create and share triggers. Our extension current supports events such as visiting a web page with particular content, at a specific time, or the available of an e-mail with particular content. We have found triggers to be useful in many situations, such as automatically augmenting web application with help procedures and resuming tasks that cannot be completed in one sitting.

This project is being done in collaboration with Jake Wobbrock, Maneesh Agrawala, and Simon Tan.

Personal Information Environments

I am also involved in the Personal Information Environments (PIEs) project, which is led by Jeff Pierce. The goal of this project is to explore new interaction styles and infrastructures to support better interoperability between a user's many devices. See the IBM project web page for more information.

CoScripter

I am peripherally involved in the CoScripter project, led by Tessa Lau and Allen Cypher. CoScripter is a system for recording, automating, and sharing processes performed in a web browser such as printing photos online, requesting a vacation hold for postal mail, or checking flight arrival times. Instructions for processes are recorded in human-readable language and stored in a wiki on the CoScripter server, thus making them available to everyone else on the web. An external version of CoScripter is available through IBM's Alphaworks Services.

The Personal Universal Controller

My thesis work at Carnegie Mellon was a part of the Pebbles Project, which is exploring how handhelds and desktops can be used in conjunction. I primarily worked on the personal universal controller, a project that explored the issues that arise when a handheld is used as remote control to manipulate all of the devices in the world around us. Our primary focus was creating a specification language that can define any device and an automatic user interface generator that can create control panels from this specification language. This paper is a good overview of the Pebbles project that discusses my work in some detail. This paper best describes the basics of the PUC system. Our evaluation of the PUC system found that automatically generated interfaces could be better than hand-designed interface in some cases.

I extended the PUC with a layer called Uniform that automatically generates remote control interfaces that are consistent with past interfaces that the user has seen. This system works for interfaces generated on both a PocketPC and mobile phone interface. See some screenshots here. An evaluation found that users performed better with the interfaces generated for consistency than with normal interfaces.

I also built the Huddle system for automatically creating combined interfaces for systems of multiple appliances. Huddle uses a model of content flow to understand the tasks that users will want to perform with the system of appliances. Based on the model, Huddle creates two kinds of interfaces: a Flow-Based Interface that allows users to accomplish their high-level goals and four kinds of Aggregate User Interfaces that give users task-specific low-level control of the appliances.

Welcome, Matt Project

This started out as a class project for HCI Processes & Theory. The work is a collaboration between Jake Wobbrock, Darren Gergle, and myself. It culminated in a paper accepted as an exhibit at the DIS'2002 conference (paper: pdf, poster: pdf).

www.metadesign.com Magic Lens

This was a short-lived project for my Graduate Design Studio I class. It is a prototype of a magic lens for use on the world wide web, done in Macromedia Director. If you click on the link above, you will be taken to a page which shows the presentation. You will need the Shockwave plug-in to view the prototype.

The class project was to explore interesting ways to show the structure and content of a particular artifact. The artifact I was given was www.MetaDesign.com. In my project, I focused on showing the structure of the site with a magic lens. The lens allows the user to see through the links on web pages to view how many links those pages have. In the case of the MetaDesign web site, this is very useful, because you can immediately see where all of the content is located within the site.

Unfortunately, while this prototype was reasonably easy to implement, a generalized version of this technique would probably not be as effective. Numbers of links are not often a good indicator of content in a web page structure. Furthermore, the layout of the information seen through the lens would be difficult to manage in the general case.

Scheme Constraints Window Manager

The Scheme Constraints Window Manager (scwm) is a window manager for the X11 windowing system (commonly associated with the UNIX operating system). Scwm has several unique features, including an embedded constraint solver for managing windows and a fully integrated scheme interpreter that can be used to program any feature of the window manager. I built, with Greg Badros, a user interface for the embedded constraint solver that made it easy for users to add constraints to their windows and view what constraints were currently acting on a window.

Baseball

In the last couple years I have become a big fan of baseball, including fantasy baseball, which I did last year for the first time. I follow the Seattle Mariners closely, but I'm also a fan of the Pirates, Padres and Red Sox.

These are baseball sites I read regularly:

• The Baseball Prospectus
• Rob Neyer, ESPN.com columnist
• ESPN.com

Ultimate

This is my favorite sport to play, which is something of a cross between football and soccer, except with a frisbee. More details are available on the Ultimate Players Association web site.

Ultimate players in Pittsburgh might want to check out the Pittsburgh Ultimate web site.

Board and Card Games

In my free time, I like to play board games. I'm especially interested in non-traditional board games, although I enjoy chess and card games as well. Mayfair Games and Rio Grande Games are two publishers that sell games that I particularly like. Among my favorites:

• Abalone
• Cathedral
• The Settlers of Catan
• Terrace
• Torres

I also learned to play Bridge recently, but I haven't found the time or people to become more involved with it. I also hope to someday learn to play Go.

Conference Papers

Tessa Lau, Julian Cerruti, Guillermo Manzato, Mateo Bengualid, Jeffrey P. Bigham, Jeffrey Nichols. "A Conversational Interface to Web Automation," in Proceedings of UIST 2010, New York, NY, October 3-6, 2010. pp. 229-238 (pdf)

Ian Li, Jeffrey Nichols, Tessa Lau, Clemens Drews, Allen Cypher. "Here's What I Did: Sharing and Reusing Web Activity with ActionShot," In Proceedings of CHI 2010. Atlanta, GA. pp. 723-732 (pdf)

Jeffrey P. Bigham, Ryan Kaminsky, Jeffrey Nichols. "Mining Web Interactions to Automatically Create Mash-Ups," In Proceedings of UIST 2009. pp. 203-212 (pdf)

Tessa Lau, Clemens Drews, and Jeffrey Nichols. "Interpreting Written How-To Instructions," In Proceedings of IJCAI 2009. pp. 1433-1438 (pdf)

Heather Wiltse, Jeffrey Nichols. "PlayByPlay: Collaborative Web Browsing for Desktop and Mobile Devices," In Proceedings of CHI 2009. Boston, MA. April 4-9. pp. 1781-1790. (pdf)

James Lin, Jeffrey Wong, Allen Cypher, Jeffrey Nichols, Tessa Lau. "End-User Programming of Mashups with Vegemite," In Proceedings of IUI 2009. Sanibel Island, Florida. February 8-11. pp. 97-106. (pdf)

Jeffrey P. Bigham, Tessa Lau, Jeffrey Nichols. "TrailBlazer: Enabling Blind Users to Blaze Trails Through the Web," In Proceedings of IUI 2009. Sanibel Island, Florida. February 8-11. pp. 177-186. (pdf)

Jeffrey Nichols, Zhigang Hua, John Barton. "Highlight: A System for Creating and Deploying Mobile Web Applications," in Proceedings of UIST'2008. Monterey, CA. October 19-22. pp. 249-258. (pdf)

Jeffrey S. Pierce, Jeffrey Nichols. "An Infrastructure for Building Applications That Span Multiple Personal Devices," in Proceedings of UIST'2008. Monterey, CA. October 19-22. pp. 101-110. (pdf)

Jeffrey Nichols, Tessa Lau. "Mobilization by Demonstration: Using Traces to Re-author Existing Web Sites," In Proceedings of IUI'2008. Maspalomas, Gran Canaria, Spain. January 13-16. pp. 149-158. (pdf)

Jeffrey Nichols, Duen Horng Chau, Brad A. Myers. "Demonstrating the Viability of Automatically Generated User Interfaces," In Proceedings of CHI'2007. San Jose, CA. April 28-May 3. pp. 1283-1292 (pdf)

Jeffrey Nichols, Brandon Rothrock, Duen Horng Chau, Brad A. Myers. "Huddle: Automatically Generating Interfaces for Systems of Multiple Connected Appliances," In Proceedings of UIST'2006. October 15-18. Montreux, Switzerland. pp 279-288 (pdf)

Jeffrey Nichols, Brad A. Myers, Brandon Rothrock. "UNIFORM: Automatically Generating Consistent Remote Control User Interfaces," In Proceedings of CHI'2006. April 22-26. Montreal, Canada. pp. 611-620 (pdf)

Jeffrey Nichols, Brad A. Myers, and Kevin Litwack. "Improving Automatic Interface Generation with Smart Templates," In Proceedings of Intelligent User Interfaces (IUI) 2004. Funchal, Portugal. January 13-16. pp. 286-288 paper: (pdf) poster: (pdf)

Jeffrey Nichols and Brad A. Myers. "Studying The Use Of Handhelds to Control Smart Appliances," In Proceedings of the 23rd International Conference on Distributed Computing Systems Workshops (ICDCS '03). Providence, RI. May 19-22, 2003. pp. 274-279 (pdf)

Brad A. Myers, Jeffrey Nichols, Jacob O. Wobbrock, Kevin Litwack, Michael Higgins, Joseph Hughes, Thomas K. Harris, Roni Rosenfeld, Mathilde Pignol. "Handheld Devices for Control," in Human-Computer Interaction Consortium (HCIC'2003), Winter Park, CO, Feb 5-9, 2003. (pdf)

Jeffrey Nichols, Brad A. Myers, Michael Higgins, Joseph Hughes, Thomas K. Harris, Roni Rosenfeld, Kevin Litwack. "Personal Universal Controllers: Controlling Complex Appliances With GUIs and Speech," In Extended Abstracts of CHI'2003, April 5-10. Ft. Lauderdale, FL. pp. 624-625 (pdf)

Jeffrey Nichols, Brad A. Myers, Michael Higgins, Joseph Hughes, Thomas K. Harris, Roni Rosenfeld, Mathilde Pignol. "Generating Remote Control Interfaces for Complex Appliances," In Proceedings of UIST'2002, Paris, France. Oct 27-30. pp. 161-170 (pdf)

Jeffrey Nichols, Brad Myers, Thomas K. Harris, Roni Rosenfeld, Stefanie Shriver, Michael Higgins, Joseph Hughes. "Requirements for Automatically Generating Multi-Modal Interfaces for Complex Appliances," In Proceedings of the Fourth IEEE International Conference on Multimodal Interfaces, October 14-16, 2002. Pittsburgh, PA. pp. 377-382 (pdf)

Brad Myers, Robert Malkin, Michael Bett, Alex Waibel, Ben Bostwick, Robert C. Miller, Jie Yang, Matthias Denecke, Edgar Seemann, Jie Zhu, Choon Hong Peck, Dave Kong, Jeffrey Nichols, Bill Scherlis. "Flexi-modal and Multi-Machine User Interfaces," In Proceedings of the Fourth IEEE International Conference on Multimodal Interfaces, October 14-16, 2002. Pittsburgh, PA. pp. 343-348 (pdf)

Jeffrey Nichols, Jacob O. Wobbrock, Darren Gergle, and Jodi Forlizzi. "Mediator and Medium: Doors as Interruption Gateways and Aesthetic Displays" In Proceedings of DIS'2002, London, UK. June 25-28. pgs 379-386. paper: (pdf) poster: (pdf)

Brad A. Myers, Jacob O. Wobbrock, Sunny Yang, Brian Yeung, Jeffrey Nichols, and Robert Miller. "Using Handhelds to Help People with Motor Impairments". In Proceedings of the ACM SIGCAPH's 5th International Conference on Assistive Technologies (ASSETS '02), Edinburgh, Scotland, July 8-10 2002. pp. 89-96 (pdf)

Brad A. Myers, Rishi Bhatnagar, Jeffrey Nichols, Choon Hong Peck, Dave Kong, Robert Miller, and A. Chris Long. "Interacting At a Distance: Measuring the Performance of Laser Pointers and Other Devices". In Proceedings of CHI'2002. Minneapolis, MN. April 20-25 pp. 33-40 (pdf)

Brad A. Myers, Choon Hong Peck, Jeffrey Nichols, Dave Kong, and Robert Miller. "Interacting At a Distance Using Semantic Snarfing," In Proceedings of Ubicomp 2001. Sept 30-Oct 2, Atlanta, Georgia. pp. 305-314. (pdf)

Greg J. Badros, Jeffrey Nichols, and Alan Borning. "Scwm: An Extensible Constraint-Enabled Window Manager", In Proceedings of Freenix track of USENIX Technical Conference. Boston, June 2001. (pdf)

Greg Badros, Jeffrey Nichols and Alan Borning, "Scwm--An Intelligent Constraint-Enabled Window Manager", AAAI Spring Symposium on Smart Graphics, March 2000 (pdf)

Journal Papers

Jeffrey Nichols and Brad A. Myers. "Creating a Lightweight User Interface Description Language: An Overview and Analysis of the Personal Universal Controller Project," In ACM Transactions on Computer-Human Interaction, Volume 16, Issue 4, Article 17. (pdf)

Jeffrey Nichols and Brad A. Myers. "Controlling Home and Office Appliances Using Smartphones". In IEEE Pervasive Computing, July-September 2006. pp. 60-67 (pdf)

Brad A. Myers, Jeffrey Nichols, Jacob O. Wobbrock, Robert C. Miller. "Taking Handheld Devices to the Next Level." IEEE Computer. December 2004. pp 40-47. (pdf)

Dissertation

Jeffrey Nichols. Automatically Generating High-Quality User Interfaces for Appliances, December 2006. (high-res pdf, low-res pdf)

Workshop Papers

Allen Cypher, Tessa Lau, Jeffrey Nichols, Mira Dontcheva. "Workshop on End User Programming on the Web" in Extended Abstracts of CHI 2009. April 3-9. Boston, MA. pp. 4779-4782. (pdf)

Kai Richter, Jeffrey Nichols, Krzysztof Gajos, Ahmed Seffah. "The Many Faces of Consistency in Cross-Platform Design" in Extended Proceedings of CHI'2006. April 22-23. Montreal, Canada. pp. 1639-1642. (pdf)

Jeffrey Nichols and Andrew Faulring. "Automatic Interface Generation and Future User Interface Tools", In Proceedings of the Workshop on the Future of User Interface Design Tools at CHI 2005. April 4. Portland, OR. (pdf)

Jeffrey Nichols and Brad A. Myers. "Generating Consistent User Interfaces for Appliances", In Proceedings of the Workshop on Multi-User and Ubiquitous User Interfaces at Intelligent User Interfaces 2005. January 9. San Diego, CA. pp. 9-10 (pdf)

Jeffrey Nichols. "Automatically Generating User Interfaces for Appliances", In UIST 2004 Conference Companion, October 24-27, 2004. Santa Fe, NM. pp 71-74 paper: (pdf) poster: (pdf)

Jeffrey Nichols, Brad A. Myers, Kevin Litwack, Michael Higgins, Joseph Hughes, Thomas K. Harris. "Describing Appliance User Interfaces Abstractly with XML," In Workshop on Developing User Interfaces with XML: Advances on User Interface Description Languages, 2004, 25 May, Gallipoli, Italy. pp. 9-16 (pdf)

Jeffrey Nichols. "Automatically Generating User Interfaces for Appliances," in Advances of Pervasive Computing edited by Alois Ferscha, Horst Hortner, and Gabriele Kotsis. April 18, 2004. pp. 105-110. (pdf)

Jeffrey Nichols, Brad A. Myers, and Rob Miller. .Personal Interfaces in Ubiquitous Environments,. in Proceedings of the CHI.2001 Workshop on Building the Ubiquitous Computing Experience. Seattle, WA. (html)

Technical Reports

Brad A. Myers, Jeff Nichols, Rob Miller. "User Interfaces that Span Hand-Held and Fixed Devices" Workshop on Distributed and Disappearing User Interfaces in Ubiquitous Computing at CHI'2001, Seattle, WA. Albrecht Schmidt, Peter Ljundgstrand, and Anind Dey, editors. University of Karlsruhe Faculty of Information Technical Report 2001-6. ISSN 1432-7864. (html)

Student Posters

Jeffrey Nichols. "Informing Automatic Generation of Remote Control Interfaces with Human Designs," In Extended Proceedings of CHI'2002. Minneapolis, MN. April 20-25 pp. 864-865 abstract: (pdf) poster: (pdf)

Jeffrey Nichols. "Using Handhelds as Controls for Everyday Appliances: A Paper Prototype Study," ACM CHI'2001 Student Posters. Seattle, WA. March 31-April 5, 2001. pp 443-444. abstract: (pdf) poster: (pdf)

Unpublished Reports

Jeffrey Nichols and Brad A. Myers. "Report on the INCITS/V2 AIAP-URC Standard," February 9, 2004. (pdf)

Jeffrey Nichols. "Automatically Generating High-Quality User Interfaces for Appliances," Thesis Proposal, April 14, 2004. (pdf)

US7664862. John Barton, Zhigang Hua, Jeffrey Nichols. Browser-based proxy server for customization and distribution of existing applications. Issued February 16, 2010.

Creating a Lightweight User Interface Description Language: An Overview and Analysis of the Personal Universal Controller Project, CHI'2010. Atlanta, Georgia. April 15, 2010. (ppt)

Highlight: A System for Creating and Deploying Mobile Web Applications, UIST'2008. Monterey, California. October 22, 2008. (ppt)

Mobilization by Demonstration: Using Traces to Re-author Existing Web Sites, IUI'2008. Maspalomas, Gran Canaria, Spain. January 15, 2008. (zip)

Huddle and Information Triggers: Helping Users Automate Complex and Temporally-Distant Tasks, Berkeley Institute of Design Seminar. Berkeley, CA. June 14, 2007. (ppt)

Demonstrating the Viability of Automatically Generated User Interfaces, CHI'2007. San Jose, CA. May 3, 2007. (ppt)

Automatically Generating High-Quality User Interfaces for Appliances, Thesis Defense. Pittsburgh, PA. December 1, 2006. (ppt)

Huddle: Automatically Generating Interfaces for Systems of Multiple Connected Appliances, at UIST'2006. Montreux, Switzerland. October 18, 2006. (ppt)

UNIFORM: Automatically Generating Consistent Remote Control User Interfaces, at CHI'2006. Montreal, Canada. April 25, 2006. (ppt)

Model-Based UIs & Automatic Generation, Workshop on the Future of User Interface Design Tools at CHI 2005. (ppt)

Generating Consistent Interfaces for Appliances, Second Workshop on Multi-User and Ubiquitous User Interfaces (MU3I) at Intelligent User Interfaces 2005. January 9, 2005. (ppt)

Automatically Generating High-Quality User Interfaces for Appliances, UIST 2004 Doctoral Colloquium. October 24, 2004. Santa Fe, NM. (ppt)

Describing Appliance User Interfaces Abstractly with XML, Workshop on Developing User Interfaces with XML: Advances on User Interface Description Langauges. May 25, 2004. Gallipoli, Italy. (ppt)

Automatically Generating High-Quality User Interfaces for Appliances, Pervasive 2004 Doctoral Colloquium. April 18, 2004. Linz, Austria. (ppt)

Automatically Generating High-Quality User Interfaces for Appliances, Thesis Proposal Presentation, April 14, 2004. (ppt)

Automatically Generating Interfaces for Multi-Device Environments, Workshop on Multi-Device Interfaces for Peripheral Interaction at the Fifth International Conference on Ubiquitous Computing. Seattle, WA. October 12, 2003. (ppt)

The Personal Universal Controller and .NET CF, Microsoft Research Faculty Summit, Redmond, Wa. July 2003 (ppt)

Studying The Use of Handhelds to Control Smart Appliances, International Workshop on Smart Appliances and Wearable Computing, Providence, RI. May 19, 2003. (ppt)

Personal Universal Controllers: Controlling Complex Interfaces With GUIs and Speech, CHI 2003, Ft. Lauderdale, FL. April 8, 2003. (ppt)

Generating Remote Control Interfaces for Complex Appliances, UIST 2002, Paris, France. October 30, 2002. (ppt)

Requirements for Automatically Generating Interfaces for Complex Appliances, ICMI 2002, Pittsburgh, PA. October 15, 2002. (ppt)

Speaking Requirement Talk, HCII August 2002. (ppt)

Using Handhelds to Help People with Motor Impairments, ASSETS'2002, Edinburgh, Scotland. July 9, 2002. (ppt)

Handheld Computers in Higher-Education, Slippery Rock University September 2001. (ppt)

Speaking Requirement Talk, HCII August 2001. (ppt)

Robert McMillan. IBM Slims Down the Web for Your Phone, IDG News Service. July 11, 2008. (html) (local pdf)
Also covered on Engadget Mobile

These are the most up-to-date ways to get in touch with me.