Churchill Elizabeth - Designing with Data : Improving the User Experience with A/B Testing

Beijing Boston Farnham Sebastopol Tokyo

• DRM-free ebooks use your ebooks across devices without restrictions or limitations
• Multiple formats use on your laptop, tablet, or phone
• Lifetime access, with free updates
• Dropbox syncing your files, anywhere

T HE MYTH OF THE genius designer, someone whose instincts and intuition lead to great design decisions, is a popular one. This seductive myth can lead some to conclude that design is never grounded in data, that design is never an empirical discipline, and that design practice stands in opposition to data and data sciences.

We understand where this myth comes from. On the surface, data science and design practice are not obviously compatible. Design philosophy and practice emphasizes empathy with users, and the creation and crafting of artful user experiences in the form of effective products. For some designers (and for many outside the design world who valorize design as inspired genius), design is a nonlinear, exploratory journey. It is a fundamentally human, fluid, and creative process, unlike the procedures and rigors of science. Design is emotional. Design cannot be articulated as a set of procedures or steps to be followed. Design cannot be rationalized and constrained. For some, incorporating data into the design process is a cause for concern.

Some concerns we have heard expressed include:

On the other side, proponents of design through experimentation and data science value measurement. Some see data as rational, and numbers as irrefutable. Data science reveals the truth data science is a proceduralized, scientific endeavor where rigor leads to irrefutable results and therefore to certainty. Data science is a trustworthy and precise craft. This view is reinforced by the increasing fascination with measures and metrics for business, commonly referred to today as big data. An extreme view is that large-scale experiments can be run to gather data from millions of users to answer all design questions and that such analytics can, therefore, replace design. Under this view, font types, colors, and sizes, and questions such as Should we have a blue or a red dialogue box?, Do people engage more with a list or a carousel?, or Does a wizard help with new user onboarding flow? fall under the purview of data science and not design practice. This could be characterized as Let the crowd speak with their clicks, and what emerges will necessarily be the best design.

We deliberately present these extreme positions to illustrate a point. We believe that the extreme views we just outlined draw a false dichotomy between data and design. In reality, data sciences and design practices are working toward the same goal: understanding users and crafting elegant experiences for them. Design is and always has been informed by data. The data may be an accumulated set of experiences and informally gathered observations that provide the basis for design genius and craft knowledge. The data may also be derived from more systematic studies of users activities and opinions, such as lab-based studies, field observations, and surveys. Design practice has always been about different forms of data. In an ever-changing marketplace and industry where new applications and new behaviors are constantly emerging, data can play a big role in helping us learn and respond in a timely way to shifts in user interests and needs. By harnessing and leveraging the power of data at scale that is, data in high volume, often arriving in streams from millions of users, and which may be of disparate types new ways to understand people, users, are emerging. Data at all scales from individuals to millions and hundreds of millions of users systematically collected, analyzed, communicated, and leveraged can empower design.