Labeling bad decisions as “unforeseen consequences” is no longer an excuse. Here’s how designers can proactively ensure that their products are ethical in every stage.
This is the second part of a two part series. If you haven’t already, check out Designing Ethically Pt. 1 for a high-level overview on why ethics in design is necessary, how to redesign our design process to include an ethical foundation, and how to sustain ethics in our practice.
For an alternative to this 13-minute read, check out the site for a more digestible version of the framework, toolkit, and more!
In Designing Ethically Pt. 1, we saw how it was necessary to integrate ethics into design, reimagined a new design process, and strategized how to sustain the practice. To summarize, ethics are crucial because our technology today has the ability to impact billions of people, for better or for worse. Nowadays, it seems like a lot of worst-case scenarios have been unveiled (at Facebook, Uber, and the like). A common response from companies in hot water is that the unethical consequences they caused were “unforeseen.” 1) I’m calling BS on that. 2) It is more imperative than ever for those in tech — not just designers — to play a role in preemptively preventing “unforeseen” issues. 3) The redesigned design process can help.
Redesigned design process
In the redesigned design process, there were several new steps introduced. The three steps that are especially actionable for designers are evaluating, forecasting, and monitoring. In Designing Ethical Pt. 2, I want to go over ways to apply those three key steps and also reframe the way we consider products. The structure of this post is as follows:
- Evaluating Problem Statements: How do you assign ethical weight to your problem statement to determine if it is worthy of being addressed?
- Layers of Effects: In this section, learn how to reframe the way you view products so that you can better forecast ethical conundrums.
- Forecasting Toolkit: Get your hands dirty by trying out the forecasting exercises for your own products.
- Monitoring Checklist: Intermittently double and triple check your product after it’s been shipped to make sure it isn’t violating anything on this list.
Evaluating Problem Statements
Once you’ve defined a legitimate problem, how do you go about figuring out if the problem ought to be addressed? This stage of the redesigned process goes after the empathize and define stages in the typical design process. After you’ve completed the define step, you’ll likely have a point of view statement, which is parsed out below.
Check out more info from the Interaction Design Foundation’s site!
The need section is the part that is important to analyze. Is it benefitting users or society? Think back to the duty-based ethics we discussed in Designing Ethically Pt. 1. Is the intention good? And while you’re still in that frame of mind, try analyzing if the identified, positive, user need has any negative collateral. For example, check out the following graphic.
Try going through these considerations and figuring out if your user need aligns with one of the categories on the left side without necessitating one of negative categories on the right side. For example, if I am trying to bring in a product like Airbnb into a new community to help a group of people (namely the hosts that live there as well as the tourists that visit)…will I be able to do so without putting down another group (put more bluntly, without gentrifying the area)?
These lists of positive needs and negative collateral are by no means set in stone and should be added to in the future. This step of evaluating problem statements does not mean that you shouldn’t develop any new product just because it might intertwine with negative collateral. If that were the case, we’d never develop anything. Evaluating just means taking the time to consider the ramifications of your user need with a wider lens. It also primes us to start thinking in a way that will come in handy when we jump into forecasting “unforeseen consequences.”
Layers of Effect
Before we start forecasting, let’s reframe the way we look at the products and experiences we either make or use. If you were to ask any layperson in the street, “What is Twitter?”, they would probably give you an answer along these lines:
- “It’s a site where I can connect with my friends and family!”
- “I use Twitter to follow up on what famous people are up to.”
- “It’s where I go for dank memes.”
Great. But Twitter is so much more than a generic social media platform. And it’s got some pretty complex effects. The following graphic shows the breakdown of products into three sections: primary, secondary, and tertiary.
Primary
These are the effects that you think of first when you think of a product or experience. Primary effects are pitched as the heart of a product. They are always intended and known. For example, Yelp’s primary effects is that it is a platform for people to review and learn about establishments. Primary effects may evolve over the course of time, but they typically do not change too drastically, especially if they have already set off and made an impression on the market. These effects are confined in the innermost, opaque green box to highlight that they are known and intentional.
Secondary
These are the effects that might not pop up immediately as the defining characteristic of a product, but are still just as relevant to the company’s shareholders. For Twitter, while its primary effect might be serving as a social media platform, a secondary effect would be how it acts as an ad revenue generator. Again, similar to primary effects, these are not set in stone and can evolve over time. These effects confined in the outermost, opaque green box to highlight that they are also known and intentional.
Tertiary
These are the effects that are either unintended or unforeseen. These can be good or bad; in any case, tertiary effects are always surprises that start cropping up after users have had their hands on the product. For example, a tertiary effect of Facebook would be the role it played in perpetuating the spread of fake news due to its algorithm. A tertiary effect of Twitter is the massive trolling and cyberbullying (as well as the ensuing emotional trauma) that happens on its platform. In the above graphic, the tertiary circle is feathered out, representing the fact that there can be an unlimited amount of tertiary effects. They aren’t quite the same as edge cases, although there is some overlap; tertiary effects are not always the result of extreme parameters.
Those who have the privilege of creating products have the responsibility of defining ethical primary and secondary effects, as well as forecasting tertiary effects and ensuring that they pose no significant harm.
An example of retrospectively mapping the layers of effect for a product
Forecasting Toolkit
Now that we’ve established how to reframe products and experiences, we can start forecasting. The forecasting part is the most complex, which is why I’ve developed a toolkit for designers to begin to wrap their minds around it.
You’re probably scratching your head wondering, how on earth do you forecast tertiary effects? How are mere mortals supposed to predict consequences and reactions that haven’t happened yet? Forecasting all the “unintended” ways users might interact with a product is tricky. But it’s not impossible. In fact, it might be more familiar than you think.
The skills involved in forecasting should not be new to anyone. If you’ve ever found yourself binge-watching a television series in one weekend, only to find out that the next season was a year away from being released (*cough* GoT *cough*), you’ve probably encountered the natural instinct to try and predict what would occur. People forecast other people all the time. Also consider this: when forecasting for tertiary effects, quantity matters more than quality. It’s not about being 100% accurate. The important part is being creative and imagining as many tertiary effects as you can, so you can hone in the harmful ones and prevent them from happening.
Motivation Matrix
People perform actions because they are triggered by motivations. There’s countless articles on the types of motivation out there, and after perusing dozens of articles, I think that all of the motivating factors out there can be distilled into six core types: incentive, achievement, social acceptance, fear, power, and growth.
Incentive: any type of reward-oriented motivating factor; can be monetary or not monetary
Achievement: the kind of motivation that’s propelled by the drive for competency
Social Acceptance: essentially the need to belong to a group and not feel ostracized
Fear: motivation that is based off of wanting to avoid certain outcomes or consequences
Power: motivation that is derived from the need to be autonomous or to gain and maintain control over others
Growth: intrinsic motivation that encapsulates wanting to become a better version of oneself
By creating a matrix that relates these motivating factors with various users of a product in different contexts, you can write statements that predict how a user might interact with the product in a particular context.
I recommend doing this exercise after empathy mapping so you have a clearer understanding of your sponsor users. In the example below, you can see that the simplified empathy maps are on the right (with the blue stickies). Within each motivation cell, there are two types of stickies: we used pink to represent the any insights around that particular motivation when applied to the user; yellow was used to depict the potential actions that the user could take place given the motivation.
Motivation Matrix applied to AngelList
Maslow Mirrored
From SimplyPsychology
You’ve heard of Maslow’s Hierarchy of Needs. It’s the go-to breakdown of human needs, broken down into five neat categories. The further down the pyramid you go, the more important and basic the need. In this exercise, I’m including an additional need that Maslow allegedly added near the end of his life: transcendence. This is the idea that, as humans, we seek to help others find self-actualization and feel the need to look beyond ourselves for purpose.
We can use Maslow’s Hierarchy to analyze how various products or features can affect users. I don’t believe that every feature is either good or bad. Most of the time, product teams would advocate how the features they are building are good and ethically sound. But as we have seen, there often is a flip side to every feature.
In this exercise, team members brainstorm the positive aspects of their product/feature as they relate to each tier of Maslow’s Hierarchy on the right side of the model. This is the easy part. Then, they write the negative aspects of their product/feature as they relate to each tier of Maslow’s Hierarchy on the left side of the model.
Maslow Mirrored applied to Instagram
360 Review
Much like the tool utilized by companies to evaluate their employees and managers, we can apply that type of systems thinking by taking your typical stakeholder map and expanding it to include shareholders and those that don’t fit in the first two categories. This will provide a comprehensive view of every entity that is affected by the ideation, creation, shipping, and scaling of the product. As you can surmise, this process is hectic due to the fact that most systems such as companies are nonlinear, which is why your end result for this exercise will likely look like a messy web. Donella Meadows has written a fantastic primer on systems thinking.
A system is a set of things — people, cells, molecules, or whatever — interconnected in such a way that they produce their own pattern of behavior over time. The system may be buffeted, constricted, triggered, or driven by outside forces. But the system’s response to these forces is characteristic of itself, and that response is seldom simple in the real world.
-Donella Meadows in “Thinking in Systems”
For a really great resource on systems mapping, check out Leyla Acaroglu’s article, Tools for Systems Thinkers: Systems Mapping. When you are compiling the 360 review, reflect on how each node (entity in the system) might be affected by the product. Again, consider how they might be affected at each stage of the product. Think about effects in terms of emotional, physical, monetary, safety, etc. Also pay attention to the various power dynamics between certain entities. For example, if you’re factoring in your company’s employees as well as management, what is the relationship between the two like? How might they influence each other as well as the product? If there’s a strained power dynamic between two nodes, draw the connecting line in red. If there’s a mutually beneficial power dynamic between two nodes, draw the connecting line in green. Revisit the red-line relationships and analyze what creates the existing and/or potential tension.
360 Review applied to Airbnb
Dichotomy Mapping
In this exercise, list all of the positive aspects of your product’s feature and then try to see what can happen if the feature is implemented to the extreme. Make sure this exercise is grounded on human needs. For example, Facebook intended for its engagement ranked newsfeed to show you the most relevant content from your friends and family…instead, it ended up showing the most outlandish and “clickbait-y” content and ended up fueling the fake news problem on its platform.
This is similar to the Maslow Mirrored exercise, but intentionally focuses in on specific features that your team is trying to implement for a product. Try to zoom in and figure out detailed variations of what beneficial and/or harmful results may occur from a product. Consider how those results might affect a single user as well as a group of users.
Afterwards, cluster the aspects that you have listed by whether they are macro or micro. Macro means that they can be applied on a wide scope; an example of this for Facebook’s newsfeed is that it’s algorithm created echo chambers and thus contributed to the polarization of our politics. Micro means that they are more focused on the individual; an example is that Facebook’s newsfeed might lead someone to feel more depressed and anxious because of the content that they’re seeing. In the below example, you’ll see that you can also start clustering these aspects as you’re writing them down; micro and macro aspects are presented on different colored sticky notes.
Dichotomy Mapping applied to IBM Watson
BJ Fogg’s Inverted Behavior Model
If you haven’t checked out BJ Fogg’s work on behavior design, do it. What I find the most fascinating in his research is his “formula” for behavior.
“Behavior (B) happens when Motivation (M), Ability (A), and a Prompt (P) come together at the same moment.”
If you’ve ever tried applying this formula to design specifically for an intended behavior, you’ll know that it is pretty fun and extremely powerful. Essentially, Fogg is advocating that an intended behavior will only happen if there is enough motivation from the users, if their ability to accomplish the behavior is sufficient, and if there is a prompt for them to do so. When applying it to forecasting, we can invert it so instead of trying to design for an intended behavior, we can leverage the formula to predict any unknown or unintended behaviors, and then forecast what consequences that behavior might trigger.
Start out listing out the prompts for which you want to forecast. These can be features or they can be holistic products. Then write out the motivating factors that might come into play with that prompt. With those two categories, try predicting some behaviors that might result. After, try to analyze and see what consequences might result from those behaviors. You can also calculate the ability levels for those behaviors to happen, and rank them from most likely to occur to least likely to occur. You can also play with ranking by the severity of the behaviors you’ve generated.
Inverted behavior model applied to Facebook
Monitoring Checklist
Monitoring should occur consistently after your team has shipped its product. For monitoring, we can we refer back to the three essential qualities for an ethical product or experience: autonomy, transparency, and safety. Much like the aforementioned considerations for evaluating problem statements, this list is not immutable and should be expanded and modified as time passes. Sit down with your team and ask yourselves these questions from the checklist:
A Note
This process can seem daunting. Intense. It’s not easy, that’s for sure. However, I think, as designers, we know that something has to change in the way companies create products and experiences for our users. Designing ethical products from the get-go ensures business success in the long-run. If we have the capability of taking any preventative measure so that “unforeseen consequences” can be avoided, then we should take those steps. And if we can advocate ethically vigilant practices in our teams and companies, then we should speak up. After all, it’s rare to regret double-checking, but extremely common to regret not double-checking.
Thanks for reading! Check out for the full framework, toolkit, and more. I would love to hear your thoughts on ethical design in tech, either via the site or in the comments below or @katherinemzhou or @ethicallydesign on Twitter.
I am incredibly lucky to be surrounded by so many inspiring people who not only have been so generous with their time but also believe in advocating for more ethical design in tech. This framework would not be possible without the following:
Nathaniel Axios, Sarah Brooks, Nicole Coumes, Zoe Fuller-Young, Jordan Higgins, Barry Lam, Twesha Mitra, Anna Miyajima, Steph Salileng, Francesca Tang, Melissa Wu, Tanya Xu, Michelle Ye, and my family.
