Companies can no longer afford to ignore the ethical ramifications of their products. Here’s how designers can pave the way for ethical decision-making in tech.
This is the first part of a two part series. Check out Designing Ethically Pt. 2 for a deeper dive into the nitty-gritty of evaluating problem statements, forecasting “unforeseen consequences,” and monitoring your product’s ethical progress.
For an alternative to this 13-minute read, check out the site for a more digestible version of the framework, toolkit, and more!
Today, it seems like there is a tech scandal every week. Uber, Facebook, Twitter, Google, Snapchat, and others have been making the rounds in the media for their wrongdoing. Now don’t get me wrong — scandals are by no means a new phenomenon for companies. However, with the pervasive power of social media (quite ironic in some cases), these tech giants are starting to get called out for their lapses in judgment.
The days of reckless tech need to come to an end. I’m no Luddite by any means, but I do think that taking time to consider the ethical consequences behind decisions is necessary. There’s just too much at stake today with the complexity and reach of our accelerating, technological advances. There’s also the fact that we often don’t know what we don’t know…until it’s too late.
For giggles…but also to get you into the “ethical design” mindset.
We are at a point where technology is advancing so quickly that governing bodies cannot keep up. Any regulation we see today seems to happen after the fact, when the damage has already been done. While “regulation” may sound scary to companies, it has its place in society for a reason. If the government cannot catch up, then companies themselves need to shoulder some ethical responsibility and regulate themselves.
Why should designers care though?
As designers, we have fought tooth and nail for a seat at the table. Some of us are still fighting the good fight. We are fighting because we realize the impact of our work. We are fighting because we know that we can exact measurable improvements. We are fighting because we believe that human-centered decision making is of utmost importance.
"And what is more human-centric than making ethical choices that will ultimately help users and society at large?"
Excerpt from John Maeda’s Design in Tech Report 2018
Designers have the privilege (and burden) of wielding a lot more power than we ever had in the existence of our profession. Now and even more so in upcoming years, we will be designing products that have the ability to dramatically influence millions of users. In John Maeda’s Design in Tech Report 2018, we see that the important design decisions of the future will be focused around incredibly advanced technology that, depending on its application, can save countless lives or wreak havoc. Our unlicensed influence as designers has been compared to that of licensed doctors and lawyers. With that amount of power comes an enormous amount of responsibility as well.
With this framework, I will show you that ethical design is necessary, possible, and beneficial. The structure of this framework is as follows:
- Ethics 101 for Designers: Exactly what it sounds like, this section is a basic ethics primer for designers.
- Redesigned Design Process: This section introduces a new design process that encourages preemptive forecasting of ethical problems.
- What’s Good: In order to apply the new design process, you need to have a baseline understanding of what is ethically good vs. bad in a product.
- Sustaining Ethical Design: My thoughts on how to keep ethical design relevant and at the forefront for every company.
Ethics 101 for Designers
What is Ethics?
Ethics generally refers to an established set of standards that allows us to determine right from wrong. These standards give us the ability to figure out what we should and should not do. It is important to note that what is ethical does not always correspond with what is legal. After all, slavery was legal at one point…but that does not mean it was ever ethical.
Schools of Ethical Thought
There are several perspectives when it comes to deciding what is ethical. The two that I think are most applicable to the process of designing a product or experience are:
- Duty-based: Also referred to as Deontological ethics, this system asserts that motives matter more than the outcome when it comes to deeming the ‘goodness’ of such action. Philosophers like Immanuel Kant and Sir David Ross defined certain duties to uphold. In any case, a good way to think about duty-based ethics is that it is intent-oriented.
- Results-based: Also referred to as consequentialism, this system evaluates actions based on their outcome. An outcome that produces the greatest good for the greatest number of people is deemed good, and validates the action that produced it. A good way to think about this is that it is results-oriented.
The thing with philosophy is that oftentimes life is way too complicated to just stick with one ethical mindset. Most people tend to subscribe to a mixed model where they apply both duty-based and results-based ethics on a case-by-case basis.
And that’s exactly what we can do with how we design. When you think about it, the actual life cycle of a product has two dimensions. There’s the part where observations about society are made and a problem is identified (that’s the intent part). Then, there’s the iterative part where solutions are ideated and eventually shipped to the public (that’s the results part).
Redesigned Design Process
What happens if we apply this intent/results paradigm to our design process? What would that look like? The fact of the matter is that our current design processes do not actively encourage us to really consider the intent and results of what we create. And even if designers do have concerns about the two, oftentimes our voices are drowned out by the prioritized voices of shareholders. However, if we intentionally redesign our own process, perhaps we can enact more of an internally organic change and develop the wherewithal to defend our choices.
Stanford d.school’s Design Thinking Process
Today, we see many design frameworks and processes that have been successful in defining north stars and outlining methods for delineating problems and developing solutions. What these frameworks share in common (when applied by businesses) is that they are often built on a motivating foundation of making more profits.
Rarely do you see an ethical component factored into the equation. I would argue that it is imperative for anyone in a decision-making role to prioritize ethical considerations. Companies today are so large and unwieldy that there can be disastrous consequences if they do not validate the ethical consequences of their actions.
By applying the intent/results paradigm to the typical design process, we have a new, in-depth process that proactively seeks to 1) evaluate the ethical validity of the selected problem and 2) ensure that the designed solution is not ethically unsound.
You’ll see that there are a few new steps integrated into this model. I’ll go into a bit more detail on the new steps but as for the original steps, they can be found in Stanford d.school’s Design Thinking process.
Intents Phase — New Steps
Evaluate: This step is critical and involves analyzing the problem statement seeing it if it is even ethically worthy of being addressed. For example, if your company is trying to develop a tool to hurt thousands of people for the sake of advancing some agenda, you might want to heavily reconsider.
The thing is, most companies will have crafted a pitch for their product that seems ethically sound. That’s how they convince venture capitalists to invest and how they get users to jump on board. It’s normally in the results phase where things go haywire. In general, the intents phase is rarely the problematic part, but it always is good to have a system of checks and balances in case defining an ethical problem statement is not obvious.
Results Phase — New Steps
Forecast: Take a look at the ideas you’ve generated with your team. For each idea, figure out any ethical weak spots or any potential ways that your product can be exploited in an unethical way (either by users, the company itself, investors, etc.).
Ship: After you’ve tested your product thoroughly, send it out! This is by no means a novel step, but just serves to indicate that this framework looks beyond the scope of the traditional process.
Monitor: Almost as crucial as forecasting, monitoring is simply the act of continually checking in with your shipped product to see if any ethical problems occur so your team can quickly mobilize to solve them.
This framework is not meant to guarantee perfection. Mistakes will be made. But a significantly higher percentage of them will be caught before they can wreak havoc. The iterative nature of the design process as it is allows for designers to refine the way they evaluate, forecast, and monitor. Furthermore, this ethical design framework will introduce a necessary way of thinking twice (or thrice) before acting…something that the tech world seems to have forgotten.
What’s Good?
First things first, before you apply the design process, you have to know what is good and what is bad. On a fundamental level, designing ethically means ideating solutions with three key concepts in mind: autonomy, transparency, and safety. Together, these three qualities empower users and ensure for trustworthy products.
Autonomy
Autonomy is crucial. And by autonomy, I’m referring to user autonomy. We as humans like to feel like we are in control of our technology, and not the other way around. When we don’t feel that way, it can get incredibly frustrating. A desirable and good product gives users the ability to make deliberate choices about how they interact with the product.
Knowing that I have the option of tipping drivers on a ride-sharing app is a positive attribute for autonomy. Knowing that I can rate a bad driver under 3 stars (and receiving confirmation that this rating notifies the company about the driver) is also a positive attribute for autonomy.
Providing autonomy includes other features such as: allowing users to customize their experiences with settings, designing helpful “opt-in” and “opt-out” choices for users, avoiding dark UX tactics that take advantage of users, etc.
Note: in providing autonomy to one user, you must make sure that does not allow the user to infringe upon the autonomy of other users (or non-users).
Transparency
Transparency is just as important. Users have a right to know what they are signing up for when they agree to use a product. Oh, but that’s what the Terms & Conditions are for! Sometimes, that’s not enough.
For example, when using a complex product such as Facebook, I want to know how I can control my privacy settings. Currently, Facebook is undergoing major criticism for playing a huge role in the data breach involving Cambridge Analytica. In a different scandal, previously, there was the case of the college students who did not realize that by joining a queer choir group on Facebook, they were unintentionally ‘outing’ themselves to their homophobic families. Users should always have a clear understanding of what the product does and how it works.
Another example that is pretty timely would be the perpetuation of fake news on social media. Companies that spread misinformation need to seriously redesign their algorithms and make sure that they are flagging incorrect content.
Safety
Safety for the user is another incredibly essential quality. Safety includes many aspects: sanctity of life, inclusion, privacy, and emotional well-being.
For example, it was revealed that Instagram strategically withholds “likes” from certain users in an attempt to get them to feel disappointed about their photo’s popularity and check in more consistently with the app. Talk about emotional manipulation.
Another example of safety in regards to inclusion is Airbnb’s struggle with racism towards people of color on its platform. No person of color should be made to feel like they are excluded from a product; and no person using a product should be allowed to exclude others via said product.
In a more futuristic (and controversial) example, if someone purchases an autonomous vehicle, are they guaranteed to have their life protected and prioritized in the case of an accident? What about a non-user, like a pedestrian? There is more urgency than ever for figuring out an approach to this problem, given that just recently, one of Uber’s test cars killed a pedestrian.
Not sure if I’d want this adorable kid to design my future autonomous vehicle…
Sustaining Ethical Design
So how can we ensure that ethical design is not only preached, but actually practiced? There’s still a long way to go, but I’m optimistic that change will occur.
Redefine how we measure success.
In the Calvin and Hobbes comic above, Calvin notes, “In the real world, people care about success, not principles.” It’s easy to forget that ROIs and KPIs are not the only things that matter nowadays. Ironically, some of the very products that have been criticized for their ethical mishaps have been the conduits for a counterpart to monetary currency: social currency. Companies can no longer afford to ignore the ethical ramifications of their products. Remember the #DeleteUber scandal, when the ride-sharing company got put on blast and lost 200,000 users for undermining the protest against Trump’s Muslim Ban? How about the mess that Facebook has gotten itself into? Looking back, when would have been a good time for Zuckerberg to heed the warnings that perhaps, just perhaps, the social network had gone a bit too far?
Inclusive and diverse teams are imperative.
This is no longer a cherry on top, this is the goddamn sundae. Time and time again, studies have shown how having inclusive and diverse teams improve collective decision making. Having a variety of different thinkers who are all empowered to speak in the brainstorm session makes a world of a difference. Recently, it was shown that facial recognition systems were biased in that they identified white men significantly better than they identified women of color. I wonder what the teams that created those systems looked like. This could have huge implications as facial recognition is being used more and more in law enforcement; in a country that already brutalizes certain people of color, this could be disastrous. Furthermore, when conducting the empathize phase of your design process, it is critical to make sure that the teams conducting and contributing to the research are representative of the user demographics as well. Participatory design is a great way to bring the users into the researching realm, so as to offer authentic insight.
In the future, I hope that everyone on a product team undergoes ethics training.
Product teams need to have a designated ethicist.
I’m hoping that in the future, prioritizing and applying ethics will be second nature to anybody in any position. However, in the present, that simply is not the case. Our education system has not prepared us for this. Consider that just recently, a computer scientist at Harvard was asked about the ethical consequences of artificial intelligence and responded by shrugging and saying, “I’m just an engineer.” Given the rapid pace at which advanced technologies are developing and the vast consequences that they can have, “I am just an [insert position]” is no longer an acceptable excuse. If you are contributing to creating a product in any manner, you are responsible for its effects on others. At the end of the day, nobody is neutral and doing nothing is not neutral. Companies need to invest and make sure that they have ethicists on each product team and in the boardroom. These ethicists would ideally be divorced from any “bottom line” motivation or profit incentive. That gives them the freedom to zoom out, forecast, and monitor while imparting that type of mindset on their coworkers.
Ethical practices need to be emphasized and prioritized in the boardroom and C-suite.
This should be so obvious, but as we have seen, that is not always the case. Corporate social responsibility is not just a catchphrase; it needs to be implemented across all channels of a company. It’s not enough to talk the talk these days. Company executives need to maintain transparency around their products and show that they prioritize stakeholders (and non-stakeholders) just as much as they prioritize shareholders.
Pay it forward by applying this ethical design framework.
A designer once told me that he chooses to work for companies that prioritize users on an ethical level, which I think is a great and empowering mindset. However, if you are a junior designer and currently do not find yourself in a position where you can easily choose a more ethical employer, consider advocating to apply the methodologies outlined in this framework on your current team. It is never too early to strengthen and assert your voice as a designer. As designers, we have a head start when it comes to taking an ethical approach in our process. The vocabulary is already there for us. After all, this framework is simply a detailed exploration of using design thinking to create human-centric solutions.
Thanks for reading! Check out for the full framework, toolkit, and more. I would love to hear your thoughts on ethical design in tech, either via the site or in the comments below or @katherinemzhou or @ethicallydesign on Twitter.
I am incredibly lucky to be surrounded by so many inspiring people who not only have been so generous with their time but also believe in advocating for more ethical design in tech. This framework would not be possible without the following:
Nathaniel Axios, Sarah Brooks, Nicole Coumes, Zoe Fuller-Young, Jordan Higgins, Barry Lam, Twesha Mitra, Anna Miyajima, Steph Salileng, Francesca Tang, Melissa Wu, Tanya Xu, Michelle Ye, and my family.
This article was originally published on Kat's Medium page.
