{"id":7002,"date":"2017-03-03T13:18:46","date_gmt":"2017-03-03T13:18:46","guid":{"rendered":"https:\/\/marvel7077.wpengine.com\/?p=7002"},"modified":"2022-03-25T07:40:59","modified_gmt":"2022-03-25T07:40:59","slug":"testing-ai-concepts-user-research","status":"publish","type":"post","link":"https:\/\/marvelapp.com\/blog\/testing-ai-concepts-user-research\/","title":{"rendered":"Testing AI Concepts in User\u00a0Research"},"content":{"rendered":"<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">It is clear that intelligent algorithms such as deep learning, machine learning, artificial intelligence, etc. will have a major impact on the world. However, what isn\u2019t clear is how these technologies can be used to solve real problems for people.<\/p>\n<div id=\"attachment_7003\" style=\"width: 810px\" class=\"wp-caption alignnone\"><a class=\"link link--blue fontWeight-4\"href=\"https:\/\/marvel7077.wpengine.com\/wp-content\/uploads\/2017\/03\/16sNLNSXCgYJwNPzzUg4GpA.png\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-7003\" class=\"wp-image-7003 size-full\" src=\"https:\/\/marvel7077.wpengine.com\/wp-content\/uploads\/2017\/03\/16sNLNSXCgYJwNPzzUg4GpA.png\" alt=\"16sNLNSXCgYJwNPzzUg4GpA\" width=\"800\" height=\"286\" srcset=\"https:\/\/marvelapp.com\/wp-content\/uploads\/2017\/03\/16sNLNSXCgYJwNPzzUg4GpA.png 800w, https:\/\/marvelapp.com\/wp-content\/uploads\/2017\/03\/16sNLNSXCgYJwNPzzUg4GpA-600x215.png 600w, https:\/\/marvelapp.com\/wp-content\/uploads\/2017\/03\/16sNLNSXCgYJwNPzzUg4GpA-768x275.png 768w\" sizes=\"auto, (max-width: 800px) 100vw, 800px\" \/><\/a><p id=\"caption-attachment-7003\" class=\"wp-caption-text\">Patrick, your friendly robot helper, as designed by my amazing teammates at Philosophie, <a class=\"link link--blue fontWeight-4\"href=\"https:\/\/medium.com\/@pigstake\">Jamie<\/a> and <a class=\"link link--blue fontWeight-4\"href=\"https:\/\/medium.com\/@leahm\">Leah<\/a>.<\/p><\/div>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">At <a class=\"link link--blue fontWeight-4\"href=\"http:\/\/philosophie.is\">Philosophie<\/a> we work on experiences that help humans in their day-to-day tasks. We hope to join the benefits of intelligent algorithms with the things that humans do best: deal with complex and creative endeavors that solve their problems towards some purpose.<\/p>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">In this post we will take you through the methods we have used at Philosophie to validate how intelligent algorithms could be used to solve real world problems.<\/p>\n<h2 class=\"pageWrap pageWrap--s marginTop-xl marginBottom-l c-black lineHeight-xl fontSize-xl fontWeight-5 breakPointM-lineHeight-xxl breakPointM-fontSize-xxl\" style=\"margin-left: auto; margin-right: auto;\">Centaurs or\u00a0cyborgs?<\/h2>\n<div id=\"attachment_7004\" style=\"width: 640px\" class=\"wp-caption alignnone\"><a class=\"link link--blue fontWeight-4\"href=\"https:\/\/marvel7077.wpengine.com\/wp-content\/uploads\/2017\/03\/1JhszxGHhre1zLtlGwFamcg.jpeg\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-7004\" class=\"size-full wp-image-7004\" src=\"https:\/\/marvel7077.wpengine.com\/wp-content\/uploads\/2017\/03\/1JhszxGHhre1zLtlGwFamcg.jpeg\" alt=\"Scene from COMPUTER CHESS\" width=\"630\" height=\"418\" srcset=\"https:\/\/marvelapp.com\/wp-content\/uploads\/2017\/03\/1JhszxGHhre1zLtlGwFamcg.jpeg 630w, https:\/\/marvelapp.com\/wp-content\/uploads\/2017\/03\/1JhszxGHhre1zLtlGwFamcg-600x398.jpeg 600w\" sizes=\"auto, (max-width: 630px) 100vw, 630px\" \/><\/a><p id=\"caption-attachment-7004\" class=\"wp-caption-text\">Scene from <a class=\"link link--blue fontWeight-4\"href=\"https:\/\/www.o-cinema.org\/event\/computer-chess\/\">COMPUTER CHESS<\/a><\/p><\/div>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">Our inspiration comes from intelligent algorithms being used in the domain of chess, one of the first \u2018hard\u2019 games to be conquered by deep learning. After Garry Kasparov was beaten by Deep Blue he was inspired to start a new form of chess that combines humans and computers together called Advanced Chess. It is also referred to as centaur, advanced, and cyborg chess.<\/p>\n<section class=\"pageWrap pageWrap--s marginBottom-l boxShadow-smoke boxShadow-smoke--xd--hover borderRadius-m transitionDuration-m transitionTimingFunction-ease\"><a class=\"link link--blue fontWeight-4\"class=\"link link--blue fontWeight-4\" href=\"https:\/\/en.wikipedia.org\/wiki\/Advanced_Chess\"><span class=\"display-block padding-m paddingRight-xl position-relative\"><span class=\"display-block padding-xs paddingRight-m\"><span class=\"marginBottom-xs display-block c-slate lineHeight-l fontSize-l fontWeight-5\">Advanced Chess - Wikipedia<\/span><span class=\"c-slate lineHeight-l fontSize-l fontWeight-3 opacity-80\">Advanced Chess is a relatively new form of chess, wherein each human player uses a computer chess program to help him\u2026<\/span><\/span><\/span><\/a><\/section>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">As in the centaur model, we look to the intelligent algorithms as assistants, helpers, and to help the humans they work with achieve the human\u2019s purposes. Not the other way around. It is much like a partnership since the algorithms need to learn from the human counterpart as well.<\/p>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">In this article I try to avoid talking about particular technologies or techniques because people don\u2019t care what you use as long as it helps solve a problem. Generally the systems we are considering use <a class=\"link link--blue fontWeight-4\"href=\"http:\/\/stats.stackexchange.com\/questions\/517\/unsupervised-supervised-and-semi-supervised-learning\">supervised or semi-supervised learning<\/a> techniques, but it could really be anything (<a class=\"link link--blue fontWeight-4\"href=\"https:\/\/blog.waya.ai\/introduction-to-gans-a-boxing-match-b-w-neural-nets-b4e5319cc935\">GANs are pretty cool right now<\/a>).<\/p>\n<h2 class=\"pageWrap pageWrap--s marginTop-xl marginBottom-l c-black lineHeight-xl fontSize-xl fontWeight-5 breakPointM-lineHeight-xxl breakPointM-fontSize-xxl\" style=\"margin-left: auto; margin-right: auto;\">Always get started with a\u00a0problem<\/h2>\n<blockquote class=\"pageWrap pageWrap--s position-relative marginTopBottom-l breakPointM-marginTopBottom-xl\"><div class=\"blog-quote-before position-absolute bg-marvel\"><\/div><div class=\"tweet-quote blog-quote-after position-absolute bg-marvel cursor-pointer transitionDuration-l transitionProperty-all transitionTimingFunction-cv-easeOutCircular scaleUp--hover zi-weak\"><svg class=\"fill-white opacity-0 pointerEvents-none position-absolute pinCenter transitionProperty-all transitionTimingFunction-easeInOut\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"24\" height=\"20\" viewBox=\"0 0 24 20\"><path d=\"M24,2.37a9.64,9.64,0,0,1-2.83.79A5,5,0,0,0,23.34.37a9.72,9.72,0,0,1-3.13,1.23A4.86,4.86,0,0,0,16.62,0a5,5,0,0,0-4.8,6.2A13.87,13.87,0,0,1,1.67.92,5.13,5.13,0,0,0,3.19,7.67,4.81,4.81,0,0,1,1,7a5,5,0,0,0,3.95,5,4.82,4.82,0,0,1-2.22.09,4.94,4.94,0,0,0,4.6,3.51A9.72,9.72,0,0,1,0,17.73,13.69,13.69,0,0,0,7.55,20c9.14,0,14.31-7.92,14-15A10.17,10.17,0,0,0,24,2.37Z\"\/><\/svg><\/div><p class=\"blog-quote position-relative textAlign-center c-marvel\"><span class=\"blog-quote-text transitionDuration-l transitionProperty-all transitionTimingFunction-easeInOut\">\"First, as with any endeavor, you need to have a real problem that you are trying to solve.\"<\/p><\/span><\/blockquote>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">At this stage of intelligent algorithms, you should be targeting the parts of a problem that are busywork. By busywork, I mean those aspects that feel like routine, are easily documented (say in a manual), or are a heuristic commonly used.<\/p>\n<h2 class=\"pageWrap pageWrap--s marginTop-xl marginBottom-l c-black lineHeight-xl fontSize-xl fontWeight-5 breakPointM-lineHeight-xxl breakPointM-fontSize-xxl\" style=\"margin-left: auto; margin-right: auto;\">Ideation without an AI\u00a0PhD<\/h2>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">Once you have identified an important problem to solve, you need to ideate solutions with intelligent algorithms in mind. We love to do <a class=\"link link--blue fontWeight-4\"href=\"https:\/\/uxdesign.cc\/honing-crazy-eights-with-randomness-1c8100b94824\">Crazy Eights when ideating<\/a> and this is a perfect time to do so.<\/p>\n<div id=\"attachment_7005\" style=\"width: 810px\" class=\"wp-caption alignnone\"><a class=\"link link--blue fontWeight-4\"href=\"https:\/\/marvel7077.wpengine.com\/wp-content\/uploads\/2017\/03\/1w3Wu9PvecR2drr1tegqPeQ.jpeg\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-7005\" class=\"wp-image-7005 size-full\" src=\"https:\/\/marvel7077.wpengine.com\/wp-content\/uploads\/2017\/03\/1w3Wu9PvecR2drr1tegqPeQ.jpeg\" alt=\"1w3Wu9PvecR2drr1tegqPeQ\" width=\"800\" height=\"600\" srcset=\"https:\/\/marvelapp.com\/wp-content\/uploads\/2017\/03\/1w3Wu9PvecR2drr1tegqPeQ.jpeg 800w, https:\/\/marvelapp.com\/wp-content\/uploads\/2017\/03\/1w3Wu9PvecR2drr1tegqPeQ-600x450.jpeg 600w, https:\/\/marvelapp.com\/wp-content\/uploads\/2017\/03\/1w3Wu9PvecR2drr1tegqPeQ-768x576.jpeg 768w\" sizes=\"auto, (max-width: 800px) 100vw, 800px\" \/><\/a><p id=\"caption-attachment-7005\" class=\"wp-caption-text\">A Crazy Eights exercise sketch by yours\u00a0truly<\/p><\/div>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">To be clear, we don\u2019t assume that in this process you are building anything yet. We are just looking for ideas that an intelligent algorithm acting like another human helping the person. When sketching (and later prototyping) we will just assume some intelligent algorithm \u2018magic\u2019 happens behind the scenes.<\/p>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">There are a few questions you should ask yourself when ideating for intelligent algorithms:<\/p>\n<ul class=\"pageWrap pageWrap--s list list--unordered marginBottom-l lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">\n<li>How would someone else help a person solve this problem?<\/li>\n<li>What information would they need to solve the problem themselves?<\/li>\n<li>How will the system describe what it is doing?<\/li>\n<li>How will the system build trust with the human operator over time?<\/li>\n<li>How will the human be able to approve or intervene?<\/li>\n<li>How will feedback be given to the system?<\/li>\n<\/ul>\n<h2 class=\"pageWrap pageWrap--s marginTop-xl marginBottom-l c-black lineHeight-xl fontSize-xl fontWeight-5 breakPointM-lineHeight-xxl breakPointM-fontSize-xxl\" style=\"margin-left: auto; margin-right: auto;\">Prototyping the algorithms<\/h2>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\"><span class=\"long-quote\">\"Building prototypes (coded or otherwise) with intelligent algorithm support is no different than hiding business or application logic behind an experience.\"<\/span><\/p>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">While designing the prototype you should be asking the same questions as during ideation: how will the system describe what it is doing, how will the human be able to approve or intervene, and how will feedback be given to the system?<\/p>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">A pattern we have used and seen is fake loading and \u2018calculating\u2019 experiences. They can help convey the work going into the suggestion, recommendation or decision to start creating trust with the person using it. This is similar to OpenTable\u2019s old \u2018looking for a table\u2019 or TurboTax \u2018checking for all possible tax breaks\u2019:<\/p>\n<section class=\"pageWrap pageWrap--s marginBottom-l boxShadow-smoke boxShadow-smoke--xd--hover borderRadius-m transitionDuration-m transitionTimingFunction-ease\"><a class=\"link link--blue fontWeight-4\"class=\"link link--blue fontWeight-4\" href=\"https:\/\/www.theatlantic.com\/technology\/archive\/2017\/02\/why-some-apps-use-fake-progress-bars\/517233\"><span class=\"display-block padding-m paddingRight-xl position-relative\"><span class=\"display-block padding-xs paddingRight-m\"><span class=\"marginBottom-xs display-block c-slate lineHeight-l fontSize-l fontWeight-5\">Why Some Apps Use Fake Progress Bars<\/span><span class=\"c-slate lineHeight-l fontSize-l fontWeight-3 opacity-80\">Curiously, the case of the TurboTax animations is a departure from most of the deceptive practices Adar studied: Rather\u2026<\/span><\/span><\/span><\/a><\/section>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">We used this concept most recently we were researching how people would work with an intelligent finding the right person to work on a job. Even though \u2018Patrick\u2019 would have already calculated the options we walk the person through the criteria he is using so they understand.<\/p>\n<div id=\"attachment_7006\" style=\"width: 810px\" class=\"wp-caption alignnone\"><a class=\"link link--blue fontWeight-4\"href=\"https:\/\/marvel7077.wpengine.com\/wp-content\/uploads\/2017\/03\/1Q-tjuQ7vjha2ij3psfpP6w.png\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-7006\" class=\"size-full wp-image-7006\" src=\"https:\/\/marvel7077.wpengine.com\/wp-content\/uploads\/2017\/03\/1Q-tjuQ7vjha2ij3psfpP6w.png\" alt=\"Patrick helping the scheduler find the perfect person to work on the job as part of a timeline.\" width=\"800\" height=\"487\" srcset=\"https:\/\/marvelapp.com\/wp-content\/uploads\/2017\/03\/1Q-tjuQ7vjha2ij3psfpP6w.png 800w, https:\/\/marvelapp.com\/wp-content\/uploads\/2017\/03\/1Q-tjuQ7vjha2ij3psfpP6w-600x365.png 600w, https:\/\/marvelapp.com\/wp-content\/uploads\/2017\/03\/1Q-tjuQ7vjha2ij3psfpP6w-768x468.png 768w\" sizes=\"auto, (max-width: 800px) 100vw, 800px\" \/><\/a><p id=\"caption-attachment-7006\" class=\"wp-caption-text\">Patrick helping the scheduler find the perfect person to work on the job as part of a timeline.<\/p><\/div>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\"><\/p>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">Also, you shouldn\u2019t ignore the way that a human will give feedback to the intelligent algorithms (aka supervised learning). If the human has to step in then we need to get the feedback to make the algorithm better. This is generally a \u2018why did you make that choice\u2019 type of question to the person using it.<\/p>\n<div id=\"attachment_7007\" style=\"width: 810px\" class=\"wp-caption alignnone\"><a class=\"link link--blue fontWeight-4\"href=\"https:\/\/marvel7077.wpengine.com\/wp-content\/uploads\/2017\/03\/1KrGOym28IEfaY_uKgSuOYg.png\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-7007\" class=\"size-full wp-image-7007\" src=\"https:\/\/marvel7077.wpengine.com\/wp-content\/uploads\/2017\/03\/1KrGOym28IEfaY_uKgSuOYg.png\" alt=\"Robots love getting feedback! \" width=\"800\" height=\"572\" srcset=\"https:\/\/marvelapp.com\/wp-content\/uploads\/2017\/03\/1KrGOym28IEfaY_uKgSuOYg.png 800w, https:\/\/marvelapp.com\/wp-content\/uploads\/2017\/03\/1KrGOym28IEfaY_uKgSuOYg-600x429.png 600w, https:\/\/marvelapp.com\/wp-content\/uploads\/2017\/03\/1KrGOym28IEfaY_uKgSuOYg-768x549.png 768w\" sizes=\"auto, (max-width: 800px) 100vw, 800px\" \/><\/a><p id=\"caption-attachment-7007\" class=\"wp-caption-text\">Robots love getting feedback!<\/p><\/div>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">In our case, we wanted to get feedback if the recommendation that Patrick gave was not used.<\/p>\n<h2 class=\"pageWrap pageWrap--s marginTop-xl marginBottom-l c-black lineHeight-xl fontSize-xl fontWeight-5 breakPointM-lineHeight-xxl breakPointM-fontSize-xxl\" style=\"margin-left: auto; margin-right: auto;\">Research with\u00a0humans<\/h2>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">Once the prototype is ready, it is time to put it front of someone to see how they perceive the intelligent system working.<\/p>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\"><span class=\"long-quote\">\"When using intelligent algorithms you want to know: 1) if the help they are giving is useful to the problem the person is trying to solve, 2) do they trust the information, and 3) do they feel comfortable giving feedback to the system.\"<\/span><\/p>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">Like any other test plan you will be focusing on the prototype, but it is important to check in on a few key questions as they go through the experience:<\/p>\n<ul class=\"pageWrap pageWrap--s list list--unordered marginBottom-l lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">\n<li>Think back to the last time you did this, how did you come to that decision?<\/li>\n<li>Do you trust these suggestions for what to do next?<\/li>\n<li>How do you think the system decided [action]?<\/li>\n<li>Was there enough information for you to [take action]?<\/li>\n<li>How much do you trust the system to make the right decision in the future? It is more or less than before?<\/li>\n<\/ul>\n<h2 class=\"pageWrap pageWrap--s marginTop-xl marginBottom-l c-black lineHeight-xl fontSize-xl fontWeight-5 breakPointM-lineHeight-xxl breakPointM-fontSize-xxl\" style=\"margin-left: auto; margin-right: auto;\">Synthesis<\/h2>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">In our most recent process we had been writing observations on post-its and took some time affinitizing them in different ways.<\/p>\n<div id=\"attachment_7008\" style=\"width: 810px\" class=\"wp-caption alignnone\"><a class=\"link link--blue fontWeight-4\"href=\"https:\/\/marvel7077.wpengine.com\/wp-content\/uploads\/2017\/03\/1G7YQAbOnwyg1ZOZFUfpTUQ.jpeg\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-7008\" class=\"size-full wp-image-7008\" src=\"https:\/\/marvel7077.wpengine.com\/wp-content\/uploads\/2017\/03\/1G7YQAbOnwyg1ZOZFUfpTUQ.jpeg\" alt=\"Affinitized, but\u00a0helpful?\" width=\"800\" height=\"600\" srcset=\"https:\/\/marvelapp.com\/wp-content\/uploads\/2017\/03\/1G7YQAbOnwyg1ZOZFUfpTUQ.jpeg 800w, https:\/\/marvelapp.com\/wp-content\/uploads\/2017\/03\/1G7YQAbOnwyg1ZOZFUfpTUQ-600x450.jpeg 600w, https:\/\/marvelapp.com\/wp-content\/uploads\/2017\/03\/1G7YQAbOnwyg1ZOZFUfpTUQ-768x576.jpeg 768w\" sizes=\"auto, (max-width: 800px) 100vw, 800px\" \/><\/a><p id=\"caption-attachment-7008\" class=\"wp-caption-text\">Affinitized, but\u00a0helpful?<\/p><\/div>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\"><\/p>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">Affinitizing is nice and comfortable, but it doesn\u2019t really help with understanding how people feel about using the intelligent algorithms in their work. Luckily, <a class=\"link link--blue fontWeight-4\"href=\"https:\/\/medium.com\/@leahm\">Leah<\/a>, my awesome designer teammate on the Philosophie team, came up with a great way to understand the intelligent-algorithm-specific observations: put them on a 2x2 for understanding vs. confusion and positive vs. negative sentiment.<\/p>\n<div id=\"attachment_7009\" style=\"width: 810px\" class=\"wp-caption alignnone\"><a class=\"link link--blue fontWeight-4\"href=\"https:\/\/marvel7077.wpengine.com\/wp-content\/uploads\/2017\/03\/1Zig68GnDIijOJZaO1DPWRg.jpeg\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-7009\" class=\"size-full wp-image-7009\" src=\"https:\/\/marvel7077.wpengine.com\/wp-content\/uploads\/2017\/03\/1Zig68GnDIijOJZaO1DPWRg.jpeg\" alt=\"Our new\u00a02x2\" width=\"800\" height=\"600\" srcset=\"https:\/\/marvelapp.com\/wp-content\/uploads\/2017\/03\/1Zig68GnDIijOJZaO1DPWRg.jpeg 800w, https:\/\/marvelapp.com\/wp-content\/uploads\/2017\/03\/1Zig68GnDIijOJZaO1DPWRg-600x450.jpeg 600w, https:\/\/marvelapp.com\/wp-content\/uploads\/2017\/03\/1Zig68GnDIijOJZaO1DPWRg-768x576.jpeg 768w\" sizes=\"auto, (max-width: 800px) 100vw, 800px\" \/><\/a><p id=\"caption-attachment-7009\" class=\"wp-caption-text\">Our new\u00a02x2<\/p><\/div>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\"><\/p>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">When looking at the observations this way we found a few different groupings:<\/p>\n<ul class=\"pageWrap pageWrap--s list list--unordered marginBottom-l lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">\n<li>Positive and understood \u2014 validation that the system\u2019s autonomy is helpful<\/li>\n<li>Negative and understood \u2014 concerns about the system taking over their jobs or some distrust of execution<\/li>\n<li>Negative\/positive confusion \u2014 issues to address in the UX or via change management of the organization<\/li>\n<\/ul>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">If you look at the discussions happening about intelligent algorithms, such as self-driving cars, it mimics the set of general topics.<\/p>\n<h2 class=\"pageWrap pageWrap--s marginTop-xl marginBottom-l c-black lineHeight-xl fontSize-xl fontWeight-5 breakPointM-lineHeight-xxl breakPointM-fontSize-xxl\" style=\"margin-left: auto; margin-right: auto;\">Dystopian possibilities<\/h2>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\"><a class=\"link link--blue fontWeight-4\"href=\"https:\/\/marvel7077.wpengine.com\/wp-content\/uploads\/2017\/03\/1VEZdHO-vfqbr-LgxnAeIgA.jpeg\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-7010\" src=\"https:\/\/marvel7077.wpengine.com\/wp-content\/uploads\/2017\/03\/1VEZdHO-vfqbr-LgxnAeIgA.jpeg\" alt=\"1VEZdHO-vfqbr-LgxnAeIgA\" width=\"800\" height=\"450\" srcset=\"https:\/\/marvelapp.com\/wp-content\/uploads\/2017\/03\/1VEZdHO-vfqbr-LgxnAeIgA.jpeg 800w, https:\/\/marvelapp.com\/wp-content\/uploads\/2017\/03\/1VEZdHO-vfqbr-LgxnAeIgA-600x338.jpeg 600w, https:\/\/marvelapp.com\/wp-content\/uploads\/2017\/03\/1VEZdHO-vfqbr-LgxnAeIgA-768x432.jpeg 768w\" sizes=\"auto, (max-width: 800px) 100vw, 800px\" \/><\/a><\/p>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\"><\/p>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">As we saw in the negative\/understood quadrant of the synthesis, people are worried about how intelligent algorithms impact their lives. This isn\u2019t just that they may lose their jobs, but that when a decision is made by an algorithm there may be no chance to understand why.<\/p>\n<blockquote class=\"pageWrap pageWrap--s position-relative marginTopBottom-l breakPointM-marginTopBottom-xl\"><div class=\"blog-quote-before position-absolute bg-marvel\"><\/div><div class=\"tweet-quote blog-quote-after position-absolute bg-marvel cursor-pointer transitionDuration-l transitionProperty-all transitionTimingFunction-cv-easeOutCircular scaleUp--hover zi-weak\"><svg class=\"fill-white opacity-0 pointerEvents-none position-absolute pinCenter transitionProperty-all transitionTimingFunction-easeInOut\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"24\" height=\"20\" viewBox=\"0 0 24 20\"><path d=\"M24,2.37a9.64,9.64,0,0,1-2.83.79A5,5,0,0,0,23.34.37a9.72,9.72,0,0,1-3.13,1.23A4.86,4.86,0,0,0,16.62,0a5,5,0,0,0-4.8,6.2A13.87,13.87,0,0,1,1.67.92,5.13,5.13,0,0,0,3.19,7.67,4.81,4.81,0,0,1,1,7a5,5,0,0,0,3.95,5,4.82,4.82,0,0,1-2.22.09,4.94,4.94,0,0,0,4.6,3.51A9.72,9.72,0,0,1,0,17.73,13.69,13.69,0,0,0,7.55,20c9.14,0,14.31-7.92,14-15A10.17,10.17,0,0,0,24,2.37Z\"\/><\/svg><\/div><p class=\"blog-quote position-relative textAlign-center c-marvel\"><span class=\"blog-quote-text transitionDuration-l transitionProperty-all transitionTimingFunction-easeInOut\">\"The ethics of why we are creating systems still matters.\"<\/p><\/span><\/blockquote>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">The ethics of why we are creating systems still matters. People\u2019s purposes are what drive the meaning of technology. Creating a person-less bureaucracy isn\u2019t better than what we have now no matter the amount of additional efficiency we get.<\/p>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">A recent article on Real Life was a great piece on how intelligent algorithms when used incorrectly can just create a bureaucracy with no people:<\/p>\n<section class=\"pageWrap pageWrap--s marginBottom-l boxShadow-smoke boxShadow-smoke--xd--hover borderRadius-m transitionDuration-m transitionTimingFunction-ease\"><a class=\"link link--blue fontWeight-4\"class=\"link link--blue fontWeight-4\" href=\"http:\/\/reallifemag.com\/rule-by-nobody\"><span class=\"display-block padding-m paddingRight-xl position-relative\"><span class=\"display-block padding-xs paddingRight-m\"><span class=\"marginBottom-xs display-block c-slate lineHeight-l fontSize-l fontWeight-5\">Rule by Nobody - Real Life<\/span><span class=\"c-slate lineHeight-l fontSize-l fontWeight-3 opacity-80\">The compensation for a death sentence is knowledge of the exact hour when one is to die. -Cincinnatus C., Invitation to\u2026<\/span><\/span><\/span><\/a><\/section>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">A short ebook by Marshall Brain speaks to the possible-post jobs world if you want to go further out:<\/p>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\"><a class=\"link link--blue fontWeight-4\"href=\"http:\/\/www.amazon.com\/Manna-Two-Visions-Humanitys-Future-ebook\/dp\/B007HQH67U\" target=\"_BLANK\" rel=\"noopener noreferrer\"><span><img loading=\"lazy\" decoding=\"async\" class=\"centered\" src=\"https:\/\/marvel7077.wpengine.com\/wp-content\/uploads\/2017\/03\/1Ox6GScNQxJUhtgnE2dcCUQ.png\" alt=\"1Ox6GScNQxJUhtgnE2dcCUQ\" width=\"200\" height=\"319\" \/><\/span><\/a><\/p>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">I don\u2019t have the answers on what this means, but as creators of these new technologies we need to try to understand and do what is best for all of us.<\/p>\n<h2 class=\"pageWrap pageWrap--s marginTop-xl marginBottom-l c-black lineHeight-xl fontSize-xl fontWeight-5 breakPointM-lineHeight-xxl breakPointM-fontSize-xxl\" style=\"margin-left: auto; margin-right: auto;\">What\u2019s next?<\/h2>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">There is a lot of focus on techniques to create intelligent algorithms technically, but we need people thinking about how they help people. We have outlined some key questions when doing this research before you build.<\/p>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">In the coming months we will post more on how to include these new technologies in products and experience patterns we are starting to see.<\/p>\n<p class=\"pageWrap pageWrap--s marginBottom-m paddingBottom-s c-slate lineHeight-l fontSize-l fontWeight-3 breakPointM-fontSize-xl breakPointM-lineHeight-xl\">Check out [Ballpark](https:\/\/ballparkhq.com\/?utm_campaign=Ballpark%20SEO&utm_source=marvelblog \"Ballpark\") for inclusive research tools for every team.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>It is clear that intelligent algorithms such as deep learning, machine learning, artificial intelligence, etc. will have a major impact on the world. However, what isn\u2019t clear is how these technologies can be used to solve real problems for people. At Philosophie we work on experiences that help humans in their day-to-day tasks. We hope to join the benefits of&#8230; <a class=\"link link--blue fontWeight-4\" href=\"https:\/\/marvelapp.com\/blog\/testing-ai-concepts-user-research\/\">Read More &#65515;<\/a><\/p>\n","protected":false},"author":92,"featured_media":7005,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[378],"tags":[],"class_list":["post-7002","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-research"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v15.5 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/marvelapp.com\/blog\/testing-ai-concepts-user-research\/\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Testing AI Concepts in User\u00a0Research | Marvel Blog\" \/>\n<meta property=\"og:description\" content=\"It is clear that intelligent algorithms such as deep learning, machine learning, artificial intelligence, etc. will have a major impact on the world. However, what isn\u2019t clear is how these technologies can be used to solve real problems for people. At Philosophie we work on experiences that help humans in their day-to-day tasks. We hope to join the benefits of... Read More &#65515;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/marvelapp.com\/blog\/testing-ai-concepts-user-research\/\" \/>\n<meta property=\"og:site_name\" content=\"Marvel Blog\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/marvelapp\" \/>\n<meta property=\"article:published_time\" content=\"2017-03-03T13:18:46+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-03-25T07:40:59+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/marvelapp.com\/wp-content\/uploads\/2017\/03\/1w3Wu9PvecR2drr1tegqPeQ.jpeg\" \/>\n\t<meta property=\"og:image:width\" content=\"800\" \/>\n\t<meta property=\"og:image:height\" content=\"600\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@marvelapp\" \/>\n<meta name=\"twitter:site\" content=\"@marvelapp\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\">\n\t<meta name=\"twitter:data1\" content=\"6 minutes\">\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https:\/\/marvelapp.com\/blog\/#organization\",\"name\":\"Marvel\",\"url\":\"https:\/\/marvelapp.com\/blog\/\",\"sameAs\":[\"https:\/\/www.facebook.com\/marvelapp\",\"https:\/\/www.instagram.com\/marvelapp\/\",\"https:\/\/www.linkedin.com\/company\/marvel-app\/\",\"https:\/\/twitter.com\/marvelapp\"],\"logo\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/marvelapp.com\/blog\/#logo\",\"inLanguage\":\"en-GB\",\"url\":\"https:\/\/marvelapp.com\/wp-content\/uploads\/2018\/06\/Logo-Light.png\",\"width\":1605,\"height\":1130,\"caption\":\"Marvel\"},\"image\":{\"@id\":\"https:\/\/marvelapp.com\/blog\/#logo\"}},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/marvelapp.com\/blog\/#website\",\"url\":\"https:\/\/marvelapp.com\/blog\/\",\"name\":\"Marvel Blog\",\"description\":\"Ideas and words on user experience, design, collaboration and more\",\"publisher\":{\"@id\":\"https:\/\/marvelapp.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":\"https:\/\/marvelapp.com\/blog\/?s={search_term_string}\",\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-GB\"},{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/marvelapp.com\/blog\/testing-ai-concepts-user-research\/#primaryimage\",\"inLanguage\":\"en-GB\",\"url\":\"https:\/\/marvelapp.com\/wp-content\/uploads\/2017\/03\/1w3Wu9PvecR2drr1tegqPeQ.jpeg\",\"width\":800,\"height\":600},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/marvelapp.com\/blog\/testing-ai-concepts-user-research\/#webpage\",\"url\":\"https:\/\/marvelapp.com\/blog\/testing-ai-concepts-user-research\/\",\"name\":\"Testing AI Concepts in User\\u00a0Research | Marvel Blog\",\"isPartOf\":{\"@id\":\"https:\/\/marvelapp.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/marvelapp.com\/blog\/testing-ai-concepts-user-research\/#primaryimage\"},\"datePublished\":\"2017-03-03T13:18:46+00:00\",\"dateModified\":\"2022-03-25T07:40:59+00:00\",\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/marvelapp.com\/blog\/testing-ai-concepts-user-research\/\"]}]},{\"@type\":\"Article\",\"@id\":\"https:\/\/marvelapp.com\/blog\/testing-ai-concepts-user-research\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/marvelapp.com\/blog\/testing-ai-concepts-user-research\/#webpage\"},\"author\":{\"@id\":\"https:\/\/marvelapp.com\/blog\/#\/schema\/person\/0545c7aafa47e4d51ac73f539a372228\"},\"headline\":\"Testing AI Concepts in User\\u00a0Research\",\"datePublished\":\"2017-03-03T13:18:46+00:00\",\"dateModified\":\"2022-03-25T07:40:59+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/marvelapp.com\/blog\/testing-ai-concepts-user-research\/#webpage\"},\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/marvelapp.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/marvelapp.com\/blog\/testing-ai-concepts-user-research\/#primaryimage\"},\"articleSection\":\"Research\",\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/marvelapp.com\/blog\/testing-ai-concepts-user-research\/#respond\"]}]},{\"@type\":\"Person\",\"@id\":\"https:\/\/marvelapp.com\/blog\/#\/schema\/person\/0545c7aafa47e4d51ac73f539a372228\",\"name\":\"Chris Butler\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/marvelapp.com\/blog\/#personlogo\",\"inLanguage\":\"en-GB\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/82c26bfcfabc12476ae345abc1cbcc3c94acd4ffbaefe906bd316ce3690f2898?s=96&d=mm&r=g\",\"caption\":\"Chris Butler\"},\"description\":\"Chris Butler is Senior Product Strategist at Philosophie in NYC. He has over 15 years of experience working in product, business development, regular development (aka engineering), and tech evangelism for large to small companies (including founding one of his own). He has worked with Microsoft, Waze, Horizon Ventures, KAYAK, and more.\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","_links":{"self":[{"href":"https:\/\/marvelapp.com\/blog\/wp-json\/wp\/v2\/posts\/7002","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/marvelapp.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/marvelapp.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/marvelapp.com\/blog\/wp-json\/wp\/v2\/users\/92"}],"replies":[{"embeddable":true,"href":"https:\/\/marvelapp.com\/blog\/wp-json\/wp\/v2\/comments?post=7002"}],"version-history":[{"count":0,"href":"https:\/\/marvelapp.com\/blog\/wp-json\/wp\/v2\/posts\/7002\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/marvelapp.com\/blog\/wp-json\/wp\/v2\/media\/7005"}],"wp:attachment":[{"href":"https:\/\/marvelapp.com\/blog\/wp-json\/wp\/v2\/media?parent=7002"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/marvelapp.com\/blog\/wp-json\/wp\/v2\/categories?post=7002"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/marvelapp.com\/blog\/wp-json\/wp\/v2\/tags?post=7002"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}