Our last EthnoBorrel focused on the problems UX and qualitative researchers face in promoting how their research is used within their companies. It’s quite common to for these researchers to feel that their work is under-utilized, and many spend a great deal of time trying to educate their colleagues, build collaborations, and trying to get a seat at the executive table to get their insights heard.
But what about the people who work closely with researchers? How do they see research and its uses?
In this follow-up session we had a panel with professionals who are not researchers themselves, but who influence how research is implemented within a company: managers, non-researchers, and practicalities. The panelists were Kenny Baas-Schwegler (Domain-Driven Design consultant & trainer, Xebia), Puja Nanda (Senior Product Manager, Booking) and Gien Verschatse (Owner, EIGHT POINT SQUARED). The panel was moderated by Melanie T. Uy.
Melanie introduced the topic with reference to the above diagram. It shows the different levels of practice that affect a research culture.
In the innermost circle are the activities that come to mind when we think of research: methods, frameworks, and so on: the actual work that goes into designing and carrying out research.
In the middle circle are the results of research that are now shared with other people: the insights gained and attempts to turn it into products and services. People working in this ring include designers, product managers, and technologists.
The outermost circle involves all the other activities that affect how research is done, from workshops with clients to stakeholder engagement and wider company involvement. A wide range of employees practice in this circle, including managers and executives.
A company’s research culture takes place across all these circles. The porous borders, indicated by the dashed lines, indicate the collaborative nature of research across different levels.
This diagram reconfigures how we see research. It isn’t just something that researchers do, but rather it involves all kinds of professionals from across the company and even stakeholders from outside. Moreover, much of what researchers do isn’t confined to core research activities, but rather involves a great deal of collaboration and stakeholder engagement.
A healthy research culture is one in which there is engagement across the three circles, and interest in research and insights flows throughout the organization. The outcome is a high level of curiosity, not just of research per se but also about who the customers are and the mission of the company across departments.
What is your relationship to research?
Melanie used this diagram to open up the panel discussion. First she asked Puja to explain her relationship to research and what it means to her. Puja said that, in her experience at Booking, research is incredibly important because without it you can’t define a strategy or make a product and ship it.
Software developers have their own need and understanding of research. Their predilection is to find ways to build better software more efficiently and effectively to solve a problem.
Gien, who has been a software developer for twelve years, said that she became interested in research when she decided to do a Masters in computer science. She realised that the academic research she was reading could be incredibly useful in her day job as there are many conventional ways to do things but little empirical evidence to back them up. Unfortunately, it’s difficult to access because it’s often behind a paywall and isn’t written in a particularly accessible way.
Kenny, who also has a background in software engineering, said that he also wanted to apply evidence to his work so he went looking for research that could help him. He found that there is a lot of research, but not on how you build software. He did, however, eventually find some, and he said it helped him in his practice, but not so much in his role as an IT consultant.
What is research, anyway?
Melanie commented that it seems like we’re using the word ‘research’ differently, and she asked the panel members to define it.
Puja responded that her definition of research usually relates to product development. For every product we develop, how do you build a hypothesis around that and how do you build an understanding of who is the user? This process can involve various methodologies–qualitative and quantitative–but what’s critical is that identification of what problems you’re trying to solve, how you arrive at a hypothesis.
Kenny commented that research is rarely carried out in technology companies. Technology is often a feature factory, and research is rarely done. There is no connection with the user, and this frustrates developers. For him the key questions are how we can bring developers and users closer together so that engineers can build better software.
Gien said she also got into research to understand users better, and she realised that there are several kinds that can be applied. One is the academic research that she mentioned earlier, although much of it is difficult to apply to industry. Another is using prototypes and experiments, which you do see being done in software development.
Melanie responded that the experiences of Puja, Kenny and Gien are united by their focus on users. How, she asked, can we improve collaborations to make research happen?
Puja explained that in most of the organizations she’s worked in research was already very embedded and was carried out by different teams. It would include market research, usability studies and qualitative research. She said in her work she focuses on two kinds of research. One is strategic research to decide where to make investments and what directions the company should go. There is also tactical research, which is carried out when you already have an idea of what your market is like and where you want to go. That is related more to product development. Puja said it’s great if you can democratize research so that many people feel enabled to carry it out. She asked:
“How do we enable everyone in the teams to do their own research and make it part of their job? In doing strategic research, how do you ensure these are high quality pieces of work that can be made more granular so that everyone can take advantage of the insights?”
Gien responded that it’s interesting how Puja makes a distinction between strategic and tactical, because these concepts also exist in Domain-Driven Design (DDD). DDD encourages developers to get involved in the strategic aspect of their company, rather than just focusing on producing features, because software can strategically benefit your company. But to do this you need to know something about your company’s strategy: what are their goals for the next five years, how they make money, how should departments be organised, and so on. The business goals need to be embedded in the software, but often people working in IT departments don’t know this information.
Melanie asked Kenny if this was also his experience. Kenny responded that in his line of work he has been in close contact with users and so could see them react. He believes that software developers can help immensely to improve products but aren’t always listened to. He gave the example of a software developer at Amazon who suggested that people shopping online should be shown items that other people bought. His idea was initially rejected, but now it’s a standard part of online shopping. So a developer’s observations of users might not be formally considered ‘research’, but in fact they also generate valuable insights.
Does research add value or is it a cost?
Melanie commented that some companies see research as valuable, whereas others see it as a cost. Kenny replied that IT usually sees it as a cost because it takes time to see the value it generates, and also is usually done by a different department. You need to quantify into value for users and battle for it. The fact that IT and research are done in separate teams makes this hard to achieve.
Puja added that another thing that is rarely discussed is that if you spend, say, 5 days conducting the research, then you should have spent 15 days discussing the research question with your colleagues and other stakeholders. Research is a partnership and takes time. How much are the researchers empowered to prioritise and spend their time on what brings the most value and drive what’s happening in the company? If researchers can only do what’s being asked of them, they don’t have the opportunity to try to add the value they are able to. We’re often not asking the right questions or using the right resources for the right needs. If researchers are given freedom to use their own judgement, research becomes embedded as part of the culture, and then it isn’t seen as an extra cost.
Bridging the gap
The panel then discussed how these differences can be reconciled. As Melanie put it:
“Are we talking about philosophical differences vs practicalities? Are there ways for us to work together to bridge that, e.g. profit/loss culture, feature factory, versus curious people?”
The panelists’s answers focused on agile work, using metrics and building a shared language. Kenny argued that the agile method can help teams to bridge this gap because:
“In essence, agile involves inspecting, researching and adapting, that’s the basic essence of agile. If you want to check if something works, especially complex systems, then adapt to it.”
Puja said that the tension between ‘fast’ and ‘slow’ research is a reality in every organization because you need to prioritise where you spend you time and the results need to ‘trickle up’. Working fast can help the company move forward. But the company’s leadership must have an understanding of what research brings and make that part of the research culture. It’s really important for researchers to talk with their managers and say, “hey, this is a given, we don’t need to do research on this, we need to spend our time and effort elsewhere”. It’s also really important for companies to use metrics as they help people to work towards a common goal. The metrics might be different depending on your job role, but “I don’t think we can succeed without a common metric”.
Kenny agreed that shared metrics are really helpful, and added that they help to create a common language. If everyone understand the metric then they can talk with each other, even if they aren’t experts in each other’s field, and understand how they are progressing and where they need to push harder.
Gien commented that she isn’t sure what the metrics are in her current workplace. She said that often IT gets left out of these discussions:
“Nobody asks what IT wants to achieve and how you’re going to translate it to your team. Often they have a separate system and you don’t have access to the data that they have, so you don’t know if you’re achieving your goal.”
In summary, Melanie observed that all three panelists are quite similar in two important ways: they focus on users and believe that research should be democratized. In this sense there is little difference there is between researchers and non-researchers: it’s about getting the job done to benefit all.
Are you interested in joining our next discussion? You can join our EthnoBorrel Meetup group to be notified of upcoming events. Currently they all take place virtually.