Have you ever worked on a large scale project and wished you had access to tips, like dos and don'ts?
Whether it's for record-keeping, to share with others or to reflect back on your own work, capturing lessons learned is often the plan, but too often forgotten.
Last Fall, our UX team built a Learner Journey Map and a Service Blueprint for our Premium pilot. As we worked on this daunting task, we took notes of a few things to learn from our mistakes (yes, we all make mistakes!). We also wanted to share these with others! Because really, that's what it's all about... learning from our successes and failures, and sharing our lessons learned with others so that they can benefit from these as well!
If you know what Premium pilot is, you can skip this introduction and go to the next section ‘Why did we do it?’
What is Premium?
Premium is a three-month learning experience at the Canada School of Public Service (CSPS) Digital Academy (DA) piloted in winter and fall 2019. Its objective was two-fold: build and strengthen knowledge and skills in the areas of design, data, AI/Machine Learning and DevOps; and provide an opportunity for learners to apply their new knowledge and skills by researching and prototyping a solution to a real business problem. More than 60 learners from across the public service participated in the first cohort (January-June 2019), and over 100 in cohort 2 (November 2019-January 2020), including 15% from outside the National Capital Region (NCR).
Our work focused on the second cohort. Each applicant had to submit a business problem. Individuals accepted into Premium were grouped into 15 multidisciplinary teams, and each team was assigned a different business problem selected from all the problems submitted during the intake process. Cohort 2 started with a 4-day in-person boot camp where teams bonded and started exploring their problem. It then moved into a 12-week period where individuals attended their specialized learning stream while also researching their problem and meeting with users as part of the problem-based learning component. The experience ended with a 15-day practicum that included an initial 4-day ideation and prototyping workshop followed by 10 days of prototyping using an agile and iterative solution development approach. On the last day, all 15 teams presented the prototype of their solution to a live audience. Throughout the learning journey, coaches, mentors and technical advisors supported the teams. Learners were also expected to ask their peers for help and provide help to other learners throughout the prototyping phase.
Why did we do it?
The experience mapping project was born for a simple reason - gain a better understanding of the learners needs and pain points, and ensure a good learning experience.
Premium was different from the type of learning the School typically offered to public servants. Lessons had been learned from the first cohort, and substantial changes had been made in response to learner feedback, although no formal mapping of the learner experience had been done. In order to assess the effectiveness of these changes from a learner’s perspective, we needed to understand the overall experience from start to finish. We decided to create a learner journey map to help us pinpoint the gaps in the overall learners’ experience and identify areas that required adjustments and refining.
As we started collecting data for the development of the journey map, it became evident that we needed to have a better shared understanding of the team’s process and challenges as we worked to deliver Premium. To help reach this understanding, we decided to widen our scope and built a service blueprint to identify opportunities to achieve a smoother process when working together to develop and deliver more advanced, experiential learning products such as Premium.
How did we do it?
The project was divided into 5 main phases.
Project Planning
In this phase, we defined the goals and objectives of the project and developed a research plan. This included defining data collection techniques and channels, what type of maps would be created and identifying teams and stakeholders with whom we would collaborate.
What we learned during this phase:
- Have a champion who advocates for the experience mapping project to the stakeholders and acts as a liaison between the team members and the stakeholders. This helps the researchers and designers keep focus on the execution of the project. The manager of the team executing this project is a good choice for a champion.
- Get buy-in from senior management. This helps get project visibility in the team and helps get more input and contribution from stakeholders.
- Involve all stakeholders in the project kickoff. This helps to ensure everyone is on the same page from the getgo and avoids scope creep, helping keep to the project timelines. This will also help establish a goal-oriented research plan and identify the required support from stakeholders for a successful project.
- Define the width vs depth for the project. To keep the project scope manageable and the maps from getting overwhelming, it’s important to find the right balance between the phases of the journey that you would like to explore and in how much detail you will explore it.
Tools used during this phase:
- Trello: a web-based Kanban-style list-making application to organize plans and projects. It is flexible, easy to use and free. We used it to track tasks and project status.
- Gsuite (Drive, Documents, Sheets): provides a variety of cloud-based apps for productivity and collaboration. We used it to document our research plan, interview guides, consent forms and share it easily across the team.
Data Collection
In this phase, we captured data about the learners experience and the DA’s process in delivering Premium.
What we learned during this phase:
- Make it a priority to capture data. Since data is what drives any mapping projects, it is very important to prioritize the need of capturing good data. To do this, it helps to assign this responsibility to specific team members who can make this a priority throughout the entire project. Asking additional people to help collect data at different stages of the project can be helpful, but can also result in insufficient or low-quality data if they cannot be fully invested in this task and aren’t following the same data collection protocols, which can result in invalid.
- Be mindful of timing when setting up interviews with stakeholders. To maximize the retention capacity of the human mind, try to schedule interviews shortly after the occurrence of the event. This will ensure you can capture the details from the events that are still fresh in the participant’s memory. However, be cognizant not to interrupt the participant’s work schedule and priorities.
- Make it easy for stakeholders to give input. When possible, establish different data collection channels based on the preferences of the stakeholders or use simple and familiar channels. . This helps to get data in quickly to ensure your project stays on track.
- Begin with a template for map visualization. The map template is a document, usually in visual form that shows what types of findings are captured and how they are displayed. Starting with a template in mind can help identify what kind of data needs to be captured. Hence, reducing the effort needed during the data synthesis phase.
Techniques used during this phase:
- Surveys: Since we had limited opportunities for primary research with learners, we collaborated with the Evaluation team to add UX-specific questions to the evaluation surveys. Learn more about creating effective surveys.
- User Interviews: User interviews are a core user experience method. They are a great way to gain foundational knowledge about the problems your users are facing. We conducted and recorded interviews remotely with the Premium project team using Zoom. Learn more about user interview basics.
- Retrospectives: Typically, an Agile retrospective is a meeting that's held at the end of an iteration in Agile software development. During the retrospective, the team reflects on what happened in the iteration and identifies actions for improvement going forward. We used it to capture learners’ experience data regarding what worked well and what didn’t in different phases of the Premium journey.
Tools used during this phase:
- Zoom: an easy, reliable cloud platform for video and audio conferencing.
- Funretro: a tool you can use to capture and discuss what went well and what didn't. We used it to capture learners’ experience data.
- Evalhalla: an Open Proof of Concept survey tool developed and managed by the Innovation Services team at CSPS Digital Academy.
Data Synthesis
In this phase, we transcribed the data collected from various sources such as interviews, surveys, categorized and tagged the data.
What we learned during this phase:
- Maintain a research database. Keep all the insights (verbatim, survey responses, etc.) collected throughout the project in one database. This makes it easier to quickly revisit the insights collected from different channels.
- Categorize and tag your data. The research findings by itself aren’t as useful unless you can deduce a relation/pattern and make it easy to consume for the stakeholders. Data tagging and categorization can help you do that. These added steps allow you to filter your data, focus on one aspect at a time and also find a relation between seemingly unrelated research findings.
Techniques used during this phase:
- Data tagging: Tagging allows you to structure your data by themes, make it searchable, and better collaborate with your colleagues during the research process. Learn more about how to do data tagging.
Tools used during this phase:
- Otter.ai: a great tool to transcribe interviews using the power of artificial intelligence. Based on the volume of the person speaking, accent and proper nouns, it doesn’t always provide perfect transcription, but it will certainly ease your job and add to your productivity.
- Airtable: a spreadsheet-database hybrid, with the features of a database but applied to a spreadsheet. It comes with a default UX Research Insights template which can help you organize your data for analysis.
Data Mapping
In this phase, we mapped out the categorized data and built the journey map.
Here’s what you should keep in mind during this phase:
- Consider using a combination of analog and digital tools. Don’t be afraid to use a combination of different tools. Each of them has pros which can be leveraged during a specific phase of the mapping project and the cons can be managed by switching to a different tool when needed. While analog tools like sticky notes and whiteboards help to get the creative juices flowing, working with online collaborative whiteboard platforms make it easier to make changes and provides the ability to work and collaborate remotely.
● Try out different tools before committing to one. More often than not, free tools are to get you hooked to the ease of the tool and then get paid for additional feature needs, e.g. downloading your work in a readable format. To avoid this, try a few tools before committing all your work to a specific one.
Techniques used during this phase:
● Affinity Mapping: Affinity mapping is a way to analyze and categorize research findings. It is similar to data tagging but it is usually done collaboratively and visually. We used Miro to do affinity mapping.
Learn more about how to do affinity mapping.
● Journey mapping: A journey map is a visualization of the process that a person goes through in order to accomplish a goal tied to a specific business or product. We used the synthesized data to build an end-to-end journey of a learner attending Premium. We used a combination of analog tools (Whiteboard, sticky notes) and Miro to do journey mapping.
Learn more about journey mapping basics.
Tools used in this phase:
● Whiteboarding with Sticky notes & sharpies: Don’t be scared to get your hands dirty with sharpie ink. These analog tools are low investment items which can help spark creativity and conversations at the beginning of journey mapping.
● Miro: a digital whiteboard for team collaboration in distributed teams. Being digital, saving work for later, making changes and remote collaboration is easy along with templates as an added advantage.
Data Visualization
In this phase, we created the final artefacts for the experience mapping.
Here’s what you should keep in mind during this phase:
● Choose the template tool carefully. Creating the visualizations using a pre-defined template can be very restrictive. Template tools are good when you don’t need to stray too much from the template since customization can be restrictive.
● Categorize research findings as per template, if using one. Fitting research insights into the structure of the template can become an additional task and require additional analysis. To avoid this, use the template structure to guide your data synthesis and data mapping phases.
Tools used during this phase:
● UXPressia: a set of online tools for visualizing customer experience. It helps you create customer journey maps, user/buyer or marketing personas, and impact maps without involving graphic designers or messing with slides, spreadsheets, or sticky notes.We used to build the final artefact for Learner Journey map using the templates available.
● Figma: a digital design and prototyping tool. Since the templates available in UXPressia were not able to show the story we were trying to depict, we created a custom design for Service Blueprint artefact using Figma.
Final Artifacts
What We Learned Overall
Throughout the project, we identified a few more general tips to keep in mind when developing experience maps:
● Keep an open mind and an adaptable attitude throughout the project. This helps in cases where there is high ambiguity, which is often the case when working on new projects being built from the ground up.
● Validate early, and frequently. To reduce extra work, make sure to have a good understanding of the journey and constituent components and to validate it with product owners and key stakeholders early in the project and as work progresses. For this, have regular checkpoints to discuss the project progress and make sure there is alignment with the business needs.
Experience mapping projects can seem overwhelming at first, followed by a lot of questioning and re-evaluating how work progresses. However, the final product and results are extremely rewarding, especially as you are seeing the artefacts being built and can see the value it brings to the team, helping them reach a common understanding of the user journey and business processes.
Have more questions about this project? You can reach out to the CSPS Digital Academy’s UX Team at csps.digitalacademy-academiedunumerique.efpc@canada.ca
