In 2020 we did a shift from in-class activities to all virtual instructor led training (VILT). One of the innovations added was moving from Word documents for scored activities to using Microsoft Forms as an online option.
This allowed us the following improvements:
Live updating to the instrument and questions
Ability to regrade assignments
Instant feedback for the participants
Live score viewing by the instructors
Ability to share performance with coaches and Supervisors throughout training
Here is an example of scores feeding to an online Excel sheet:
Microsoft Stream is something I just started utilizing. We were starting out with 3 main purposes:
Example calls: These include screen recordings and audio from client interactions. We primarily use these calls for Quality Assurance (QA) but they can also be using across classes for observational examples as well as for individual coaching.
Class recordings: One of our 2 year goals was to start recording each of our lessons as part of our walkthrough and review process. This would allow us to have a “clean” copy of the material to help people who have missed class, want to self-review, or as a coaching tool.
Training modules: These are videos or Articulate modules used in training that had previously been in multiple locations. Creating a consistent library will allow for easier updating as well as easier modularity.
Here is an example of the 3 Stream channels:
Here is an example of a list of calls in our call library:
I saw this marvelous demo from AirBNB about going directly from a sketch to an interactive prototype. It was what LEAN UX folks dream of! There is such a tight design language and asset library that it is easy to adjust sections to their responsive locations.
The bulk of the sites I have designed in my career have been in Adobe DreamWeaver. The corporate sites I have interacted with have utilized Adobe Enterprise Manager (AEM). Most of my small consulting sites (and my lazy portfolio) have used WordPress.
The concept of being able to easily go from a concept in Photoshop/InDesign/Illustrator to a coded system such as DreamWeaver/AEM seemed to have been addressed in the Adobe family by XD. When I started working with it around 2016 it was not very robust. It was still more of a visual representation and it still required front-end and back-end programmers and graphic artists to create web versions of the concepts. They have come a LONG way since then and are more quickly approaching that seamless transition from concept to code.
I love using Usability Hub because I can set up a test at the beginning of a meeting and have the results by the end of the meeting. I used it mostly for preference testing between two alternatives or for first click testing to see if people would know where to start a process. I would most often pick a sample size of 50.
I used this tool both in corporate life and in teaching. It allows for a webiste to be provided, questions presented one at a time, and the ability to record the participant going through the site and answering the questions. I would use it for unmoderated user tests.
For example, one could ask someone to fill out an online application and make comments throughout the process. Also, one could ask what someone would EXPECT to see on a page, click on that page, and then indicate their reaction on what they saw compared to what they expected.
For students I would have them run 3 rounds of tests.
Round 1: For the first round I would introduce them to the tool, we would use the existing website as the baseline model, I would give them the questions to enter, and I would walk them through how to analyze and present the results.
Round 2: For the second round, they would provide their own updated website, they would use comparator questions from the baseline site, and I would help them create additional questions. Guidance was given on how to report the changes from baseline to updated site.
Round 3: For the third round, the students provided the questions, the updated site, the updated questions, and presented their results.
I liked using UserZoom for moderated online tests where I would want to select a panel of people and administer questions to them directly. One of the benefits of using this model is that we would not have the logistic issue of bringing people into a lab setting. It also allowed for us to do deeper dives or explore different emergent flows.
My coworker, Ryan Johnston, turned me on to Atomic Design as we were working on the digital library for corporate projects. I had been familiar with brand guidelines and reusable code but this took it to the next level. It truly took the individual pieces such as font and then scaled them up to form elements to banners to pages to sites.
Airbnb has this down to a super lean process where their design language system is so tight that they can take a sketch and convert it to code across devices. Check it out!
This is the capstone of web analytics tools. I have used this for YEARS and it keeps getting better. One of my favorite visualizations is the user flow.
Omniture Site Catalyst was bought out by Adobe and is now called Adobe Analytics. I worked with a REALLY talented SEO guy who showed me a lot of powerful things that SiteCat could do! As a UX or Marketing person, it is great to be able to baseline and then compare after a site layout or process is changed. It also would help validate items that were screen clutter and NOT being used on a page or section.
At the University I was the survey administrator and used a tool called Select Survey. I had trained 100s of faculty to use it so I was reluctant to switch to a new tool. A few professors in the Marketing Department in the College of Business really wanted to use Qualtrics. I set up a 50 user license to test it out. It blew me away with how easy it was to use. I also had the opportunity to use this a bit with our primary research gathering in a corporate setting.
I did not administrate Clarabridge but used it for client experience dashboards and insights. I worked with a GREAT team that would build customer dashboards and do monthly reports and breakout analysis sessions with my quality assurance team.
I wish I would have had more time to play with Tableau. It seems I would get a basic dashboard set up and not get the time to make it even more robust. My basic usage was to have data fed into an Excel sheet or a SharePoint list and then feeding it to a dashboard and displaying it in a usable manner. The tool can certainly do a LOT more than what I used it for.
This tool is super cool! I have done only the most basic processes. I have a colleague who was able to create a connection from her Forms item, have that put an alert on a Teams channel, have it add an intake item on her Planner board, and assign the task to people based on the fields selected in the form. Wow! She shared her flow with me but I am still looking for the time to perfect it for my needs.
My talented colleagues at Illinois State University set up the Social Media Analytics Command Center (SMACC). Very few universities have a facility set up like this. I brought multiple tours from my existing teams in my corporate roles to see this HIGHLY useful tool and to give kudos to this center as an educational experience.
Here is an excerpt from their website: “Students, faculty, and researchers use the SMACC to identify public conversation from major social media platforms such as Facebook, Twitter, and Instagram, popular discussion forums, major blogs, and online news sources. Using specialized software they can then analyze these conversations for key data such as top influencers, trending keywords and concepts, audience metrics, geographic location of posts, user sentiment, and share of voice in the social web.”
Techsmith Morae is a data analytics software that can be used for testing the usability and market satisfaction level of a product. Users get to understand their customer’s experiences and thought processes related to a particular product in real-time and make relevant changes. The software captures all the minute details of its user’s product testing sessions and offers hard and vivid data, consisting of real-life usability problems.
I used this tool at my IT facility at Illinois State University, for teaching my classes to run live subject usability studies, and as part of the UX team in my corporate roles. It has the ability for the facilitator to speak to the participant outside of the testing room, the ability to tag behaviors, the ability for remote observers to watch live testing, and the generation of video clips.
I read that Techsmith stopped selling Morae in 2020. I have seen a similar tool in use called Observer by Noldus but haven’t run a study with it myself.
Many moons ago I did some qualitative research with NVivo and Qualrus but haven’t used them since I had adopted Morae.
From the Plan-Do-Check-Act of Walter Demig to the efficiencies of Lean and Agile to the principle of a learning organization, a central theme is empowering the teams, iterative design, and automating data.
I listed this under collaboration because I think it is good to have agreement across teams on what model/s we are using. Being an Agile leader encouraging a SAM process with a coworker who is used to a Waterfall/ADDIE approach can be a BIG challenge! It is good to have consensus ahead of time.
Hierarchical Task Analysis is a systematic way of describing how work is arranged in order to fulfill the job’s general aim. It involves identifying in a top down manner the overall aim of the job, then the various sub-jobs as well as the conditions under which they need to be completed to reach that goal. For example, you may have 5 basic duties numbered 1 through 5. Duty 3 might have 3 main tasks which would be numbered 3.1, 3.2, and 3.3. Each main task might have subtasks which would be numbered 3.1.1, 3.1.2, and 3.1.3.
When doing an HTA for the purpose of defining training needs, the DACUM method helps to prioritize tasks and assign them to audience needs such as new hires or veteran employees. One of the differences over traditional HTA is that this method utilizes people who are proficient incumbents within a job and is a rapid 2 day workshop. It is usually limited to 12 main duties, that are each broken into tasks, and tasks are broken into steps.
Both of these basic approaches require more validation steps such as card sorting and scenario testing.
