Category Archives: Diary Studies

Front-of-house

War stories. ‘Those’ user research projects…

There are user research projects where the exact objectives fade over time.

…but certain moments, and the people leave lingering bright spots, ripe to be shared.

I’m talking about those design research projects, where you meet people you could never have imagined, or entered into a person’s life so unfamiliar to you …

These experiences leave a dent – especially when people open up, sharing deep or private stories. Stories that stick, or are even hard-to-shake,

I’ve experienced my share  of these, like during (often teary) interviews with offenders in corrections facilities, patients in hospitals and with teenagers on both sides of bullying.

Then there are the memorable moments you simply can’t prepare for. The ones you can laugh about afterwards, and which remind you to stay open and flexible.

Like the time I arrived at the house in the photo above:

Everyone has their storage and parking preferences, but it was clear nobody had parked in either of these garages for a loooong time.

…in fact, it was even difficult for a person to enter this ‘hoarders’ house… as I found out when I helped the research participant make space for us to sit down to map her customer journey.

Part-moving-stuff

 

I stood outside, while she passed me boxes of stuff out the door to pile up outside so I could enter the house, (without climbing over relics from the 70s) and we could both sit at this table.

As you can see in the photo(bottom right), it took so long to get across the mountain of stuff that she used a thermos flask when she bought me a cup of tea so it was still hot when it arrived.

High-shot-room

She was the kindest woman, bringing me cheese and crackers, then going on to tell me her heartbreaking story of a series of family and financial events which resulted in a series or mortgage rejections, stone-walling her to move to a smaller house (presumably staging the epic garage sale before calling the moving truck).

She never mentioned her surroundings or her house became this way, but there were long, telling pauses and distant, sad stares which had us both welling up.

War stories

Every researcher has these stories, some sad, some strange, some confronting… all which test your social skills.

Steve Portigal calls these ‘War Stories’ and has one of mine in his new book: Danger, doorbells and dead batteries, a book of awkward, affronting and amusing anecdotes from design research fieldwork.

He shouted out for one where the user research participant did something illegal.

I had the story of ‘Aaron’ who decided to crank his diary study up to motorway speed, making selfie films of himself for a video diary study * while watching videos… while driving in fast moving traffic.

‘Aaron’ (Not his real name), happily filming himself as he records diary entries in his motorway down-time…

Aarondash

While catching up on some home-brewing vlogs on YouTube…

Aaronmway

He’ll be first on the list for a driverless car at this rate.

A few minutes into the home visit he said “I could show you something which might blow you away right now if you like” … when I knew this would be one of ‘those’ interviews, and Aaron one of those people.

Now that story and a bunch of other user research war stories are in print, and there’s an archive online too.

Thanks Steve!

*This links to an article about using what are known as ‘cultural probes’ where you send out a diary pack including a camera for research participants to make a visual journal… Nowadays, I might consider doing the same thing using Dscout, or even Voxpopme

Video diary study highlights screening

Capture the moment with video diary studies

Two and three at a time, 25 diary study packs arrived back by courier – this time containing not just logbook and photos but a video camera – 300+ self-recorded clips from selected moments during the previous week.

I had no idea what to expect from the video, but very quickly began to wonder why I hadn’t done this before?
Yep, after watching the first few clips I had struck ethno-gold by using video-selfies in a diary study project.

Diary studies are great for capturing interactions with a product or service which play out over time. I’ve tended towards keeping things old-school with paper based diary studies followed by exit interviews and always been pleased with the results, but after adding video to the mix, it’ll be hard to look back from here.

What resulted was a raft of in-the-moment, rich and raw footage offering an intimate, personal perspective which unfolds beautifully over time. With clips from different times of day, contexts etc. you really get a feel for the way the person’s week went and feel somehow more connected to their mindset during each interaction or moment they documented.

workshop ladies with wayne

Self-shot video combined with stills and printed verbatims combine to provide a powerful platform for conversation.


Here’s how the project unfolded, and a few things I learned along the way:

How to capture the footage?

We looked at a few options, with our main goal being to keep things as simple as possible for the participants capturing the data, and us wrangling it later.
We considered using their own phones, setting up video blogs, and even using managed services like www.watchmethink.com, but we needed to move fast and to have control over the technology and format, so opted for buying a fleet of cameras. No guesswork!.
We went for fairly basic and compact point-and-shoot cameras with HD video and of course stills too. Looking back it’s hard to think of a cleaner, simpler way to go about it.

Setting up the diary packs:

Diary study laid bare

Camera-CHECK, Charger-CHECK, Logbook-CHECK etc….

We put together 25 identical packs, containing;

  • Camera with stills and video capability.
  • Intro sheet – describing the objectives of the project and where the participant fits in.
  • Idiot-proof instructions on how to use the camera – recording video, charging etc.
  • Guidelines on some types of moments to capture with video.
  • Log book and pens, with a mix of simplified multi-choice checkboxes for recording the what, where, when, and how, plus an open text field for the why?…which is what we were most interested in.

Keep it loose.

The last thing you want is a participant thinking twice whether the thing they think is important will be useful to you, so don’t be too prescriptive with your suggestions of which moments are worth capturing.

I had figured on filming a sample clip to give people an idea what I was looking for, but am glad I didn’t as our sample exercised their creative freedom to capture some surprising moments in ways and from contexts we never would have imagined.

We let our participants decide themselves which moments were important / relevant to them. This seemed a little ‘open to interpretation’ but paid off in spades as it revealed key differences between individuals – super relevant to our study.

Unboxing and setup.

boxes and packaging

Allow several hours for getting the cameras ready. (I completely underestimated this).
Next time I’d make this a two person production line – opening boxes and packaging, charging batteries, prying SD cards from impenetrable plastic shells, printing info sheets, numbering and assembling all the kits.

sd cards clipped

Come face to face with consumer guilt as you unpack all this guff.

If you’re of the green persuasion, perhaps go plant a tree afterwards to get over the consumer guilt of dealing with all the packaging. Ok, better make that two trees actually.
Set up every camera the same, particularly the video capture resolution. This saves handling different image formats during editing.

Test pilot.

Absolutely DO Run a pilot. Ask a friend or two to follow your instruction sheet to shoot a couple of clips. You’ll quickly discover where more information, (or less) is needed in your supporting material.

Say what?

Get creative with your prompts, but let your participants do their own thing too.

Get creative with your prompts, but let your participants do their own thing too.

We put a sticker next to the lens with some very loose prompts to help our participants get to the ‘why?’. We did this after the pilot session and it worked a treat across the sample.

 

Packs away!

Include your contact details in each pack, on the camera if you can, so participants can let you know if something’s up. I’ve been contacted on every diary study I’ve done.

Also, a day or so after they’ve received the pack, call the participants to make sure they are in the groove with what’s expected of them. Keep this call short, a minute or two should do the trick.

Primed to share.

I’ve found diary study data is pretty bland on it’s own and the real flavour of the individual comes through in the exit interview. Somehow the act of logging their actions raises participants awareness of their intentions and behaviours. While there may be some downside to this, I believe this ‘priming’ opens some doors in their mind, making for deeper and more valuable, access-all-areas exit interview conversations.

Say vs. do.

I like to think I’ve got a great bullshit detector, but hearing participants speak in retrospect about their behaviour during the week, and comparing that to the self footage was a good reminder that what people say they do and what they actually do can be very different. For this reason, be sure to watch some of the footage before the exit interview, and even better then watch it again without he participant. They might even surprise themselves.

Pulling it all together.

clips in window

With so many clips to sort through I key-worded the file names to remind me what they contained.

How to handle and share all that footage?.
From the 300 or so short clips I weaved together a carefully edited highlight reel of a few dozen ‘moments’ from the video-selfies. When combined with snippets from the exit interviews, this offered a colourful and authentic ‘voice of the customer’ narrative as a ‘week in the life’ unfolded across many contexts.

Being there in the moment.

I’ll definitely be doing this again, but welcome any other techniques for getting into people’s lives when it’s just not possible to be there in the moment. I know there are other ways to approach this, so please hit me with your top tips in the comments below…

Or go ahead and record a selfie?

diary close up

Keeping it ‘old school’ with Diary Studies

Oops, The original article is now about halfway down this page because:

UPDATE 2016. Six years on and many of these products and services have changed shape. (And I’ve added a few).

The latest of these is Streetbees, ‘your eyes and ears on the street’ which has coverage in 87 countries and looks like a ‘selfie’ version of remote usability testing service usertesting.com, where consumer ‘bees’ get paid a small sum to film their impressions while they complete a task. I’m seeing insights dripping in honey.

… and Over the Shoulder, less of a DIY tech platform and more of a managed service for qualitative mobile ethnography.

Also… I’ve now had a crack at running a video diary study, with great success, and in the ‘how-to’ article show you what’s involved with some tips from my experiences.

user-experience-diary-studyI’m always looking for better ways to understand customer behaviour and the experiences customers have in their own context, out in the wild.

 

REMOTE DIARY SERVICES / PLATFORMS:
There are some interesting ‘mobile ethnography’ tools on the market, using web and mobile channels to capture, collate and filter all this stuff as it occurs in the field:

7daysinmylife …Which might be more ‘manual’ than it looks.
Consumerthink …Which appears to do everything. Hmm. I guess you have an account manager to build out the tool to fit your project.
CX Workout A mobile diary and ‘co-design’ platform which apparently builds journey maps. 
D Scout Which has fast become the UX researcher’s favourite way to capture participants’ ‘mobile moments’ during diaries, lifestyle and retail studies.
Experiencefellow …Why not slip into your customers’ shoes?
ethosapp …’An ethnographer in your pocket’, apparently.
Overtheshoulder … Seems similar to Streetbees, but less of a DIY platform and more of managed service.
Revelation … A versatile platform at a premium price …with what look like some useful data analysis tools.
Streetbees, more (and probably more reach) in the moment video vox-pop.
thethinkingshed …Which seems to be like a blog platform for your participants.
Watchmethink …for some in the moment video vox-pop.
webnographer …Which smells like a remote usability tool.

From speaking to John, the founder of ‘Watchmethink’, it seems quality and motivation of the sample is the big ‘gotcha’ here for serious researchers – as opposed to vox-pop collectors – so quite possibly the way to get the most from these services is to do your own recruit and just use the tool as a platform. 

WEARABLE GADGETS:
Oh, and here are some filming gadgets which ‘log’ activity from the subjects’ point of view:

Autographer
Narrative Clip
Looxcie

Keeping it old school…

Impressive technology, but ‘whizz-bang’ isn’t always the answer. I sometimes stick to the traditional pen and paper approach for a diary study, as this also has it’s benefits over the digital tools which seem so tempting…

For the uninitiated, diary studies in UX are a qualitative research method where participants record events, interactions, attitudes etc. in diary format over days or weeks. They are a great way to study customer behaviours in the context using a product or service over time, as opposed to during a traditional in-context interview.

In the study which prompted this article back in 2010 I was interested in how and where people used a prototype mouse in their day to day activities and how well it performed in each situation.

Here are some ways I feel the old-school method holds its’ own:

It’s human
No learning curve, no teething problems. Paper and pen doesn’t require login details, needs almost no instructions, is ultra portable, and doesn’t rely on web or mobile coverage. Participants don’t have to think about or remember anything other than jotting down their thoughts.

It’s flexible
Photos are great, and really help to add context, but I’m always amazed at how pictorial some people make their diary notes. Sketching and doodling on a blank sheet of paper is always going to win over an online text-entry box.

‘till the fat lady sings
The real gems from these studies emerge in the exit interview with each participant. When they’re looking back over their own handwriting, these paper diaries transport people back to those moments in time, where you can access the rich detail needed to paint the full picture. It’s literally a trigger for them to share stories, which is where the ‘gold’ always reveals itself in interviews.

It’s immersive
I love the process of pinning-up diary data around a room.
Having met each participant, built rapport and empathy, this is somehow retained when you’re surrounded in their scrawl. All those attitudes and responses pop back in your mind, help you get into their character, see things through their eyes and in relation to their context.

When the right type of project comes along, I’ll give the ‘digital ethnography’ tools a shot, but until then, I know I’ve got paper.

UPDATE 2015: I’ve now had a crack at running a video diary study, with great success.

Even so …I’d love to hear from someone who’s run an ethno project using any of these tools.