We may have achieved a small but significant first. We have used big data methods to predict which social housing occupants are at greatest risk of abandoning their home without warning. It doesn’t happen often, but when it does there can be months of work ahead. There may be a trail of unpaid rent, an empty property to clean and repair, uncertainty about what to do with it, and concern for the person who has gone.

As a first attempt the prediction was far from perfect. It was also at the extreme end of what’s possible. Within this Housing Association Group there are about 100 cases of abandonment each year out of many thousands of residents. This prediction challenge compares with that for rates of credit card fraud and rare disease infection.

Those predicted to go fall into three groups: residents subsequently found to be still occupying their home with no plans to leave (a prediction error); those already confirmed as abandonees (but not flagged correctly within the system), and tenants who we now realise have gone or who are very close to going.

This last group is the one of interest. Those who abandon their home probably have a chaotic lifestyle and may be in a state of desperation. Catch them before they go and they might think twice. This will hopefully keep them in a safer place where help is at hand. And if they haven’t being paying their rent we may have more time to recover what’s owed.

It’s about knowing how and where to look

This predictive capability is needed because resources are being squeezed and the number of people getting into difficulties is set to grow. E.g. Universal Credit is expected to push arrears up significantly.

The limited number of interventions that will be possible will need to be targeted where they can do the most good. This means getting smart with the data and knowing how and where to look.

That said, we didn’t predict many abandonments the first time around. But those we did are being checked out by people knocking on doors. This will bring in new data which will improve our accuracy when we repeat the exercise again.

.. and it’s about killing a misconception

But there’s a big misconception slowing down progress. It is that Business Intelligence of this sort costs millions.

It doesn’t. But some investment is required. E.g.

  • In educating senior management teams to understand what’s needed and how it can be achieved;
  • To build relevant skills and experience. These are expensive to procure but straightforward to grow from within. People who do this stuff are part programmer and data wrangler, part statistician, and part business consultant able to tell a story about how the business works. We don’t yet have the job descriptions, training programmes and career paths that capture this mix. With help from a well informed HR service, this situation can change.
  • In specialist technology. Some is needed but there’s a wide and accessible choice with all of it available either Open Source (i.e. free) or at a low cost.
  • In educating Business Managers to be hungry for information and to be prepared to improve data capture out in the field in order to build the right information. Prepare to be surprised. Once there is evidence that something is working results will start to be used. And once that happens there will be no looking back. Those providing the results will find their hands bitten off.

It’s not sexy, it’s fraught with difficulty, and it’s hard work. But it’s time to use big data to better serve the dispossessed and vulnerable. We have made a start, we are starting to see some results, and we hope that we can encourage others to follow.
Paul Clarke
Develin Consulting

Identifying cost reductions in an organisation is never easy. But happy days !! – an initiative to do just that has just concluded successfully, with all cost reduction targets agreed in full. Such a good feeling. And this was in a place that seeks consensus at every step within a management decision. No top down diktats here. Full agreement had to be reached.

BI, or ‘Business Intelligence’ made this possible.

It’s not about standard reports

Every business system worth its salt will be said to support BI. But the insights needed are rarely found in standard reports. They typically have to be handcrafted and involve originality, innovation, imagination and hard work. And the result needs to strike right at the heart of the issue. Nothing can be left to interpretation. It must answer the question – exactly.

This often means data that’s hard to find. For example, this business has a lot of people on the road visiting people in their homes. No-one doubted that costs could come down if visits were scheduled better. Benchmarks pointing the way were plentiful.

But it took the building of an ad hoc scheduling simulator that modelled a range of scheduling options to nail down exactly what could be achieved.

The result still had to be cross referenced to all those benchmarks and compared with related pieces of work from others. But the influence over the debate from this little piece of BI was profound. Reasons became clear. Particular defences could no longer be sustained.

It is about showing the way

Senior managers are perhaps much happier to embrace change than they are credited for. The problem is in how the argument is put to them. It’s often expressed in the language of failure, even blame. E.g.

‘That group over there operates at a lower cost than you. You need to get your costs down to their level.’

A valid point, sure. But if that’s the starting point then you know that you are in for the long haul. No-one is going to shake your hand and say ..  ‘Thanks for pointing that out.   What have I been thinking all these years!   I’ll get onto it right now’.

A more intelligent way to serve intelligent people is not to show them what’s wrong but to show them the way to somewhere better. That’s the role of BI. And the clue is in the letter ‘I’.

Here are some rules of thumb for putting the intelligence into BI.

  • Tap into the intelligence of those you are seeking to influence. BI is about building a simple model of the business, not on paper, but in peoples’ heads. What you build must resonate with how they see the business, what matters most to them, the metrics and imagery that will exert the greatest influence over their thoughts.
  • Look for the pool of people they will want to influence. BI may be focused towards a particular senior Executive. But they will need help to influence others, in particular within their own teams.
  • Stay small. BI is about simple models that contain just what’s needed and no more.
  • Be prepared to search for new data and keep an open mind about what you might need. The ad hoc scheduler drew upon public records for routes between locations, driving speeds, even walking distances – none of which had been used before.
    BI about abandoned properties drew upon a count of spiders webs on a front door. A quirky piece of data, but known by Field Agents to be a good indicator that no-one has been home for a while.
  • Welcome extensive trial and error. Conspicuous effort to get it right will help to sell the result.
  • Show options. It’s not only about showing there’s a better place to go to. It’s about showing the best way to get there.
  • Be obsessive about presentation. If it’s clear you took the time to get the font right, then you probably took the time to get it all right.

And finally, it doesn’t have to be perfect. Intelligence is about making connections between scattered pieces of information. You may not have many, and the component pieces may not be ideal, but if they come together to tell a compelling story then job done.
Paul Clarke
Develin Consulting