By June Lau and Sanjid Rahman
Posted: Updated:

Did you ever play cops and robbers as a kid? After a few rounds, you probably worked out who the fastest runners were, those that bunched up together only to split away as a distraction, and the agile ones that always dodged and wove just out of reach.Cops and Robbers

While chasing each other around, your brain was capturing and processing information about your surroundings. You worked out which trees people climbed, their tendency to dodge left first and so on. That’s the human brain at work, identifying patterns in information. Those patterns were formed over many observations, each happening quickly.

In the real world, catching thieves is similar to those childhood games of cops and robbers. Each incident is solved individually by applying whatever is known about an offender’s history. However there is generally little information about an offender’s preferences for different goods or time of day or proximity to specific places.

This is where big data can help. Each incident captured and entered into the Auror platform about an offender helps us to learn a little bit more about an offender and others that exhibit a similar profile. With a variety of data points, including external events and physical surroundings, we are able to build a richer picture of crime and its influencing factors.

Databases of information about offenders aren’t new. However, the primary challenge has always been quantity and reliability of data. Two-thirds of all theft goes unreported, and the numbers are even more disappointing in retail crime, which under- or misreports crimes at a higher rate than any other industry.

The bottom line is that retail theft is severely underreported because it has been too tedious and time-consuming. 

For busy store managers, incentive to take the time to compile lengthy reports for relatively minor incidents is simply lacking. For police, solving cases by reviewing hard-copy reports is a time-consuming process.

The perceived low value of each theft contributes to a sense of apathy and overall ‘numbness’ towards retail theft in general. However, the cost of replacing the stolen goods are borne by customers in the form of higher prices for the goods they purchase.

Retail thefts quickly add up to a significant national issue. Security executives estimate that at least one-third of ‘inventory shrinkage’ (loss of products between point of manufacture and purchase from a store’s point of sale) can be attributed to shoplifting incidents, costing New Zealand businesses more than $2 million every day.

The following stacked barplot shows how an offender becomes increasingly expensive as they commit more crimes. It demonstrates how shoplifting is not as petty as it seems and the extent it can impact the economy.Theft Bar Graph

Although a huge portion of the offenders are recorded as one-offs (91%), they are responsible for 66% of shoplifting losses. Whereas offenders who commit five or more crimes (classified as catastrophic) are only 0.66% of the entire offender population, but responsible for 9% of crimes and as much as 15% of the total value stolen.

In the UK, the Magistrates Association has advised retailers to report the full extent of crimes, including threats of violence, intimidation or abuse. Such information was recently critical in supporting more severe sentencing as a stronger deterrent to potential thieves.

Knowing the full extent of offences including associates, tools, products stolen and prevention measures in place all adds to building a more detailed picture of retail crime. A global report (Alan Fanarof, 2015) (PricewaterhouseCoopers, 2015) identified business intelligence and predictive analytics as the next frontier in improving shrink-reduction programs.

Overseas research (National Retail Federation, 2013) (Taylor, Kelly, Valescu, Reynolds, & al, 2001) has indicated strongly that retail theft can be a gateway crime.

There is future potential here to connect Auror’s database with a national database of criminals to test this theory and add to the understanding of how an offender progresses from a seemingly victimless crime (like shoplifting) to those with more significant social costs.

Understanding The Offender Psyche

Studies on character traits of offenders have shown they are strongly motivated by financial benefit, with those economically disadvantaged being more likely to become chronic shoplifters.

Interestingly, there are two types of offenders that are not typically motivated by need or financial gain; these are the ‘nonsensical’ shoplifters, who shoplift due to other personal or physical stressors, or out of ‘thrill-seeking’ behaviour.

The variety of information captured in Auror opens new perspectives in understanding offenders that moves away from demographic pro ling and instead focuses on behavioural patterns and preferences.

This has previously only been available through time-motion studies of CCTV footage or intensive surveys and questionnaires with anonymised offenders.

It’s easy to forget that retailers are often already collecting this information. It’s just been too difficult to analyse.

Understanding Behavioural Patterns

Let’s look at three perspectives to gain an understanding into the behavioural patterns of repeat offenders.

First, we analyse their time preferences (preferred time of day to steal), then quantify the severity of incidents during those same time periods. Finally, we look at patterns found in their history and associates.

Time of Day

The chart above shows when offenders strike. The larger the circle, the more incidents they have been involved with during a three-hour period. Each vertical slice is an individual offender, with offenders with the most number of incidents on the right.

Through this analysis we can begin to form an understanding of whether they like to hide in crowds when shoplifting or choose quieter hours when there are no guards on duty, or preferences for working with others or sharing vehicles. The ability to overlay a financial impact on these offenders means that we can prioritise preventative actions accordingly.

The most obvious pattern here is that most incidents take place between 12 and 6 pm. However, there are certain individuals who prefer the 6-9 pm time period.

Those with a more spread out time profile (i.e. where all the circles in a vertical slice are of a similar size) are opportunistic; they will shoplift at any time of day. Different shaped circles in a vertical slice point to shoplifters that have a routine.

This analysis can be strengthened by looking at how costly crimes are by each hour of the day. The graphic below does just that:

The Clocks

The rows group the clocks into three major store categories: petrol stations, grocery stores and ‘other stores’. The clocks on the left show the mean value of stolen goods during the day from 6 am to 5 pm, and the ones on the right show the same information overnight from 6 pm to 5 am. The longer the lines, the more valuable the goods stolen during that time period.

The expensive crimes for each store type each occur at different times. While midday to 6 pm and evenings from 6 to 9 pm were generally preferred times for offenders, they become more severe at 4 pm and 8 pm in grocery stores. It’s worth noting that offenders also take advantage of the rush hour before offices open at 8 am and often target petrol stations around the 3 am mark. This is also a peak theft hour for other stores types that are open late at night.

Understanding an offender’s psyche leads to developing more effective deterrents. Novice shoplifters are easily deterred by fear of being caught and by guilt. More hardened offenders (five or more repeat offences) lack such adrenaline-inducing emotions, and tend to be more considered and strategic in their approach. This speaks to their ability to steal a higher value of products. These offenders view store personnel and security devices as obstacles to circumvent and are the most creative in developing new schemes to beat the system.

A centralised database offers a unique opportunity to study career criminals and assist in the battle against organised crime. Automated image processing and streamlined reporting permits new perspectives in understanding behaviour and repeat offending.

Repeat Offender Profiling

For example, automated video and image processing has the ability to spot anomalies in a customer’s path through a store, identify potential concealment opportunities and trigger actions. These technologies can even spot people scanning for surveillance cameras, notice heavy clothing in the middle of summer, or identify pacing around a known theft hotspot within a store.

Global studies and our own analysis has shown that when offenders work together, they are capable of more damage. The influence of their accomplices has two strong effects: firstly, together they can make off with a higher value of goods, and secondly, accomplices who are very active influence an offender to also become more active.

We know that organised crime relies on new recruits targeting goods that are easily distributed.

However, there are two possible interpretations for this relationship. This offender could either be a low-level criminal that is frequently hired and therefore not the central leader figure. Or, they could be an instigator of crimes and involved with many cliques of offenders.

We can test these hypotheses by visualising how these relationships formed. Focusing on this cluster of offenders, a temporal analysis shows how relationships evolve and who makes the relationships. On these pages we can only show two frames: the first of an initial incident where eight offenders were involved, and their extended relationships as we know it now. To see how these relationships developed over time, visit

Affiliated Offenders

Final Words

We have just scratched the surface in identifying patterns of offenders and using data science techniques to shed light on their psyche. Though we like to think of everyone as an individual with unique preferences and a specific way of doing things, the reality is that there are commonalities across all of us that are often unseen. This is the power of data mining: the ability to test theories of offender motivation and identify patterns and influencing factors that drive shoplifters to do what they do, ultimately helping all of us to do our part in helping to prevent crime.



Dabney, D. A., Hollinger, R. C., & Dugan, L. (2004). Who actually steals? A study of covertly observed shoplifters. Justice Quarterly, 21(4), 693-728.

Alan Fanarof. (2015, March 23). How to GHT shrink in a complex retail world. Retrieved from CSO:

National Retail Federation. (2013). 2013 Organized Retail Crime Survey. Washington, D.C.: National Retail Federation. PricewaterhouseCoopers. (2015). Empower loss prevention with strategic data analytics. United States: PricewaterhouseCoopers.

Taylor, E. R., Kelly, J., Valescu, S., Reynolds, G. S., & al, e. (2001). Is stealing a gateway crime? Community Mental Health Journal, 347-58.

The Author

June Lau and Sanjid Rahman are both Data Scientists working at Auror thanks to a Callaghan Innovation grant.

Help us to improve

Related Posts

Revisiting crime rates as a useful prevention metric.

How the NZ government improves social services with data.

Leave a Reply

Want more great content delivered to your inbox?