A habit is basically a learned association between a stimulus or cue in the environment and a behavioral action or set of actions (a routine). The actions triggered by the cue can be relatively automatic. So, for example, your alarm clock goes off and you find yourself in the bathroom, still groggy, already brushing your teeth. In order to develop a habit, we first have to begin with intentional, goal-directed behavior. This simply means we do something in order to receive a rewarding outcome. So perhaps at some point when you were younger, your alarm clock went off and a parent praised you for getting up and brushing your teeth. Initially, you may have brushed your teeth for the good feeling you got from your parent’s praise or maybe it was just to try that fun toothpaste flavor.
Over time we learn to repeat behaviors more when they come with a positive reward and avoid the behaviors that lead to negative consequences, and this fundamental principle, known as Thorndike’s Law of Effect, shapes how we choose “purposeful” actions.
An example of a behavioral sequence that involves a cue, an action, and a response. Goal-directed behavior describes an action (here brushing teeth) that is performed in order to receive a future reward (praise, fun toothpaste flavors, clean teeth).
Crucially, a habit begins to be formed when the actions are done regardless of the reward. This is known as outcome insensitivity. With repeated training or practice, rats will continue to press a lever [1] or traverse a maze [2] in order to obtain food even though they have already eaten and are no longer hungry. Whereas before, during goal-directed behavior, the action was done for the reward, now the action is triggered by the cue. For the rats, this cue may be seeing the lever or a light that was paired with the location of the food in the maze.
Environmental cues that have been associated with a reward can produce cravings or feelings of ‘wanting’. Think of the smell of freshly baked cookies. In cases of addiction, these environmental cues (for example, seeing someone light up a cigarette) can acquire motivational properties in and of themselves, even if the person no longer experiences pleasure or ‘liking’ from the action [3]. Of course, cues can prompt healthy habits as well: according to one study, almost 90% of regular exercisers had a time or location cue, such as running on the beach, to exercise [4]. These automatic cue-response associations allow the brain to free up resources for more efficient processing [5].
Habits are triggered by previously occurring cues (here the ringing alarm clock) and are not performed in order to obtain a future reward.
How can the timing of rewards increase habit formation?
How behavior is influenced by a reward’s dependence on the previous action has been studied extensively in behavioral psychology. Interestingly, certain “schedules” of rewards that specify when a reward is delivered to an animal have been shown to promote habit formation. The possible reward schedules can be classified according to two dimensions: fixed vs. variable and ratio vs. interval. The first dimension describes how deterministic the reward delivery is. The second dimension describes whether the reward is given based on a certain number of actions or a certain amount of time.
We can find some examples of these reward schedules in the daily patterns of our lives. In fixed interval schedules, a reward is always given after a certain amount of time, such as when someone is paid a fixed salary at the end of each month. In fixed ratio schedules, a reward is always given for a certain number of actions performed, such when a salesperson is paid a commission for every sale. It gets more interesting when we look at the variable schedules. Variable ratio schedules give a reward probabilistically after a certain number of actions. The action may need to be repeated five or twenty times in order to get a reward. This type of reinforcement schedule can be found in gambling. Finally, there are the variable interval schedules, where the first action that is performed after a random amount of time has passed can lead to a reward delivery. While there are not as many examples of variable interval schedules in daily life, one occurrence can be found when checking emails. If you are waiting for a response from an important person, you may repeatedly check your email. However, regardless of how many times you check, you will only get a message after a variable amount of time has passed between when you sent your message and when the other person has chosen to respond.
Interval reward schedules have been shown to promote habit formation [6]. These schedules model naturally depleting resources in the environment such as when an animal is foraging for food. Here animals have to perform an action to obtain a reward, but the reward is not always available and is depleted and replenished at consistent intervals (berries for example). Interval schedules generate predictable patterns of behavior [7], with variable interval schedules potentially creating the most long-lasting habits [1]. One reason for this is that in interval schedules, particularly when they are random, the correlation as well as the closeness in time between the action and the reward is limited [1]. Therefore, it is in the animal’s best interest to perform the action as often as possible. Think again of checking your email for that important message. You can’t control when the message will arrive in your inbox, but the more often you check, the shorter the time will be between when the message arrives and when you get to read it.
Possible reward schedules can vary based on with what uncertainty the reward is given and whether it is given based on time or quantity of actions. Examples of reward schedules in daily life: Fixed ratio — getting paid a commission for every sale. Fixed Interval — getting a fixed monthly paycheck. Variable ratio — gambling. Variable interval — checking for an important email message. Interval schedules have been shown to promote habits.
How does the brain control habits?
Learning goal-directed associations depends on networks in the brain between the basal ganglia and the cortex. The basal ganglia are important for selecting actions, and the cortex is involved in higher-level functions like planning as well as execution of movements. In particular, a component of the basal ganglia, known as the striatum, plays a key role in learning behavior that is modified by rewards. This region receives massive dopaminergic projections from the midbrain [8]. While popularly depicted as a chemical of pleasure, dopamine serves as a teaching signal about how the reward stacks up to what you expected as an outcome from the action. This helps animals learn and reinforces certain behaviors.
Although an intricate network of areas is involved, the dorsomedial striatum (caudate) seems to be a crucial region for learning goal-directed actions. Lesions to this area result in cue-based responding in rats [9] and an early emergence of habitual behavior [10]. Stimulation of this area has been shown to be effective in treating diseases that may be characterized by reduced goal-directed behavior and an over-reliance on habit, such as obsessive-compulsive disorder and major depression [11]. During goal-directed learning, the dorsomedial striatum (caudate) communicates with prefrontal cortical areas responsible for higher cognitive processes like planning. On the other hand, the dorsolateral striatum (putamen) mediates habitual behavior. This area is capable of “chunking” smaller tasks together to see them as one big task [5]. Lesions to this area prevent rats from using a habit strategy of always turning in the same direction to find food in a maze, and instead they go back to their earlier goal-directed strategy of remembering the location of the food [12]. Patients with Parkinson’s disease, that have damage to this area, have trouble with habit acquisition and learning from cues in some situations [13,14].
During habit responses, the dorsolateral striatum (putamen) communicates with prefrontal cortical areas involved in movement. As behavior shifts from goal-directed to more automatic, brain activity shifts from the network involving higher level planning to the network involved in making movements [10]. Previously, neuroscientists thought that these two brain networks competed with each other for control over behavior, but more recently the view has emerged that the systems can work together and complement each other [10]. For example, a goal-directed action can then trigger a habit sequence.
Brain regions involved in goal-directed and habit behavior. Goal-directed actions involve communication between the dorsomedial striatum (caudate) and prefrontal areas involved in higher cognition like planning. Habits involve communication between the dorsolateral striatum (putamen) and areas involved in movement. As behavior becomes more automatic, neural activity shifts from the network involved in planning to the network involved in movement.
What does this mean for medication adherence and Collabree?
Taking medications can also be broken down into behavioral sequences involving cues, actions, and rewards. However, for patients with chronic conditions, the reward for taking medications can be far delayed into the future. In addition, while a symptom can serve as a cue to take a medication, patients with chronic conditions don’t necessarily experience daily symptoms. Therefore, they are responsible for creating their own cues to take the medication.
Interventions for medication adherence can provide both the external cues to initiate action to take the medication as well as rewards for doing so. Studies have shown that providing reminders in combination with rewards can be effective at improving medication adherence [15,16]. Rewards can take the form of praise, financial incentives, a lottery for prizes, or social support.
For the Collabree mobile application, patients with chronic conditions will undergo a 90-day intervention program to promote medication adherence. Reminders (cues) will be paired with both financial incentives and instant gratification (animation + praise) as reward for timely medication intake. At first the behavior of the patients may be governed by more goal-directed systems and done for the reward.
The Collabree mobile phone application will enroll patients with chronic conditions in a 90-day intervention program. The app will provide the cues (reminders) and rewards for timely medication intake (instant gratification and financial incentives) to make the behavior sequences easier and more practiced for patients. Initially behavior may be driven by goal-directed systems and done for the reward.
After the 90-day intervention, patients will continue to receive reminders, but there will no longer be financial incentives. At this point, behavioral control may have gradually shifted to habit systems. Medication intake may then be triggered by the reminders alone and done regardless of the reward. To test whether sticky habits have been formed and medication adherence continues to be increased after the intervention, there will be a 90-day follow-up period.
After the intervention, patients will continue to receive reminders from the Collabree mobile phone application, but there will no longer be financial incentives. Behavioral control may have then gradually shifted to habit systems. Medication intake may be triggered by the reminders and done regardless of the reward. To test whether sticky habits for medication adherence have formed, there will be a 90-day follow up period.
The timing of the rewards in Collabree also takes advantage of the fact that interval reward schedules are known to promote habit formation. The financial incentives are delivered in a fixed interval at the end of each day, regardless of how many medications the patient took, as long as they were adherent to their medication plan for the day. The instant gratification animations happen according to a variable interval schedule. It is not guaranteed to be shown after each medication intake, but after a variable amount of time has passed and the patient takes a medication, they can see the instant gratification. By using interval schedules for rewards, the Collabree mobile phone application is more likely to promote sticky habits for medication adherence.
The Collabree mobile phone application provides rewards in both fixed interval (financial incentives) and variable interval (instant gratification) schedules. The timing of these rewards helps promote sticky habits for medication adherence.
[1] A.L. Derusso, D. Fan, J. Gupta, O. Shelest, R.M. Costa, H.H. Yin, Front. Integr. Neurosci. 4 (2010).
[2] J.R. Sage, B.J. Knowlton, Behav. Neurosci. 114 (2000) 295–306.
[3] T.E. Robinson, K.C. Berridge, Annu. Rev. Psychol. 54 (2003) 25–53.
[4] K. Tappe, E. Tarves, J. Oltarzewski, D. Frum, J. Phys. Act. Health 10 (2013) 607–613.
[5] K.S. Smith, A.M. Graybiel, Dialogues Clin. Neurosci. 18 (2016) 33–43.
[6] A. Dickinson, L. Weiskrantz, Philos. Trans. R. Soc. London. B, Biol. Sci. 308 (1985) 67–78.
[7] C.B. Ferster, B.F. Skinner, Schedules of Reinforcement., Appleton-Century-Crofts, East Norwalk, CT, US, 1957.
[8] J.R. Wickens, J.C. Horvitz, R.M. Costa, S. Killcross, J. Neurosci. 27 (2007) 8181–8183.
[9] B.D. Devan, R.J. McDonald, N.M. White, Behav. Brain Res. 100 (1999) 5–14.
[10] H.H. Yin, B.J. Knowlton, Nat. Rev. Neurosci. 7 (2006) 464–476.
[11] B. Aouizerate, E. Cuny, C. Martin-Guehl, D. Guehl, H. Amieva, A. Benazzouz, C. Fabrigoule, M. Allard, A. Rougier, B. Bioulac, J. Tignol, P. Burbaud, J. Neurosurg. 101 (2004) 682–686.
[12] M.G. Packard, J.L. McGaugh, Neurobiol. Learn. Mem. 65 (1996) 65–72.
[13] B.J. Knowlton, J.A. Mangels, L.R. Squire, Science (80-. ). 273 (1996) 1399–1402.
[14] D. Shohamy, C.E. Myers, S. Grossman, J. Sage, M.A. Gluck, R.A. Poldrack, Brain 127 (2004) 851–859.
[15] R. Nieuwlaat, N. Wilczynski, T. Navarro, N. Hobson, R. Jeffery, A. Keepanasseril, T. Agoritsas, N. Mistry, A. Iorio, S. Jack, B. Sivaramalingam, E. Iserman, R.A. Mustafa, D. Jedraszewski, C. Cotoi, R.B. Haynes, Cochrane Database Syst. Rev. 2014 (2014).
[16] J. Roseleur, G. Harvey, N. Stocks, J. Karnon, Patient (2019).
