Using User Behavior Analytics to Detect Authentication Anomalies
Authors: Aron Anderson, Enterprise Security Lead
You may think detecting user authentication anomalies is as simple as identifying a red egg in a carton of white ones, but it’s nowhere near that easy. On the other hand, it’s not impossible, especially when you implement user behavior analytics (UBA). In a nutshell, UBA helps detect anomalies in user behavior through a combination of data science, machine learning (ML) algorithms, and AI. When integrated into a company’s overall security procedures, UBA can help security professionals proactively detect potential perimeter issues, suspicious lateral movement, possible insider threats, and other anomalies that may be malicious. At a higher level, UBA enables you to simultaneously strengthen your proactive security efforts and enhance your defensive security solutions. For example, using ML and AI, you can identify activity that, if allowed to continue unabated, might become a full-blown security incident.
Here at Adobe, the Security Intelligence team uses UBA to perform daily anomaly detection in our user authentication logs, which are generated by Okta. We use a custom portal — the Identity & Access Management Portal — to track user interactions and alerts, and we take user feedback into account when weighting anomalies. We added functionality to the IAM portal so users can view their own historical data about their anomalies and how they’ve responded to them. This direct involvement makes it easier for the user see how they are adding value to what we are doing behind the scenes. Plus, most users want to do the right thing security-wise, but they’re often unaware of what the best practice is or they’re making honest mistakes. Engaging the user and making them aware of how their activities might directly affect or impact what could be identified as an anomaly is important in this process.
So, based on user feedback about whether an anomaly is legitimate or malicious, we can adjust the model to weight features appropriately: either that the activity is no longer considered anomalous and we should treat it differently the next time it appears, or that it should be escalated to another team for analysis using an automated workflow. Being able to close the loop and bring the user into the process helps eliminate false positives and improves our overall security efforts.
Now let’s dig deeper into the technical aspects of the anomaly detection and alerts workflow we use at Adobe:
The first step is to pull the logs of all successful authentications in the last 24 hours and filter them according to employee type. This is accomplished via an API call to Okta, which provides a robust, valuable data set that helps us understand what the authentication process or the session might look like for a user. Details about geography, applications they’ve accessed, IP addresses, and such make it easy to come up with a strong methodology for analyzing those events.
Then, we note the features — or detail data — for each successful authentication and add these, along with the Okta logs, to a MongoDB database.
Next, we append the Okta data from the past 24 hours to last six months of historical Okta data for a user in the MongoDB database and run a machine learning algorithm against those data sets, using User Agent String (UAS), country, and app as features. We then use statistical calculations to check if the country in the anomaly is the least count country for the user or if the UAS is least count UAS and we also use travel data to account for international business travel to remove false positives. These anomalies are then added to the MongoDB database. This step in the workflow typically takes 30 minutes to two hours, with an average runtime of one hour.
Now comes the process of alerting the user and getting their feedback. IAM team pulls all the anomalies in the past 24 hours from the MongoDB database and sends email to each user with all anomaly associated with their username. The user then logs into the IAM portal and either confirms or denies the anomaly, and the MongoDB is updated with the user’s response. If the user responds that he recognizes the login and that it’s legitimate, future anomaly with same parameters will not be escalated to the user for the next 2 weeks. If the user responds otherwise, the anomaly is automatically escalated to the Adobe SCC in the workflow execution. And if the user responds with NOT SURE, the alert is assessed for compromise or better classification by the Security Intelligence team.
Now let’s look at some examples of user authentication activities that would trigger various levels of anomalies:
High Severity
- User travels to a different country and uses new device/browser
- User attempts to login from a blacklisted country or using a blacklisted UAS
Medium Severity
- User travels to a different country and uses existing device/browser
- User logs in from a new device/browser even if in their home country
It is important to note that just because an anomaly is flagged as “high severity” does not mean that it is malicious.
Since implementing UBA within Adobe, we’ve seen a number of business benefits, not the least of which is that our user community is now engaged in the security process. Our password policy now more closely mirrors the latest NIST standards, ending arbitrary password rotation and improving overall security. We have greater visibility into business processes and possible security policy violations, such as an attempt to automate the authentication process using a workflow that hasn’t undergone a security review. We’ve been able to identify a number of previously undetected malicious activities, and we’ve improved other authentication workflows by applying the UBA results, including adaptive authentication based on user/device risk and history.
In conclusion, this project has helped us at Adobe gain valuable insight into possible user behavior anomalies, how to detect them, and how to avoid issues that could trigger a needless reaction. First, sometimes the results of our analytics simply reflected unexpected or poor work processes or policy issues rather than truly malicious activity. Second, ensuring automated and repeatable results takes time. And finally, occasionally legitimate activity will trigger an alert and it’s important to determine, as a company, your acceptable rate of false alarms.
