Threat Detection

Home » Threat Detection

Using PacketSled to detect Golden Ticket Attacks

in Network Visibility, Threat Detection by Patrick Kelley Comments are off
2016 was a year of exciting news in Information Security, and unfortunately one of many breaches. As I write this, I’m reading news about continuing leaks from Yahoo and new SCADA attacks.

With over 20 years as a professional in Information Technology, I’m always looking at new methodologies.  However, I’m also looking at the ways things haven’t changed. 

For as long as I remember, privileged account exploitation has always been at the center of targeted cyber attacks. This provides InfoSec Professionals with a constant pattern of events to look for on their networks.

In typical fashion, attackers penetrate the network perimeter using phishing attacks or attacks on mobile devices, hijack credentials and use them to move laterally throughout the network, taking additional credentials and escalating privileges along the way.

During my time as a Penetration Tester, combining privileged accounts with attacks on the Kerberos authentication in Windows domains was a method I always employed, in hopes of compromising the entire network. In fact, my team used to joke that lunch wouldn’t be served until we had “Domain Admin”. During such attacks, my team would target domain administrator privileges, which provide unrestricted access and control of the IT landscape. Armed with these privileges, my team could manipulate Domain Controllers (and Active Directory) and generate Kerberos tickets to obtain unauthorized access.  Our ultimate goal was the “Golden Ticket”!  That’s the ticket that gives us 10 years of unfettered access.



In order to successfully collect one of these tickets, the four things necessary to formulate one is:

·       the account name of a domain administrator

·       the domain name

·       the SID for the domain

·       the password hash of the krbtgt user from the Domain Controller

The good news is that Golden Ticket Attacks are particularly noisy, both in system logs and on the network. With the ability to perform deep packet inspection on Kerberos and DCE-RPC logs, this gives users of PacketSled the ability to look for indicators of these attacks across their entire network, instead of searching through system logs which are often dispersed and not centrally stored while attempting to piece together the pattern for themselves 

In particular, Golden Ticket Attacks require a particular DCE-RPC command on the network, defined as, “DRSGetNCChanges”. This command obtains updates for a specified naming context (NC = partition of the AD database), typically between domain controllers.  This behavior is particularly useful, as knowing that this data would not be transferred between a workstation and a domain controller would be anomalous.



Now, how do you get visibility into these attacks from a central point? Well, deploying a PacketSled sensor to establish a perimeter has never been easier. We have an IR deployment package that can have you up and running in minutes. Provide a SPAN/TAP feed to our sensor(s) and you have perimeter network visibility when you need it most.

Using this approach to develop the security detection moves the security controls from making decisions based on circumstantial, atomic elements and moves towards being more contextual in nature. This can greatly reduce the false-positive rate and make notifications more actionable.

At PacketSled, our researchers are continually pushing the boundaries of using our IRES-based security platform to provide greater value in security notifications. We do so by not just notifying InfoSec Professionals with alarms of potentially malicious behavior, but putting the context of those events in their hands as quickly as possible. 

Let us show you the value of PacketSled network visibility tools. To arrange a product demonstration or talk IR strategies give us a call.

Analyzing Unknown Protocols – Finding a Piece of Hay in a Hay Stack

in Network Visibility, Threat Detection by Patrick Kelley Comments are off
Over the years, I’ve had the fortune of performing in several differing capacities in the Information Security community.  Some positions have been on “blue teams”, whereas others have been performed more “red team” roles.  One thing that always rings true is the importance of visibility. 

You can’t make decisions around data that you can’t see. 

When PacketSled chose to build our sensors on Bro, we knew that significant effort would be necessary, if we were going to meet our high level of expectations.  Bro provides visibility for around 60 protocols, at this time. Unfortunately, there are literally thousands of protocols. That’s a lot of missed network traffic. 

For those that aren’t aware, WinRM is a remote management service for Windows that is installed in Windows XP and higher versions. Both of these authenticated services are tied to an HTTP or HTTPS SOAP listener and support Kerberos and NTLM authentication by default. It’s important to note that many attacks start with NTLM authentication since it’s already supported in Metasploit.



As our security team began launching live attacks in networks observed by Bro sensors, we realized that many of the Metasploit attacks weren’t being properly observed and tagged.  WinRM traffic is based on HTTP and HTTPS, but isn’t formatted in a manner that is properly recognized and analyzed by Bro. As a company focused on security solutions, we realized that something had to be done to permit our clients to see actionable data in this attack, as well as attacks on other protocols.

Therefore, we built raw TCP and UDP analyzers.  With these analyzers, it’s now possible to build IRES rules or bro-formatted detection scripts to alert on attacks in protocols that aren’t yet supported by the Bro framework.  So, how do you do it? 

By collecting a Packet Capture (pcap) of the attack traffic using our raw analyzers, we were able to extract the associated User-Agents and URI strings associated with these attacks.  

In our case, these were defined as:



User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)

URI and Action: POST /wsman HTTP/1.1

Host: 192.168.1.3:5985

Though this was an authenticated session, which later became encrypted, we were still able to see some interesting indicators as the session was established.  The most important being a Mozilla User-Agent connecting to a service that would normally be engaged using a variation WinRM.* as the User-Agent.  Also, we noticed that the listening port was TCP/5985, which is a non-standard HTTP port, and repeated failed authentication attempts.  Knowing this was anomalous and potentially malicious, our team was able to create a simple IRES rule to pull these indicators from our raw TCP analyzed traffic.  By doing so, we were immediately able to recognize a large amount of attacks from Metasploit against WinRM services in our extracted TCP payloads.



The single piece of hay was found in the haystack and what would normally be unrecognizable by standard installations of Bro was quickly brought to the surface.  That is the unmatched power of the PacketSled platform. 

At PacketSled, we’re always striving to provide the greatest visibility with our award-winning platform by continually developing additional protocol analyzers, allowing you to continuously monitor for advanced threats and policy violations missed by other defenses. Allowing you to analyze and remediate in record time. Don’t settle with being an Analyst. Be a Hero!

It’s 10 o’clock; do you know where your data is?

The Biggest Companies in the World Are Failing at Basic Detection

in Incident Response, Malware, PacketSled, Threat Detection by Web Admin Comments are off
We received a really interesting email from our upstream Internet provider yesterday – AT&T, indicating that we were effectively infected with Tinba malware. For those of you who don’t remember it, Tinba is 4 year old Windows specific Malware based on the old Blackhole exploit kit that does all kinds of fun stuff like steal facebook credentials, attempt to access your online banking accounts, etc. The email was as follows: att-incident Technical and operational observations are as follows:
  • First, we don’t possess any Windows machines capable of executing this Malware, with the exception of some VMs that are turned off unless otherwise needed.
  • Second, all of those machines have adequate endpoint protection on them, such that they would have averted this ancient attack, should it have been, you know, actually happening.
Regarding the email we received, if you haven’t already found the funny part, I’d like to point that out first. Obfuscating the IP address of the site that (according to them) indicates an infection is extraordinarily unhelpful, almost to the point where I feel like they might be trying to live up to some weird obtuse legacy of what it was like to order circuits in the 1990s. Thankfully, they took the time to include the domain name, which, you know…resolves to the obfuscated IP address listed directly above it. The unfortunate part about this situation is that they simply didn’t have the proper tools to make a quick determination that this was in fact, not a threat at all, and that we were running some tests on this network against legacy threat intel. Of course, we knew that, and we could easily validate that such testing was occurring during the timeframe that they mention with a simple query in the investigator as follows:
Querying specific IPs during specific timeframes using natural language

Querying specific IPs during specific timeframes using natural language

We get one session record returned:
PacketSled Investigator

PacketSled Investigator

And immediately determine that this “threat” originated from a Mac running Wget:
Hey look, not a Windows machine. Not even a real browser.

Hey look, not a Windows machine. Not even a real browser.

Nevertheless, we received this email, meaning that someone in the AT&T SOC had to investigate this non-incident, put in a ticket, send the email and manage the eventual closure of that ticket. This scenario is extremely commonplace. The number of SOC personnel, incident responders, and general infosec professionals that routinely chase non-incidents due to only having partial data is extremely perplexing. The data is in the packets. Making it accessible to analysts and automated processes such that it can be intelligently used is the goal. With something this simple getting incorrectly bubbled up to the top, we are clearly a long way off from that goal.

An Analysis of SPAM Across the Entire Organization in 5 Minutes

in PacketSled, Threat Detection by Web Admin Comments are off
One of the neat things we get to do at PacketSled is work with our customers to address a very large variety of problems that don’t always immediately manifest themselves as obvious security incidents. This last week we had a customer experiencing an unusually large volume of email spam, some of which had malware payloads, some of which had adware payloads, and some of which was, well, just spam. The customer’s question was very interesting: “Do any of the spam senders have anything in common?” To answer this question, we turn to a core feature of PacketSled’s natural language search called “clustering.” Clustering enables a security analyst or incident responder to see the relationship between normally otherwise disparate data that traverses their network. Because PacketSled creates a semantic record of every conversation on the network by extracting all the meaningful attributes from its observed flows, we can establish the relationship between two or more otherwise seemingly disparate activities on any network. For example, clustering filenames on destination ip addresses will show us the nature of the files that our users are downloading, irrespective of protocol (HTTP, FTP, dropbox, etc). This feature is immensely helpful during breach analysis and incident response to determine the “who, what, when, where, how” in an incident. In this case, we’re trying to define why the spam filter isn’t working, and what the resolution should be. The obvious places to look for commonalities are in the characteristics of the email itself. First, we issue the search “cluster subject on [receiver_email]” and we return a list of all email subjects, followed by a count of those subjects. Clicking on that result shows us the recipients that received any email with that subject in it. redacted-cluster-on-no-geo (note: the recipients and subjects of legitimate emails have been redacted) As we examine the data, we start to notice that there seems to be a heavy correlation between any email that originates outside of the recipient’s country (in this case, non-US traffic), and the content of the emails. To validate this, we simply add the modifiers src_geo != lo (for local), and src_geo != US (to eliminate any emails that originated from the recipient’s geo, and we add the country of origin to the cluster query (src_geo). redacted-cluster-on-src_ip This is where the results get -very interesting-. There is not a single email in the entirety of 24 hours that originates outside of the customer’s native geography which is a legitimate email. Subjects range from telepathy classes to blue pill sales. A simple right click on the IP address in the visualization, and we can lookup the originator of over 90% of this traffic: Screen Shot 2015-02-18 at 3.13.55 PM Seems that LeaseWeb’s customers are our primary offenders. In this case, our customer had a legacy rule deployed in their message gateway (that is managed by a third party) to allow anything from that class A, irrespective of content. Nobody at the organization is quite sure why, but they are sure that this would have taken several days, if not weeks to hunt down, and exposed most of their users to large amounts of unnecessary risk. PacketSled enabled this customer to resolve this issue in record time, keeping the customer safe and productive.

Detecting ShellShock Using PacketSled

in Articles, PacketSled, Threat Detection by Web Admin Comments are off
Knock Knock. Bash Bash.  The scope and scale of the bash attack is fairly well known even though its only been public for 24 hours, but in case you’ve missed out on it somehow, here’s a quick recap of the CVE summary that will provide some context:
GNU Bash through 4.3 processes trailing strings after function definitions in the values of environment variables, which allows remote attackers to execute arbitrary code via a crafted environment, as demonstrated by vectors involving the ForceCommand feature in OpenSSH sshd, the mod_cgi and mod_cgid modules in the Apache HTTP Server, scripts executed by unspecified DHCP clients, and other situations in which setting the environment occurs across a privilege boundary from Bash execution.
Most of the early exploit activity has leveraged CGI to exploit shell commands. Exploit POCs for other protocols will follow for sure, and we’ll be posting about detections for those in real-time. For the time being, we can make one broad assertion, which is that the payload of the attack will exist in a relatively obvious place, as it did with the HTTP proof that’s out. In order to properly detect and investigate sessions that leverage this vulnerability in your environment, the first step is to isolate the various attributes and their values (HTTP headers in this example). The good news –  If you already have PacketSled, you have all of this data preloaded, not only for HTTP, but for telnet, DHCP, and 2000 other protocols. As Troy Hunt mentioned in his post today, there will likely be additional iterations of the vulnerability. Automatic detection of ShellShock atttempts on a running system: PacketSled behavioral detections were updated for all customers today with a level 10 alert that will fire for ShellShock exploit attempts. The severity of this attack will likely go down over time, as systems become immune or at least improve their resistance to the problem. If you see this detection on your dashboard,
ShellShock Detection in Dashboard

ShellShock Detection in Overview

then the good news is that you already have the detection built in, and any incidents that have been identified are made available for investigation simply by clicking them. If you do not see this security incident, and want to manually identify any sessions that may be exhibiting shellshock behavior, you can also use the natural language search engine in investigator to look through all of your network traffic, with one easy query. For instance:
today http user_agent like /\(\)\s{\s.*;/
 
ShellShock-New

ShellShock Investigation in PacketSled

This same query can be run against any attribute in an http flow. Other common places to look include:
  • The User-Agent HTTP header (user_agent)
  • Cookie (cookie)
  • Host (server)
  • Referrer (referrer)
  Finding your highest risk hosts: Now that we’ve identified attempts to exploit this vulnerability, the next logical step is to determine how many hosts we have that are actively executing code out of /cgi-bin. Simply use the investigator to search through all the web transactions that happened in your production environment:
today 10.1.1.0/24 http user_agent like /\(\)\s{\s.*;/ uri like /cgi-bin 
This search will identify hosts with active applications that run out of cgi-bin, who are currently targets of the attack. Total elapsed time: 30 seconds. 

The Challenges of Threat Detection in a Changing Landscape

in Articles, Big Data Security Analytics, PacketSled, Threat Detection by Web Admin Comments are off
Having spent 20 years being exposed to the real world security problems of customers as a trusted advisor, I’ve had the unique privilege of observing the common challenges that large financial, retail, defense, and critical infrastructure organizations face when trying to address complex, evolving threats. I’ve also watched the security community go from a niche, perhaps misunderstood group of ex-hackers trying to solve the problems that they discover in the soft underbelly of the early Internet to building an entire multi-billion dollar industry who’s goal it is to address the problems that plagues every employee, officer, director, and consumer. Companies have spent millions of dollars on security products to protect themselves and their customer’s data. However, there are still holes, and so-called advanced threats are getting through the cracks. What is needed, once and for all, is a solution that does a better job of detecting advanced threats, as well as providing a means for containing, analyzing and remediating unknown threats that have gotten through traditional defenses. Today’s security practitioners, engineers, and operating teams are currently using SIEM and/or full packet network forensics. Let’s talk about these options. Screen Shot 2013-12-17 at 6.15.33 PMOption 1- SIEM Tools – The idea here is that millions and millions of events being generated by hundreds or thousands of devices are cataloged in a central repository for correlation and processing. Let’s consider this for a minute; In order for technologies that rely on log data to correlate events, three things have to happen: 1. The devices that matter in the context of a security event have to be pointing log data at the SIEM. Most organizations typically prioritize this by starting with their security infrastructure (FW, IPS/IDS, Endpoint). This requirement to prioritize the output of these technologies seems to ignore the fact that the attack vector may have been designed specifically to avoid or elude them, and that the entry point for advanced threats may be somewhere else entirely. 2. Accuracy of the logs you require relies on integrity of the endpoints generating these logs – Attackers compromise IPS/FW technologies routinely, pivoting from internally compromised hosts into the admin interface of these technologies. At that point, its fairly trivial to create or disable rules that enable them to add a pathway for tunneled traffic that goes undetected by said devices, and certainly isn’t generating any useful log data that might be caught by the SIEM. 3. Alerts setup correctly – So what do you do with all the data you now have in one pane of glass? (if you’re even able to do that). Most analysts will tell you that the bulk of alerts they currently look at are not valuable, and that a very small percentage of the data coming across the console pertains to real security threats. Getting meaningful event correlation out of standalone SIEM using noise reduction techniques is difficult to do correctly, and even some of the best security organizations struggle with it.   full packetOption 2- Full Packet Solutions– Traditional full packet technologies offer the promise of much deeper insight into the activities of attackers than traditional log-only solutions, but they exhibit some fundamental problems that preclude them from operating in a manner that is consistent with anything other than “after the fact” scenarios. Areas of concern include: 1. Historical forensic record length – If a user issues a SQL query that results in a dump of thousands of records, and the network forensics solution has visibility into that session, guess what – you’re storing those thousands of records now. If you place a sensor on the ingress/egress points of your network, and someone watches a 40 minute youtube video on bonsai gardening, you get to keep that too. Neither of these things are valuable in the context of a security incident, but nevertheless, they’re quickly eating up expensive storage on your network. While your SOC team may learn some new root trimming techniques, they aren’t getting security value from the data. The fundamental challenge here is that storage requirements exceed the need to retain the data. One saturated 10Gbps link can produce up to 108TB of stored data per day, or 3.2 petabytes per month. It’s no mystery why these tools aren’t being deployed enterprise-wide. The incidents are buried in forests of tiny trees. 2. Fast, intelligent access to the data – Full IP packet data is by definition binary. Other than augmentation of the data with descriptors, full packet solutions have no way of answering user queries or machine generated analytical inquires in a reasonable timeframe that is acceptable for a typical incident response plan. In short – pulling back full packet is slow and ineffective in a real-time solution. It also makes it impossible to distribute the solution over very large environments, because sending the same data to a remote central repository would absolutely destroy even the most resilient networks. 3. Cost of retention –78% of all breaches are being discovered months, or sometimes even years after they’ve occurred. The data is already gone, and the damage is already done. Buying more disks may sound like a cheap option, but ask anyone in charge of buying these solutions, and you will quickly be corrected to understand that an enterprise-wide deployment of full packet capture simply isn’t an option financially. At PacketSled, we care deeply about the issues that plague today’s technology-enabled companies. We’re delivering solutions to these very difficult problems by building a next generation of products that can produce answers about attacker behavior, assets that are or may be impacted, and fastest path to threat eradication / remediation. We do it at scale, in real-time, for the entire enterprise, and in the cloud, at a cost that is commensurate with the security budgets of our customers. I’m very excited to be here, and I look forward to working with you. Matt Harrigan President & CEO PacketSled

© 2017 PacketSled, Inc.