Dr Jim Treinen
Carbon Black
Dr. Jim Treinen is Carbon Black’s Vice President of Threat. A security industry veteran with 15 years of experience, Jim is based in Boulder, Colorado, and is responsible for original threat research and operationalising threat intelligence. Additionally, he leads the team in charge of publishing research findings and emerging trends as they pertain to the cyber security industry. As a holder of a Ph.D. and M.S. in Computer Science, with a focus on machine learning and graph algorithms, Jim is an expert in data analysis related to enterprise security.   AusCERT18_Jim_Treinen_Analyzing-The-Killchain-AusCert-CB-Template

Analyzing the Kill Chain: How to Load Your Gun when you can’t buy a silver bullet

This session is a formal survey of the security analytics landscape through both an industry and academic lens. Dr. Jim Treinen will discuss the strengths and failings of numerous techniques and proactively answer the questions you should be asking. The session will conclude with a proposed framework for considering analytics in your environment, and reserve time for answering practical questions. Dr. Jim Treinen, an expert in security analytics and industry veteran, will provide a no-nonsense survey of the most popular security analytics techniques in use today.  Aimed at the experienced security practitioner hoping to depart with a greater understanding of the landscape, this session’s primary goal is education. To that end, Dr. Treinen will reinforce a few simple points: Point 1: Analytics is a Spectrum.  Different approaches have different strengths, and they also have individual weaknesses.   Each of the following subject areas will be reviewed in detail:
  • Intelligence Analysis
  • Signatures
  • Heuristics
  • Device and User Based Behavioral Modeling
  • Machine Learning
  • Deep Learning
  • Regression Analysis
  • Predictive Analysis
  • Structural Analysis
Given the strengths and weaknesses of each of these approaches, we should think of them as neither better or worse than the other, but as complimentary components of a holistic solution. Point 2: Defense in depth requires analytics in depth. See Point 1.  Each technique is good at catching certain classes of attacks, but falls short at detecting others.  Just as we deploy defense in depth as an overarching defense strategy, we need to start considering analytics in depth to support the same. Point 3:  You can’t catch everything in real time. The differences between fast analytics and slow analytics will be discussed, with an overview of data mining and Lambda Architectures, and how they are relevant to the field of security analytics. Point 4: Keep it simple Our field is in the middle of an esoteric algorithms arms race.  More often than not, if the inputs are designed correctly, the simple techniques are the best techniques.  Feature design will be discussed, as well as a deciphering of receiver operating characteristics for assessing accuracy. Point 5: Every output is an input. There are no silver bullets: if you invent a deterministic detection algorithm; the world is quite literally yours.  We should all keep in mind that the output of any analytics solution requires interpretation of a secondary decision process.  Sometimes this is another machine, often times it is a human.  We will discuss hierarchical analysis systems, how they relate to false positive reduction, and how we can incorporate feedback loops for accuracy refinement. Finally, a conclusion and proposed architecture will be presented. The discussion will include use of the above-mentioned techniques as complements to each other in the context of a Lambda Architecture.