Contents

Extending Snort

Plugins

Plugins have an associated API defined for each type, all of which share a common header, called the BaseApi. A dynamic library makes its plugins available by exporting the snort_plugins symbol, which is a null terminated array of BaseApi pointers.

The BaseApi includes type, name, API version, plugin version, and function pointers for constructing and destructing a Module. The specific API add various other data and functions for their given roles.

Modules

If we are defining a new Inspector called, say, gadget, it might be configured in snort.lua like this:

gadget =
{
    brain = true,
    claw = 3
}

When the gadget table is processed, Snort will look for a module called gadget. If that Module has an associated API, it will be used to configure a new instance of the plugin. In this case, a GadgetModule would be instantiated, brain and claw would be set, and the Module instance would be passed to the GadgetInspector constructor.

Module has three key virtual methods:

  • begin() - called when Snort starts processing the associated Lua table. This is a good place to allocate any required data and set defaults.

  • set() - called to set each parameter after validation.

  • end() - called when Snort finishes processing the associated Lua table. This is where additional integrity checks of related parameters should be done.

The configured Module is passed to the plugin constructor which pulls the configuration data from the Module. For non-trivial configurations, the working paradigm is that Module hands a pointer to the configured data to the plugin instance which takes ownership.

Note that there is at most one instance of a given Module, even if multiple plugin instances are created which use that Module. (Multiple instances require Snort binding configuration.)

Inspectors

There are several types of inspector, which determines which inspectors are executed when:

  • IT_BINDER - determines which inspectors apply to given flows

  • IT_WIZARD - determines which service inspector to use if none explicitly bound

  • IT_PACKET - used to process all packets before session and service processing (e.g. normalize)

  • IT_NETWORK - processes packets w/o service (e.g. arp_spoof, back_orifice)

  • IT_STREAM - for flow tracking, ip defrag, and tcp reassembly

  • IT_SERVICE - for http, ftp, telnet, etc.

  • IT_PROBE - process all packets after all the above (e.g. perf_monitor, port_scan)

  • IT_PASSIVE - for configuration only or data consuming

Codecs

The Snort Codecs decipher raw packets. These Codecs are now completely pluggable; almost every Snort Codec can be built dynamically and replaced with an alternative, customized Codec. The pluggable nature has also made it easier to build new Codecs for protocols without having to touch the Snort code base.

The first step in creating a Codec is defining its class and protocol. Every Codec must inherit from the Snort Codec class defined in "framework/codec.h". The following is an example Codec named "example" and has an associated struct that is 14 bytes long.

#include <cstdint>
#include <arpa/inet.h>
#include “framework/codec.h”
#include "main/snort_types.h"
#define EX_NAME “example”
#define EX_HELP “example codec help string”
struct Example
{
    uint8_t dst[6];
    uint8_t src[6];
    uint16_t ethertype;
    static inline uint8_t size()
    { return 14; }
}
class ExCodec : public Codec
{
public:
    ExCodec() : Codec(EX_NAME) { }
    ~ExCodec() { }
    bool decode(const RawData&, CodecData&, DecodeData&) override;
    void get_protocol_ids(std::vector<uint16_t>&) override;
};

After defining ExCodec, the next step is adding the Codec’s decode functionality. The function below does this by implementing a valid decode function. The first parameter, which is the RawData struct, provides both a pointer to the raw data that has come from a wire and the length of that raw data. The function takes this information and validates that there are enough bytes for this protocol. If the raw data’s length is less than 14 bytes, the function returns false and Snort discards the packet; the packet is neither inspected nor processed. If the length is greater than 14 bytes, the function populates two fields in the CodecData struct, next_prot_id and lyr_len. The lyr_len field tells Snort the number of bytes that this layer contains. The next_prot_id field provides Snort the value of the next EtherType or IP protocol number.

bool ExCodec::decode(const RawData& raw, CodecData& codec, DecodeData&)
{
   if ( raw.len < Example::size() )
       return false;
    const Example* const ex = reinterpret_cast<const Example*>(raw.data);
    codec.next_prot_id = ntohs(ex->ethertype);
    codec.lyr_len = ex->size();
    return true;
}

For instance, assume this decode function receives the following raw data with a validated length of 32 bytes:

00 11 22 33 44 55 66 77    88 99 aa bb 08 00 45 00
00 38 00 01 00 00 40 06    5c ac 0a 01 02 03 0a 09

The Example struct’s EtherType field is the 13 and 14 bytes. Therefore, this function tells Snort that the next protocol has an EtherType of 0x0800. Additionally, since the lyr_len is set to 14, Snort knows that the next protocol begins 14 bytes after the beginning of this protocol. The Codec with EtherType 0x0800, which happens to be the IPv4 Codec, will receive the following data with a validated length of 18 ( == 32 – 14):

45 00 00 38 00 01 00 00    40 06 5c ac 0a 01 02 03
0a 09

How does Snort know that the IPv4 Codec has an EtherType of 0x0800? The Codec class has a second virtual function named get_protocol_ids(). When implementing the function, a Codec can register for any number of values between 0x0000 - 0xFFFF. Then, if the next_proto_id is set to a value for which this Codec has registered, this Codec’s decode function will be called. As a general note, the protocol ids between [0, 0x00FF] are IP protocol numbers, [0x0100, 0x05FF] are custom types, and [0x0600, 0xFFFF] are EtherTypes.

For example, in the get_protocol_ids function below, the ExCodec registers for the protocols numbers 17, 787, and 2054. 17 happens to be the protocol number for UDP while 2054 is ARP’s EtherType. Therefore, this Codec will now attempt to decode UDP and ARP data. Additionally, if any Codec sets the next_protocol_id to 787, ExCodec’s decode function will be called. Some custom protocols are already defined in the file "protocols/protocol_ids.h"

void ExCodec::get_protocol_ids(std::vector<uint16_t>&v)
{
    v.push_back(0x0011); // == 17  == UDP
    v.push_back(0x1313); // == 787  == custom
    v.push_back(0x0806); // == 2054  == ARP
}

To register a Codec for Data Link Type’s rather than protocols, the function get_data_link_type() can be similarly implemented.

The final step to creating a pluggable Codec is the snort_plugins array. This array is important because when Snort loads a dynamic library, the program only find plugins that are inside the snort_plugins array. In other words, if a plugin has not been added to the snort_plugins array, that plugin will not be loaded into Snort.

Although the details will not be covered in this post, the following code snippet is a basic CodecApi that Snort can load. This snippet can be copied and used with only three minor changes. First, in the function ctor, ExCodec should be replaced with the name of the Codec that is being built. Second, EX_NAME must match the Codec’s name or Snort will be unable to load this Codec. Third, EX_HELP should be replaced with the general description of this Codec. Once this code snippet has been added, ExCodec is ready to be compiled and plugged into Snort.

static Codec* ctor(Module*)
{ return new ExCodec; }
static void dtor(Codec *cd)
{ delete cd; }
static const CodecApi ex_api =
{
    {
        PT_CODEC,
        EX_NAME,
        EX_HELP,
        CDAPI_PLUGIN_V0,
        0,
        nullptr,
        nullptr,
    },
    nullptr, // pointer to a function called during Snort's startup.
    nullptr, // pointer to a function called during Snort's exit.
    nullptr, // pointer to a function called during thread's startup.
    nullptr, // pointer to a function called during thread's destruction.
    ctor, // pointer to the codec constructor.
    dtor, // pointer to the codec destructor.
};
SO_PUBLIC const BaseApi* snort_plugins[] =
{
    &ex_api.base,
    nullptr
};

Two example Codecs are available in the extra directory on git and the extra tarball on the Snort page. One of those examples is the Token Ring Codec while the other example is the PIM Codec.

As a final note, there are four more virtual functions that a Codec should implement: encode, format, update, and log. If the functions are not implemented Snort will not throw any errors. However, Snort may also be unable to accomplish some of its basic functionality.

  • encode is called whenever Snort actively responds and needs to builds a packet, i.e. whenever a rule using an IPS ACTION like react, reject, or rewrite is triggered. This function is used to build the response packet protocol by protocol.

  • format is called when Snort is rebuilding a packet. For instance, every time Snort reassembles a TCP stream or IP fragment, format is called. Generally, this function either swaps any source and destination fields in the protocol or does nothing.

  • update is similar to format in that it is called when Snort is reassembling a packet. Unlike format, this function only sets length fields.

  • log is called when either the log_codecs logger or a custom logger that calls PacketManager::log_protocols is used when running Snort.

IPS Actions

Action plugins specify a builtin action in the API which is used to determine verdict. (Conversely, builtin actions don’t have an associated plugin function.)

Trace Loggers

The Trace Loggers print trace messages. They can be implemented as inspector plugins.

The first step is creating a custom logger by inheriting from the Snort TraceLogger class. The following is an example TraceLogger.

class FooLogger : public TraceLogger
{
public:
    void log(const char*, const char*, uint8_t, const char*, const Packet*) override
    { printf("%s%s\n", "Foo", "Bar"); }
};

To instantiate logger objects it’s needed to create a logger factory derived from the Snort TraceLoggerFactory class.

class FooFactory : public TraceLoggerFactory
{
public:
    TraceLogger* instantiate() override
    { return new FooLogger(); }
};

Once the Factory is created, Inspector and appropriate Module are needed. Inspector::configure() must initialize the logger factory.

bool FooInspector::configure(SnortConfig* sc) override
{
    return TraceApi::override_logger_factory(sc, new FooFactory());
}

Piglet Test Harness

In order to assist with plugin development, an experimental mode called "piglet" mode is provided. With piglet mode, you can call individual methods for a specific plugin. The piglet tests are specified as Lua scripts. Each piglet test script defines a test for a specific plugin.

Here is a minimal example of a piglet test script for the IPv4 Codec plugin:

plugin =
{
    type = "piglet",
    name = "codec::ipv4",
    use_defaults = true,
    test = function()
        local daq_header = DAQHeader.new()
        local raw_buffer = RawBuffer.new("some data")
        local codec_data = CodecData.new()
        local decode_data = DecodeData.new()
        return Codec.decode(
            daq_header,
            raw_buffer,
            codec_data,
            decode_data
        )
    end
}

To run snort in piglet mode, first build snort with the ENABLE_PIGLET option turned on (pass the flag -DENABLE_PIGLET:BOOL=ON in cmake).

Then, run the following command:

snort --script-path $test_scripts --piglet

(where $test_scripts is the directory containing your piglet tests).

The test runner will generate a check-like output, indicating the the results of each test script.

Piglet Lua API

This section documents the API that piglet exposes to Lua. Refer to the piglet directory in the source tree for examples of usage.

Note: Because of the differences between the Lua and C++ data model and type system, not all parameters map directly to the parameters of the underlying C\++ member functions. Every effort has been made to keep the mappings consist, but there are still some differences. They are documented below.

Plugin Instances

For each test, piglet instantiates plugin specified in the name field of the plugin table. The virtual methods of the instance are exposed in a table unique to each plugin type. The name of the table is the CamelCase name of the plugin type.

For example, codec plugins have a virtual method called decode. This method is called like this:

Codec.decode(...)

Codec

  • Codec.get_data_link_type() → { int, int, … }

  • Codec.get_protocol_ids() → { int, int, … }

  • Codec.decode(DAQHeader, RawBuffer, CodecData, DecodeData) → bool

  • Codec.log(RawBuffer, uint[lyr_len])

  • Codec.encode(RawBuffer, EncState, Buffer) → bool

  • Codec.update(uint[flags_hi], uint[flags_lo], RawBuffer, uint[lyr_len] → int

  • Codec.format(bool[reverse], RawBuffer, DecodeData)

Differences:

  • In Codec.update(), the (uint64_t) flags parameter has been split into flags_hi and flags_lo

Inspector

  • Inspector.configure()

  • Inspector.tinit()

  • Inspector.tterm()

  • Inspector.likes(Packet)

  • Inspector.eval(Packet)

  • Inspector.clear(Packet)

  • Inspector.get_buf_from_key(string[key], Packet, RawBuffer) → bool

  • Inspector.get_buf_from_id(uint[id], Packet, RawBuffer) → bool

  • Inspector.get_buf_from_type(uint[type], Packet, RawBuffer) → bool

  • Inspector.get_splitter(bool[to_server]) → StreamSplitter

Differences: * In Inspector.configure(), the SnortConfig* parameter is passed implicitly. * the overloaded get_buf() member function has been split into three separate methods.

IpsOption

  • IpsOption.hash() → int

  • IpsOption.is_relative() → bool

  • IpsOption.fp_research() → bool

  • IpsOption.get_cursor_type() → int

  • IpsOption.eval(Cursor, Packet) → int

  • IpsOption.action(Packet)

IpsAction

  • IpsAction.exec(Packet)

Logger

  • Logger.open()

  • Logger.close()

  • Logger.reset()

  • Logger.alert(Packet, string[message], Event)

  • Logger.log(Packet, string[message], Event)

SearchEngine

Currently, SearchEngine does not expose any methods.

SoRule

Currently, SoRule does not expose any methods.

Interface Objects

Many of the plugins take C++ classes and structs as arguments. These objects are exposed to the Lua API as Lua userdata. Exposed objects are instantiated by calling the new method from each object’s method table.

For example, the DecodeData object can be instantiated and exposed to Lua like this:

local decode_data = DecodeData.new(...)

Each object also exposes useful methods for getting and setting member variables, and calling the C++ methods contained in the the object. These methods can be accessed using the : accessor syntax:

decode_data:set({ sp = 80, dp = 3500 })

Since this is just syntactic sugar for passing the object as the first parameter of the function DecodeData.set, an equivalent form is:

decode_data.set(decode_data, { sp = 80, dp = 3500 })

or even:

DecodeData.set(decode_data, { sp = 80, dp = 3500 })

Buffer

  • Buffer.new(string[data]) → Buffer

  • Buffer.new(uint[length]) → Buffer

  • Buffer.new(RawBuffer) → Buffer

  • Buffer:allocate(uint[length]) → bool

  • Buffer:clear()

CodecData

  • CodecData.new() → CodecData

  • CodecData.new(uint[next_prot_id]) → CodecData

  • CodecData.new(fields) → CodecData

  • CodecData:get() → fields

  • CodecData:set(fields)

fields is a table with the following contents:

  • next_prot_id

  • lyr_len

  • invalid_bytes

  • proto_bits

  • codec_flags

  • ip_layer_cnt

  • ip6_extension_count

  • curr_ip6_extension

  • ip6_csum_proto

Cursor

  • Cursor.new() → Cursor

  • Cursor.new(Packet) → Cursor

  • Cursor.new(string[data]) → Cursor

  • Cursor.new(RawBuffer) → Cursor

  • Cursor:reset()

  • Cursor:reset(Packet)

  • Cursor:reset(string[data])

  • Cursor:reset(RawBuffer)

DAQHeader

  • DAQHeader.new() → DAQHeader

  • DAQHeader.new(fields) → DAQHeader

  • DAQHeader:get() → fields

  • DAQHeader:set(fields)

fields is a table with the following contents:

  • caplen

  • pktlen

  • ingress_index

  • egress_index

  • ingress_group

  • egress_group

  • flags

  • opaque

DecodeData

  • DecodeData.new() → DecodeData

  • DecodeData.new(fields) → DecodeData

  • DecodeData:reset()

  • DecodeData:get() → fields

  • DecodeData:set(fields)

  • DecodeData:set_ipv4_hdr(RawBuffer, uint[offset])

fields is a table with the following contents:

  • sp

  • dp

  • decode_flags

  • type

EncState

  • EncState.new() → EncState

  • EncState.new(uint[flags_lo]) → EncState

  • EncState.new(uint[flags_lo], uint[flags_hi]) → EncState

  • EncState.new(uint[flags_lo], uint[flags_hi], uint[next_proto]) → EncState

  • EncState.new(uint[flags_lo], uint[flags_hi], uint[next_proto], uint[ttl]) → EncState

  • EncState.new(uint[flags_lo], uint[flags_hi], uint[next_proto], uint[ttl], uint[dsize]) → EncState

Event

  • Event.new() → Event

  • Event.new(fields) → Event

  • Event:get() → fields

  • Event:set(fields)

fields is a table with the following contents:

  • event_id

  • event_reference

  • sig_info

    • generator

    • id

    • rev

    • class_id

    • priority

    • text_rule

    • num_services

Flow

  • Flow.new() → Flow

  • Flow:reset()

Packet

  • Packet.new() → Packet

  • Packet.new(string[data]) → Packet

  • Packet.new(uint[size]) → Packet

  • Packet.new(fields) → Packet

  • Packet.new(RawBuffer) → Packet

  • Packet.new(DAQHeader) → Packet

  • Packet:set_decode_data(DecodeData)

  • Packet:set_data(uint[offset], uint[length])

  • Packet:set_flow(Flow)

  • Packet:get() → fields

  • Packet:set()

  • Packet:set(string[data])

  • Packet:set(uint[size])

  • Packet:set(fields)

  • Packet:set(RawBuffer)

  • Packet:set(DAQHeader)

fields is a table with the following contents:

  • packet_flags

  • xtradata_mask

  • proto_bits

  • application_protocol_ordinal

  • alt_dsize

  • num_layers

  • iplist_id

  • user_policy_id

  • ps_proto

Note: Packet.new() and Packet:set() accept multiple arguments of the types described above in any order

RawBuffer

  • RawBuffer.new() → RawBuffer

  • RawBuffer.new(uint[size]) → RawBuffer

  • RawBuffer.new(string[data]) → RawBuffer

  • RawBuffer:size() → int

  • RawBuffer:resize(uint[size])

  • RawBuffer:write(string[data])

  • RawBuffer:write(string[data], uint[size])

  • RawBuffer:read() → string

  • RawBuffer:read(uint[end]) → string

  • RawBuffer:read(uint[start], uint[end]) → string

Note: calling RawBuffer.new() with no arguments returns a RawBuffer of size 0

StreamSplitter

  • StreamSplitter:scan(Flow, RawBuffer) → int, int

  • StreamSplitter:scan(Flow, RawBuffer, uint[len]) → int, int

  • StreamSplitter:scan(Flow, RawBuffer, uint[len], uint[flags]) → int, int

  • StreamSplitter:reassemble(Flow, uint[total], uint[offset], RawBuffer) → int, RawBuffer

  • StreamSplitter:reassemble(Flow, uint[total], uint[offset], RawBuffer, uint[len]) → int, RawBuffer

  • StreamSplitter:reassemble(Flow, uint[total], uint[offset], RawBuffer, uint[len], uint[flags]) → int, RawBuffer

  • StreamSplitter:finish(Flow) → bool

Note: StreamSplitter does not have a new() method, it must be created by an inspector via Inspector.get_splitter()

Performance Considerations for Developers

  • Since C compilers evaluate compound conditional expression from left to right, put the costly condition last. Put the often-false condition first in && expression. Put the often-true condition first in || expression.

  • Use emplace_back/emplace instead of push_back/insert on STL containers.

  • In general, unordered_map is faster than map for frequent lookups using integer key on relatively static collection of unsorted elements. Whereas, map is faster for frequent insertions/deletions/iterations and for non-integer key such as string or custom objects. Consider the same factors when deciding ordered vs. unordered multimap and set.

  • Iterate using range-based for loop with reference (i.e., auto&).

  • Be mindful of construction and destruction of temporary objects which can be wasteful. Consider using std::move, std::swap, lvalue reference (&), and rvalue reference (&&).

  • Avoid thread-local storage. When unavoidable, minimize frequent TLS access by caching it to a local variable.

  • When writing inter-library APIs, consider interfaces depending on use cases to minimize context switching. For example, if two APIs foo() and bar() are needed to call, combine these into a single API to minimize jumps.

Coding Style

All new code should try to follow these style guidelines. These are not yet firm so feedback is welcome to get something we can live with.

General

  • Generally try to follow https://google.github.io/styleguide/cppguide.html, but there are some differences documented here.

  • Each source directory should have a dev_notes.txt file summarizing the key points and design decisions for the code in that directory. These are built into the developers guide.

  • Makefile.am and CMakeLists.txt should have the same files listed in alpha order. This makes it easier to maintain both build systems.

  • All new code must come with unit tests providing 95% coverage or better.

  • Generally, Catch is preferred for tests in the source file and CppUTest is preferred for test executables in a test subdirectory.

C++ Specific

  • Do not use exceptions. Exception-safe code is non-trivial and we have ported legacy code that makes use of exceptions unwise. There are a few exceptions to this rule for the memory manager, shell, etc. Other code should handle errors as errors.

  • Do not use dynamic_cast or RTTI. Although compilers are getting better all the time, there is a time and space cost to this that is easily avoided.

  • Use smart pointers judiciously as they aren’t free. If you would have to roll your own, then use a smart pointer. If you just need a dtor to delete something, write the dtor.

  • Prefer and over && and or over || for new source files.

  • Use nullptr instead of NULL.

  • Use new, delete, and their [] counterparts instead of malloc and free except where realloc must be used. But try not to use realloc. New and delete can’t return nullptr so no need to check. And Snort’s memory manager will ensure that we live within our memory budget.

  • Use references in lieu of pointers wherever possible.

  • Use the order public, protected, private top to bottom in a class declaration.

  • Keep inline functions in a class declaration very brief, preferably just one line. If you need a more complex inline function, move the definition below the class declaration.

  • The goal is to have highly readable class declarations. The user shouldn’t have to sift through implementation details to see what is available to the client.

  • Any using statements in source files should be added only after all includes have been declared.

Naming

  • Use camel case for namespaces, classes, and types like WhizBangPdfChecker.

  • Use lower case identifiers with underscore separators, e.g. some_function() and my_var.

  • Do not start or end variable names with an underscore. This has a good chance of conflicting with macro and/or system definitions.

  • Use lower case filenames with underscores.

Comments

  • Write comments sparingly with a mind towards future proofing. Often the comments can be obviated with better code. Clear code is better than a comment.

  • Heed Tim Ottinger’s Rules on Comments (https://disqus.com/by/tim_ottinger/):

    1. Comments should only say what the code is incapable of saying.

    2. Comments that repeat (or pre-state) what the code is doing must be removed.

    3. If the code CAN say what the comment is saying, it must be changed at least until rule #2 is in force.

  • Function comment blocks are generally just noise that quickly becomes obsolete. If you absolutely must comment on parameters, put each on a separate line along with the comment. That way changing the signature may prompt a change to the comments too.

  • Use FIXIT (not FIXTHIS or TODO or whatever) to mark things left for a day or even just a minute. That way we can find them easily and won’t lose track of them.

  • Presently using FIXIT-X where X is one of the characters below. Place A and W comments on the exact warning line so we can match up comments and build output. Supporting comments can be added above.

  • A = known static analysis issue

  • D = deprecated - code to be removed after users update

  • E = enhancement - next steps for incomplete features (not a bug)

  • H = high priority - urgent deficiency

  • L = low priority - cleanup or similar technical debt (not a bug)

  • M = medium priority - suspected non-urgent deficiency

  • P = performance issue (not a bug)

  • W = warning - known compiler warning

  • Put the copyright(s) and license in a comment block at the top of each source file (.h and .cc). Don’t bother with trivial scripts and make foo. Some interesting Lua code should get a comment block too. Copy and paste exactly from src/main.h (don’t reformat).

  • Put author, description, etc. in separate comment(s) following the license. Do not put such comments in the middle of the license foo. Be sure to put the author line ahead of the header guard to exclude them from the developers guide. Use the following format, and include a mention to the original author if this is derived work:

    // ips_dnp3_obj.cc author Maya Dagon <mdagon@cisco.com>
    // based on work by Ryan Jordan
  • Each header should have a comment immediately after the header guard to give an overview of the file so the reader knows what’s going on.

  • Use the following comment on switch cases that intentionally fall through to the next case to suppress compiler warning on known valid cases:

    // fallthrough

Logging

  • Messages intended for the user should not look like debug messages. Eg, the function name should not be included. It is generally unhelpful to include pointers.

  • Most debug messages should just be deleted.

  • Don’t bang your error messages (no !). The user feels bad enough about the problem already w/o you shouting at him.

Types

  • Use logical types to make the code clearer and to help the compiler catch problems. typedef uint16_t Port; bool foo(Port) is way better than int foo(int port).

  • Use forward declarations (e.g. struct SnortConfig;) instead of void*.

  • Try not to use extern data unless absolutely necessary and then put the extern in an appropriate header. Exceptions for things used in exactly one place like BaseApi pointers.

  • Use const liberally. In most cases, const char* s = "foo" should be const char* const s = "foo". The former goes in the initialized data section and the latter in read only data section.

  • But use const char s[] = "foo" instead of const char* s = "foo" when possible. The latter form allocates a pointer variable and the data while the former allocates only the data.

  • Use static wherever possible to minimize public symbols and eliminate unneeded relocations.

  • Declare functions virtual only in the parent class introducing the function (not in a derived class that is overriding the function). This makes it clear which class introduces the function.

  • Declare functions as override if they are intended to override a function. This makes it possible to find derived implementations that didn’t get updated and therefore won’t get called due a change in the parent signature.

  • Use bool functions instead of int unless there is truly a need for multiple error returns. The C-style use of zero for success and -1 for error is less readable and often leads to messy code that either ignores the various errors anyway or needlessly and ineffectively tries to do something about them. Generally that code is not updated if new errors are added.

Macros (aka defines)

  • In many cases, even in C++, use #define name "value" instead of a const char* const name = "value" because it will eliminate a symbol from the binary.

  • Use inline functions instead of macros where possible (pretty much all cases except where stringification is necessary). Functions offer better typing, avoid re-expansions, and a debugger can break there.

  • All macros except simple const values should be wrapped in () and all args should be wrapped in () too to avoid surprises upon expansion. Example:

    #define SEQ_LT(a,b)  ((int)((a) - (b)) <  0)
  • Multiline macros should be blocked (i.e. inside { }) to avoid if-else type surprises.

Formatting

  • Try to keep all source files under 2500 lines. 3000 is the max allowed. If you need more lines, chances are that the code needs to be refactored.

  • Indent 4 space chars … no tabs!

  • If you need to indent many times, something could be rewritten or restructured to make it clearer. Fewer indents is generally easier to write, easier to read, and overall better code.

  • Braces go on the line immediately following a new scope (function signature, if, else, loop, switch, etc.

  • Use consistent spacing and line breaks. Always indent 4 spaces from the breaking line. Keep lines less than 100 chars; it greatly helps readability.

    No:
        calling_a_func_with_a_long_name(arg1,
                                        arg2,
                                        arg3);
    Yes:
        calling_a_func_with_a_long_name(
            arg1, arg2, arg3);
  • Put function signature on one line, except when breaking for the arg list:

    No:
        inline
        bool foo()
        { // ...
    Yes:
        inline bool foo()
        { // ...
  • Put conditional code on the line following the if so it is easy to break on the conditional block:

    No:
        if ( test ) foo();
    Yes:
        if ( test )
            foo();

Headers

  • Don’t hesitate to create a new header if it is needed. Don’t lump unrelated stuff into an header because it is convenient.

  • Write header guards like this (leading underscores are reserved for system stuff). In my_header.h:

    #ifndef MY_HEADER_H
    #define MY_HEADER_H
    // ...
    #endif
  • Includes from a different directory should specify parent directory. This makes it clear exactly what is included and avoids the primordial soup that results from using -I this -I that -I the_other_thing … .

    // given:
    src/foo/foo.cc
    src/bar/bar.cc
    src/bar/baz.cc
    // in baz.cc
    #include "bar.h"
    // in foo.cc
    #include "bar/bar.h"
  • Includes within installed headers should specify parent directory.

  • Just because it is a #define doesn’t mean it goes in a header. Everything should be scoped as tightly as possible. Shared implementation declarations should go in a separate header from the interface. And so on.

  • All .cc files should include config.h with the standard block shown below immediately following the initial comment blocks and before anything else. This presents a consistent view of all included header files as well as access to any other configure-time definitions. No .h files should include config.h unless they are guaranteed to be local header files (never installed).

    #ifdef HAVE_CONFIG_H
    #include "config.h"
    #endif
  • A .cc should include its own .h before any others aside from the aforementioned config.h (including system headers). This ensures that the header stands on its own and can be used by clients without include prerequisites and the developer will be the first to find a dependency issue.

  • Split headers included from the local directory into a final block of headers. For a .cc file, the final order of sets of header includes should look like this:

    1. config.h

    2. its own .h file

    3. system headers (.h/.hpp/.hxx)

    4. C++ standard library headers (no file extension)

    5. Snort headers external to the local directory (path-prefixed)

    6. Snort headers in the local directory

  • Include required headers, all required headers, and nothing but required headers. Don’t just clone a bunch of headers because it is convenient.

  • Keep includes in alphabetical order. This makes it easier to maintain, avoid duplicates, etc.

  • Do not put using statements in headers unless they are tightly scoped.

Warnings

  • With g++, use at least these compiler flags:

    -Wall -Wextra -pedantic -Wformat -Wformat-security
    -Wunused-but-set-variable -Wno-deprecated-declarations
    -fsanitize=address -fno-omit-frame-pointer
  • With clang, use at least these compiler flags:

    -Wall -Wextra -pedantic -Wformat -Wformat-security
    -Wno-deprecated-declarations
    -fsanitize=address -fno-omit-frame-pointer
  • Two macros (PADDING_GUARD_BEGIN and PADDING_GUARD_END) are provided by utils/cpp_macros.h. These should be used to surround any structure used as a hash key with a raw comparator or that would otherwise suffer from unintentional padding. A compiler warning will be generated if any structure definition is automatically padded between the macro invocations.

  • Then Fix All Warnings and Aborts. None Allowed.

Uncrustify

Currently using uncrustify from at https://github.com/bengardner/uncrustify to reformat legacy code and anything that happens to need a makeover at some point.

The working config is crusty.cfg in the top level directory. It does well but will munge some things. Specially formatted INDENT-OFF comments were added in 2 places to avoid a real mess.

You can use uncrustify something like this:

uncrustify -c crusty.cfg --replace file.cc

Source Code

src/

This directory contains the program entry point, thread management, and control functions.

  • The main / foreground thread services control inputs from signals, the command line shell (if enabled), etc.

  • The packet / background threads service one input source apiece.

The main_loop() starts a new Pig when a new source (interface or pcap, etc.) is available if the number of running Pigs is less than configured.

It also does housekeeping functions like servicing signal flags, shell commands, etc.

The shell has to be explicitly enabled at build time to be available and then must be configured at run time to be activated. Multiple simultaneous remote shells are supported.

Unit test and piglet test harness build options also impact actual execution.

Reload is implemented by swapping a thread local config pointer by each running Pig. The inspector manager is called to empty trash if the main loop is not otherwise busy.

Reload policy is implemented by cloning the thread local config and overwriting the policy map and the inspection policy in the main thread. The inspector list from the old config’s inspection policy is copied into the inspection policy of the new config. After the config pointer is cloned, the new inspection policy elements (reloadable) such as inspectors, binder, wizard etc are read and instantiated. The inspector list of the new config is updated by swapping out the old inspectors, binder etc. with the newly instantiated elements. The reloaded inspectors, binders and other inspection policy elements are marked for deletion. After the new inspection policy is loaded, the thread local config pointer is swapped with the new cloned config by running Pig. This happens in the packet thread. The inspector manager is then called to delete any reloaded policy elements and empty trash.

main.h

Path = src/main.h

#ifndef MAIN_H
#define MAIN_H

#include "main/request.h"

struct lua_State;

const char* get_prompt();
SharedRequest get_current_request();

// commands provided by the snort module
int main_delete_inspector(lua_State* = nullptr);
int main_dump_stats(lua_State* = nullptr);
int main_rotate_stats(lua_State* = nullptr);
int main_reload_config(lua_State* = nullptr);
int main_reload_policy(lua_State* = nullptr);
int main_reload_module(lua_State* = nullptr);
int main_reload_daq(lua_State* = nullptr);
int main_reload_hosts(lua_State* = nullptr);
int main_process(lua_State* = nullptr);
int main_pause(lua_State* = nullptr);
int main_resume(lua_State* = nullptr);
int main_quit(lua_State* = nullptr);
int main_help(lua_State* = nullptr);

#ifdef SHELL
int main_dump_plugins(lua_State* = nullptr);
int main_detach(lua_State* = nullptr);
#endif

void main_poke(unsigned);

#endif

actions/

IPS actions allow you to execute custom responses to events. Unlike loggers, these are invoked before thresholding and can be used to control external agents (including loggers).

IPS rules have an associated type that determines what kind of action they trigger. The rule types defined in this module are:

  • log

  • pass

  • alert

  • drop

  • block

  • reset

There is also a "none" rule type, which is a no-op.

It also defines 3 active responses: * react * reject * rewrite

Reject performs active response to shutdown hostile network session by injecting TCP resets (TCP connections) or ICMP unreachable packets.

React sends an HTML page to the client, a RST to the server and blocks the flow. It is using payload_injector utilty. payload_injector should be configured when react is used.

Rewrite enables overwrite packet contents based on "replace" option in the rules.

actions.h

Path = src/actions/actions.h

#ifndef ACTIONS_H
#define ACTIONS_H

// Define action types and provide hooks to apply a given action to a packet

#include <cstdint>

#include "main/snort_types.h"

struct OptTreeNode;

namespace snort
{
struct Packet;

class SO_PUBLIC Actions
{
public:
    // FIXIT-L if Type is changed, RateFilterModule and type in actions.cc must be updated
    enum Type : uint8_t
    { NONE = 0, LOG, PASS, ALERT, DROP, BLOCK, RESET, MAX };

    static const char* get_string(Type);
    static Type get_type(const char*);

    static void execute(Type, struct Packet*, const struct OptTreeNode*,
        uint16_t event_id);

    static void apply(Type, struct Packet*);

    static inline bool is_pass(Type a)
    { return ( a == PASS ); }
};
}
#endif

ips_actions.h

Path = src/actions/ips_actions.h

#ifndef IPS_ACTIONS_H
#define IPS_ACTIONS_H

void load_actions();

#endif

catch/

Unit Test

This directory contains the unit-test interface.

snort_catch.h

Path = src/catch/snort_catch.h

#ifndef SNORT_CATCH_H
#define SNORT_CATCH_H

// This header adds a wrapper around Catch to facilitate testing in dynamic
// modules. Because of quirks in global storage (as used by Catch), an
// installer in the main binary is necessary to access Catch's global state and
// provide test cases from dynamic plugins to the global list of tests to be
// run. This header should be used instead of including catch.hpp directly.

// pragma for running unit tests on dynamic modules
#pragma GCC visibility push(default)
#include "catch.hpp"
#pragma GCC visibility pop

#include "main/snort_types.h"

namespace snort
{
class TestCaseInstaller
{
public:
    SO_PUBLIC TestCaseInstaller(void(*fun)(), const char* name);
    SO_PUBLIC TestCaseInstaller(void(*fun)(), const char* name, const char* group);
};

/*  translate this:
        SNORT_TEST_CASE("my_test", "[snort_tests"])
        {
            REQUIRE(true);
        }

    to this:
        static void snort_test1();
        static TestCaseInstaller test_test_installer1(snort_test1, "my_test", "[snort_tests]");
        static void snort_test1()
        {
            REQUIRE(true);
        }
*/
#define SNORT_CATCH_UNIQUE_NAME_LINE2( name, line ) name##line
#define SNORT_CATCH_UNIQUE_NAME_LINE( name, line ) SNORT_CATCH_UNIQUE_NAME_LINE2(name, line)
#define SNORT_CATCH_UNIQUE_NAME( name ) SNORT_CATCH_UNIQUE_NAME_LINE(name, __LINE__)
#define MAKE_SNORT_TEST_CASE(fun, ...) \
    static void fun(); \
    static snort::TestCaseInstaller SNORT_CATCH_UNIQUE_NAME(test_case_installer)(fun, __VA_ARGS__); \
    static void fun()

#undef TEST_CASE
#define TEST_CASE(...) MAKE_SNORT_TEST_CASE(SNORT_CATCH_UNIQUE_NAME(snort_test), __VA_ARGS__)
}

#endif

unit_test.h

Path = src/catch/unit_test.h

#ifndef UNIT_TEST_H
#define UNIT_TEST_H

#include "main/snort_types.h"

// Unit test interface

void catch_set_filter(const char* s);

bool catch_enabled();

int catch_test();

#endif

codecs/

This directory contains all of the codecs used for parsing raw fields contained within captured frames. The codecs contained do not perform processing beyond basic per-frame validation. They handle protocols from link layer through transport.

Codecs here conform to the API defined by src/framework/codec.h.

codec_api.h

Path = src/codecs/codec_api.h

#ifndef CODEC_API_H
#define CODEC_API_H

void load_codecs();

#endif

codec_module.h

Path = src/codecs/codec_module.h

#ifndef CODECS_CODEC_MODULE_H
#define CODECS_CODEC_MODULE_H

#include <cstdint>

#include "framework/module.h"
#include "main/snort_types.h"

namespace snort
{
class Trace;
}

extern THREAD_LOCAL const snort::Trace* decode_trace;

namespace snort
{
constexpr int GID_DECODE = 116;

//-----------------------------------------------------
// remember to add rules to preproc_rules/decoder.rules
// add the new decoder rules to the following enum.

enum CodecSid : uint32_t
{
    DECODE_NOT_IPV4_DGRAM = 1,
    DECODE_IPV4_INVALID_HEADER_LEN = 2,
    DECODE_IPV4_DGRAM_LT_IPHDR = 3,
    DECODE_IPV4OPT_BADLEN = 4,
    DECODE_IPV4OPT_TRUNCATED = 5,
    DECODE_IPV4_DGRAM_GT_CAPLEN = 6,

    DECODE_TCP_DGRAM_LT_TCPHDR = 45,
    DECODE_TCP_INVALID_OFFSET = 46,
    DECODE_TCP_LARGE_OFFSET = 47,

    DECODE_TCPOPT_BADLEN = 54,
    DECODE_TCPOPT_TRUNCATED = 55,
    DECODE_TCPOPT_TTCP = 56,
    DECODE_TCPOPT_OBSOLETE = 57,
    DECODE_TCPOPT_EXPERIMENTAL = 58,
    DECODE_TCPOPT_WSCALE_INVALID = 59,

    DECODE_UDP_DGRAM_LT_UDPHDR = 95,
    DECODE_UDP_DGRAM_INVALID_LENGTH = 96,
    DECODE_UDP_DGRAM_SHORT_PACKET = 97,
    DECODE_UDP_DGRAM_LONG_PACKET = 98,

    DECODE_ICMP_DGRAM_LT_ICMPHDR = 105,
    DECODE_ICMP_DGRAM_LT_TIMESTAMPHDR = 106,
    DECODE_ICMP_DGRAM_LT_ADDRHDR = 107,

    DECODE_ARP_TRUNCATED = 109,
    DECODE_EAPOL_TRUNCATED = 110,
    DECODE_EAPKEY_TRUNCATED = 111,
    DECODE_EAP_TRUNCATED = 112,

    DECODE_BAD_PPPOE = 120,
    DECODE_BAD_VLAN = 130,
    DECODE_BAD_LLC_HEADER = 131,
    DECODE_BAD_LLC_OTHER = 132,
    DECODE_BAD_80211_ETHLLC = 133,
    DECODE_BAD_80211_OTHER = 134,

    DECODE_BAD_TRH = 140,
    DECODE_BAD_TR_ETHLLC = 141,
    DECODE_BAD_TR_MR_LEN = 142,
    DECODE_BAD_TRHMR = 143,

    DECODE_BAD_TRAFFIC_LOOPBACK = 150,
    DECODE_BAD_TRAFFIC_SAME_SRCDST = 151,

    DECODE_GRE_DGRAM_LT_GREHDR = 160,
    DECODE_GRE_MULTIPLE_ENCAPSULATION = 161,
    DECODE_GRE_INVALID_VERSION = 162,
    DECODE_GRE_INVALID_HEADER = 163,
    DECODE_GRE_V1_INVALID_HEADER = 164,
    DECODE_GRE_TRANS_DGRAM_LT_TRANSHDR = 165,

    DECODE_BAD_MPLS = 170,
    DECODE_BAD_MPLS_LABEL0 = 171,
    DECODE_BAD_MPLS_LABEL1 = 172,
    DECODE_BAD_MPLS_LABEL2 = 173,
    DECODE_BAD_MPLS_LABEL3 = 174,
    DECODE_MPLS_RESERVED_LABEL = 175,
    DECODE_MPLS_LABEL_STACK = 176,

    DECODE_ICMP_ORIG_IP_TRUNCATED = 250,
    DECODE_ICMP_ORIG_IP_VER_MISMATCH = 251,
    DECODE_ICMP_ORIG_DGRAM_LT_ORIG_IP = 252,
    DECODE_ICMP_ORIG_PAYLOAD_LT_64 = 253,
    DECODE_ICMP_ORIG_PAYLOAD_GT_576 = 254,
    DECODE_ICMP_ORIG_IP_WITH_FRAGOFFSET = 255,

    DECODE_IPV6_MIN_TTL = 270,
    DECODE_IPV6_IS_NOT = 271,
    DECODE_IPV6_TRUNCATED_EXT = 272,
    DECODE_IPV6_TRUNCATED = 273,
    DECODE_IPV6_DGRAM_LT_IPHDR = 274,
    DECODE_IPV6_DGRAM_GT_CAPLEN = 275,
    DECODE_IPV6_DST_ZERO = 276,
    DECODE_IPV6_SRC_MULTICAST = 277,
    DECODE_IPV6_DST_RESERVED_MULTICAST = 278,
    DECODE_IPV6_BAD_OPT_TYPE = 279,
    DECODE_IPV6_BAD_MULTICAST_SCOPE = 280,
    DECODE_IPV6_BAD_NEXT_HEADER = 281,
    DECODE_IPV6_ROUTE_AND_HOPBYHOP = 282,
    DECODE_IPV6_TWO_ROUTE_HEADERS = 283,

    DECODE_ICMPV6_TOO_BIG_BAD_MTU = 285,
    DECODE_ICMPV6_UNREACHABLE_NON_RFC_2463_CODE = 286,
    DECODE_ICMPV6_SOLICITATION_BAD_CODE = 287,
    DECODE_ICMPV6_ADVERT_BAD_CODE = 288,
    DECODE_ICMPV6_SOLICITATION_BAD_RESERVED = 289,
    DECODE_ICMPV6_ADVERT_BAD_REACHABLE = 290,

    DECODE_IPV6_TUNNELED_IPV4_TRUNCATED = 291,
    DECODE_IPV6_DSTOPTS_WITH_ROUTING = 292,
    DECODE_IP_MULTIPLE_ENCAPSULATION = 293,

    DECODE_ESP_HEADER_TRUNC = 294,
    DECODE_IPV6_BAD_OPT_LEN = 295,
    DECODE_IPV6_UNORDERED_EXTENSIONS = 296,

    DECODE_GTP_MULTIPLE_ENCAPSULATION = 297,
    DECODE_GTP_BAD_LEN = 298,

    DECODE_TCP_XMAS = 400,
    DECODE_TCP_NMAP_XMAS,
    DECODE_DOS_NAPTHA,
    DECODE_SYN_TO_MULTICAST,
    DECODE_ZERO_TTL,
    DECODE_BAD_FRAGBITS,
    DECODE_UDP_IPV6_ZERO_CHECKSUM,
    DECODE_IP4_LEN_OFFSET,
    DECODE_IP4_SRC_THIS_NET,
    DECODE_IP4_DST_THIS_NET,
    DECODE_IP4_SRC_MULTICAST,
    DECODE_IP4_SRC_RESERVED,
    DECODE_IP4_DST_RESERVED,
    DECODE_IP4_SRC_BROADCAST,
    DECODE_IP4_DST_BROADCAST,
    DECODE_ICMP4_DST_MULTICAST,
    DECODE_ICMP4_DST_BROADCAST,
    DECODE_ICMP4_TYPE_OTHER = 418,
    DECODE_TCP_BAD_URP,
    DECODE_TCP_SYN_FIN,
    DECODE_TCP_SYN_RST,
    DECODE_TCP_MUST_ACK,
    DECODE_TCP_NO_SYN_ACK_RST,
    DECODE_ETH_HDR_TRUNC,
    DECODE_IP4_HDR_TRUNC,
    DECODE_ICMP4_HDR_TRUNC,
    DECODE_ICMP6_HDR_TRUNC,
    DECODE_IP4_MIN_TTL,
    DECODE_IP6_ZERO_HOP_LIMIT,
    DECODE_IP4_DF_OFFSET, // = 430
    DECODE_ICMP6_TYPE_OTHER,
    DECODE_ICMP6_DST_MULTICAST,
    DECODE_TCP_SHAFT_SYNFLOOD,
    DECODE_ICMP_PING_NMAP,
    DECODE_ICMP_ICMPENUM,
    DECODE_ICMP_REDIRECT_HOST,
    DECODE_ICMP_REDIRECT_NET,
    DECODE_ICMP_TRACEROUTE_IPOPTS,
    DECODE_ICMP_SOURCE_QUENCH,
    DECODE_ICMP_BROADSCAN_SMURF_SCANNER, // = 440
    DECODE_ICMP_DST_UNREACH_ADMIN_PROHIBITED,
    DECODE_ICMP_DST_UNREACH_DST_HOST_PROHIBITED,
    DECODE_ICMP_DST_UNREACH_DST_NET_PROHIBITED,
    DECODE_IP_OPTION_SET,
    DECODE_UDP_LARGE_PACKET,
    DECODE_TCP_PORT_ZERO,
    DECODE_UDP_PORT_ZERO,
    DECODE_IP_RESERVED_FRAG_BIT,
    DECODE_IP_UNASSIGNED_PROTO,
    DECODE_IP_BAD_PROTO, // = 450
    DECODE_ICMP_PATH_MTU_DOS,
    DECODE_ICMP_DOS_ATTEMPT,
    DECODE_IPV6_ISATAP_SPOOF,
    DECODE_PGM_NAK_OVERFLOW,
    DECODE_IGMP_OPTIONS_DOS,
    DECODE_IP6_EXCESS_EXT_HDR,
    DECODE_ICMPV6_UNREACHABLE_NON_RFC_4443_CODE,
    DECODE_IPV6_BAD_FRAG_PKT,
    DECODE_ZERO_LENGTH_FRAG,
    DECODE_ICMPV6_NODE_INFO_BAD_CODE, // = 460
    DECODE_IPV6_ROUTE_ZERO,
    DECODE_ERSPAN_HDR_VERSION_MISMATCH,
    DECODE_ERSPAN2_DGRAM_LT_HDR,
    DECODE_ERSPAN3_DGRAM_LT_HDR,
    DECODE_AUTH_HDR_TRUNC,
    DECODE_AUTH_HDR_BAD_LEN,
    DECODE_FPATH_HDR_TRUNC,
    DECODE_CISCO_META_HDR_TRUNC,
    DECODE_CISCO_META_HDR_OPT_LEN,
    DECODE_CISCO_META_HDR_OPT_TYPE, // = 470
    DECODE_CISCO_META_HDR_SGT,
    DECODE_TOO_MANY_LAYERS,
    DECODE_BAD_ETHER_TYPE,
    DECODE_ICMP6_NOT_IP6,
    DECODE_MIPV6_BAD_PAYLOAD_PROTO,
    DECODE_INDEX_MAX
};

//-------------------------------------------------------------------------
// module
//-------------------------------------------------------------------------

class BaseCodecModule : public Module
{
public:
    BaseCodecModule(const char* s, const char* h) : Module(s, h)
    { }

    BaseCodecModule(const char* s, const char* h, const Parameter* p, bool is_list = false)
        : Module(s, h, p, is_list) { }

    unsigned get_gid() const override
    { return GID_DECODE; }

    Usage get_usage() const override
    { return CONTEXT; }
};

class SO_PUBLIC CodecModule : public BaseCodecModule
{
public:
    CodecModule();

    const RuleMap* get_rules() const override;

    void set_trace(const Trace*) const override;
    const TraceOption* get_trace_options() const override;
};
}

#endif

codecs/ip/

All codecs under this directory handle data that would be seen directly following or under IP headers.

checksum.h

Path = src/codecs/ip/checksum.h

#ifndef CODECS_CHECKSUM_H
#define CODECS_CHECKSUM_H

#include <cstddef>

#include <protocols/protocol_ids.h>

namespace checksum
{
union Pseudoheader
{
    struct
    {
        uint32_t sip;
        uint32_t dip;
        uint8_t zero;
        IpProtocol protocol;
        uint16_t len;
    } hdr;
    uint16_t arr[6];
    static_assert(sizeof(hdr) == sizeof(arr), "IPv4 pseudoheader must be 12 bytes");
};

union Pseudoheader6
{
    struct
    {
        uint32_t sip[4];
        uint32_t dip[4];
        uint8_t zero;
        IpProtocol protocol;
        uint16_t len;
    } hdr;
    uint16_t arr[18];
    static_assert(sizeof(hdr) == sizeof(arr), "IPv6 pseudoheader must be 36 bytes");
};

//  calculate the checksum for this general case.
static uint16_t cksum_add(const uint16_t* buf, std::size_t buf_len);
inline uint16_t tcp_cksum(const uint16_t* buf, std::size_t len, const Pseudoheader&);
inline uint16_t tcp_cksum(const uint16_t* buf, std::size_t len, const Pseudoheader6&);
inline uint16_t udp_cksum(const uint16_t* buf, std::size_t len, const Pseudoheader&);
inline uint16_t udp_cksum(const uint16_t* buf, std::size_t len, const Pseudoheader6&);
inline uint16_t icmp_cksum(const uint16_t* buf, std::size_t len, const Pseudoheader6&);
inline uint16_t icmp_cksum(const uint16_t* buf, std::size_t len);
inline uint16_t ip_cksum(const uint16_t* buf, std::size_t len);

/*
 *  NOTE: Since multiple dynamic libraries use checksums, the choice
 *          is to either include all of the checksum details in a header,
 *          or ensure I include these symbols for every linker which
 *          can be used. Obviously, setting correct linker flags is
 *          significantly more difficult, so these functions will all
 *          stay in a header file
 */

/*
 *  IT IS HIGHLY RECOMMENDED to use the above API. Rathern than calling
 *  any of of the following recommendations directly
 */
namespace detail
{
inline uint16_t cksum_add(const uint16_t* buf, std::size_t len, uint32_t cksum)
{
    const uint16_t* sp = buf;

    // if pointer is 16 bit aligned calculate checksum in tight loop...
    // gcc 5.4 -O3 generates unaligned quadword instructions that crash; fixed in gcc 8.0.1
    if ( !( reinterpret_cast<std::uintptr_t>(sp) & 0x01 ) )
    {
        while ( len > 1 )
        {
            cksum += *sp++;
            len -= 2;
        }
    }
    else if ( len > 1 )
    {
        std::size_t sn = ((len / 2) & 0xF);  // == len/2 % 16
        std::size_t n = (((len / 2) + 15) / 16);   // ceiling of (len / 2) / 16

        switch (sn)
        {
        case 0:
            sn = 16;
            cksum += sp[15];    // fallthrough
        case 15:
            cksum += sp[14];    // fallthrough
        case 14:
            cksum += sp[13];    // fallthrough
        case 13:
            cksum += sp[12];    // fallthrough
        case 12:
            cksum += sp[11];    // fallthrough
        case 11:
            cksum += sp[10];    // fallthrough
        case 10:
            cksum += sp[9];     // fallthrough
        case 9:
            cksum += sp[8];     // fallthrough
        case 8:
            cksum  += sp[7];    // fallthrough
        case 7:
            cksum += sp[6];     // fallthrough
        case 6:
            cksum += sp[5];     // fallthrough
        case 5:
            cksum += sp[4];     // fallthrough
        case 4:
            cksum += sp[3];     // fallthrough
        case 3:
            cksum += sp[2];     // fallthrough
        case 2:
            cksum += sp[1];     // fallthrough
        case 1:
            cksum += sp[0];
        }
        sp += sn;

        /* unroll loop using Duff's device. */
        while (--n > 0)
        {
            cksum += sp[0];
            cksum += sp[1];
            cksum += sp[2];
            cksum += sp[3];
            cksum += sp[4];
            cksum += sp[5];
            cksum += sp[6];
            cksum += sp[7];
            cksum += sp[8];
            cksum += sp[9];
            cksum += sp[10];
            cksum += sp[11];
            cksum += sp[12];
            cksum += sp[13];
            cksum += sp[14];
            cksum += sp[15];
            sp += 16;
        }
    }

    // if len is odd, sum in the last byte...
    if ( len & 0x01 )
        cksum += *((const uint8_t*) sp);

    cksum  = (cksum >> 16) + (cksum & 0x0000ffff);
    cksum += (cksum >> 16);

    return (uint16_t)(~cksum);
}

inline void add_ipv4_pseudoheader(const Pseudoheader& ph4, uint32_t& cksum)
{
    const uint16_t* h = ph4.arr;

    /* ipv4 pseudo header must have 12 bytes */
    cksum += h[0];
    cksum += h[1];
    cksum += h[2];
    cksum += h[3];
    cksum += h[4];
    cksum += h[5];
}

inline void add_ipv6_pseudoheader(const Pseudoheader6& ph6, uint32_t& cksum)
{
    const uint16_t* h = ph6.arr;

    /* PseudoHeader must have 36 bytes */
    cksum += h[0];
    cksum += h[1];
    cksum += h[2];
    cksum += h[3];
    cksum += h[4];
    cksum += h[5];
    cksum += h[6];
    cksum += h[7];
    cksum += h[8];
    cksum += h[9];
    cksum += h[10];
    cksum += h[11];
    cksum += h[12];
    cksum += h[13];
    cksum += h[14];
    cksum += h[15];
    cksum += h[16];
    cksum += h[17];
}

inline void add_tcp_header(const uint16_t*& d,
    std::size_t& len,
    uint32_t& cksum)
{
    /* TCP hdr must have 20 hdr bytes */
    cksum += d[0];
    cksum += d[1];
    cksum += d[2];
    cksum += d[3];
    cksum += d[4];
    cksum += d[5];
    cksum += d[6];
    cksum += d[7];
    cksum += d[8];
    cksum += d[9];
    d += 10;
    len -= 20;
}

inline void add_udp_header(const uint16_t*& d,
    size_t& len,
    uint32_t& cksum)
{
    /* UDP must have 8 hdr bytes */
    cksum += d[0];
    cksum += d[1];
    cksum += d[2];
    cksum += d[3];
    len -= 8;
    d += 4;
}

inline void add_ip_header(const uint16_t*& d,
    std::size_t& len,
    uint32_t& cksum)
{
    /* IP must be >= 20 bytes */
    cksum += d[0];
    cksum += d[1];
    cksum += d[2];
    cksum += d[3];
    cksum += d[4];
    cksum += d[5];
    cksum += d[6];
    cksum += d[7];
    cksum += d[8];
    cksum += d[9];
    d += 10;
    len -= 20;
}
} // namespace detail

inline uint16_t icmp_cksum(const uint16_t* buf,
    std::size_t len,
    const Pseudoheader6& ph)
{
    uint32_t cksum = 0;

    detail::add_ipv6_pseudoheader(ph, cksum);
    return detail::cksum_add(buf, len, cksum);
}

inline uint16_t icmp_cksum(const uint16_t* buf, size_t len)
{
    return detail::cksum_add(buf, len, 0);
}

inline uint16_t tcp_cksum(const uint16_t* h,
    std::size_t len,
    const Pseudoheader& ph)
{
    uint32_t cksum = 0;

    detail::add_ipv4_pseudoheader(ph, cksum);
    detail::add_tcp_header(h, len, cksum);
    return detail::cksum_add(h, len, cksum);
}

inline uint16_t tcp_cksum(const uint16_t* buf,
    std::size_t len,
    const Pseudoheader6& ph)
{
    uint32_t cksum = 0;

    detail::add_ipv6_pseudoheader(ph, cksum);
    detail::add_tcp_header(buf, len, cksum);
    return detail::cksum_add(buf, len, cksum);
}

inline uint16_t udp_cksum(const uint16_t* buf,
    std::size_t len,
    const Pseudoheader& ph)
{
    uint32_t cksum = 0;

    detail::add_ipv4_pseudoheader(ph, cksum);
    detail::add_udp_header(buf, len, cksum);
    return detail::cksum_add(buf, len, cksum);
}

inline uint16_t udp_cksum(const uint16_t* buf,
    std::size_t len,
    const Pseudoheader6& ph)
{
    uint32_t cksum = 0;

    detail::add_ipv6_pseudoheader(ph, cksum);
    detail::add_udp_header(buf, len, cksum);
    return detail::cksum_add(buf, len, cksum);
}

inline uint16_t ip_cksum(const uint16_t* buf, std::size_t len)
{
    uint32_t cksum = 0;

    detail::add_ip_header(buf, len, cksum);
    return detail::cksum_add(buf, len, cksum);
}

inline uint16_t cksum_add(const uint16_t* buf, std::size_t len)
{ return detail::cksum_add(buf, len, 0); }
} // namespace checksum

#endif  /* CODECS_CHECKSUM_H */

These codecs handle link-layer protocols that would be presented beyond the root encapsulation defined by the capture data-link type.

codecs/misc/

This directory contains codecs that do not fall under the classifications of the other codec directories. These codecs primarily handle IP tunneling protocols.

codecs/root/

This directory contains codecs for all of the top-level codecs used. These codecs represent the first layer of encapsulation encountered within a captured frame, as defined by interface data-link types.

connectors/

Connectors house the collection of the plugin-type Connector. As defined …/framework/connector.h, the Connector object implements a simplex communications channel that is in-turn used by SideChannel objects.

Connectors have the standard plugin api and instantiation/destruction protocols.

The file_connector writes messages to a file and reads messages from a file.

Configuration entries map side channels to connector instances.

connectors.h

Path = src/connectors/connectors.h

#ifndef CONNECTORS_H
#define CONNECTORS_H

void load_connectors();

#endif

connectors/file_connector/

Implement a connector plugin that reads/writes side channel messages from/to files.

Each connector implements a simplex channel, either transmit or receive. In- turn, each SideChannel owns a transmit and/or a receive connector object.

The "format = text" option in the configuration sets text file input/output else binary file format is used.

In binary files, an additional FileConnector message header is pre-pended to each side channel message written to the output file. This header specifies the file format version and the length of the side channel message. This length does not include the file connector message header, but does include the side channel message header.

The utility get_instance_file() is used to uniquely name the files. The complete file name convention is:

The input file is named file_connector_<name>_receive and the output file is named file_connector_<name>_transmit. Where the <name> is specified "name" field in the configuration. These names are in turn pre-pended with the get_instance_file() naming convention.

The file_connector Connector configuration results in ONE ConnectorCommon object which is used to contain a list of all Connectors being configured. A vector<> in the ConnectorCommon object holds individual Connector config objects. The ConnectorManager then uses this vector<> to instantiate the set of desired Connectors.

file_connector_config.h

Path = src/connectors/file_connector/file_connector_config.h

#ifndef FILE_CONNECTOR_CONFIG_H
#define FILE_CONNECTOR_CONFIG_H

#include <string>
#include <vector>

#include "framework/connector.h"

class FileConnectorConfig : public snort::ConnectorConfig
{
public:
    FileConnectorConfig()
    { direction = snort::Connector::CONN_UNDEFINED; text_format = false; }

    bool text_format;
    std::string name;

    typedef std::vector<FileConnectorConfig*> FileConnectorConfigSet;
};

#endif

file_connector.h

Path = src/connectors/file_connector/file_connector.h

#ifndef FILE_CONNECTOR_H
#define FILE_CONNECTOR_H

#include <fstream>

#include "framework/connector.h"

#include "file_connector_config.h"

#define FILE_FORMAT_VERSION (1)

//-------------------------------------------------------------------------
// class stuff
//-------------------------------------------------------------------------

class __attribute__((__packed__)) FileConnectorMsgHdr
{
public:
    FileConnectorMsgHdr(uint32_t length)
    { version = FILE_FORMAT_VERSION; connector_msg_length = length; }

    uint16_t version;
    uint32_t connector_msg_length;
};

class FileConnectorMsgHandle : public snort::ConnectorMsgHandle
{
public:
    FileConnectorMsgHandle(const uint32_t length);
    ~FileConnectorMsgHandle();
    snort::ConnectorMsg connector_msg;
};

class FileConnectorCommon : public snort::ConnectorCommon
{
public:
    FileConnectorCommon(FileConnectorConfig::FileConnectorConfigSet*);
    ~FileConnectorCommon();
};

class FileConnector : public snort::Connector
{
public:
    FileConnector(FileConnectorConfig*);
    snort::ConnectorMsgHandle* alloc_message(const uint32_t, const uint8_t**) override;
    void discard_message(snort::ConnectorMsgHandle*) override;
    bool transmit_message(snort::ConnectorMsgHandle*) override;
    snort::ConnectorMsgHandle* receive_message(bool) override;

    snort::ConnectorMsg* get_connector_msg(snort::ConnectorMsgHandle* handle) override
    { return( &((FileConnectorMsgHandle*)handle)->connector_msg ); }
    Direction get_connector_direction() override
    { return( ((const FileConnectorConfig*)config)->direction ); }

    std::fstream file;

private:
    snort::ConnectorMsgHandle* receive_message_binary();
    snort::ConnectorMsgHandle* receive_message_text();
};

#endif

file_connector_module.h

Path = src/connectors/file_connector/file_connector_module.h

#ifndef FILE_CONNECTOR_MODULE_H
#define FILE_CONNECTOR_MODULE_H

#include "framework/module.h"

#include "file_connector_config.h"

#define FILE_CONNECTOR_NAME "file_connector"
#define FILE_CONNECTOR_HELP "implement the file based connector"

class FileConnectorModule : public snort::Module
{
public:
    FileConnectorModule();
    ~FileConnectorModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    FileConnectorConfig::FileConnectorConfigSet* get_and_clear_config();

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;

    snort::ProfileStats* get_profile() const override;

    Usage get_usage() const override
    { return GLOBAL; }

private:
    FileConnectorConfig::FileConnectorConfigSet* config_set;
    FileConnectorConfig* config;
};

#endif

connectors/file_connector/test/

connectors/tcp_connector/

Implement a connector plugin that reads and writes side channel messages across a TCP stream channel.

Each connector implements a duplex channel, both transmit and receive. When used by a side_channel object, a single TcpConnector object is used for both the transmit and receive connectors.

An additional TcpConnector message header is pre-pended to each side channel message transmitted. This header specifies the protocol format version and the length of the side channel message. This length does not include the tcp connector message header, but does include the side channel message header.

The tcp_connector Connector configuration results in ONE ConnectorCommon object which is used to contain a list of all Connectors being configured. A vector<> in the ConnectorCommon object holds individual Connector config objects. The ConnectorManager then uses this vector<> to instantiate the set of desired Connectors.

TCP connector configuration includes a partner address, base port numbers, and connection setup direction. The actual port number used is the base port number added to the thread instance value.

A TCP connector can be either the active partner and initiate the TCP connection or can be the passive partner and expect to be called by the active side. This is controlled by the setup configuration element.

Receive messages are managed via separate thread and ring buffer queue structure. The thread’s purpose is to read whole side channel messages from the stream and insert them into the queue. Then the packet processing thread is able to read whole side messages from the queue.

tcp_connector_config.h

Path = src/connectors/tcp_connector/tcp_connector_config.h

#ifndef TCP_CONNECTOR_CONFIG_H
#define TCP_CONNECTOR_CONFIG_H

#include <vector>

#include "framework/connector.h"

class TcpConnectorConfig : public snort::ConnectorConfig
{
public:
    enum Setup { CALL, ANSWER };
    TcpConnectorConfig()
    { direction = snort::Connector::CONN_DUPLEX; async_receive = true; }

    uint16_t base_port = 0;
    std::string address;
    Setup setup = {};
    bool async_receive;

    typedef std::vector<TcpConnectorConfig*> TcpConnectorConfigSet;
};

#endif

tcp_connector.h

Path = src/connectors/tcp_connector/tcp_connector.h

#ifndef TCP_CONNECTOR_H
#define TCP_CONNECTOR_H

#include <thread>

#include "framework/connector.h"
#include "helpers/ring.h"

#include "tcp_connector_config.h"

#define TCP_FORMAT_VERSION (1)

//-------------------------------------------------------------------------
// class stuff
//-------------------------------------------------------------------------

class __attribute__((__packed__)) TcpConnectorMsgHdr
{
public:
    TcpConnectorMsgHdr() = default;
    TcpConnectorMsgHdr(uint32_t length)
    { version = TCP_FORMAT_VERSION; connector_msg_length = length; }

    uint8_t version;
    uint16_t connector_msg_length;
};

class TcpConnectorMsgHandle : public snort::ConnectorMsgHandle
{
public:
    TcpConnectorMsgHandle(const uint32_t length);
    ~TcpConnectorMsgHandle();
    snort::ConnectorMsg connector_msg;
};

class TcpConnectorCommon : public snort::ConnectorCommon
{
public:
    TcpConnectorCommon(TcpConnectorConfig::TcpConnectorConfigSet*);
    ~TcpConnectorCommon();
};

class TcpConnector : public snort::Connector
{
public:
    typedef Ring<TcpConnectorMsgHandle*> ReceiveRing;

    TcpConnector(TcpConnectorConfig*, int sock_fd);
    ~TcpConnector() override;
    snort::ConnectorMsgHandle* alloc_message(const uint32_t, const uint8_t**) override;
    void discard_message(snort::ConnectorMsgHandle*) override;
    bool transmit_message(snort::ConnectorMsgHandle*) override;
    snort::ConnectorMsgHandle* receive_message(bool) override;

    snort::ConnectorMsg* get_connector_msg(snort::ConnectorMsgHandle* handle) override
    { return( &((TcpConnectorMsgHandle*)handle)->connector_msg ); }
    Direction get_connector_direction() override
    { return Connector::CONN_DUPLEX; }
    void process_receive();

    int sock_fd;

private:
    bool run_thread = false;
    std::thread* receive_thread;
    void start_receive_thread();
    void stop_receive_thread();
    void receive_processing_thread();
    ReceiveRing* receive_ring;
};

#endif

tcp_connector_module.h

Path = src/connectors/tcp_connector/tcp_connector_module.h

#ifndef TCP_CONNECTOR_MODULE_H
#define TCP_CONNECTOR_MODULE_H

#include "framework/module.h"

#include "tcp_connector_config.h"

#define TCP_CONNECTOR_NAME "tcp_connector"
#define TCP_CONNECTOR_HELP "implement the tcp stream connector"

class TcpConnectorModule : public snort::Module
{
public:
    TcpConnectorModule();
    ~TcpConnectorModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    TcpConnectorConfig::TcpConnectorConfigSet* get_and_clear_config();

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;

    snort::ProfileStats* get_profile() const override;

    Usage get_usage() const override
    { return GLOBAL; }

private:
    TcpConnectorConfig::TcpConnectorConfigSet* config_set;
    TcpConnectorConfig* config;
};

#endif

connectors/tcp_connector/test/

control/

This module provides functions for registering and running handlers that are called when Snort is not doing anything more important.

idle_processing.h

Path = src/control/idle_processing.h

#ifndef IDLE_PROCESSING_H
#define IDLE_PROCESSING_H

using IdleHook = void (*)();

class IdleProcessing
{
public:
    static void register_handler(IdleHook);
    static void execute();

    // only needs to be called if changing out the handler set
    static void unregister_all();
};

#endif

decompress/

The components in this area implement several file decompression mechanisms. They provide real-time decompression to permit inspection (rules) of the decompressed content.

In particular the components support these decompression options:

  1. Decompress SWF (Adobe Flash) files compressed with the ZLIB algorithm

  2. Optionally decompress SWF files compressed with the LZMA algorithm. This is only available if Snort ++ is built with the optional LZMA support.

  3. Decompress the Deflate compressed portions of PDF files.

The three modes are individually enabled/disabled at initialization time.

All parsing and decompression is incremental and allows inspection to proceed as the file is received and processed.

SWF File Processing:

SWF files exist in three forms: 1) uncompressed, 2) ZLIB compressed, and 3) LZMA compressed. SWF files begin with a file signature block (always uncompressed) to indicate the format of the balance of the file. The balance of the file is formatted and processed as specified.

PDF files are significantly more complex as the compressed content is embedded within the PDF syntax and one file may contain one or many compressed segments.

Thus the PDF decompression engine implements a lightweight PDF file parser to locate the PDF Stream segments and then attempt to decompress Streams that are filtered with the FlateDecode compression. Streams are binary objects that are used for much of the PDF actual content. A Stream object can be labeled with a Filter option to indicate that the Steam is encoded in some fashion, perhaps having multiple cascaded Filters.

The current implementation supports the most common FlateDecode Filter option and does not support cascaded Filters (including cascaded FlateDecode’s).

The decompressor processors can indicate several error situations. There are two mechanisms used to relay these error codes to the calling context. Some errors terminate processing and are passed to the caller in the decompression context structure. Other errors are inline and may be passed back to the caller via a provided call-back function.

Possible error conditions are:

  • FILE_DECOMP_ERR_SWF_ZLIB_FAILURE - The ZLIB decompression engine returned an error.

  • FILE_DECOMP_ERR_SWF_LZMA_FAILURE - The LZMA decompression engine returned an error.

  • FILE_DECOMP_ERR_PDF_DEFL_FAILURE - The Deflate (form of ZLIB) decompression engine returned at error.

  • FILE_DECOMP_ERR_PDF_UNSUP_COMP_TYPE - An unsupported PDF Stream Filter type was encountered,

  • FILE_DECOMP_ERR_PDF_CASC_COMP - Cascasded FlateDecode Stream Filters were encountered.

  • FILE_DECOMP_ERR_PDF_PARSE_FAILURE - Error while parsing the PDF file.

file_decomp.h

Path = src/decompress/file_decomp.h

#ifndef FILE_DECOMP_H
#define FILE_DECOMP_H

// File_Decomp global typedefs (used in child objects)

#include <cstring>

#include "main/snort_types.h"

/* Function return codes used internally and with caller */
// FIXIT-L these need to be split into internal-only codes and things that may be returned to the
// application. The codes used by PDF and SWF should be standardized. PDF is returning BlockIn and
// BlockOut while SWF is using OK. There also needs to be clarity about what Error means and what
// should be done about it. Is it just an indicator of programming error or are there operational
// cases where it occurs? No idea whether Complete and Eof are real things and what should be done
// about them.

enum fd_status_t
{
    File_Decomp_DecompError = -2,  /* Error from decompression */
    File_Decomp_Error = -1,        /* Error from decompression */
    File_Decomp_OK = 0,
    File_Decomp_NoSig = 1,         /* No file signature located */
    File_Decomp_Complete = 2,      /* Completed */
    File_Decomp_BlockOut = 3,      /* Blocked due to lack of output space */
    File_Decomp_BlockIn = 4,       /* Blocked due to lack in input data */
    File_Decomp_Eof = 5            /* End of file located */
};

enum file_compression_type_t
{
    FILE_COMPRESSION_TYPE_NONE,
    FILE_COMPRESSION_TYPE_DEFLATE,
    FILE_COMPRESSION_TYPE_ZLIB,
    FILE_COMPRESSION_TYPE_LZMA,
    FILE_COMPRESSION_TYPE_MAX
};

/* Potential decompression modes, passed in at initialization time. */
#define FILE_SWF_LZMA_BIT    (0x00000001)
#define FILE_SWF_ZLIB_BIT    (0x00000002)
#define FILE_PDF_DEFL_BIT    (0x00000004)
#define FILE_ZIP_DEFL_BIT    (0x00000008)

#define FILE_PDF_ANY         (FILE_PDF_DEFL_BIT)
#define FILE_SWF_ANY         (FILE_SWF_LZMA_BIT | FILE_SWF_ZLIB_BIT)
#define FILE_ZIP_ANY         (FILE_ZIP_DEFL_BIT)

/* Error codes either passed to caller via the session->Error_Alert of
   the File_Decomp_Alert() call-back function. */
enum FileDecompError
{
    FILE_DECOMP_ERR_SWF_ZLIB_FAILURE,
    FILE_DECOMP_ERR_SWF_LZMA_FAILURE,
    FILE_DECOMP_ERR_PDF_DEFL_FAILURE,
    FILE_DECOMP_ERR_PDF_UNSUP_COMP_TYPE,
    FILE_DECOMP_ERR_PDF_CASC_COMP,
    FILE_DECOMP_ERR_PDF_PARSE_FAILURE,
    FILE_DECOMP_ERR_ZIP_PARSE_FAILURE,
    FILE_DECOMP_ERR_ZIP_DEFL_FAILURE
};

/* Private Types */
enum file_type_t
{
    FILE_TYPE_NONE,
    FILE_TYPE_SWF,
    FILE_TYPE_PDF,
    FILE_TYPE_ZIP,
    FILE_TYPE_MAX
};

enum fd_states_t
{
    STATE_NEW,        /* Session created */
    STATE_READY,      /* Session created and ready for content, no file/decomp selected */
    STATE_ACTIVE,     /* Decompressor inited and ready for content */
    STATE_COMPLETE    /* Decompression completed */
};

/* Primary file decompression session state context */
struct fd_session_t
{
    // FIXIT-L replace with abstract base class pointer used for
    // PDF or SWF subclass and eliminate switches on File_Type
    union
    {
        struct fd_PDF_t* PDF;
        struct fd_SWF_t* SWF;
        struct fd_ZIP_t* ZIP;
    };

    const uint8_t* Next_In;     // next input byte
    uint8_t* Next_Out;          // next output byte should be put there

    // Alerting callback
    void (* Alert_Callback)(void* Context, int Event);
    void* Alert_Context;

    uint32_t Avail_In;   // number of bytes available at next_in
    uint32_t Total_In;   // total number of input bytes read so far

    uint32_t Avail_Out;  // remaining free space at next_out
    uint32_t Total_Out;  // total number of bytes output so far

    // Configuration settings
    // FIXIT-RC Compr_Depth and Decompr_Depth only support OHI and eventually should be removed
    uint32_t Compr_Depth;
    uint32_t Decompr_Depth;
    uint32_t Modes;      // Bit mapped set of potential file/algo modes

    int Error_Event;     // Specific event indicated by DecomprError return

    // Internal State
    uint8_t File_Type;   // Active file type
    uint8_t Decomp_Type; // Active decompression type
    uint8_t Sig_State;   // Sig search state machine
    uint8_t State;       // main state machine
};

/* Macros */

/* Macros used to sync my decompression context with that
   of the underlying decompression engine context. */
#ifndef SYNC_IN
#define SYNC_IN(dest) \
    dest->next_in = const_cast<Bytef*>(SessionPtr->Next_In); \
    (dest)->avail_in = SessionPtr->Avail_In; \
    (dest)->total_in = SessionPtr->Total_In; \
    (dest)->next_out = SessionPtr->Next_Out; \
    (dest)->avail_out = SessionPtr->Avail_Out; \
    (dest)->total_out = SessionPtr->Total_Out;
#endif

#ifndef SYNC_OUT
#define SYNC_OUT(src) \
    SessionPtr->Next_In = (const uint8_t*)(src)->next_in; \
    SessionPtr->Avail_In = (src)->avail_in; \
    SessionPtr->Total_In = (src)->total_in; \
    SessionPtr->Next_Out = (uint8_t*)(src)->next_out; \
    SessionPtr->Avail_Out = (src)->avail_out; \
    SessionPtr->Total_Out = (src)->total_out;
#endif

/* Inline Functions */

/* If available, look at the next available byte in the input queue */
inline bool Peek_1(fd_session_t* SessionPtr, uint8_t* c)
{
    if ( (SessionPtr->Next_In != nullptr) && (SessionPtr->Avail_In > 0) )
    {
        *c = *(SessionPtr->Next_In);
        return( true );
    }
    else
        return( false );
}

/* If available, get a byte from the input queue */
inline bool Get_1(fd_session_t* SessionPtr, uint8_t* c)
{
    if ( (SessionPtr->Next_In != nullptr) && (SessionPtr->Avail_In > 0) )
    {
        *c = *(SessionPtr->Next_In)++;
        SessionPtr->Avail_In -= 1;
        SessionPtr->Total_In += 1;
        return( true );
    }
    else
        return( false );
}

/* If available, get N bytes from the input queue.  All N must be
   available for this call to succeed. */
inline bool Get_N(fd_session_t* SessionPtr, const uint8_t** c, uint16_t N)
{
    if ( (SessionPtr->Next_In != nullptr) && (SessionPtr->Avail_In >= N) )
    {
        *c = SessionPtr->Next_In;
        SessionPtr->Next_In += N;
        SessionPtr->Avail_In -= N;
        SessionPtr->Total_In += N;
        return( true );
    }
    else
        return( false );
}

/* If there's room in the output queue, put one byte. */
inline bool Put_1(fd_session_t* SessionPtr, uint8_t c)
{
    if ( (SessionPtr->Next_Out != nullptr) && (SessionPtr->Avail_Out > 0) )
    {
        *(SessionPtr->Next_Out)++ = c;
        SessionPtr->Avail_Out -= 1;
        SessionPtr->Total_Out += 1;
        return( true );
    }
    else
        return( false );
}

/* If the output queue has room available, place N bytes onto the queue.
   The output queue must have space for N bytes for this call to succeed. */
inline bool Put_N(fd_session_t* SessionPtr, const uint8_t* c, uint16_t N)
{
    if ( (SessionPtr->Next_Out != nullptr) && (SessionPtr->Avail_Out >= N) )
    {
        strncpy( (char*)SessionPtr->Next_Out, (const char*)c, N);
        SessionPtr->Next_Out += N;
        SessionPtr->Avail_Out -= N;
        SessionPtr->Total_Out += N;
        return( true );
    }
    else
        return( false );
}

/* If the input queue has at least one byte available AND there's at
   space for at least one byte in the output queue, then move one byte. */
inline bool Move_1(fd_session_t* SessionPtr)
{
    if ( (SessionPtr->Next_Out != nullptr) && (SessionPtr->Avail_Out > 0) &&
        (SessionPtr->Next_In != nullptr) && (SessionPtr->Avail_In > 0) )
    {
        *(SessionPtr->Next_Out) = *(SessionPtr->Next_In);
        SessionPtr->Next_Out += 1;
        SessionPtr->Next_In += 1;
        SessionPtr->Avail_In -= 1;
        SessionPtr->Avail_Out -= 1;
        SessionPtr->Total_In += 1;
        SessionPtr->Total_Out += 1;
        return( true );
    }
    else
        return( false );
}

/* If the input queue has at least N bytes available AND there's at
   space for at least N bytes in the output queue, then move all N bytes. */
inline bool Move_N(fd_session_t* SessionPtr, uint16_t N)
{
    if ( (SessionPtr->Next_Out != nullptr) && (SessionPtr->Avail_Out >= N) &&
        (SessionPtr->Next_In != nullptr) && (SessionPtr->Avail_In >= N) )
    {
        memcpy( (char*)SessionPtr->Next_Out, (const char*)SessionPtr->Next_In, N);
        SessionPtr->Next_Out += N;
        SessionPtr->Next_In += N;
        SessionPtr->Avail_In -= N;
        SessionPtr->Avail_Out -= N;
        SessionPtr->Total_In += N;
        SessionPtr->Total_Out += N;
        return( true );
    }
    else
        return( false );
}

/* API Functions */
namespace snort
{
/* Create a new decompression session object */
SO_PUBLIC fd_session_t* File_Decomp_New();

/* Initialize the session */
SO_PUBLIC fd_status_t File_Decomp_Init(fd_session_t*);

/* Run the incremental decompression engine */
SO_PUBLIC fd_status_t File_Decomp(fd_session_t*);

/* Close the decomp session processing */
SO_PUBLIC fd_status_t File_Decomp_End(fd_session_t*);

/* Close the current decomp session, but setup for another */
SO_PUBLIC fd_status_t File_Decomp_Reset(fd_session_t*);

/* Abort and delete the session */
SO_PUBLIC fd_status_t File_Decomp_StopFree(fd_session_t*);

/* Delete the session object */
SO_PUBLIC void File_Decomp_Free(fd_session_t*);

/* Call the error alerting call-back function */
SO_PUBLIC void File_Decomp_Alert(fd_session_t*, int Event);
}
#endif

file_decomp_pdf.h

Path = src/decompress/file_decomp_pdf.h

#ifndef FILE_DECOMP_PDF_H
#define FILE_DECOMP_PDF_H

#include <zlib.h>

#include "file_decomp.h"

#define ELEM_BUF_LEN        (12)
#define FILTER_SPEC_BUF_LEN (40)
#define PARSE_STACK_LEN     (12)

/* FIXIT-RC Other than the API prototypes, the other parts of this header should
   be private to file_decomp_pdf. */

enum fd_PDF_States
{
    PDF_STATE_NEW,
    PDF_STATE_LOCATE_STREAM,     /* Found sig bytes, looking for dictionary & stream */
    PDF_STATE_INIT_STREAM,       /* Init stream */
    PDF_STATE_PROCESS_STREAM     /* Processing stream */
};

struct fd_PDF_Parse_Stack_t
{
    uint8_t State;
    uint8_t Sub_State;
};

struct fd_PDF_Parse_t
{
    const uint8_t* xref_tok;
    uint32_t Obj_Number;
    uint32_t Gen_Number;
    uint8_t Parse_Stack_Index;
    uint8_t Sub_State;
    uint8_t State;
    uint8_t Dict_Nesting_Cnt;
    uint8_t Elem_Index;
    uint8_t Filter_Spec_Index;
    uint8_t Elem_Buf[ELEM_BUF_LEN];
    uint8_t Filter_Spec_Buf[FILTER_SPEC_BUF_LEN+1];
    fd_PDF_Parse_Stack_t Parse_Stack[PARSE_STACK_LEN];
};

struct fd_PDF_Deflate_t
{
    z_stream StreamDeflate;
};

struct fd_PDF_t
{
    union
    {
        fd_PDF_Deflate_t Deflate;
    } PDF_Decomp_State;
    fd_PDF_Parse_t Parse;
    uint8_t Decomp_Type;
    uint8_t State;
};

/* API Functions */

/* Init the PDF decompressor */
fd_status_t File_Decomp_Init_PDF(fd_session_t*);

/* Run the incremental PDF file parser/decompressor */
fd_status_t File_Decomp_PDF(fd_session_t*);

/* End the decompressor */
fd_status_t File_Decomp_End_PDF(fd_session_t*);

#endif

file_decomp_swf.h

Path = src/decompress/file_decomp_swf.h

#ifndef FILE_DECOMP_SWF_H
#define FILE_DECOMP_SWF_H

#ifdef HAVE_LZMA
#include <lzma.h>
#endif
#include <zlib.h>

#include "file_decomp.h"

/* FIXIT-RC Other than the API prototypes, the other parts of this header should
   be private to file_decomp_swf. */

/* Both ZLIB & LZMA files have an uncompressed eight byte header.  The signature is
   three bytes.  The header consists of a three byte sig, a one byte version,
   and a four byte uncompressed length (little-endian).  */

#define SWF_SIG_LEN       (3)
#define SWF_VER_LEN       (1)
#define SWF_UCL_LEN       (4)

/* LZMA Files have an additional nine bytes of header prior to the compressed data.
   This includes a four byte compressed length (little-endian) and five bytes
   of LZMA properties. */
#define SWF_LZMA_CML_LEN  (4)
#define SWF_LZMA_PRP_LEN  (5)

/* AFTER the sig, the max number of header bytes to fetch.
   VER+UCL+CML+LZMA_PRP -> 14 bytes. */
#define SWF_MAX_HEADER    (14)

/* Types */

enum fd_SWF_States
{
    SWF_STATE_NEW,
    SWF_STATE_GET_HEADER,     /* Found sig bytes, looking for end of uncomp header */
    SWF_STATE_PROC_HEADER,    /* Found header bytes, now process the header */
    SWF_STATE_DATA            /* Done with header, looking for start of data */
};

struct fd_SWF_t
{
    z_stream StreamZLIB;
#ifdef HAVE_LZMA
    lzma_stream StreamLZMA;
#endif
    uint8_t Header_Bytes[SWF_MAX_HEADER];
    uint8_t State;
    uint8_t Header_Len;
    uint8_t Header_Cnt;
};

/* API Functions */

/* Initialize the SWF file decompressor */
fd_status_t File_Decomp_Init_SWF(fd_session_t*);

/* Process the file incrementally */
fd_status_t File_Decomp_SWF(fd_session_t*);

/* End the SWF file decompression */
fd_status_t File_Decomp_End_SWF(fd_session_t*);

#endif

file_decomp_zip.h

Path = src/decompress/file_decomp_zip.h

#ifndef FILE_DECOMP_ZIP_H
#define FILE_DECOMP_ZIP_H

#include "file_decomp.h"

#include <zlib.h>

static const uint32_t ZIP_LOCAL_HEADER = 0x04034B50;

enum fd_ZIP_states
{
    ZIP_STATE_LH,             // local header (4 bytes)

    // skipped:
    // ZIP_STATE_VER,         // version (2 bytes)
    // ZIP_STATE_BITFLAG,     // bitflag (2 bytes)

    ZIP_STATE_METHOD,         // compression method (2 bytes)

    // skipped:
    // ZIP_STATE_MODTIME,     // modification time (2 bytes)
    // ZIP_STATE_MODDATE,     // modification date (2 bytes)
    // ZIP_STATE_CRC,         // CRC-32 (4 bytes)

    ZIP_STATE_COMPSIZE,       // compressed size (4 bytes)

    // skipped:
    // ZIP_STATE_UNCOMPSIZE,  // uncompressed size (4 bytes)

    ZIP_STATE_FILENAMELEN,    // filename length (2 bytes)
    ZIP_STATE_EXTRALEN,       // extra field length (2 bytes)

    // skipped:
    // ZIP_STATE_FILENAME,    // filename field (filenamelen bytes)
    // ZIP_STATE_EXTRA,       // extra field (extralen bytes)
    // ZIP_STATE_STREAM,      // compressed stream (compsize bytes)

    ZIP_STATE_INFLATE_INIT,   // initialize zlib inflate
    ZIP_STATE_INFLATE,        // perform zlib inflate
    ZIP_STATE_SKIP            // skip state
};

struct fd_ZIP_t
{
    // zlib stream
    z_stream Stream;

    // decompression progress
    unsigned progress;

    // ZIP fields
    uint32_t local_header;
    uint16_t method;
    uint32_t compressed_size;
    uint16_t filename_length;
    uint16_t extra_length;

    // field index
    unsigned Index;

    // current parser state
    fd_ZIP_states State;
    unsigned Length;

    // next parser state
    fd_ZIP_states Next;
    unsigned Next_Length;
};

// allocate and set initial ZIP state
fd_status_t File_Decomp_Init_ZIP(fd_session_t*);

// end ZIP processing
fd_status_t File_Decomp_End_ZIP(fd_session_t*);

// run the ZIP state machine
fd_status_t File_Decomp_ZIP(fd_session_t*);

#endif

detection/

Rules are grouped by ports and services. An MPSE instance is created for:

  • protocol and source port(s)

  • protocol and dest ports(s)

  • protocol any ports

  • service to server

  • service to client

For each fast pattern match state, a detection option tree is created which allows Snort to efficiently evaluate a set of rules. The non-leaf nodes in this tree reference an IpsOption instance. The leaf nodes are OTNs, which represents the rule body. Attached to the OTN is one or more RTNs, which represent the rule head. There is one RTN for each policy in which the rule appears. (There is just one instance of each unique RTN in each policy to save space.) The RTN criteria are evaluated last to determine if an event should be generated.

Note that the fast pattern detection code refers to qualified events and non-qualified events. The latter are just fast pattern hits for which no rule fired. The former are fast pattern hits for which a rule actually fired.

Rules w/o fast patterns are grouped per the above and evaluated for each packet for which the group is selected. These are definitely bad for performance.

The following was written by Norton and Roelker on 2002/05/15 and predates the use of services but is still applicable.

Fast Packet Classification for Rule and Pattern Matching in SNORT

A simple method for grouping rules into lists and looking them up quickly in realtime.

There is a natural problem when aggregating rules into pattern groups for performing multi-pattern matching not seen with single pattern Boyer-Moore strategies. The problem is how to group the rules efficiently when considering that there are multiple parameters which govern what rules to apply to each packet or connection. The parameters sip, dip, sport, dport, and flags form an enormous address space of possible packets that must be tested in realtime against a subset of rule patterns. Methods to group patterns precisely based on all of these parameters can quickly become complicated by both algorithmic implications and implementation details. The procedure described herein is quick and simple.

Some implementations of MPSE have the capability to perform searches for multiple pattern groups over the same piece of data efficiently, rather than requiring a separate search for each group. A batch of searches can be built up from a single piece of data and passed to the search_batch() MPSE method, allowing MPSE specific optimization of how to carry out the searches to be performed.

The methodology presented here to solve this problem is based on the premise that we can use the source and destination ports to isolate pattern groups for pattern matching, and rely on an event validation procedure to authenticate other parameters such as sip, dip and flags after a pattern match is made. An intrinsic assumption here is that most sip and dip values will be acceptable and that the big gain in performance is due to the fact that by isolating traffic based on services (ports) we gain the most benefit. Additionally, and just as important, is the requirement that we can perform a multi-pattern recognition-inspection phase on a large set of patterns many times quicker than we can apply a single pattern test against many single patterns.

The current implementation assumes that for each rule the src and dst ports each have one of 2 possible values. Either a specific port number or the ANYPORT designation. This does allow us to handle port ranges and NOT port rules as well.

We make the following assumptions about classifying packets based on ports:

  1. There are Unique ports which represent special services. For example, ports 21,25,80,110,etc.

  2. Patterns can be grouped into Unique Pattern groups, and a Generic Pattern Group

    1. Unique pattern groups exist for source ports 21,25,80,110,etc.

    2. Unique pattern groups exist for destination ports 21,25,80,etc.

    3. A Generic pattern group exists for rules applied to every combination of source and destination ports.

We make the following assumptions about packet traffic:

  1. Well behaved traffic has one Unique port and one ephemeral port for most packets and sometimes legitimately, as in the case of DNS, has two unique ports that are the same. But we always determine that packets with two different but Unique ports is bogus, and should generate an alert. For example, if you have traffic going from port 80 to port 20.

  2. In fact, state could tell us which side of this connection is a service and which side is a client. Than we could handle this packet more precisely, but this is a rare situation and is still bogus. We can choose not to do pattern inspections on these packets or to do complete inspections.

Rules are placed into each group as follows:

  1. Src Port == Unique Service, Dst Port == ANY → Unique Src Port Table Src Port == Unique Service, Dst Port == Unique → Unique Src & Dst Port Tables

  2. Dst Port == Unique Service, Src Port == ANY → Unique Dst Port Table Dst Port == Unique Service, Src Port == Unique → Unique Dst & Src Port Tables

  3. Dst Port == ANY, Src Port == ANY → Generic Rule Set, And add to all Unique Src/Dst Rule Sets that have entries

  4. !Dst or !Src Port is the same as ANY Dst or ANY Src port respectively

  5. DstA:DstB is treated as an ANY port group, same for SrcA:SrcB

Initialization

For each rule check the dst-port, if it’s specific, then add it to the dst table. If the dst-port is Any port, then do not add it to the dst port table. Repeat this for the src-port.

If the rule has Any for both ports then it’s added generic rule list.

Also, fill in the Unique-Conflicts array, this indicates if it’s OK to have the same Unique service port for both destination and source. This will force an alert if it’s not ok. We optionally pattern match against this anyway.

Processing Rules

When packets arrive:

  1. Categorize the Port Uniqueness:

    1. Check the DstPort[DstPort] for possible rules, if no entry,then no rules exist for this packet with this destination.

    2. Check the SrcPort[SrcPort] for possible rules, if no entry,then no rules exist for this packet with this source.

  2. Process the Uniqueness:

    If a AND !b has rules or !a AND b has rules then
       match against those rules
    If a AND b have rules then
        if( sourcePort != DstPort )
            Alert on this traffic and optionally match both rule sets
       else if( SourcePort == DstPort )
           Check the Unique-Conflicts array for allowable conflicts
           if( NOT allowed )
               Alert on this traffic, optionally match the rules
    else
        match both sets of rules against this traffic
    If( !a AND ! b )  then
        Pattern Match against the Generic Rules ( these apply to all packets)

Pseudocode

PORT_RULE_MAP * prm;
PortGroup  *src, *dst, *generic;
RULE * prule; //user defined rule structure for user rules
prm = prmNewMap();
for( each rule )
{
    prule = ....get a rule pointer
    prmAddRule( prm, prule->dport, prule->sport, prule );
}
prmCompileGroups( prm );
while( sniff-packets )
{
    ....
    stat = prmFindRuleGroup( prm, dport, sport, &src, &dst, &generic );
    switch( stat )
    {
    case 0:  // No rules at all
        break;
    case 1:  // Dst Rules
        // pass 'dst->pgPatData', 'dst->pgPatDataUri' to the pattern engine
        break;
    case 2:  // Src Rules
        // pass 'src->pgPatData', 'src->pgPatDataUri' to the pattern engine
        break;
    case 3:  // Src/Dst Rules - Both ports represent Unique service ports
        // pass 'src->pgPatData' ,'src->pgPatDataUri' to the pattern engine
        // pass 'dst->pgPatData'  'src->pgPatDataUri' to the pattern engine
        break;
    case 4:  // Generic Rules Only
        // pass 'generic->pgPatData' to the pattern engine
        // pass 'generic->pgPatDataUri' to the pattern engine
        break;
    }
}

context_switcher.h

Path = src/detection/context_switcher.h

#ifndef CONTEXT_SWITCHER_H
#define CONTEXT_SWITCHER_H

// ContextSwitcher maintains a set of contexts, only one of which can be
// active at any time. the normal workflow is:
//
// 1.  start and stop are called at the beginning and end of each wire
// packet which activates and releases one context from among those
// available.
//
// 2.  during processing interrupt and complete should be called to start
// and finish processing of a generated pseudo packet. it is possible to
// interrupt pseudo packets. complete may return without doing anything if
// dependent contexts were suspended.
//
// 3.  suspend may be called to pause the current context and activate the
// prior. multiple contexts may be suspended.
//
// 4.  there is no ordering of idle contexts. busy contexts are in strict LIFO
// order. context dependency chains are maintained in depth-first order by Flow.

#include <vector>

#include "detection/ips_context_chain.h"
#include "utils/primed_allocator.h"

namespace snort
{
class Flow;
class IpsContext;
class IpsContextData;
}

// FIXIT-E add the hold to catch offloads that don't return
class ContextSwitcher
{
public:
    ~ContextSwitcher();

    void push(snort::IpsContext*);

    void start();
    void stop();
    void abort();

    snort::IpsContext* interrupt();
    snort::IpsContext* complete();

    void suspend();
    void resume(snort::IpsContext*);

    snort::IpsContext* get_context() const;
    snort::IpsContext* get_next() const;

    snort::IpsContextData* get_context_data(unsigned id) const;
    void set_context_data(unsigned id, snort::IpsContextData*) const;

    unsigned idle_count() const;
    unsigned busy_count() const;

public:
    snort::IpsContextChain non_flow_chain;

private:
    std::vector<snort::IpsContext*> idle;
    std::vector<snort::IpsContext*> busy;
    std::vector<snort::IpsContext*> contexts;
};

#endif

detect.h

Path = src/detection/detect.h

#ifndef DETECT_H
#define DETECT_H

#include "detection/rules.h"
#include "main/snort_types.h"
#include "main/thread.h"

namespace snort
{
struct Packet;
struct ProfileStats;
}

extern THREAD_LOCAL snort::ProfileStats eventqPerfStats;

// main loop hooks
bool snort_ignore(snort::Packet*);
bool snort_log(snort::Packet*);

// alerts
void CallLogFuncs(snort::Packet*, ListHead*, struct Event*, const char*);
void CallLogFuncs(snort::Packet*, const OptTreeNode*, ListHead*);
void CallAlertFuncs(snort::Packet*, const OptTreeNode*, ListHead*);

void enable_tags();
void check_tags(snort::Packet*);

#endif

detection_engine.h

Path = src/detection/detection_engine.h

#ifndef DETECTION_ENGINE_H
#define DETECTION_ENGINE_H

// DetectionEngine manages a detection context.  To detect a rebuilt
// packet (PDU), first call set_next_packet().  If rebuild is successful,
// then instantiate a new DetectionEngine to detect that packet.

#include "actions/actions.h"
#include "detection/detection_util.h"
#include "detection/ips_context.h"
#include "main/snort_types.h"

struct DataPointer;
struct Replacement;

namespace snort
{
struct Packet;
class Flow;
class IpsContext;
class IpsContextChain;
class IpsContextData;

class SO_PUBLIC DetectionEngine
{
public:
    DetectionEngine();
    ~DetectionEngine();

public:
    static void thread_init();
    static void thread_term();

    static void reset();

    static IpsContext* get_context();

    static Packet* get_current_packet();
    static Packet* get_current_wire_packet();
    static Packet* set_next_packet(Packet* parent = nullptr);
    static uint8_t* get_next_buffer(unsigned& max);

    static bool offload(Packet*);

    static void onload(Flow*);
    static void onload();
    static void idle();

    static void set_encode_packet(Packet*);
    static Packet* get_encode_packet();

    static void set_file_data(const DataPointer& dp);
    static DataPointer& get_file_data(IpsContext*);

    static uint8_t* get_buffer(unsigned& max);
    static struct DataBuffer& get_alt_buffer(Packet*);

    static void set_data(unsigned id, IpsContextData*);
    static IpsContextData* get_data(unsigned id);
    static IpsContextData* get_data(unsigned id, IpsContext*);

    static void add_replacement(const std::string&, unsigned);
    static bool get_replacement(std::string&, unsigned&);
    static void clear_replacement();

    static bool detect(Packet*, bool offload_ok = false);
    static bool inspect(Packet*);

    static int queue_event(const struct OptTreeNode*);
    static int queue_event(unsigned gid, unsigned sid, Actions::Type = Actions::NONE);

    static void disable_all(Packet*);
    static bool all_disabled(Packet*);

    static void disable_content(Packet*);
    static void enable_content(Packet*);
    static bool content_enabled(Packet*);

    static IpsContext::ActiveRules get_detects(Packet*);
    static void set_detects(Packet*, IpsContext::ActiveRules);

    static void set_check_tags(bool enable = true);
    static bool get_check_tags();

    static void wait_for_context();

private:
    static struct SF_EVENTQ* get_event_queue();
    static bool do_offload(snort::Packet*);
    static void offload_thread(IpsContext*);
    static void complete(snort::Packet*);
    static void resume(snort::Packet*);
    static void resume_ready_suspends(const IpsContextChain&);

    static int log_events(Packet*);
    static void clear_events(Packet*);
    static void finish_inspect_with_latency(Packet*);
    static void finish_inspect(Packet*, bool inspected);
    static void finish_packet(Packet*, bool flow_deletion = false);

private:
    IpsContext* context;
};

static inline void set_file_data(const uint8_t* p, unsigned n)
{
    DataPointer dp { p, n };
    DetectionEngine::set_file_data(dp);
}

static inline void clear_file_data()
{ set_file_data(nullptr, 0); }

} // namespace snort
#endif

detection_module.h

Path = src/detection/detection_module.h

#ifndef DETECTION_MODULE_H
#define DETECTION_MODULE_H

#include "framework/module.h"

namespace snort
{
class DetectionModule : public Module
{
public:
    DetectionModule();

    bool set(const char*, Value&, SnortConfig*) override;
    bool end(const char*, int, SnortConfig*) override;

    const PegInfo* get_pegs() const override
    { return pc_names; }

    PegCount* get_counts() const override
    { return (PegCount*) &pc; }

    Usage get_usage() const override
    { return GLOBAL; }

    void set_trace(const Trace*) const override;
    const TraceOption* get_trace_options() const override;
};
}

#endif // DETECTION_MODULE_H

detection_options.h

Path = src/detection/detection_options.h

#ifndef DETECTION_OPTIONS_H
#define DETECTION_OPTIONS_H

// Support functions for rule option tree
//
// This implements tree processing for rule options, evaluating common
// detection options only once per pattern match.
//
// These trees are instantiated at parse time, one per MPSE match state.
// Eval, profiling, and latency data are attached in an array sized per max
// packet threads.

#include <sys/time.h>

#include "detection/rule_option_types.h"
#include "time/clock_defs.h"
#include "main/snort_debug.h"

namespace snort
{
class HashNode;
class XHash;
struct Packet;
struct SnortConfig;
}
struct RuleLatencyState;

typedef int (* eval_func_t)(void* option_data, class Cursor&, snort::Packet*);

// this is per packet thread
struct dot_node_state_t
{
    int result;
    struct
    {
        struct timeval ts;
        uint64_t context_num;
        uint32_t rebuild_flag;
        uint16_t run_num;
        char result;
        char flowbit_failed;
    } last_check;

    // FIXIT-L perf profiler stuff should be factored of the node state struct
    hr_duration elapsed;
    hr_duration elapsed_match;
    hr_duration elapsed_no_match;
    uint64_t checks;
    uint64_t disables;

    unsigned latency_timeouts;
    unsigned latency_suspends;

    // FIXIT-L perf profiler stuff should be factored of the node state struct
    void update(hr_duration delta, bool match)
    {
        elapsed += delta;

        if ( match )
            elapsed_match += delta;
        else
            elapsed_no_match += delta;

        ++checks;
    }
};

struct detection_option_tree_node_t
{
    eval_func_t evaluate;
    detection_option_tree_node_t** children;
    void* option_data;
    dot_node_state_t* state;
    struct OptTreeNode* otn;
    int is_relative;
    int num_children;
    int relative_children;
    option_type_t option_type;
};

struct detection_option_tree_root_t
{
    int num_children;
    detection_option_tree_node_t** children;
    RuleLatencyState* latency_state;

    struct OptTreeNode* otn;  // first rule in tree
};

struct detection_option_eval_data_t
{
    void* pmd;
    snort::Packet* p;
    char flowbit_failed;
    char flowbit_noalert;
};

// return existing data or add given and return nullptr
void* add_detection_option(struct snort::SnortConfig*, option_type_t, void*);
void* add_detection_option_tree(struct snort::SnortConfig*, detection_option_tree_node_t*);

int detection_option_node_evaluate(
    detection_option_tree_node_t*, detection_option_eval_data_t&, const class Cursor&);

void print_option_tree(detection_option_tree_node_t*, int level);
void detection_option_tree_update_otn_stats(snort::XHash*);

detection_option_tree_root_t* new_root(OptTreeNode*);
void free_detection_option_root(void** existing_tree);

detection_option_tree_node_t* new_node(option_type_t, void*);
void free_detection_option_tree(detection_option_tree_node_t*);

#endif

detection_util.h

Path = src/detection/detection_util.h

#ifndef DETECTION_UTIL_H
#define DETECTION_UTIL_H

// this is a legacy junk-drawer file that needs to be refactored
// it provides file and alt data and event trace foo.

#include "main/snort_config.h"

#define DECODE_BLEN 65535

struct DataPointer
{
    const uint8_t* data;
    unsigned len;
};

struct DataBuffer
{
    uint8_t data[DECODE_BLEN];
    unsigned len;
};

// FIXIT-RC event trace should be placed in its own files
void EventTrace_Init();
void EventTrace_Term();

void EventTrace_Log(const snort::Packet*, const OptTreeNode*, int action);

inline int EventTrace_IsEnabled(const snort::SnortConfig* sc)
{
    return ( sc->event_trace_max > 0 );
}

#endif

detect_trace.h

Path = src/detection/detect_trace.h

#ifndef DETECT_TRACE_H
#define DETECT_TRACE_H

// Detection trace utility

#include "framework/cursor.h"
#include "main/snort_types.h"
#include "main/thread.h"

namespace snort
{
struct Packet;
class Trace;
}

extern THREAD_LOCAL const snort::Trace* detection_trace;

struct detection_option_tree_node_t;
struct PatternMatchData;

enum
{
    TRACE_DETECTION_ENGINE = 0,
    TRACE_RULE_EVAL,
    TRACE_BUFFER,
    TRACE_RULE_VARS,
    TRACE_FP_SEARCH,
    TRACE_PKT_DETECTION,
    TRACE_OPTION_TREE,
    TRACE_TAG,
};

void clear_trace_cursor_info();
void print_pkt_info(snort::Packet* p, const char*);
void print_pattern(const PatternMatchData* pmd, snort::Packet*);
void dump_buffer(const uint8_t* buff, unsigned len, snort::Packet*);
void node_eval_trace(const detection_option_tree_node_t* node, const Cursor& cursor, snort::Packet*);

#endif

fp_config.h

Path = src/detection/fp_config.h

#ifndef FP_CONFIG_H
#define FP_CONFIG_H

namespace snort
{
    struct MpseApi;
}

// this is a basically a factory for creating MPSE

#define PL_BLEEDOVER_WARNINGS_ENABLED        0x01
#define PL_DEBUG_PRINT_NC_DETECT_RULES       0x02
#define PL_DEBUG_PRINT_RULEGROUP_BUILD       0x04
#define PL_DEBUG_PRINT_RULEGROUPS_UNCOMPILED 0x08
#define PL_DEBUG_PRINT_RULEGROUPS_COMPILED   0x10
#define PL_SINGLE_RULE_GROUP                 0x20

class FastPatternConfig
{
public:
    FastPatternConfig();

    void set_debug_mode()
    { debug = true; }

    bool get_debug_mode() const
    { return debug; }

    void set_stream_insert(bool enable)
    { inspect_stream_insert = enable; }

    bool get_stream_insert() const
    { return inspect_stream_insert; }

    void set_max_queue_events(unsigned num_events)
    { max_queue_events = num_events; }

    unsigned get_max_queue_events() const
    { return max_queue_events; }

    void set_bleed_over_port_limit(unsigned n)
    { bleedover_port_limit = n; }

    int get_bleed_over_port_limit() const
    { return bleedover_port_limit; }

    int get_single_rule_group() const
    { return portlists_flags & PL_SINGLE_RULE_GROUP; }

    int get_bleed_over_warnings() const
    { return portlists_flags & PL_BLEEDOVER_WARNINGS_ENABLED; }

    int get_debug_print_nc_rules() const
    { return portlists_flags & PL_DEBUG_PRINT_NC_DETECT_RULES; }

    int get_debug_print_rule_group_build_details() const
    { return portlists_flags & PL_DEBUG_PRINT_RULEGROUP_BUILD; }

    int get_debug_print_rule_groups_compiled() const
    { return portlists_flags & PL_DEBUG_PRINT_RULEGROUPS_COMPILED; }

    int get_debug_print_rule_groups_uncompiled() const
    { return portlists_flags & PL_DEBUG_PRINT_RULEGROUPS_UNCOMPILED; }

    void set_debug_print_fast_patterns(bool b)
    { debug_print_fast_pattern = b; }

    bool get_debug_print_fast_patterns() const
    { return debug_print_fast_pattern; }

    void set_split_any_any(bool enable)
    { split_any_any = enable; }

    bool get_split_any_any() const
    { return split_any_any; }

    void set_single_rule_group()
    { portlists_flags |= PL_SINGLE_RULE_GROUP; }

    void set_bleed_over_warnings()
    { portlists_flags |= PL_BLEEDOVER_WARNINGS_ENABLED; }

    void set_debug_print_nc_rules()
    { portlists_flags |= PL_DEBUG_PRINT_NC_DETECT_RULES; }

    void set_debug_print_rule_group_build_details()
    { portlists_flags |= PL_DEBUG_PRINT_RULEGROUP_BUILD; }

    void set_debug_print_rule_groups_compiled()
    { portlists_flags |= PL_DEBUG_PRINT_RULEGROUPS_COMPILED; }

    void set_debug_print_rule_groups_uncompiled()
    { portlists_flags |= PL_DEBUG_PRINT_RULEGROUPS_UNCOMPILED; }

    void set_search_opt(bool flag)
    { search_opt = flag; }

    bool get_search_opt() const
    { return search_opt; }

    bool set_search_method(const char*);
    const char* get_search_method();

    bool set_offload_search_method(const char*);
    void set_max_pattern_len(unsigned);
    void set_queue_limit(unsigned);

    unsigned get_queue_limit() const
    { return queue_limit; }

    const snort::MpseApi* get_search_api() const
    { return search_api; }

    const snort::MpseApi* get_offload_search_api() const
    { return offload_search_api; }

    int get_num_patterns_truncated() const
    { return num_patterns_truncated; }

    unsigned set_max(unsigned bytes);

private:
    const snort::MpseApi* search_api = nullptr;
    const snort::MpseApi* offload_search_api = nullptr;

    bool inspect_stream_insert = true;
    bool split_any_any = false;
    bool debug_print_fast_pattern = false;
    bool debug = false;
    bool search_opt = false;

    unsigned max_queue_events = 5;
    unsigned bleedover_port_limit = 1024;
    unsigned max_pattern_len = 0;

    unsigned queue_limit = 0;

    int portlists_flags = 0;
    int num_patterns_truncated = 0;  // due to max_pattern_len
};

#endif

fp_create.h

Path = src/detection/fp_create.h

#ifndef FPCREATE_H
#define FPCREATE_H

// this is where rule groups are compiled and MPSE are instantiated

#include <string>
#include "ports/port_group.h"

namespace snort
{
struct SnortConfig;
}

struct PMX
{
    struct PatternMatchData* pmd;
    RULE_NODE rule_node;
};

/* Used for negative content list */
struct NCListNode
{
    PMX* pmx;
    NCListNode* next;
};

/*
**  This is the main routine to create a FastPacket inspection
**  engine.  It reads in the snort list of RTNs and OTNs and
**  assigns them to PORT_MAPS.
*/
int fpCreateFastPacketDetection(snort::SnortConfig*);
void fpDeleteFastPacketDetection(snort::SnortConfig*);
void get_pattern_info(const PatternMatchData* pmd,
    const char* pattern, int pattern_length, std::string& hex, std::string& txt,
    std::string& opts);

#endif

fp_detect.h

Path = src/detection/fp_detect.h

#ifndef FPDETECT_H
#define FPDETECT_H

// this is where the high-level fast pattern matching action is
// rule groups are selected based on traffic and any fast pattern
// matches trigger rule tree evaluation.

#include "main/thread.h"
#include "profiler/profiler_defs.h"

#define REBUILD_FLAGS (PKT_REBUILT_FRAG | PKT_REBUILT_STREAM)

namespace snort
{
class IpsContext;
struct Packet;
struct ProfileStats;
}

class Cursor;
struct PortGroup;
struct OptTreeNode;

extern THREAD_LOCAL snort::ProfileStats mpsePerfStats;
extern THREAD_LOCAL snort::ProfileStats rulePerfStats;

struct RuleTreeNode;
int fpLogEvent(const RuleTreeNode*, const OptTreeNode*, snort::Packet*);
bool fp_eval_rtn(RuleTreeNode*, snort::Packet*, int check_ports);
int fp_eval_option(void*, Cursor&, snort::Packet*);

#define MAX_NUM_RULE_TYPES 16   // max number of allowed rule types

/*
**  This define is for the number of unique events
**  to match before choosing which event to log.
**  (Since we can only log one.) This define is the limit.
*/
#define MAX_EVENT_MATCH 100

/*
**  The events that are matched get held in this structure,
**  and iMatchIndex gets set to the event that holds the
**  highest priority.
*/
struct MatchInfo
{
    const OptTreeNode* MatchArray[MAX_EVENT_MATCH];
    unsigned iMatchCount;
    unsigned iMatchIndex;
    unsigned iMatchMaxLen;
};

/*
**  This structure holds information that is
**  referenced during setwise pattern matches.
**  It also contains information regarding the
**  number of matches that have occurred and
**  the event to log based on the event comparison
**  function.
*/
struct OtnxMatchData
{
    MatchInfo* matchInfo;
    bool have_match;
};

int fpAddMatch(OtnxMatchData*, const OptTreeNode*);

void fp_set_context(snort::IpsContext&);
void fp_clear_context(snort::IpsContext&);

void fp_full(snort::Packet*);
void fp_partial(snort::Packet*);
void fp_complete(snort::Packet*, bool search = false);

#endif

fp_utils.h

Path = src/detection/fp_utils.h

#ifndef FP_UTILS_H
#define FP_UTILS_H

// fast pattern utilities
#include <vector>
#include "framework/ips_option.h"
#include "framework/mpse.h"
#include "ports/port_group.h"

struct OptFpList;
struct OptTreeNode;

struct PatternMatchData* get_pmd(OptFpList*, SnortProtocolId, snort::RuleDirection);

bool make_fast_pattern_only(const OptFpList*, const PatternMatchData*);
bool is_fast_pattern_only(const OptTreeNode*, const OptFpList*, snort::Mpse::MpseType);

PmType get_pm_type(snort::CursorActionType);

bool set_fp_content(OptTreeNode*);

std::vector <PatternMatchData*> get_fp_content(
    OptTreeNode*, OptFpList*&, bool srvc, bool only_literals, bool& exclude);

void queue_mpse(snort::Mpse*);
unsigned compile_mpses(struct snort::SnortConfig*, bool parallel = false);

void validate_services(struct snort::SnortConfig*, OptTreeNode*);

#endif

ips_context_chain.h

Path = src/detection/ips_context_chain.h

#ifndef IPS_CONTEXT_CHAIN_H
#define IPS_CONTEXT_CHAIN_H

// IpsContextChain provides an interface for maintaining dependencies between
// IpsContexts. This class is provided to handle all linking and ensure only
// the tips of dependency chains are able to be processed, enforcing strict
// processing order.

namespace snort
{
class IpsContext;
class IpsContextChain
{
public:
    void abort()
    { _front = _back = nullptr; }

    IpsContext* front() const
    { return _front; }

    IpsContext* back() const
    { return _back; }

    void pop();
    void push_back(IpsContext*);

private:
    IpsContext* _front = nullptr;
    IpsContext* _back = nullptr;
};
}

#endif

ips_context_data.h

Path = src/detection/ips_context_data.h

#ifndef IPS_CONTEXT_DATA_H
#define IPS_CONTEXT_DATA_H

#include "main/snort_types.h"

#include "detection/detection_engine.h"

namespace snort
{
class SO_PUBLIC IpsContextData
{
public:
    virtual ~IpsContextData() = default;

    static unsigned get_ips_id();
    // Only unit tests can call this function to clear the id
    static void clear_ips_id();

    template<typename T>
    static T* get(unsigned ips_id)
    {
        T* data = (T*)DetectionEngine::get_data(ips_id);
        if ( ! data )
        {
            data = new T;
            DetectionEngine::set_data(ips_id, data);
        }
        return data;
    }
    virtual void clear() {}

protected:
    IpsContextData() = default;

private:
    static unsigned ips_id;
};
}
#endif

ips_context.h

Path = src/detection/ips_context.h

#ifndef IPS_CONTEXT_H
#define IPS_CONTEXT_H

// IpsContext provides access to all the state required for detection of a
// single packet.  the state is stored in IpsContextData instances, which
// are accessed by id.

#include <list>

#include "detection/detection_util.h"
#include "framework/codec.h"
#include "framework/mpse.h"
#include "framework/mpse_batch.h"
#include "main/snort_types.h"
#include "protocols/packet.h" // required to get a decent decl of pkth

class MpseStash;
struct OtnxMatchData;
struct SF_EVENTQ;
struct RegexRequest;

namespace snort
{
class IpsContextData;
struct SnortConfig;
struct Replacement
{
    std::string data;
    unsigned offset;
};

struct FlowSnapshot
{
    uint32_t session_flags;
    SnortProtocolId proto_id;
};

class SO_PUBLIC IpsContext
{
public:
    using Callback = void(*)(IpsContext*);
    enum State { IDLE, BUSY, SUSPENDED };

    IpsContext(unsigned size = 0);  // defaults to max id
    ~IpsContext();

    IpsContext(const IpsContext&) = delete;
    IpsContext& operator=(const IpsContext&) = delete;

    void setup();
    void clear();

    void set_context_data(unsigned id, IpsContextData*);
    IpsContextData* get_context_data(unsigned id) const;

    void snapshot_flow(Flow*);

    uint32_t get_session_flags()
    { return flow.session_flags; }

    SnortProtocolId get_snort_protocol_id()
    { return flow.proto_id; }

    void disable_detection();
    void disable_inspection();

    enum ActiveRules
    { NONE, NON_CONTENT, CONTENT };

    void register_post_callback(Callback callback)
    { post_callbacks.emplace_back(callback); }

    void clear_callbacks()
    { post_callbacks.clear(); }

    bool has_callbacks() const
    { return !post_callbacks.empty(); }

    void post_detection();

    void link(IpsContext* next)
    {
        assert(!next->depends_on);
        assert(!next->next_to_process);
        assert(!next_to_process);

        next->depends_on = this;
        next_to_process = next;
    }

    void unlink()
    {
        assert(!depends_on);
        if ( next_to_process )
        {
            assert(next_to_process->depends_on == this);
            next_to_process->depends_on = nullptr;
        }
        next_to_process = nullptr;
    }

    IpsContext* dependencies() const
    { return depends_on; }

    IpsContext* next() const
    { return next_to_process; }

    void abort()
    {
        if ( next_to_process )
            next_to_process->depends_on = depends_on;

        if ( depends_on )
            depends_on->next_to_process = next_to_process;

        depends_on = next_to_process = nullptr;
    }

public:
    std::vector<Replacement> rpl;

    Packet* packet;
    Packet* wire_packet = nullptr;
    Packet* encode_packet;
    DAQ_PktHdr_t* pkth;
    uint8_t* buf;

    const SnortConfig* conf = nullptr;
    MpseBatch searches;
    MpseStash* stash;
    OtnxMatchData* otnx;
    std::list<RegexRequest*>::iterator regex_req_it;
    SF_EVENTQ* equeue;

    DataPointer file_data = {};
    DataBuffer alt_data = {};

    uint64_t context_num;
    uint64_t packet_number = 0;
    ActiveRules active_rules;
    State state;
    bool check_tags;
    bool clear_inspectors;

    static const unsigned buf_size = Codec::PKT_MAX;

    // FIXIT-L eliminate max_ips_id and just resize data vector.
    // Only 5 inspectors currently use the ips context data.
    static constexpr unsigned max_ips_id = 8;

private:
    FlowSnapshot flow = {};
    std::vector<IpsContextData*> data;
    std::vector<unsigned> ids_in_use;  // for indirection; FIXIT-P evaluate alternatives
    std::vector<Callback> post_callbacks;
    IpsContext* depends_on;
    IpsContext* next_to_process;
    bool remove_gadget = false;
};
}
#endif

pattern_match_data.h

Path = src/detection/pattern_match_data.h

#ifndef PATTERN_MATCH_DATA_H
#define PATTERN_MATCH_DATA_H

#include <sys/time.h>
#include <vector>

#include "framework/ips_option.h"  // FIXIT-L not a good dependency

struct PmdLastCheck
{
    struct timeval ts;
    uint64_t context_num;
    uint32_t rebuild_flag;
    uint16_t run_num;
};

struct PatternMatchData
{
    const char* pattern_buf; // app layer pattern to match on

    // FIXIT-L wasting some memory here:
    // - this is not used by content option logic directly
    // - and only used on current eval (not across packets)
    // (partly mitigated by only allocating if exception_flag is set)
    //
    /* Set if fast pattern matcher found a content in the packet,
       but the rule option specifies a negated content. Only
       applies to negative contents that are not relative */
    PmdLastCheck* last_check;

    unsigned pattern_size;   // size of app layer pattern

    int offset;              // pattern search start offset
    int depth;               // pattern search depth

    enum
    {
        NEGATED  = 0x01,
        NO_CASE  = 0x02,
        RELATIVE = 0x04,
        LITERAL  = 0x08,
        FAST_PAT = 0x10,
        NO_FP    = 0x20,
    };

    uint16_t flags;          // from above enum
    uint16_t mpse_flags;     // passed through to mpse

    uint16_t fp_offset;
    uint16_t fp_length;

    // not used by ips_content
    uint8_t pm_type;

    bool is_unbounded() const
    { return !depth; }

    void set_fast_pattern()
    { flags |= FAST_PAT; }

    void set_negated()
    { flags |= NEGATED; }

    void set_no_case()
    { flags |= NO_CASE; }

    void set_relative()
    { flags |= RELATIVE; }

    void set_literal()
    { flags |= LITERAL; }

    bool is_fast_pattern() const
    { return (flags & FAST_PAT) != 0; }

    bool is_negated() const
    { return (flags & NEGATED) != 0; }

    bool is_no_case() const
    { return (flags & NO_CASE) != 0; }

    bool is_relative() const
    { return (flags & RELATIVE) != 0; }

    bool is_literal() const
    { return (flags & LITERAL) != 0; }

    bool can_be_fp() const;
};

typedef std::vector<PatternMatchData*> PatternMatchVector;

inline bool PatternMatchData::can_be_fp() const
{
    if ( !pattern_buf or !pattern_size )
        return false;

    if ( flags & NO_FP )
        return false;

    if ( !is_negated() )
        return true;

    // Negative contents can only be considered if they are not
    // relative and don't have any offset or depth.  This is because
    // the pattern matcher does not take these into consideration and
    // may find the content in a non-relevant section of the payload
    // and thus disable the rule when it shouldn't be.

    // Also case sensitive patterns cannot be considered since patterns
    // are inserted into the pattern matcher without case which may
    // lead to false negatives.

    if ( is_relative() or !is_no_case() or offset or depth )
        return false;

    return true;
}

#endif

pcrm.h

Path = src/detection/pcrm.h

#ifndef PCRM_H
#define PCRM_H

// Packet Classification-Rule Manager
// rule groups by source and dest ports as well as any
// (generic refers to any)

#include "ports/port_group.h"
#include "protocols/packet.h"

#define ANYPORT (-1)

struct PORT_RULE_MAP
{
    int prmNumDstRules;
    int prmNumSrcRules;
    int prmNumGenericRules;

    int prmNumDstGroups;
    int prmNumSrcGroups;

    PortGroup* prmSrcPort[snort::MAX_PORTS];
    PortGroup* prmDstPort[snort::MAX_PORTS];
    PortGroup* prmGeneric;
};

PORT_RULE_MAP* prmNewMap();

int prmFindRuleGroupTcp(PORT_RULE_MAP*, int, int, PortGroup**, PortGroup**, PortGroup**);
int prmFindRuleGroupUdp(PORT_RULE_MAP*, int, int, PortGroup**, PortGroup**, PortGroup**);
int prmFindRuleGroupIp(PORT_RULE_MAP*, int, PortGroup**, PortGroup**);
int prmFindRuleGroupIcmp(PORT_RULE_MAP*, int, PortGroup**, PortGroup**);

#endif

regex_offload.h

Path = src/detection/regex_offload.h

#ifndef REGEX_OFFLOAD_H
#define REGEX_OFFLOAD_H

// RegexOffload provides an interface to fast pattern search accelerators.
// There are two flavors: MPSE and thread.  The MpseRegexOffload interfaces to
// an MPSE that is capable of regex offload such as the RXP whereas
// ThreadRegexOffload implements the regex search in auxiliary threads w/o
// requiring extra MPSE instances.  presently all offload is per packet thread;
// packet threads do not share offload resources.

#include <condition_variable>
#include <list>
#include <mutex>
#include <thread>

namespace snort
{
class Flow;
struct Packet;
struct SnortConfig;
}
struct RegexRequest;

class RegexOffload
{
public:
    static RegexOffload* get_offloader(unsigned max, bool async);
    virtual ~RegexOffload();

    virtual void stop();

    virtual void put(snort::Packet*) = 0;
    virtual bool get(snort::Packet*&) = 0;

    unsigned available() const
    { return idle.size(); }

    unsigned count() const
    { return busy.size(); }

    bool on_hold(snort::Flow*) const;

protected:
    RegexOffload(unsigned max);

protected:
    std::list<RegexRequest*> busy;
    std::list<RegexRequest*> idle;
};

class MpseRegexOffload : public RegexOffload
{
public:
    MpseRegexOffload(unsigned max);

    void put(snort::Packet*) override;
    bool get(snort::Packet*&) override;
};

class ThreadRegexOffload : public RegexOffload
{
public:
    ThreadRegexOffload(unsigned max);
    ~ThreadRegexOffload() override;

    void stop() override;

    void put(snort::Packet*) override;
    bool get(snort::Packet*&) override;

private:
    static void worker(RegexRequest*, const snort::SnortConfig*, unsigned id);
};

#endif

rtn_checks.h

Path = src/detection/rtn_checks.h

#ifndef RTN_CHECKS_H
#define RTN_CHECKS_H

namespace snort
{
    struct Packet;
}
struct RuleFpList;
struct RuleTreeNode;

// parsing
int RuleListEnd(snort::Packet*, RuleTreeNode*, RuleFpList*, int);
int OptListEnd(void* option_data, class Cursor&, snort::Packet*);

// detection
int CheckBidirectional(snort::Packet*, RuleTreeNode*, RuleFpList*, int);

int CheckProto(snort::Packet*, RuleTreeNode*, RuleFpList*, int);
int CheckSrcIP(snort::Packet*, RuleTreeNode*, RuleFpList*, int);
int CheckDstIP(snort::Packet*, RuleTreeNode*, RuleFpList*, int);

int CheckSrcPortEqual(snort::Packet*, RuleTreeNode*, RuleFpList*, int);
int CheckDstPortEqual(snort::Packet*, RuleTreeNode*, RuleFpList*, int);
int CheckSrcPortNotEq(snort::Packet*, RuleTreeNode*, RuleFpList*, int);
int CheckDstPortNotEq(snort::Packet*, RuleTreeNode*, RuleFpList*, int);

#endif

rule_option_types.h

Path = src/detection/rule_option_types.h

#ifndef RULE_OPTION_TYPES_H
#define RULE_OPTION_TYPES_H

// if you change this, you must also update detection_options.cc::option_type_str[].
enum option_type_t
{
    RULE_OPTION_TYPE_LEAF_NODE,    // internal use by rule compiler
    RULE_OPTION_TYPE_BUFFER_SET,   // sets sticky buffer
    RULE_OPTION_TYPE_BUFFER_USE,   // uses sticky buffer
    RULE_OPTION_TYPE_CONTENT,      // ideally would be eliminated (implies _BUFFER_USE)
    RULE_OPTION_TYPE_FLOWBIT,      // ideally would be eliminated
    RULE_OPTION_TYPE_OTHER         // for all new buffer independent rule options
};

#endif

rules.h

Path = src/detection/rules.h

#ifndef RULES_H
#define RULES_H

// misc rule and rule list support
// FIXIT-L refactor this header

#include <map>
#include <string>

#include "actions/actions.h"
#include "main/policy.h"

#define GID_DEFAULT          1
#define GID_SESSION        135

#define GID_BUILTIN_MIN    100
#define GID_BUILTIN_MAX    999

// should be revoked in the future
#define GID_EXCEPTION_SDF  138

#define SESSION_EVENT_SYN_RX 1
#define SESSION_EVENT_SETUP  2
#define SESSION_EVENT_CLEAR  3

#define EventIsInternal(gid) ((gid) == GID_SESSION)

namespace snort
{
    class IpsAction;
    struct SnortConfig;
}
struct OutputSet;
struct RuleTreeNode;

struct ListHead
{
    OutputSet* LogList;
    OutputSet* AlertList;
    struct RuleListNode* ruleListNode;
    bool is_plugin_action = false;
};

// for top-level rule lists by type (alert, drop, etc.)
struct RuleListNode
{
    ListHead* RuleList;   /* The rule list associated with this node */
    snort::Actions::Type mode;        /* the rule mode */
    unsigned evalIndex;        /* eval index for this rule set */
    char* name;           /* name of this rule list */
    RuleListNode* next;   /* the next RuleListNode */
};

struct RuleKey
{
    unsigned policy_id;
    unsigned gid;
    unsigned sid;

    friend bool operator< (const RuleKey&, const RuleKey&);
};

struct RuleState
{
    std::string rule_action;
    snort::Actions::Type action;
    IpsPolicy::Enable enable;
};

class RuleStateMap
{
public:
    void add(const RuleKey& key, const RuleState& state)
    { map[key] = state; }

    void apply(snort::SnortConfig*);

private:
    RuleTreeNode* dup_rtn(RuleTreeNode*);
    void update_rtn(RuleTreeNode*, const RuleState&);
    void apply(snort::SnortConfig*, OptTreeNode*, unsigned ips_num, const RuleState&);

private:
    std::map<RuleKey, RuleState> map;
};

#endif

service_map.h

Path = src/detection/service_map.h

#ifndef SERVICE_MAP_H
#define SERVICE_MAP_H

//  for managing rule groups by service
//  direction to client and to server are separate

#include <vector>

#include "target_based/snort_protocols.h"

namespace snort
{
struct SnortConfig;
class GHash;
}
struct PortGroup;

//  Service Rule Map Master Table
struct srmm_table_t
{
    snort::GHash* to_srv;
    snort::GHash* to_cli;
};

srmm_table_t* ServiceMapNew();
void ServiceMapFree(srmm_table_t*);

srmm_table_t* ServicePortGroupMapNew();
void ServicePortGroupMapFree(srmm_table_t*);

void fpPrintServicePortGroupSummary(snort::SnortConfig*);
void fpCreateServiceMaps(snort::SnortConfig*);

//  Service/Protocol Ordinal To PortGroup table
typedef std::vector<PortGroup*> PortGroupVector;

struct sopg_table_t
{
    sopg_table_t(unsigned size);
    PortGroup* get_port_group(bool c2s, SnortProtocolId svc);

    PortGroupVector to_srv;
    PortGroupVector to_cli;
};


#endif

sfrim.h

Path = src/detection/sfrim.h

#ifndef SFRIM_H
#define SFRIM_H

// provides an ordinal for each rule so they can be looked up by a number
// used during parse time when rules are compiled

struct rule_index_map_t;

rule_index_map_t* RuleIndexMapCreate();
void RuleIndexMapFree(rule_index_map_t*);

int RuleIndexMapAdd(rule_index_map_t*, unsigned gid, unsigned sid);
bool RuleIndexMapGet(rule_index_map_t* map, int index, unsigned& gid, unsigned& sid);

#endif

signature.h

Path = src/detection/signature.h

#ifndef SIGNATURE_H
#define SIGNATURE_H

// basic non-detection signature info:  gid, sid, rev, class, priority, etc.

#include <cstdint>
#include <cstdio>
#include <string>

#include "target_based/snort_protocols.h"

namespace snort
{
class GHash;
struct SnortConfig;
}

struct OptTreeNode;

struct ReferenceSystem
{
    ReferenceSystem(const std::string& n, const char* u) : name(n), url(u) { }
    std::string name;
    std::string url;
};

const ReferenceSystem* reference_system_add(snort::SnortConfig*, const std::string&, const char* = "");

struct ReferenceNode
{
    ReferenceNode(const ReferenceSystem* sys, const std::string& id) : system(sys), id(id) { }
    const ReferenceSystem* system;
    std::string id;
};

void add_reference(snort::SnortConfig*, OptTreeNode*, const std::string& sys, const std::string& id);

struct ClassType
{
    ClassType(const char* s, const char* txt, unsigned pri, int id) :
        name(s), text(txt), priority(pri), id(id) { }

    std::string name;
    std::string text;
    unsigned priority;
    int id;
};

void add_classification(snort::SnortConfig*, const char* name, const char* text, unsigned priority);

const ClassType* get_classification(snort::SnortConfig*, const char*);

struct SignatureServiceInfo
{
    SignatureServiceInfo(const char* s, SnortProtocolId proto) :
        service(s), snort_protocol_id(proto) { }
    std::string service;
    SnortProtocolId snort_protocol_id;
};

struct OtnKey
{
    uint32_t gid;
    uint32_t sid;
};

enum Target
{ TARGET_NONE, TARGET_SRC, TARGET_DST, TARGET_MAX = TARGET_DST };

struct SigInfo
{
    std::string message;
    std::string* body = nullptr;

    std::vector<const ReferenceNode*> refs;
    std::vector<SignatureServiceInfo> services;

    const ClassType* class_type = nullptr;

    uint32_t gid = 0;
    uint32_t sid = 0;
    uint32_t rev = 0;

    uint32_t class_id = 0;
    uint32_t priority = 0;

    bool builtin = false;
    Target target = TARGET_NONE;
};

snort::GHash* OtnLookupNew();
void OtnLookupAdd(snort::GHash*, OptTreeNode*);
OptTreeNode* OtnLookup(snort::GHash*, uint32_t gid, uint32_t sid);
void OtnLookupFree(snort::GHash*);
void OtnRemove(snort::GHash*, OptTreeNode*);

OptTreeNode* GetOTN(uint32_t gid, uint32_t sid);

void dump_msg_map(const snort::SnortConfig*);
void dump_rule_deps(const snort::SnortConfig*);
void dump_rule_meta(const snort::SnortConfig*);
void dump_rule_state(const snort::SnortConfig*);

#endif

tag.h

Path = src/detection/tag.h

#ifndef TAG_H
#define TAG_H

// rule option tag causes logging of some number of subsequent packets
// following an alert.  this module is use by the tag option to implement
// that functionality.  uses its own hash table.
//
// FIXIT-L convert tags to use flow instead of hash table.

#include <cstdint>

namespace snort
{
struct Packet;
}

struct OptTreeNode;
struct Event;

#define TAG_SESSION   1
#define TAG_HOST      2
#define TAG_HOST_SRC  3
#define TAG_HOST_DST  4

#define TAG_METRIC_SECONDS    0x01
#define TAG_METRIC_PACKETS    0x02
#define TAG_METRIC_BYTES      0x04
#define TAG_METRIC_UNLIMITED  0x08
#define TAG_METRIC_SESSION    0x10

struct TagData
{
    int tag_type;       /* tag type (session/host) */
    int tag_metric;     /* (packets | seconds | bytes) units */
    int tag_direction;  /* source or dest, used for host tagging */

    uint32_t tag_seconds;    /* number of "seconds" units to tag for */
    uint32_t tag_packets;    /* number of "packets" units to tag for */
    uint32_t tag_bytes;      /* number of "type" units to tag for */
};

void InitTag();
void CleanupTag();
int CheckTagList(snort::Packet*, Event&, void**);
void SetTags(const snort::Packet*, const OptTreeNode*, uint16_t);

#endif

treenodes.h

Path = src/detection/treenodes.h

#ifndef TREENODES_H
#define TREENODES_H

// rule header (RTN) and body (OTN) nodes

#include "actions/actions.h"
#include "detection/signature.h"
#include "detection/rule_option_types.h"
#include "main/policy.h"
#include "main/snort_types.h"
#include "ports/port_group.h"
#include "time/clock_defs.h"

namespace snort
{
class IpsOption;
struct Packet;
}
struct RuleTreeNode;
struct PortObject;
struct OutputSet;
struct TagData;
struct sfip_var_t;

/* same as the rule header FP list */
struct OptFpList
{
    snort::IpsOption* ips_opt;

    int (* OptTestFunc)(void* option_data, class Cursor&, snort::Packet*);

    OptFpList* next;

    unsigned char isRelative;
    option_type_t type;
};

struct OtnState
{
    // profiling
    // FIXIT-L factor the profiling stuff out
    hr_duration elapsed = 0_ticks;
    hr_duration elapsed_match = 0_ticks;
    hr_duration elapsed_no_match = 0_ticks;

    uint64_t checks = 0;
    uint64_t matches = 0;
    uint8_t noalerts = 0;
    uint64_t alerts = 0;

    uint64_t latency_timeouts = 0;
    uint64_t latency_suspends = 0;

    operator bool() const
    { return elapsed > 0_ticks || checks > 0; }
};

/* function pointer list for rule head nodes */
// FIXIT-L use bit mask to determine what header checks to do
// cheaper than traversing a list and uses much less memory
struct RuleFpList
{
    /* context data for this test */
    void* context = nullptr;

    /* rule check function pointer */
    int (* RuleHeadFunc)(snort::Packet*, RuleTreeNode*, RuleFpList*, int) = nullptr;

    /* pointer to the next rule function node */
    RuleFpList* next = nullptr;
};

struct RuleHeader
{
    RuleHeader(const char* s) : action(s) { }

    std::string action;
    std::string proto;
    std::string src_nets;
    std::string src_ports;
    std::string dir;
    std::string dst_nets;
    std::string dst_ports;
};

// one of these per rule per policy
// represents head part of rule
struct RuleTreeNode
{
    using Flag = uint8_t;
    static constexpr Flag ENABLED       = 0x01;
    static constexpr Flag ANY_SRC_PORT  = 0x02;
    static constexpr Flag ANY_DST_PORT  = 0x04;
    static constexpr Flag ANY_FLAGS     = 0x08;
    static constexpr Flag BIDIRECTIONAL = 0x10;
    static constexpr Flag ANY_SRC_IP    = 0x20;
    static constexpr Flag ANY_DST_IP    = 0x40;
    static constexpr Flag USER_MODE     = 0x80;

    RuleFpList* rule_func = nullptr; /* match functions.. (Bidirectional etc.. ) */
    RuleHeader* header = nullptr;

    sfip_var_t* sip = nullptr;
    sfip_var_t* dip = nullptr;

    PortObject* src_portobject = nullptr;
    PortObject* dst_portobject = nullptr;

    struct ListHead* listhead = nullptr;

    SnortProtocolId snort_protocol_id = 0;

    // reference count from otn.
    // Multiple OTNs can reference this RTN with the same policy.
    unsigned int otnRefCount = 0; // FIXIT-L shared_ptr?

    snort::Actions::Type action = snort::Actions::Type::NONE;

    uint8_t flags = 0;

    void set_enabled()
    { flags |= ENABLED; }

    void clear_enabled()
    { flags &= (~ENABLED); }

    bool enabled() const
    { return (flags & ENABLED) != 0; }

    bool user_mode() const
    { return (flags & USER_MODE) != 0; }

    bool any_src_port() const
    { return (flags & ANY_SRC_PORT) != 0; }

    bool any_dst_port() const
    { return (flags & ANY_DST_PORT) != 0; }

    bool any_any_port() const
    { return any_src_port() and any_dst_port(); }
};

// one of these for each rule
// represents body part of rule
struct OptTreeNode
{
    ~OptTreeNode();

    using Flag = uint8_t;
    static constexpr Flag WARNED_FP  = 0x01;
    static constexpr Flag STATELESS  = 0x02;
    static constexpr Flag RULE_STATE = 0x04;
    static constexpr Flag META_MATCH = 0x08;
    static constexpr Flag TO_CLIENT  = 0x10;
    static constexpr Flag TO_SERVER  = 0x20;
    static constexpr Flag BIT_CHECK  = 0x40;

    /* metadata about signature */
    SigInfo sigInfo;
    char* soid = nullptr;

    /* plugin/detection functions go here */
    OptFpList* opt_func = nullptr;
    OutputSet* outputFuncs = nullptr; /* per sid enabled output functions */
    snort::IpsOption* agent = nullptr;

    OptFpList* normal_fp_only = nullptr;
    OptFpList* offload_fp_only = nullptr;

    struct THD_NODE* detection_filter = nullptr; /* if present, evaluated last, after header checks */
    TagData* tag = nullptr;

    // ptr to list of RTNs (head part); indexed by policyId
    RuleTreeNode** proto_nodes = nullptr;
    OtnState* state = nullptr;

    unsigned evalIndex = 0;       /* where this rule sits in the evaluation sets */
    unsigned ruleIndex = 0; // unique index
    uint32_t num_detection_opts = 0;
    SnortProtocolId snort_protocol_id = 0;    // Added for integrity checks during rule parsing.
    unsigned short proto_node_num = 0;
    uint16_t longestPatternLen = 0;
    IpsPolicy::Enable enable;
    Flag flags = 0;

    uint8_t sticky_buf = PM_TYPE_PKT; // parsing only

    void set_warned_fp()
    { flags |= WARNED_FP; }

    bool warned_fp() const
    { return (flags & WARNED_FP) != 0; }

    void set_stateless()
    { flags |= STATELESS; }

    bool stateless() const
    { return (flags & STATELESS) != 0; }

    void set_enabled(IpsPolicy::Enable e)
    { enable = e; flags |= RULE_STATE; }

    bool is_rule_state_stub() const
    { return (flags & RULE_STATE) != 0; }

    bool enabled_somewhere() const
    {
        for ( unsigned i = 0; i < proto_node_num; i++ )
            if ( proto_nodes[i] and proto_nodes[i]->enabled() )
                return true;

        return false;
    }

    void set_metadata_match()
    { flags |= META_MATCH; }

    bool metadata_matched() const
    { return (flags & META_MATCH) != 0; }

    void set_to_client()
    { flags |= TO_CLIENT; }

    bool to_client() const
    { return (flags & TO_CLIENT) != 0; }

    void set_to_server()
    { flags |= TO_SERVER; }

    bool to_server() const
    { return (flags & TO_SERVER) != 0; }

    void set_flowbits_check()
    { flags |= BIT_CHECK; }

    bool checks_flowbits() const
    { return (flags & BIT_CHECK) != 0; }

    void update_fp(snort::IpsOption*);
};

typedef int (* RuleOptEvalFunc)(void*, Cursor&, snort::Packet*);
OptFpList* AddOptFuncToList(RuleOptEvalFunc, OptTreeNode*);

void* get_rule_type_data(OptTreeNode*, const char* name);

namespace snort
{
SO_PUBLIC bool otn_has_plugin(OptTreeNode* otn, const char* name);
}

bool otn_set_agent(OptTreeNode*, snort::IpsOption*);

void otn_trigger_actions(const OptTreeNode*, snort::Packet*);

#endif

dump_config/

This directory contains classes related to Snort configuration dump.

Snort supports dumping config into a file for JSON and text formats.

  • ConfigData

    The ConfigData structure represents the internal config data of a particular Lua file.
    A unique shell is associated with every base and targeted policy file.
    Every module in a Lua file is represented by the General Tree.
    The Tree node can be one of the following types: TreeConfigNode, ValueConfigNode.
    The TreeConfigNode represents a table or list.
    The ValueConfigNode represents a config value itself.
  • ConfigOutput

    The ConfigOutput class is a base class that encapsulates dumping config into a file.
    Pure virtual function dump() should be overridden by derived classes for particular
    output format.
  • JsonAllConfigOutput

    The JsonAllConfigOutput class inherits from ConfigOutput and dumps base and targeted
    policy file in JSON format.
  • TextConfigOutput

    The TextConfigOutput class inherits from ConfigOutput and dumps base and targeted
    policy file in text format.

config_data.h

Path = src/dump_config/config_data.h

#ifndef CONFIG_DATA_H
#define CONFIG_DATA_H

#include <list>

#include "framework/value.h"

class BaseConfigNode;

using ConfigTrees = std::list<BaseConfigNode*>;

class BaseConfigNode
{
public:
    BaseConfigNode(BaseConfigNode* parent);
    virtual ~BaseConfigNode() = default;

    virtual std::string get_name() const = 0;
    virtual snort::Parameter::Type get_type() const = 0;
    virtual BaseConfigNode* get_node(const std::string& name) = 0;
    virtual void set_value(const snort::Value&) {}
    virtual const snort::Value* get_value() const { return nullptr; }

    const ConfigTrees& get_children() const
    { return children; }

    BaseConfigNode* get_parent_node() const
    { return parent; }

    void add_child_node(BaseConfigNode* node);

    static void clear_nodes(BaseConfigNode* root);
    static void sort_nodes(BaseConfigNode* node);

protected:
    ConfigTrees children;
    BaseConfigNode* parent = nullptr;
};

class TreeConfigNode : public BaseConfigNode
{
public:
    TreeConfigNode(BaseConfigNode* parent, const std::string& node_name,
        const snort::Parameter::Type node_type);

private:
    virtual std::string get_name() const override
    { return name; }

    virtual snort::Parameter::Type get_type() const override
    { return type; }

    virtual BaseConfigNode* get_node(const std::string& name) override;

private:
    std::string name;
    snort::Parameter::Type type = snort::Parameter::PT_MAX;
};

class ValueConfigNode : public BaseConfigNode
{
public:
    ValueConfigNode(BaseConfigNode* parent, const snort::Value& value,
        const std::string& name = "");

private:
    virtual std::string get_name() const override
    { return !custom_name.empty() ? custom_name : value.get_name(); }

    virtual snort::Parameter::Type get_type() const override
    { return value.get_param_type(); }

    virtual const snort::Value* get_value() const override
    { return &value; }

    virtual void set_value(const snort::Value& v) override;
    virtual BaseConfigNode* get_node(const std::string& name) override;

private:
    snort::Value value;
    bool multi_value = false;
    std::string custom_name;
};

class ConfigData
{
public:
    ConfigData(const char* file_name);

    void add_config_tree(BaseConfigNode* root)
    { config_trees.push_back(root); }

    void sort();
    void clear();

public:
    std::string file_name;
    ConfigTrees config_trees;
};

#endif // CONFIG_DATA_H

config_output.h

Path = src/dump_config/config_output.h

#ifndef CONFIG_OUTPUT_H
#define CONFIG_OUTPUT_H

class ConfigData;

class ConfigOutput
{
public:
    ConfigOutput() = default;
    virtual ~ConfigOutput() = default;

    void dump_config(ConfigData&);

private:
    virtual void dump(const ConfigData&) = 0;
};

#endif // CONFIG_OUTPUT_H

json_config_output.h

Path = src/dump_config/json_config_output.h

#ifndef JSON_CONFIG_OUTPUT_H
#define JSON_CONFIG_OUTPUT_H

#include "config_output.h"
#include "helpers/json_stream.h"

class BaseConfigNode;

class JsonAllConfigOutput : public ConfigOutput
{
public:
    JsonAllConfigOutput();
    ~JsonAllConfigOutput() override;

private:
    void dump(const ConfigData&) override;

private:
    snort::JsonStream json;
};

class JsonTopConfigOutput : public ConfigOutput
{
public:
    JsonTopConfigOutput() : ConfigOutput(), json(std::cout) {}

private:
    void dump(const ConfigData&) override;

private:
    snort::JsonStream json;
};

#endif // JSON_CONFIG_OUTPUT_H

text_config_output.h

Path = src/dump_config/text_config_output.h

#ifndef TEXT_CONFIG_OUTPUT_H
#define TEXT_CONFIG_OUTPUT_H

#include <string>

#include "config_output.h"

class BaseConfigNode;

class TextConfigOutput : public ConfigOutput
{
public:
    TextConfigOutput() = default;

private:
    void dump(const ConfigData&) override;
};

#endif // TEXT_CONFIG_OUTPUT_H

events/

This unit manages the event queue. Widely used utility GenerateSnortEvent() is in event_wrapper.h.

The event queue has a configurable maximum number of events, which are preallocated and stored in a linked list.

There are multiple instances of the event queue accessed via a simple stack. A push is done before processing a rebuilt packet or rebuilt payload after which a pop is done. During that time any wire packet events are still pending in the event queue higher up the stack. This ensures that the events for each packet (wire or rebuilt) are processed separately.

event.h

Path = src/events/event.h

#ifndef EVENT_H
#define EVENT_H

#include "main/thread.h"

struct SigInfo;
extern THREAD_LOCAL uint16_t event_id;

/* we must use fixed size of 32 bits, because on-disk
 * format of savefiles uses 32-bit tv_sec (and tv_usec)
 */
struct sf_timeval32
{
    uint32_t tv_sec;      /* seconds */
    uint32_t tv_usec;     /* microseconds */
};

struct Event
{
    SigInfo* sig_info = nullptr;
    uint32_t event_id = 0;
    uint32_t event_reference = 0; // reference to other events that have gone off,
                              // such as in the case of tagged packets...
    struct sf_timeval32 ref_time = { 0, 0 };   /* reference time for the event reference */
    const char* alt_msg = nullptr;

    Event() = default;
    Event(SigInfo& si)
    { sig_info = &si; }
};

void SetEvent(
    Event&, uint32_t gid, uint32_t sid, uint32_t rev,
    uint32_t classification, uint32_t priority, uint32_t event_ref);

#endif

event_queue.h

Path = src/events/event_queue.h

#ifndef EVENT_QUEUE_H
#define EVENT_QUEUE_H

#include "actions/actions.h"
#include "main/snort_types.h"

#define SNORT_EVENTQ_PRIORITY    1
#define SNORT_EVENTQ_CONTENT_LEN 2

struct EventQueueConfig
{
    unsigned max_events;
    unsigned log_events;
    int order;
    int process_all_events;
};

struct EventNode
{
    const struct OptTreeNode* otn;
    const struct RuleTreeNode* rtn;
    snort::Actions::Type type;
};

EventQueueConfig* EventQueueConfigNew();
void EventQueueConfigFree(EventQueueConfig*);

#endif

sfeventq.h

Path = src/events/sfeventq.h

#ifndef SFEVENTQ_H
#define SFEVENTQ_H

struct SF_EVENTQ_NODE
{
    void* event;

    SF_EVENTQ_NODE* prev;
    SF_EVENTQ_NODE* next;
};

struct SF_EVENTQ
{
    /*
    **  Handles the actual ordering and memory
    **  of the event queue and it's nodes.
    */
    SF_EVENTQ_NODE* head;
    SF_EVENTQ_NODE* last;

    SF_EVENTQ_NODE* node_mem;
    char* event_mem;

    /*
    **  The reserve event allows us to allocate one extra node
    **  and compare against the last event in the queue to determine
    **  if the incoming event is a higher priority than the last
    **  event in the queue.
    */
    char* reserve_event;

    /*
    **  Queue configuration
    */
    int max_nodes;
    int log_nodes;
    int event_size;

    /*
    **  This element tracks the current number of
    **  nodes in the event queue.
    */
    int cur_nodes;
    int cur_events;
    unsigned fails;
};

SF_EVENTQ* sfeventq_new(int max_nodes, int log_nodes, int event_size);
void* sfeventq_event_alloc(SF_EVENTQ*);
unsigned sfeventq_reset(SF_EVENTQ*);  // returns fail count since last reset
int sfeventq_add(SF_EVENTQ*, void* event);
int sfeventq_action(SF_EVENTQ*, int (* action_func)(void* event, void* user), void* user);
void sfeventq_free(SF_EVENTQ*);

#endif

file_api/

This directory contains all the file processing related classes and APIs

  • file_api: provides the interfaces for file processing, used by service inspectors such as HTTP, SMTP, POP, IMAP, SMB, and FTP etc.

  • File capture: provides the ability to capture file data and save them in the mempool, then they can be stored to disk. Currently, files can be saved to the logging folder. Writing to disk is done by a separate thread that will not block packet thread. When a file is available to store, it will be put into a queue. The writer thread will read from this queue to write to disk. In the multiple packet thread case, many threads will write into this queue and one writer thread serves all of them. Thread synchronization is done by mutex and conditional variables for the queue. In the future, we will add support for multiple writer threads to improve performance when multiple disks are used.

  • File libraries: provides file type identification and file signature calculation

circular_buffer.h

Path = src/file_api/circular_buffer.h

#ifndef CIRCULAR_BUFFER_H
#define CIRCULAR_BUFFER_H

//  Circular buffer is thread safe for one writer and one reader thread
//  This implementation is inspired by one slot open approach.
//  See http://en.wikipedia.org/wiki/Circular_buffer

#include "main/snort_types.h"

#define CB_SUCCESS    0  // FIXIT-RC use bool
#define CB_FAIL      (-1)

// Opaque buffer element type.  This would be defined by the application.
typedef void* ElemType;

struct _CircularBuffer;
typedef struct _CircularBuffer CircularBuffer;

// Initialize buffer based on number of elements
CircularBuffer* cbuffer_init(uint64_t size);

void cbuffer_free(CircularBuffer* cb);

int cbuffer_is_full(CircularBuffer* cb); // FIXIT-RC use bool
int cbuffer_is_empty(CircularBuffer* cb); // FIXIT-RC use bool

// Returns number of elements in use
uint64_t cbuffer_used(CircularBuffer* cb);

// Returns total number of elements
uint64_t cbuffer_size(CircularBuffer* cb);

// Returns CB_SUCCESS or CB_FAIL
int cbuffer_write(CircularBuffer* cb, const ElemType elem);

// Read one element from the buffer and remove it from buffer
// Returns CB_SUCCESS or CB_FAIL
int cbuffer_read(CircularBuffer* cb, ElemType* elem);

#endif

file_api.h

Path = src/file_api/file_api.h

#ifndef FILE_API_H
#define FILE_API_H

// File API provides all the convenient functions that are used by inspectors.
// Currently, it provides three sets of APIs: file processing, MIME processing,
// and configurations.

#include <bitset>
#include <cstring>
#include <string>

#include "main/snort_config.h"
#include "main/snort_types.h"

#define     ENABLE_FILE_TYPE_IDENTIFICATION      0x1
#define     ENABLE_FILE_SIGNATURE_SHA256         0x2
#define     ENABLE_FILE_CAPTURE                  0x4
#define     FILE_ALL_ON                          0xFFFFFFFF
#define     FILE_ALL_OFF                         0x00000000

enum FileAction
{
    FILE_ACTION_DEFAULT = 0,
    FILE_RESUME_BLOCK,
    FILE_RESUME_LOG
};

#define UTF_16_LE_BOM "\xFF\xFE"
#define UTF_16_LE_BOM_LEN 2

enum FileVerdict
{
    FILE_VERDICT_UNKNOWN = 0,
    FILE_VERDICT_LOG,
    FILE_VERDICT_STOP,
    FILE_VERDICT_BLOCK,
    FILE_VERDICT_REJECT,
    FILE_VERDICT_PENDING,
    FILE_VERDICT_STOP_CAPTURE,
    FILE_VERDICT_MAX
};

enum FilePosition
{
    SNORT_FILE_POSITION_UNKNOWN,
    SNORT_FILE_START,
    SNORT_FILE_MIDDLE,
    SNORT_FILE_END,
    SNORT_FILE_FULL
};

enum FileCaptureState
{
    FILE_CAPTURE_SUCCESS = 0,
    FILE_CAPTURE_MIN,                 /*smaller than file capture min*/
    FILE_CAPTURE_MAX,                 /*larger than file capture max*/
    FILE_CAPTURE_MEMCAP,              /*memcap reached, no more file buffer*/
    FILE_CAPTURE_FAIL                 /*Other file capture failures*/
};

enum FileSigState
{
    FILE_SIG_PROCESSING = 0,
    FILE_SIG_DEPTH_FAIL,              /*larger than file signature depth*/
    FILE_SIG_FLUSH,
    FILE_SIG_DONE
};

enum FileProcessType
{
    SNORT_FILE_TYPE_ID,
    SNORT_FILE_SHA256,
    SNORT_FILE_CAPTURE
};

enum FileDirection
{
    FILE_DOWNLOAD,
    FILE_UPLOAD
};

enum FileCharEncoding
{
    SNORT_CHAR_ENCODING_ASCII,
    SNORT_CHAR_ENCODING_UTF_16LE
};

struct FileState
{
    FileCaptureState capture_state;
    FileSigState sig_state;
};

namespace snort
{
#define FILE_ID_MAX          1024
typedef std::bitset<FILE_ID_MAX> FileTypeBitSet;

class FileContext;
class FileInfo;
class Flow;
struct Packet;

class SO_PUBLIC FilePolicyBase
{
public:

    FilePolicyBase() = default;
    virtual ~FilePolicyBase() = default;

    // This is called when a new flow is queried for the first time
    // Check & update what file policy is enabled on this flow/file
    virtual void policy_check(Flow*, FileInfo*) { }

    // This is called after file type is known
    virtual FileVerdict type_lookup(Packet*, FileInfo*)
    { return FILE_VERDICT_UNKNOWN; }

    // This is called after file signature is complete
    virtual FileVerdict signature_lookup(Packet*, FileInfo*)
    { return FILE_VERDICT_UNKNOWN; }

    virtual void log_file_action(Flow*, FileInfo*, FileAction) { }
};

inline void initFilePosition(FilePosition* position, uint64_t processed_size)
{
    *position = SNORT_FILE_START;
    if (processed_size)
        *position = SNORT_FILE_MIDDLE;
}

inline void updateFilePosition(FilePosition* position, uint64_t processed_size)
{
    if ((*position == SNORT_FILE_END) || (*position == SNORT_FILE_FULL))
        *position = SNORT_FILE_START;
    else if (processed_size)
        *position = SNORT_FILE_MIDDLE;
}

inline void finalFilePosition(FilePosition* position)
{
    if (*position == SNORT_FILE_START)
        *position = SNORT_FILE_FULL;
    else if (*position != SNORT_FILE_FULL)
        *position = SNORT_FILE_END;
}

inline bool isFileStart(FilePosition position)
{
    return ((position == SNORT_FILE_START) || (position == SNORT_FILE_FULL));
}

inline bool isFileEnd(FilePosition position)
{
    return ((position == SNORT_FILE_END) || (position == SNORT_FILE_FULL));
}

inline FileCharEncoding get_character_encoding(const char* file_name, size_t length)
{
    FileCharEncoding encoding = SNORT_CHAR_ENCODING_ASCII;
    if (length >= UTF_16_LE_BOM_LEN)
    {
        if (memcmp(file_name, UTF_16_LE_BOM, UTF_16_LE_BOM_LEN) == 0)
            encoding = SNORT_CHAR_ENCODING_UTF_16LE;
    }

    return encoding;
}

SO_PUBLIC uint64_t get_file_processed_size(Flow* flow);
SO_PUBLIC FilePosition get_file_position(Packet* pkt);
SO_PUBLIC void get_magic_rule_ids_from_type(const std::string& type,
    const std::string& version, FileTypeBitSet& ids_set, SnortConfig*);
}
#endif

file_cache.h

Path = src/file_api/file_cache.h

#ifndef FILE_CACHE_H
#define FILE_CACHE_H

#include <mutex>

#include "sfip/sf_ip.h"
#include "utils/cpp_macros.h"

#include "file_config.h"

class ExpectedFileCache;

class FileCache
{
public:

PADDING_GUARD_BEGIN
    struct FileHashKey
    {
        snort::SfIp sip;
        int16_t sgroup;
        snort::SfIp dip;
        int16_t dgroup;
        uint64_t file_id;
        uint16_t asid;
        uint16_t padding[3];
    };
PADDING_GUARD_END

    struct FileNode
    {
        struct timeval cache_expire_time = {0, 0};
        snort::FileContext* file;
    };

    FileCache(int64_t max_files_cached);
    ~FileCache();

    void set_block_timeout(int64_t);
    void set_lookup_timeout(int64_t);
    void set_max_files(int64_t);

    snort::FileContext* get_file(snort::Flow*, uint64_t file_id, bool to_create);
    FileVerdict cached_verdict_lookup(snort::Packet*, snort::FileInfo*,
        snort::FilePolicyBase*);
    bool apply_verdict(snort::Packet*, snort::FileContext*, FileVerdict, bool resume,
        snort::FilePolicyBase*);

private:
    snort::FileContext* add(const FileHashKey&, int64_t timeout);
    snort::FileContext* find(const FileHashKey&, int64_t);
    snort::FileContext* get_file(snort::Flow*, uint64_t file_id, bool to_create, int64_t timeout);
    FileVerdict check_verdict(snort::Packet*, snort::FileInfo*, snort::FilePolicyBase*);
    int store_verdict(snort::Flow*, snort::FileInfo*, int64_t timeout);

    /* The hash table of expected files */
    ExpectedFileCache* fileHash = nullptr;
    int64_t block_timeout = DEFAULT_FILE_BLOCK_TIMEOUT;
    int64_t lookup_timeout = DEFAULT_FILE_LOOKUP_TIMEOUT;
    int64_t max_files = DEFAULT_MAX_FILES_CACHED;
    std::mutex cache_mutex;
};

#endif

file_capture.h

Path = src/file_api/file_capture.h

#ifndef FILE_CAPTURE_H
#define FILE_CAPTURE_H

// There are several steps for file capture:
// 1) To improve performance, file data are stored in file mempool first by
//    calling file_capture_process() during file data processing.
// 2) If file capture is needed, file_capture_reserve() should be called to
//    allow file data remains in mempool. Even if a session is closed, the file
//     data will stay in the mempool.
// 3) Then file data can be read through file_capture_read()
// 4) Finally, file data must be released from mempool file_capture_release()

#include <condition_variable>
#include <mutex>
#include <queue>
#include <thread>

#include "file_api.h"

class FileMemPool;

namespace snort
{
class FileInfo;

struct FileCaptureBlock
{
    uint32_t length;
    FileCaptureBlock* next;  /* next block of file data */
};

class SO_PUBLIC FileCapture
{
public:
    FileCapture(int64_t capture_min_size, int64_t capture_max_size);
    ~FileCapture();

    // this must be called during snort init
    static void init(int64_t memcap, int64_t block_size);

    // Capture file data to local buffer
    // This is the main function call to enable file capture
    FileCaptureState process_buffer(const uint8_t* file_data, int data_size,
        FilePosition pos);

    // Preserve the file in memory until it is released
    FileCaptureState reserve_file(const snort::FileInfo*);

    // Get the file that is reserved in memory, this should be called repeatedly
    // until nullptr is returned to get the full file
    // Returns:
    //   the next memory block
    //   nullptr: end of file or fail to get file
    FileCaptureBlock* get_file_data(uint8_t** buff, int* size);

    // Store files on local disk
    void store_file();

    // Store file to disk asynchronously
    void store_file_async();

    // Log file capture mempool usage
    static void print_mem_usage();

    // Exit file capture, release all file capture memory etc,
    // this must be called when snort exits
    static void exit();

    static FileCaptureState error_capture(FileCaptureState);

    static int64_t get_block_size() { return capture_block_size; }

    snort::FileInfo* get_file_info() { return file_info; }

    int64_t get_max_file_capture_size() { return capture_max_size; }
    int64_t get_file_capture_size() { return capture_size; }
    void get_file_reset() { current_block = head; }

private:

    static void init_mempool(int64_t max_file_mem, int64_t block_size);
    static void writer_thread();
    inline FileCaptureBlock* create_file_buffer();
    inline FileCaptureState save_to_file_buffer(const uint8_t* file_data, int data_size,
        int64_t max_size);
    void write_file_data(uint8_t* buf, size_t buf_len, FILE* fh);

    static FileMemPool* file_mempool;
    static int64_t capture_block_size;
    static std::mutex capture_mutex;
    static std::condition_variable capture_cv;
    static std::thread* file_storer;
    static std::queue<FileCapture*> files_waiting;
    static bool running;

    uint64_t capture_size;
    FileCaptureBlock* last;  /* last block of file data */
    FileCaptureBlock* head;  /* first block of file data */
    FileCaptureBlock* current_block = nullptr;  /* current block of file data */
    const uint8_t* current_data;  /*current file data*/
    uint32_t current_data_len;
    FileCaptureState capture_state;
    snort::FileInfo* file_info = nullptr;
    int64_t capture_min_size;
    int64_t capture_max_size;
};
}

#endif

file_config.h

Path = src/file_api/file_config.h

#ifndef FILE_CONFIG_H
#define FILE_CONFIG_H

// This provides the basic configuration for file processing
#include "main/snort_config.h"
#include "file_api/file_identifier.h"
#include "file_api/file_policy.h"

#define DEFAULT_FILE_TYPE_DEPTH 1460
#define DEFAULT_FILE_SIGNATURE_DEPTH 10485760 /*10 Mbytes*/
#define DEFAULT_FILE_SHOW_DATA_DEPTH 100
#define DEFAULT_FILE_BLOCK_TIMEOUT 86400 /*1 day*/
#define DEFAULT_FILE_LOOKUP_TIMEOUT 2    /*2 seconds*/
#define DEFAULT_FILE_CAPTURE_MEM            100         // 100 MiB
#define DEFAULT_FILE_CAPTURE_MAX_SIZE       1048576     // 1 MiB
#define DEFAULT_FILE_CAPTURE_MIN_SIZE       0           // 0
#define DEFAULT_FILE_CAPTURE_BLOCK_SIZE     32768       // 32 KiB
#define DEFAULT_MAX_FILES_CACHED            65536
#define DEFAULT_MAX_FILES_PER_FLOW          128

#define FILE_ID_NAME "file_id"
#define FILE_ID_HELP "configure file identification"

class FileConfig
{
public:
    const FileMagicRule* get_rule_from_id(uint32_t) const;
    void get_magic_rule_ids_from_type(const std::string&, const std::string&,
        snort::FileTypeBitSet&) const;
    void process_file_rule(FileMagicRule&);
    void process_file_policy_rule(FileRule&);
    bool process_file_magic(FileMagicData&);
    uint32_t find_file_type_id(const uint8_t* buf, int len, uint64_t file_offset, void** context);
    FilePolicy& get_file_policy() { return filePolicy; }
    const FilePolicy& get_file_policy() const { return filePolicy; }
    std::string file_type_name(uint32_t id) const;

    int64_t file_type_depth = DEFAULT_FILE_TYPE_DEPTH;
    int64_t file_signature_depth = DEFAULT_FILE_SIGNATURE_DEPTH;
    int64_t file_block_timeout = DEFAULT_FILE_BLOCK_TIMEOUT;
    int64_t file_lookup_timeout = DEFAULT_FILE_LOOKUP_TIMEOUT;
    bool block_timeout_lookup = false;
    int64_t capture_memcap = DEFAULT_FILE_CAPTURE_MEM;
    int64_t capture_max_size = DEFAULT_FILE_CAPTURE_MAX_SIZE;
    int64_t capture_min_size = DEFAULT_FILE_CAPTURE_MIN_SIZE;
    int64_t capture_block_size = DEFAULT_FILE_CAPTURE_BLOCK_SIZE;
    int64_t file_depth =  0;
    int64_t max_files_cached = DEFAULT_MAX_FILES_CACHED;
    uint64_t max_files_per_flow = DEFAULT_MAX_FILES_PER_FLOW;

    int64_t show_data_depth = DEFAULT_FILE_SHOW_DATA_DEPTH;
    bool trace_type = false;
    bool trace_signature = false;
    bool trace_stream = false;
    int64_t verdict_delay = 0;

private:
    FileIdentifier fileIdentifier;
    FilePolicy filePolicy;
};

std::string file_type_name(uint32_t id);
FileConfig* get_file_config(const snort::SnortConfig* sc = nullptr);
#endif

file_flows.h

Path = src/file_api/file_flows.h

#ifndef FILE_FLOWS_H
#define FILE_FLOWS_H

// This provides a wrapper to manage several file contexts

#include "flow/flow.h"
#include "main/snort_types.h"
#include "utils/event_gen.h"

#include "file_api.h"
#include "file_module.h"
#include "file_policy.h"

#include <map>

using FileEventGen = EventGen<EVENT__MAX_VALUE, EVENT__NONE, FILE_ID_GID>;

namespace snort
{
class FileContext;
class Flow;

class FileInspect : public Inspector
{
public:
    FileInspect(FileIdModule*);
    ~FileInspect() override;
    void eval(Packet*) override { }
    bool configure(SnortConfig*) override;
    void show(const SnortConfig*) const override;
    FileConfig* config;
};

class SO_PUBLIC FileFlows : public FlowData
{
public:

    FileFlows(Flow* f, FileInspect* inspect) : FlowData(file_flow_data_id, inspect), flow(f) { }
    ~FileFlows() override;
    static void init()
    { file_flow_data_id = FlowData::create_flow_data_id(); }

    void handle_retransmit(Packet*) override;

    // Factory method to get file flows
    static FileFlows* get_file_flows(Flow*, bool to_create=true);
    static FilePolicyBase* get_file_policy(Flow*);

    FileContext* get_current_file_context();

    void set_current_file_context(FileContext*);

    // Get file context based on file id, create it if does not exist
    FileContext* get_file_context(uint64_t file_id, bool to_create,
        uint64_t multi_file_processing_id=0);
    // Get a partially processed file context from the flow object
    FileContext* get_partially_processed_context(uint64_t file_id);
    // Remove a file from the flow object when processing is complete
    void remove_processed_file_context(uint64_t file_id);

    void remove_processed_file_context(uint64_t file_id, uint64_t multi_file_processing_id);

    uint64_t get_new_file_instance();

    void set_file_name(const uint8_t* fname, uint32_t name_size, uint64_t file_id=0);

    void set_sig_gen_state( bool enable )
    {
        gen_signature = enable;
    }

    void add_pending_file(uint64_t file_id);

    // This is used when there is only one file per session
    bool file_process(Packet* p, const uint8_t* file_data, int data_size, FilePosition,
        bool upload, size_t file_index = 0);

    // This is used for each file context. Support multiple files per session
    bool file_process(Packet* p, uint64_t file_id, const uint8_t* file_data,
        int data_size, uint64_t offset, FileDirection, uint64_t multi_file_processing_id=0,
        FilePosition=SNORT_FILE_POSITION_UNKNOWN);

    static unsigned file_flow_data_id;

    void set_file_policy(FilePolicyBase* fp) { file_policy = fp; }
    FilePolicyBase* get_file_policy() { return file_policy; }

    size_t size_of() override
    { return sizeof(*this); }

private:
    void init_file_context(FileDirection, FileContext*);
    FileContext* find_main_file_context(FilePosition, FileDirection, size_t id = 0);
    FileContext* main_context = nullptr;
    FileContext* current_context = nullptr;
    uint64_t current_file_id = 0;
    uint64_t pending_file_id = 0;
    bool gen_signature = false;
    Flow* flow = nullptr;
    FilePolicyBase* file_policy = nullptr;

    std::unordered_map<uint64_t, FileContext*> partially_processed_contexts;
    bool current_context_delete_pending = false;
    FileEventGen events;
};
}
#endif

file_identifier.h

Path = src/file_api/file_identifier.h

#ifndef FILE_IDENTIFIER_H
#define FILE_IDENTIFIER_H

// File type identification is based on file magic. To improve the detection
// performance, a trie is created to scan file data once. Currently, only the
// most specific file type is returned.

#include <list>
#include <vector>

#include "file_lib.h"

namespace snort
{
class GHash;
}

#define MAX_BRANCH (UINT8_MAX + 1)

enum IdNodeState
{
    ID_NODE_NEW,
    ID_NODE_USED,
    ID_NODE_SHARED
};

class FileMagicData
{
public:
    void clear();
    std::string content_str;   /* magic content to match*/
    std::string content;       /* magic content raw values*/
    uint32_t offset;           /* pattern search start offset */
    bool operator <(const FileMagicData& magic) const
    {
        return (offset < magic.offset);
    }
};

typedef std::vector<FileMagicData> FileMagics;

class FileMagicRule
{
public:
    void clear();
    uint32_t rev = 0;
    uint32_t id = 0;
    std::string message;
    std::string type;
    std::string category;
    std::string version;
    std::vector<std::string> groups;
    FileMagics file_magics;
};

struct IdentifierNode
{
    uint32_t type_id;       /* magic content to match*/
    IdNodeState state;
    uint32_t offset;            /* offset from file start */
    struct IdentifierNode* next[MAX_BRANCH]; /* pointer to an array of 256 identifiers pointers*/
};

typedef std::list<void* >  IDMemoryBlocks;

class FileIdentifier
{
public:
    ~FileIdentifier();
    uint32_t memory_usage() const { return memory_used; }
    void insert_file_rule(FileMagicRule& rule);
    uint32_t find_file_type_id(const uint8_t* buf, int len, uint64_t offset, void** context);
    const FileMagicRule* get_rule_from_id(uint32_t) const;
    void get_magic_rule_ids_from_type(const std::string&, const std::string&,
        snort::FileTypeBitSet&) const;

private:
    void init_merge_hash();
    void* calloc_mem(size_t size);
    void set_node_state_shared(IdentifierNode* start);
    IdentifierNode* clone_node(IdentifierNode* start);
    bool update_next(IdentifierNode* start, IdentifierNode** next_ptr, IdentifierNode* append);
    IdentifierNode* create_trie_from_magic(FileMagicRule& rule, uint32_t type_id);
    void update_trie(IdentifierNode* start, IdentifierNode* append);

    /*properties*/
    IdentifierNode* identifier_root = nullptr; /*Root of magic tries*/
    uint32_t memory_used = 0; /*Track memory usage*/
    snort::GHash* identifier_merge_hash = nullptr;
    FileMagicRule file_magic_rules[FILE_ID_MAX + 1];
    IDMemoryBlocks id_memory_blocks;
};

#endif

file_lib.h

Path = src/file_api/file_lib.h

#ifndef FILE_LIB_H
#define FILE_LIB_H

// This will be basis of file class

#include <ostream>
#include <string>

#include "file_api/file_api.h"
#include "utils/util.h"

#define SNORT_FILE_TYPE_UNKNOWN          UINT16_MAX
#define SNORT_FILE_TYPE_CONTINUE         0

class FileConfig;
class FileSegments;

namespace snort
{
class FileCapture;
class FileInspect;
class Flow;

class SO_PUBLIC FileInfo
{
public:
    virtual ~FileInfo();
    FileInfo() = default;
    FileInfo(const FileInfo& other);
    FileInfo& operator=(const FileInfo& other);
    uint32_t get_file_type() const;
    void set_file_name(const char* file_name, uint32_t name_size);
    std::string& get_file_name();
    // Whether file name has been set (could be empty file name)
    bool is_file_name_set() const { return file_name_set; }

    void set_file_size(uint64_t size);
    uint64_t get_file_size() const;
    void set_file_direction(FileDirection dir);
    FileDirection get_file_direction() const;
    uint8_t* get_file_sig_sha256() const;
    std::string sha_to_string(const uint8_t* sha256);
    void set_file_id(uint64_t index);
    uint64_t get_file_id() const;

    // Configuration functions
    void config_file_type(bool enabled);
    bool is_file_type_enabled();
    void config_file_signature(bool enabled);
    bool is_file_signature_enabled();
    void config_file_capture(bool enabled);
    bool is_file_capture_enabled();

    // Preserve the file in memory until it is released
    // The file reserved will be returned and it will be detached from file context/session
    FileCaptureState reserve_file(FileCapture*& dest);
    int64_t get_max_file_capture_size();

    FileState get_file_state() { return file_state; }

    FileVerdict verdict = FILE_VERDICT_UNKNOWN;
    bool processing_complete = false;
    struct timeval pending_expire_time = {0, 0};

protected:
    std::string file_name;
    bool file_name_set = false;
    uint64_t file_size = 0;
    FileDirection direction = FILE_DOWNLOAD;
    uint32_t file_type_id = SNORT_FILE_TYPE_CONTINUE;
    uint8_t* sha256 = nullptr;
    uint64_t file_id = 0;
    FileCapture* file_capture = nullptr;
    bool file_type_enabled = false;
    bool file_signature_enabled = false;
    bool file_capture_enabled = false;
    FileState file_state = { FILE_CAPTURE_SUCCESS, FILE_SIG_PROCESSING };

private:
    void copy(const FileInfo& other);
};

class SO_PUBLIC FileContext : public FileInfo
{
public:
    FileContext();
    ~FileContext() override;

    void check_policy(Flow*, FileDirection, FilePolicyBase*);

    // main processing functions

    // Return:
    //    true: continue processing/log/block this file
    //    false: ignore this file
    bool process(Packet*, const uint8_t* file_data, int data_size, FilePosition, FilePolicyBase*);
    bool process(Packet*, const uint8_t* file_data, int data_size, uint64_t offset, FilePolicyBase*,
        FilePosition position=SNORT_FILE_POSITION_UNKNOWN);
    void process_file_type(const uint8_t* file_data, int data_size, FilePosition);
    void process_file_signature_sha256(const uint8_t* file_data, int data_size, FilePosition);
    void update_file_size(int data_size, FilePosition position);
    void stop_file_capture();
    FileCaptureState process_file_capture(const uint8_t* file_data, int data_size, FilePosition);
    void log_file_event(Flow*, FilePolicyBase*);
    FileVerdict file_signature_lookup(Packet*);

    void set_signature_state(bool gen_sig);

    //File properties
    uint64_t get_processed_bytes();

    void print_file_sha256(std::ostream&);
    void print_file_name(std::ostream&);
    static void print_file_data(FILE* fp, const uint8_t* data, int len, int max_depth);
    void print(std::ostream&);
    char* get_UTF8_fname(size_t* converted_len);
    void set_not_cacheable() { cacheable = false; }
    bool is_cacheable() { return cacheable; }

private:
    uint64_t processed_bytes = 0;
    void* file_type_context;
    void* file_signature_context;
    FileSegments* file_segments;
    FileInspect* inspector;
    FileConfig*  config;
    bool cacheable = true;

    inline void finalize_file_type();
    inline void finish_signature_lookup(Packet*, bool, FilePolicyBase*);
};
}
#endif

file_mempool.h

Path = src/file_api/file_mempool.h

#ifndef FILE_MEMPOOL_H
#define FILE_MEMPOOL_H

//  This mempool implementation has very efficient alloc/free operations.
//  In addition, it provides thread-safe alloc/free for one allocation/free
//  thread and one release thread.
//  One more bonus: Double free detection is also added into this library
//  This is a thread safe version of memory pool for one writer and one reader thread

#include <mutex>

#include "circular_buffer.h"

#define FILE_MEM_SUCCESS    0  // FIXIT-RC use bool
#define FILE_MEM_FAIL      (-1)

class FileMemPool
{
public:

    FileMemPool(uint64_t num_objects, size_t obj_size);
    ~FileMemPool();

    // Allocate a new object from the FileMemPool
    // Note: Memory block will not be zeroed for performance
    // Returns: a pointer to the FileMemPool object on success, nullptr on failure
    void* m_alloc();

    // This must be called by the same thread calling file_mempool_alloc()
    // Return: FILE_MEM_SUCCESS or FILE_MEM_FAIL
    int m_free(void* obj);

    // This can be called by a different thread calling file_mempool_alloc()
    // Return: FILE_MEM_SUCCESS or FILE_MEM_FAIL
    int m_release(void* obj);

    //Returns number of elements allocated
    uint64_t allocated();

    // Returns number of elements freed in current buffer
    uint64_t freed();

    // Returns number of elements released in current buffer
    uint64_t released();

    // Returns total number of elements in current buffer
    uint64_t total_objects() { return total; }

private:

    void free_pools();
    int remove(CircularBuffer* cb, void* obj);

    void** datapool = nullptr; /* memory buffer */
    uint64_t total = 0;
    CircularBuffer* free_list = nullptr;
    CircularBuffer* released_list = nullptr;
    size_t obj_size = 0;
    std::mutex pool_mutex;
};

#endif

file_module.h

Path = src/file_api/file_module.h

#ifndef FILE_MODULE_H
#define FILE_MODULE_H

#include "framework/module.h"

#include "file_config.h"
#include "file_identifier.h"
#include "file_policy.h"

//-------------------------------------------------------------------------
// file_id module
//-------------------------------------------------------------------------

static const uint32_t FILE_ID_GID = 150;

class FileIdModule : public snort::Module
{
public:
    FileIdModule();
    ~FileIdModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;

    void sum_stats(bool) override;

    void load_config(FileConfig*& dst);

    Usage get_usage() const override
    { return GLOBAL; }

    void show_dynamic_stats() override;

    unsigned get_gid() const override
    { return FILE_ID_GID; }

    const snort::RuleMap* get_rules() const override;

private:
    FileMagicRule rule;
    FileMagicData magic;
    FileRule file_rule;
    FileConfig *fc = nullptr;
    bool need_active = false;
};

enum FileSid
{
    EVENT__NONE = -1,
    EVENT_FILE_DROPPED_OVER_LIMIT = 1,
    EVENT__MAX_VALUE
};

#endif

file_policy.h

Path = src/file_api/file_policy.h

#ifndef FILE_POLICY_H
#define FILE_POLICY_H

#include <map>
#include <vector>

#include "file_api.h"

namespace snort
{
class FileInfo;
}

struct FileVerdictWhen
{
    uint32_t type_id;
    std::string sha256;
};

struct FileVerdictUse
{
    FileVerdict verdict = FILE_VERDICT_UNKNOWN;
    bool type_enabled = false;
    bool signature_enabled = false;
    bool capture_enabled = false;
};

class FileRule
{
public:
    FileVerdictWhen when;
    FileVerdictUse use;

    FileRule();
    void clear();
};

class FilePolicy: public snort::FilePolicyBase
{
public:

    FilePolicy() = default;
    ~FilePolicy() override = default;

    void policy_check(snort::Flow*, snort::FileInfo*) override;

    // This is called after file type is known
    FileVerdict type_lookup(snort::Packet*, snort::FileInfo*) override;

    // This is called after file signature is complete
    FileVerdict signature_lookup(snort::Packet*, snort::FileInfo*) override;

    void insert_file_rule(FileRule&);
    void set_file_type(bool enabled);
    void set_file_signature(bool enabled);
    void set_file_capture(bool enabled);
    bool get_file_type() const;
    bool get_file_signature() const;
    bool get_file_capture() const;
    void load();
    void set_verdict_delay(int64_t delay) { verdict_delay = delay; }

private:
    FileRule& match_file_rule(snort::Flow*, snort::FileInfo*);
    FileVerdict match_file_signature(snort::Flow*, snort::FileInfo*);
    std::vector<FileRule> file_rules;
    std::map<std::string, FileVerdict> file_shas;
    bool type_enabled = false;
    bool signature_enabled = false;
    bool capture_enabled = false;
    int64_t verdict_delay = 0;

};

#endif

file_segment.h

Path = src/file_api/file_segment.h

#ifndef FILE_SEGMENT_H
#define FILE_SEGMENT_H

// Segmented file data reassemble and processing

#include <string>

#include "file_api.h"

namespace snort
{
class Flow;
}
class FileConfig;

class FileSegment
{
public:
    FileSegment() = default;
    ~FileSegment();

    // Use single list for simplicity
    FileSegment* next = nullptr;
    uint32_t offset = 0;
    std::string* data = nullptr;
};

class FileSegments
{
public:
    FileSegments(snort::FileContext*);
    ~FileSegments();

    void clear();

    // Process file segments with current_offset specified. If file segment is out of order,
    // it will be put into the file segments queue.
    int process(snort::Packet*, const uint8_t* file_data, uint64_t data_size, uint64_t offset,
        snort::FilePolicyBase*, FilePosition position=SNORT_FILE_POSITION_UNKNOWN);

private:
    FileSegment* head = nullptr;
    uint64_t current_offset;
    snort::FileContext* context = nullptr;

    void add(const uint8_t* file_data, uint64_t data_size, uint64_t offset);
    FilePosition get_file_position(uint64_t data_size, uint64_t file_size);
    int process_one(snort::Packet*, const uint8_t* file_data, int data_size, snort::FilePolicyBase*,
        FilePosition position=SNORT_FILE_POSITION_UNKNOWN);
    int process_all(snort::Packet*, snort::FilePolicyBase*);
};

#endif

file_service.h

Path = src/file_api/file_service.h

#ifndef FILE_SERVICE_H
#define FILE_SERVICE_H

// This provides a wrapper to start/stop file service

#include "file_api/file_policy.h"
#include "main/snort_config.h"
#include "main/snort_types.h"
#include "mime/file_mime_config.h"

class FileEnforcer;
class FileCache;

namespace snort
{
class SO_PUBLIC FileService
{
public:
    // This must be called when snort restarts
    static void init();

    // Called after permission is dropped
    static void post_init(const SnortConfig*);

    // Called during reload
    static void verify_reload(const SnortConfig*);

    // This must be called when snort exits
    static void close();

    static void thread_init();
    static void thread_term();

    static void enable_file_type();
    static void enable_file_signature();
    static void enable_file_capture();
    static bool is_file_type_id_enabled() { return file_type_id_enabled; }
    static bool is_file_signature_enabled() { return file_signature_enabled; }
    static bool is_file_capture_enabled() { return file_capture_enabled; }
    static bool is_file_service_enabled();
    static int64_t get_max_file_depth();
    static void reset_depths();

    static FileCache* get_file_cache() { return file_cache; }
    static DecodeConfig decode_conf;

private:
    static bool file_type_id_enabled;
    static bool file_signature_enabled;
    static bool file_capture_enabled;
    static bool file_processing_initiated;
    static FileCache* file_cache;
};
} // namespace snort
#endif

file_stats.h

Path = src/file_api/file_stats.h

#ifndef FILE_STATS_H
#define FILE_STATS_H

#include "framework/counts.h"
#include "main/thread.h"

#include "file_api.h"
#include "file_config.h"

#define MAX_PROTOCOL_ORDINAL 8192  // FIXIT-L use std::vector and get_protocol_count()

struct FileCounts
{
    PegCount files_total;
    PegCount file_data_total;
    PegCount cache_add_fails;
    PegCount files_over_flow_limit_not_processed;
    PegCount max_concurrent_files_per_flow;
    PegCount files_buffered_total;
    PegCount files_released_total;
    PegCount files_freed_total;
    PegCount files_captured_total;
    PegCount file_memcap_failures_total;
    PegCount file_memcap_failures_reserve;  // This happens during reserve
    PegCount file_reserve_failures;         // This happens during reserve
    PegCount file_size_min;                 // This happens during reserve
    PegCount file_size_max;                 // This happens during reserve
    PegCount file_within_packet;
    PegCount file_buffers_used_max;         // maximum buffers used simultaneously
    PegCount file_buffers_allocated_total;
    PegCount file_buffers_freed_total;
    PegCount file_buffers_released_total;
    PegCount file_buffers_free_errors;
    PegCount file_buffers_release_errors;
};

struct FileStats
{
    PegCount files_processed[FILE_ID_MAX + 1][2];
    PegCount signatures_processed[FILE_ID_MAX + 1][2];
    PegCount verdicts_type[FILE_VERDICT_MAX];
    PegCount verdicts_signature[FILE_VERDICT_MAX];
    PegCount files_by_proto[MAX_PROTOCOL_ORDINAL + 1];
    PegCount signatures_by_proto[MAX_PROTOCOL_ORDINAL + 1];
    PegCount data_processed[FILE_ID_MAX + 1][2];
};

extern THREAD_LOCAL FileCounts file_counts;
extern THREAD_LOCAL FileStats* file_stats;

void file_stats_init();
void file_stats_term();

void file_stats_sum();
void file_stats_print();

#endif

filters/

A collection of several different event and detection filtering function. The types of filters implemented here include:

Detection Filter - One of the last steps of the rule evaluation process. A detection filter can prevent a rule from firing based on a simple threshold. For example, only generate an alert if the filter has been evaluated N times in M time period.

Rate Filter - Based on configuration options, generically track multiple occurrences of the same event/address tuples. The configuration can specify a limit where-by if the tracked limit is exceeded, the action of the event is changed. For instance, the first N occurrences of Event X in time period Y can Alert, but if this rate is exceeded subsequent occurrence will Drop. This function can be used to protect against DOS type of attacks.

Event Filter - After the rules engine generates whatever actions it needs to, the Event Filter is then invoked to filter the logging of these events. Once again, tracking by event/address tuples, block the logging of events if the configured counts per time is exceeded. This will tend to reduce the logging system load for rules that fire too often.

All of the filters in this area are a collection of similar services brought together to share the same event tracking logic. sfthreshold.cc implements a generic threshold tracking mechanism using a hash table. This hash structure permits the various filter/threshold components to build event tracking facilities.

Detection filter support the detection_filter rule option. Rate and event filters have builtin modules defined in main/modules.cc. Those module definitions should be refactored into the appropriate filter directory.

detection_filter.h

Path = src/filters/detection_filter.h

#ifndef DETECTION_FILTER_H
#define DETECTION_FILTER_H

// the use of threshold standalone, in config, and in rules is deprecated.
// - standalone and config are replaced with event_filter.
// - within a rule is replaced with detection_filter.
//
// both detection_filter and event_filter use the same basic mechanism.
// however, detection_filter is evaluated as the final step in rule matching.
// and thereby controls event generation.  event_filter is evaluated after
// the event is queued, and thereby controls which events get logged.

namespace snort
{
struct SfIp;
}

struct DetectionFilterConfig
{
    unsigned memcap;
    int count;
    int enabled;
};

DetectionFilterConfig* DetectionFilterConfigNew();
void DetectionFilterConfigFree(DetectionFilterConfig*);

void detection_filter_init(DetectionFilterConfig*);
void detection_filter_term();

int detection_filter_test(void*, const snort::SfIp* sip, const snort::SfIp* dip, long curtime);
struct THD_NODE* detection_filter_create(DetectionFilterConfig*, struct THDX_STRUCT*);

#endif

rate_filter.h

Path = src/filters/rate_filter.h

#ifndef RATE_FILTER_H
#define RATE_FILTER_H

// rate filter interface for Snort
namespace snort
{
struct Packet;
struct SnortConfig;
}
struct RateFilterConfig;
struct tSFRFConfigNode;
struct OptTreeNode;

RateFilterConfig* RateFilter_ConfigNew();
void RateFilter_ConfigFree(RateFilterConfig*);
void RateFilter_Cleanup();

int RateFilter_Create(snort::SnortConfig* sc, RateFilterConfig*, tSFRFConfigNode*);
int RateFilter_Test(const OptTreeNode*, snort::Packet*);

#endif

sfrf.h

Path = src/filters/sfrf.h

#ifndef SFRF_H
#define SFRF_H

// Implements rate_filter feature for snort

#include <ctime>

#include "actions/actions.h"
#include "framework/counts.h"
#include "main/policy.h"

namespace snort
{
class GHash;
struct SfIp;
struct SnortConfig;
}

// define to use over rate threshold
#define SFRF_OVER_RATE

// used for the dimensions of the gid lookup array.
#define SFRF_MAX_GENID 8129

// rate_filter tracking by src, by dst, or by rule
typedef enum
{
    SFRF_TRACK_BY_SRC = 1,
    SFRF_TRACK_BY_DST,
    SFRF_TRACK_BY_RULE,
    SFRF_TRACK_BY_MAX
} SFRF_TRACK;

/* Type of operation for threshold tracking nodes.
 */
typedef enum
{
    SFRF_COUNT_NOP,
    SFRF_COUNT_RESET,
    SFRF_COUNT_INCREMENT,
    SFRF_COUNT_DECREMENT,
    SFRF_COUNT_MAX
} SFRF_COUNT_OPERATION;

typedef enum
{
    FS_NEW = 0, FS_OFF, FS_ON, FS_MAX
} FilterState;

/* A threshold configuration object, created for each configured rate_filter.
 * These are created at initialization, and remain static.
 */
struct tSFRFConfigNode
{
    // Internally generated unique threshold identity
    int tid;

    // Generator id from configured threshold
    unsigned gid;

    // Signature id from configured threshold
    unsigned sid;

    // Signature id from configured threshold
    PolicyId policyId;

    // Threshold tracking by src, dst or rule
    SFRF_TRACK tracking;

    // Number of rule matching before rate limit is reached.
    unsigned count;

    // Duration in seconds for determining rate of rule matching
    unsigned seconds;

    // Action that replaces original rule action on reaching threshold
    snort::Actions::Type newAction;

    // Threshold action duration in seconds before reverting to original rule action
    unsigned timeout;

    // ip set to restrict rate_filter
    sfip_var_t* applyTo;
};

/* tSFRFSidNode acts as a container of gid+sid based threshold objects,
 * this allows multiple threshold objects to be applied to a single
 * gid+sid pair. This is static data elements, built at initialization.
 */
struct tSFRFSidNode
{
    // List of threshold configuration nodes of type tSFRFConfigNode
    PolicyId policyId;

    // Generator id from configured threshold
    unsigned gid;

    // Signature id from configured threshold
    unsigned sid;

    // List of threshold configuration nodes of type tSFRFConfigNode
    struct sf_list* configNodeList;
};

struct tSFRFGenHashKey
{
    ///policy identifier
    PolicyId policyId;

    // Signature id from configured threshold
    unsigned sid;
};

/* Single global context containing rate_filter configuration nodes.
 */
struct RateFilterConfig
{
    /* Array of hash, indexed by gid. Each array element is a hash, which
     * is keyed on sid/policyId and data is a tSFRFSidNode node.
     */
    snort::GHash* genHash [SFRF_MAX_GENID];

    unsigned memcap;
    unsigned noRevertCount;
    int count;
    int internal_event_mask;
};

struct RateFilterStats
{
    PegCount xhash_nomem_peg = 0;
};

/*
 * Prototypes
 */
void SFRF_Delete();
void SFRF_Flush();
int SFRF_ConfigAdd(snort::SnortConfig*, RateFilterConfig*, tSFRFConfigNode*);

int SFRF_TestThreshold(
    RateFilterConfig *config,
    unsigned gid,
    unsigned sid,
    const snort::SfIp *sip,
    const snort::SfIp *dip,
    time_t curTime,
    SFRF_COUNT_OPERATION);

void SFRF_ShowObjects(RateFilterConfig*);

inline void enable_internal_event(RateFilterConfig* config, uint32_t sid)
{
    if (config == nullptr)
        return;

    config->internal_event_mask |= (1 << sid);
}

inline bool is_internal_event_enabled(RateFilterConfig* config, uint32_t sid)
{
    if (config == nullptr)
        return false;

    return (config->internal_event_mask & (1 << sid));
}

int SFRF_Alloc(unsigned int memcap);

#endif

sfthd.h

Path = src/filters/sfthd.h

#ifndef SFTHD_H
#define SFTHD_H

#include "framework/counts.h"
#include "main/policy.h"
#include "sfip/sf_ip.h"
#include "utils/cpp_macros.h"

namespace snort
{
class GHash;
class XHash;
struct SnortConfig;
}

typedef struct sf_list SF_LIST;

/*!
    Max GEN_ID value - Set this to the Max Used by Snort, this is used for the
    dimensions of the gen_id lookup array.

    Rows in each hash table, by gen_id.
*/
#define THD_MAX_GENID     8129
#define THD_GEN_ID_1_ROWS 4096
#define THD_GEN_ID_ROWS   512

#define THD_NO_THRESHOLD (-1)

#define THD_TOO_MANY_THDOBJ (-15)

/*!
   Type of Thresholding
*/
enum
{
    THD_TYPE_LIMIT,
    THD_TYPE_THRESHOLD,
    THD_TYPE_BOTH,
    THD_TYPE_SUPPRESS,
    THD_TYPE_DETECT
};

/*
   Very high priority for suppression objects
   users priorities are limited to this minus one
*/
#define THD_PRIORITY_SUPPRESS 1000000

/*!
   Tracking by src, or by dst
*/
enum
{
    THD_TRK_NONE,  // suppress only
    THD_TRK_SRC,
    THD_TRK_DST
};

/*!
    THD_IP_NODE

    Dynamic hashed node data - added and deleted during runtime
    These are added during run-time, and recycled if we max out memory usage.
*/
struct THD_IP_NODE
{
    unsigned count;
    unsigned prev;
    time_t tstart;
    time_t tlast;
};

/*!
    THD_IP_NODE_KEY

    HASH Key to lookup and store Ip nodes. The structure now tracks thresholds for different
    policies. This destroys locality of reference and may cause poor performance.
*/
PADDING_GUARD_BEGIN
struct THD_IP_NODE_KEY
{
    int thd_id;
    PolicyId policyId;
    snort::SfIp ip;
    uint16_t padding;
};

struct THD_IP_GNODE_KEY
{
    unsigned gen_id;
    unsigned sig_id;
    PolicyId policyId;
    snort::SfIp ip;
    uint16_t padding;
};
PADDING_GUARD_END

/*!
    A Thresholding Object
    These are created at program startup, and remain static.
    The THD_IP_NODE elements are dynamic.
*/
struct THD_NODE
{
    int thd_id;        /* Id of this node */
    unsigned gen_id;   /* Keep these around if needed */
    unsigned sig_id;
    int tracking;      /* by_src, by_dst */
    int type;
    int priority;
    int count;
    unsigned seconds;
    sfip_var_t* ip_address;
};

/*!
    The THD_ITEM acts as a container of gen_id+sig_id based threshold objects,
    this allows multiple threshold objects to be applied to a single
    gen_id+sig_id pair. The sflist is created using the priority field,
    so highest priority objects are first in the list. When processing the
    highest priority object will trigger first.

    These are static data elements, built at program startup.
*/
struct THD_ITEM
{
    PolicyId policyId;
    unsigned gen_id; /* just so we know what gen_id we are */
    unsigned sig_id;
    /*
     * List of THD_NODE's - walk this list and hash the
     * 'THD_NODE->sfthd_id + src_ip or dst_ip' to get the correct THD_IP_NODE.
     */
    SF_LIST* sfthd_node_list;
};

// Temporary structure useful when parsing the Snort rules
struct THDX_STRUCT
{
    unsigned gen_id;
    unsigned sig_id;
    unsigned seconds;

    int type;
    int count;
    int tracking;
    int priority;

    sfip_var_t* ip_address;
};

struct tThdItemKey
{
    PolicyId policyId;
    unsigned sig_id;
};

/*!
    THD_STRUCT

    The main thresholding data structure.

    Local and global threshold thd_id's are all unique, so we use just one
    ip_nodes lookup table
 */
struct THD_STRUCT
{
    snort::XHash* ip_nodes;   /* Global hash of active IP's key=THD_IP_NODE_KEY, data=THD_IP_NODE */
    snort::XHash* ip_gnodes;  /* Global hash of active IP's key=THD_IP_GNODE_KEY, data=THD_IP_GNODE */
};

struct ThresholdObjects
{
    int count;  /* Total number of thresholding/suppression objects */
    snort::GHash* sfthd_array[THD_MAX_GENID];    /* Local Hash of THD_ITEM nodes,  lookup by key=sig_id
                                              */

    /* Double array of THD_NODE pointers. First index is policyId and therefore variable length.
     * Second index is genId and of fixed length, A simpler definition could be
     * THD_NODE * sfthd_garray[0][THD_MAX_GENID] and we could intentionally overflow first index,
     * but we may have to deal with compiler warnings if the array is indexed by value known at
     * compile time.
     */
    //THD_NODE * (*sfthd_garray)[THD_MAX_GENID];
    THD_NODE*** sfthd_garray;
    PolicyId numPoliciesAllocated;
};

struct EventFilterStats
{
    PegCount xhash_nomem_peg_local = 0;
    PegCount xhash_nomem_peg_global = 0;
};

/*
 * Prototypes
 */
// lbytes = local threshold memcap
// gbytes = global threshold memcap (0 to disable global)
THD_STRUCT* sfthd_new(unsigned lbytes, unsigned gbytes);
snort::XHash* sfthd_local_new(unsigned bytes);
snort::XHash* sfthd_global_new(unsigned bytes);
void sfthd_free(THD_STRUCT*);
ThresholdObjects* sfthd_objs_new();
void sfthd_objs_free(ThresholdObjects*);

int sfthd_test_rule(snort::XHash* rule_hash, THD_NODE* sfthd_node,
    const snort::SfIp* sip, const snort::SfIp* dip, long curtime, PolicyId policy_id);

THD_NODE* sfthd_create_rule_threshold(
    int id,
    int tracking,
    int type,
    int count,
    unsigned int seconds
    );
void sfthd_node_free(THD_NODE*);

int sfthd_create_threshold(snort::SnortConfig*, ThresholdObjects*, unsigned gen_id,
    unsigned sig_id, int tracking, int type, int priority, int count,
    int seconds, sfip_var_t* ip_address, PolicyId policy_id);

//  1: don't log due to event_filter
//  0: log
// -1: don't log due to suppress
int sfthd_test_threshold(ThresholdObjects*, THD_STRUCT*, unsigned gen_id, unsigned sig_id,
    const snort::SfIp* sip, const snort::SfIp* dip, long curtime, PolicyId policy_id);

snort::XHash* sfthd_new_hash(unsigned, size_t, size_t);

int sfthd_test_local(snort::XHash* local_hash, THD_NODE* sfthd_node, const snort::SfIp* sip,
    const snort::SfIp* dip, time_t curtime, PolicyId policy_id);

#ifdef THD_DEBUG
int sfthd_show_objects(THD_STRUCT* thd);
#endif

#endif

sfthreshold.h

Path = src/filters/sfthreshold.h

#ifndef SFTHRESHOLD_H
#define SFTHRESHOLD_H

#include "main/policy.h"

namespace snort
{
struct SfIp;
struct SnortConfig;
}
struct THDX_STRUCT;
struct ThresholdObjects;

struct ThresholdConfig
{
    ThresholdObjects* thd_objs;
    unsigned memcap;
    int enabled;
};

ThresholdConfig* ThresholdConfigNew();
void ThresholdConfigFree(ThresholdConfig*);
void sfthreshold_reset();
int sfthreshold_create(snort::SnortConfig*, ThresholdConfig*, THDX_STRUCT*, PolicyId);
int sfthreshold_test(
    unsigned int, unsigned int, const snort::SfIp*, const snort::SfIp*, long curtime,
    PolicyId);
void sfthreshold_free();

int sfthreshold_alloc(unsigned int l_memcap, unsigned int g_memcap);

#endif

flow/

Flows are preallocated at startup and stored in protocol specific caches. FlowKey is used for quick look up in the cache hash table.

Each flow may have associated inspectors:

  • clouseau is the Wizard bound to the flow to help determine the appropriate service inspector

  • gadget is the service inspector

  • data is a passive service inspector such as a client config.

FlowData is used by various inspectors to store specific data on the flow for later use. Any inspector may store data on the flow, not just clouseau gadget.

FlowData reference counts the associated inspector so that the inspector can be freed (via garbage collection) after a reload.

There are many flags that may be set on a flow to indicate session tracking state, disposition, etc.

High Availability

HighAvailability (ha.cc, ha.h) serves to synchronize session state between high availability partners. The primary purpose is to exchange session state with the goal of keeping the session caches in sync. Other clients may also register with the HA service to exchange additional session data by implementing one or more FlowHAClient classes.

HA uses Side Channel Connectors and/or DAQ module IOCTLs to transmit and receive HA state messages. A full duplex Side Channel is required for Side Channel communications. A DAQ module that supports the IOCTLs for setting and getting HA state is required for a functional DAQ-backed setup. Modules that do not support these IOCTLs will silently fail and present no HA state on future packets in the flow. HA state will be queried from the DAQ module prior to new flow creation when the appropriate DAQ packet message flag has been set on the initiating packet.

The HA subsystem exchanges two high level message types: - DELETE: Indicate to the partner that a session has neen removed. No additional HA client status will be exchanged. (Not used for DAQ-backed storage.) - UPDATE: Indicate all other state changes. The message always includes the session state and optionally may include state from other HA clients. By default, the update messages are incremental. In the case of DAQ-backed storage, the update messages are always fully formed.

The HA subsystem implements these classes: - HighAvailabilityManager - A collection of static elements providing the top-most interface to HA capabilities. - HAMessage - A wrapper around the actual message and includes a cursor and convenience functions for producer/consumer activity. Passed around among all message handing classes/methods. - HighAvailability - If HA is enabled, instantiated in each packet thread and provides all primary HA functionality for the thread. Referenced via a THREAD_LOCAL object pointer. - FlowHAState - One per flow and referenced via a pointer in the Flow object. Contains all dynamic state information. A set of get’ers and set’ers are implemented to help manage the flags, client pending flags, and timing information. - FlowHAClient - All HA coordination is managed via a set of FlowHAClient’s. Every function that desires to use HA capabilities defines one FlowHAClient object within the packet processing thread. Each client is provided a handle bit-mask as part of the pending logic. The client may set the pending bit to indicate the desire to add content to the HA message. A maximum of 17 clients may existing (numbers 1-16, corresponding to the 16 bits of the pending flags. Client 0 is the session, is always present, and is handled as a special case. Client 0 is the fundamental session HA state sync functionality. Other clients are optional.

deferred_trust.h

Path = src/flow/deferred_trust.h

#ifndef DEFERRED_TRUST_H
#define DEFERRED_TRUST_H

#include <cstdint>
#include <forward_list>

#include "main/snort_types.h"

namespace snort
{

class Active;
class Flow;
struct Packet;

class DeferredTrust
{
    // This class is used to delay session trust. This is used in cases where
    // a module needs to continue inspecting a session to enforce policy.
    // A module calls set_deferred_trust with the on parameter true to begin
    // deferring. The sets the state to TRUST_DEFER_ON. If trust session is called
    // while deferring, the state is changed to TRUST_DEFER_DEFERRING to the trust action.
    // A module calls set_deferred_trust with the on parameter false to stop deferring.
    // When all modules have stopped deferring, the state is checked. If the state is
    // TRUST_DEFER_DEFERRING, the state is changed to TRUST_DEFER_DO_TRUST. Otherwise, the state
    // is set to TRUST_DEFER_OFF.
    // The TRUST_DEFER_DO_TRUST state is checked at the end of packet processing. If the state
    // is TRUST_DEFER_DO_TRUST and the action is ACT_ALLOW, the session is trusted.
    // If a drop, block or reset action occurs while deferring, deferring is stopped and the
    // block or blocklist version is enforced.
    // The module_id, a unique module identifier created by calling
    // FlowData::create_flow_data_id(), is used to track the modules that are currently deferring.
    // This allows the module to use trusted deferring without needing to track the deferring
    // state of the module.
    enum DeferredTrustState : uint8_t
    {
        TRUST_DEFER_OFF = 0,
        TRUST_DEFER_ON,
        TRUST_DEFER_DEFERRING,
        TRUST_DEFER_DO_TRUST,
    };

public:
    DeferredTrust() = default;
    ~DeferredTrust() = default;
    SO_PUBLIC void set_deferred_trust(unsigned module_id, bool on);
    bool is_active()
    { return TRUST_DEFER_ON == deferred_trust || TRUST_DEFER_DEFERRING == deferred_trust; }
    bool try_trust()
    {
        if (TRUST_DEFER_ON == deferred_trust)
            deferred_trust = TRUST_DEFER_DEFERRING;
        return TRUST_DEFER_DEFERRING != deferred_trust;
    }
    bool is_deferred()
    { return TRUST_DEFER_DEFERRING == deferred_trust; }
    void clear()
    {
        deferred_trust = TRUST_DEFER_OFF;
        deferred_trust_modules.clear();
    }
    void finalize(Active&);

protected:
    std::forward_list<unsigned> deferred_trust_modules;
    DeferredTrustState deferred_trust = TRUST_DEFER_OFF;
};

}

#endif

expect_cache.h

Path = src/flow/expect_cache.h

#ifndef EXPECT_CACHE_H
#define EXPECT_CACHE_H

// ExpectCache is used to track anticipated flows (like ftp data channels).
// when the flow is found, it updated with the given info.

//-------------------------------------------------------------------------
// data structs
// -- key has IP address and port pairs; one port must be zero (wild card)
//    forming a 3-tuple
// -- node struct is stored in hash table by key
// -- each node struct has one or more list structs linked together
// -- each list struct has a list of flow data
// -- when a new expect is added, a new list struct is created if a new
//    node is created or the last list struct of an existing node already
//    has the same preproc id in the flow data list
// -- when a new expect is added, the last list struct is used if the
//    given preproc id is not already in the flow data list
// -- nodes are preallocated and stored in hash table; if there is no node
//    available when an expect is added, LRU nodes are pruned
// -- list structs are also preallocated and stored in free list; if there
//    is no list struct available when an expect is added, LRU nodes are
//    pruned freeing up both nodes and list structs
// -- the number of list structs per node is capped at MAX_LIST; once
//    reached, requests to add new expects requiring new list structs fail
// -- the number of data structs per list struct is not capped
// -- example:  ftp preproc adds a new 3-tuple twice for 2 expected data
//    channels -> new node with 2 list structs linked to it
// -- example:  ftp preproc adds a new 3-tuple once and then another
//    preproc expects the same 3-tuple -> new node with one list struct
//    is created for ftp and the next request goes in that same list
//    struct
// -- new list structs are appended to node's list struct chain
// -- matching expected sessions are pulled off from the head of the node's
//    list struct chain
//
// FIXIT-M expiration is by node struct but should be by list struct, ie
//    individual sessions, not all sessions to a given 3-tuple
//    (this would make pruning a little harder unless we add linkage
//    a la FlowCache)
//-------------------------------------------------------------------------
#include <vector>
#include "flow/flow_key.h"
#include "target_based/snort_protocols.h"

struct ExpectNode;

namespace snort
{
class Flow;
class FlowData;
struct Packet;

struct SO_PUBLIC ExpectFlow
{
    struct ExpectFlow* next;
    snort::FlowData* data;

    ~ExpectFlow();
    void clear();
    int add_flow_data(snort::FlowData*);
    snort::FlowData* get_flow_data(unsigned);
    static std::vector<ExpectFlow*>* get_expect_flows();
    static void reset_expect_flows();
};
}

class ExpectCache
{
public:
    ExpectCache(uint32_t max);
    ~ExpectCache();

    ExpectCache(const ExpectCache&) = delete;
    ExpectCache& operator=(const ExpectCache&) = delete;

    int add_flow(const snort::Packet *ctrlPkt, PktType, IpProtocol, const snort::SfIp* cliIP,
        uint16_t cliPort, const snort::SfIp* srvIP, uint16_t srvPort, char direction,
        snort::FlowData*, SnortProtocolId snort_protocol_id = UNKNOWN_PROTOCOL_ID,
        bool swap_app_direction = false);

    bool is_expected(snort::Packet*);
    bool check(snort::Packet*, snort::Flow*);

    unsigned long get_expects() { return expects; }
    unsigned long get_realized() { return realized; }
    unsigned long get_prunes() { return prunes; }
    unsigned long get_overflows() { return overflows; }

private:
    void prune_lru();

    ExpectNode* get_node(snort::FlowKey&, bool&);
    snort::ExpectFlow* get_flow(ExpectNode*, uint32_t, int16_t);
    bool set_data(ExpectNode*, snort::ExpectFlow*&, snort::FlowData*);
    ExpectNode* find_node_by_packet(snort::Packet*, snort::FlowKey&);
    bool process_expected(ExpectNode*, snort::FlowKey&, snort::Packet*, snort::Flow*);

private:
    class ZHash* hash_table;
    ExpectNode* nodes;
    snort::ExpectFlow* pool;
    snort::ExpectFlow* free_list;

    unsigned long expects = 0;
    unsigned long realized = 0;
    unsigned long prunes = 0;
    unsigned long overflows = 0;
};

#endif

flow_cache.h

Path = src/flow/flow_cache.h

#ifndef FLOW_CACHE_H
#define FLOW_CACHE_H

// there is a FlowCache instance for each protocol.
// Flows are stored in a ZHash instance by FlowKey.

#include <ctime>
#include <type_traits>

#include "framework/counts.h"

#include "flow_config.h"
#include "prune_stats.h"

namespace snort
{
class Flow;
struct FlowKey;
}

class FlowUniList;

class FlowCache
{
public:
    FlowCache(const FlowCacheConfig&);
    ~FlowCache();

    FlowCache(const FlowCache&) = delete;
    FlowCache& operator=(const FlowCache&) = delete;

    snort::Flow* find(const snort::FlowKey*);
    snort::Flow* allocate(const snort::FlowKey*);

    bool release(snort::Flow*, PruneReason = PruneReason::NONE, bool do_cleanup = true);

    unsigned prune_stale(uint32_t thetime, const snort::Flow* save_me);
    unsigned prune_excess(const snort::Flow* save_me);
    bool prune_one(PruneReason, bool do_cleanup);
    unsigned timeout(unsigned num_flows, time_t cur_time);
    unsigned delete_flows(unsigned num_to_delete);

    unsigned purge();
    unsigned get_count();

    unsigned get_max_flows() const
    { return config.max_flows; }

    PegCount get_total_prunes() const
    { return prune_stats.get_total(); }

    PegCount get_prunes(PruneReason reason) const
    { return prune_stats.get(reason); }

    PegCount get_total_deletes() const
    { return delete_stats.get_total(); }

    PegCount get_deletes(FlowDeleteState state) const
    { return delete_stats.get(state); }

    void reset_stats()
    {
        prune_stats = PruneStats();
        delete_stats = FlowDeleteStats();
    }

    void unlink_uni(snort::Flow*);

    void set_flow_cache_config(const FlowCacheConfig& cfg)
    { config = cfg; }

    const FlowCacheConfig& get_flow_cache_config() const
    { return config; }

    unsigned get_flows_allocated() const
    { return flows_allocated; }

private:
    void delete_uni();
    void push(snort::Flow*);
    void link_uni(snort::Flow*);
    void remove(snort::Flow*);
    void retire(snort::Flow*);
    unsigned prune_unis(PktType);
    unsigned delete_active_flows
        (unsigned mode, unsigned num_to_delete, unsigned &deleted);

private:
    static const unsigned cleanup_flows = 1;
    FlowCacheConfig config;
    uint32_t flags;

    class ZHash* hash_table;
    unsigned flows_allocated = 0;
    FlowUniList* uni_flows;
    FlowUniList* uni_ip_flows;

    PruneStats prune_stats;
    FlowDeleteStats delete_stats;
};
#endif

flow_config.h

Path = src/flow/flow_config.h

#ifndef FLOW_CONFIG_H
#define FLOW_CONFIG_H

#include "framework/decode_data.h"

// configured by the stream module
struct FlowTypeConfig
{
    unsigned nominal_timeout = 0;
    unsigned cap_weight = 0;
};

struct FlowCacheConfig
{
    unsigned max_flows = 0;
    unsigned pruning_timeout = 0;
    FlowTypeConfig proto[to_utype(PktType::MAX)];
};

#endif

flow_control.h

Path = src/flow/flow_control.h

#ifndef FLOW_CONTROL_H
#define FLOW_CONTROL_H

// this is where all the flow caches are managed and where all flows are
// processed.  flows are pruned as needed to process new flows.

#include <cstdint>
#include <vector>

#include "flow/flow_config.h"
#include "framework/counts.h"
#include "framework/decode_data.h"
#include "framework/inspector.h"

namespace snort
{
class Flow;
class FlowData;
struct FlowKey;
struct Packet;
struct SfIp;
}
class FlowCache;

enum class PruneReason : uint8_t;
enum class FlowDeleteState : uint8_t;

class FlowControl
{
public:
    FlowControl(const FlowCacheConfig& fc);
    ~FlowControl();

    void set_flow_cache_config(const FlowCacheConfig& cfg);
    const FlowCacheConfig& get_flow_cache_config() const;
    void init_proto(PktType, snort::InspectSsnFunc);
    void init_exp(uint32_t max);
    unsigned get_flows_allocated() const;

    bool process(PktType, snort::Packet*, bool* new_flow = nullptr);
    snort::Flow* find_flow(const snort::FlowKey*);
    snort::Flow* new_flow(const snort::FlowKey*);
    void release_flow(const snort::FlowKey*);
    void release_flow(snort::Flow*, PruneReason);
    void purge_flows();
    unsigned delete_flows(unsigned num_to_delete);
    bool prune_one(PruneReason, bool do_cleanup);
    snort::Flow* stale_flow_cleanup(FlowCache*, snort::Flow*, snort::Packet*);
    void timeout_flows(time_t cur_time);
    void check_expected_flow(snort::Flow*, snort::Packet*);
    bool is_expected(snort::Packet*);

    int add_expected_ignore(
        const snort::Packet* ctrlPkt, PktType, IpProtocol,
        const snort::SfIp *srcIP, uint16_t srcPort,
        const snort::SfIp *dstIP, uint16_t dstPort,
        char direction, snort::FlowData*);

    int add_expected(const snort::Packet* ctrlPkt, PktType, IpProtocol, const snort::SfIp *srcIP,
        uint16_t srcPort, const snort::SfIp *dstIP, uint16_t dstPort,
        SnortProtocolId snort_protocol_id, snort::FlowData*, bool swap_app_direction = false);

    class ExpectCache* get_exp_cache()
    { return exp_cache; }

    PegCount get_flows()
    { return num_flows; }

    PegCount get_total_prunes() const;
    PegCount get_prunes(PruneReason) const;
    PegCount get_total_deletes() const;
    PegCount get_deletes(FlowDeleteState state) const;
    void clear_counts();

private:
    void set_key(snort::FlowKey*, snort::Packet*);
    unsigned process(snort::Flow*, snort::Packet*);
    void preemptive_cleanup();
    void update_stats(snort::Flow*, snort::Packet*);

private:
    snort::InspectSsnFunc get_proto_session[to_utype(PktType::MAX)] = {};
    PegCount num_flows = 0;
    FlowCache* cache = nullptr;
    snort::Flow* mem = nullptr;
    class ExpectCache* exp_cache = nullptr;
    PktType last_pkt_type = PktType::NONE;
};

#endif

flow_data.h

Path = src/flow/flow_data.h

#ifndef FLOW_DATA_H
#define FLOW_DATA_H

#include "main/snort_types.h"

namespace snort
{
class Inspector;
struct Packet;

class SO_PUBLIC FlowData
{
public:
    FlowData(unsigned u, Inspector* = nullptr);
    virtual ~FlowData();

    unsigned get_id()
    { return id; }

    static unsigned create_flow_data_id()
    { return ++flow_data_id; }

    void update_allocations(size_t);
    void update_deallocations(size_t);
    Inspector* get_handler() { return handler; }

    // return fixed size (could be an approx avg)
    // this must be fixed for life of flow data instance
    // track significant supplemental allocations with the above updaters
    virtual size_t size_of() = 0;

    virtual void handle_expected(Packet*) { }
    virtual void handle_retransmit(Packet*) { }
    virtual void handle_eof(Packet*) { }

public:  // FIXIT-L privatize
    FlowData* next;
    FlowData* prev;

private:
    static unsigned flow_data_id;
    Inspector* handler;
    size_t mem_in_use = 0;
    unsigned id;
};

// The flow data created from SO rules must use RuleFlowData
// to support reload
class SO_PUBLIC RuleFlowData : public FlowData
{
protected:
    RuleFlowData(unsigned u);
public:
    virtual ~RuleFlowData() { }
};

}
#endif

flow.h

Path = src/flow/flow.h

#ifndef FLOW_H
#define FLOW_H

// Flow is the object that captures all the data we know about a session,
// including IP for defragmentation and TCP for desegmentation.  For all
// protocols, it used to track connection status bindings, and inspector
// state.  Inspector state is stored in FlowData, and Flow manages a list
// of FlowData items.

#include <sys/time.h>

#include "detection/ips_context_chain.h"
#include "flow/deferred_trust.h"
#include "flow/flow_data.h"
#include "flow/flow_stash.h"
#include "framework/data_bus.h"
#include "framework/decode_data.h"
#include "framework/inspector.h"
#include "protocols/layer.h"
#include "sfip/sf_ip.h"
#include "target_based/snort_protocols.h"

#define SSNFLAG_SEEN_CLIENT         0x00000001
#define SSNFLAG_SEEN_SENDER         0x00000001
#define SSNFLAG_SEEN_SERVER         0x00000002
#define SSNFLAG_SEEN_RESPONDER      0x00000002

#define SSNFLAG_ESTABLISHED         0x00000004
#define SSNFLAG_MIDSTREAM           0x00000008 /* picked up midstream */

#define SSNFLAG_ECN_CLIENT_QUERY    0x00000010
#define SSNFLAG_ECN_SERVER_REPLY    0x00000020
#define SSNFLAG_CLIENT_FIN          0x00000040 /* server sent fin */
#define SSNFLAG_SERVER_FIN          0x00000080 /* client sent fin */

#define SSNFLAG_COUNTED_INITIALIZE  0x00000100
#define SSNFLAG_COUNTED_ESTABLISH   0x00000200
#define SSNFLAG_COUNTED_CLOSING     0x00000400
#define SSNFLAG_COUNTED_CLOSED      0x00000800

#define SSNFLAG_TIMEDOUT            0x00001000
#define SSNFLAG_PRUNED              0x00002000
#define SSNFLAG_RESET               0x00004000

#define SSNFLAG_DROP_CLIENT         0x00010000
#define SSNFLAG_DROP_SERVER         0x00020000

#define SSNFLAG_STREAM_ORDER_BAD    0x00100000
#define SSNFLAG_CLIENT_SWAP         0x00200000
#define SSNFLAG_CLIENT_SWAPPED      0x00400000

#define SSNFLAG_PROXIED             0x01000000
#define SSNFLAG_NO_DETECT_TO_CLIENT 0x02000000
#define SSNFLAG_NO_DETECT_TO_SERVER 0x04000000

#define SSNFLAG_ABORT_CLIENT        0x10000000
#define SSNFLAG_ABORT_SERVER        0x20000000

#define SSNFLAG_HARD_EXPIRATION     0x40000000
#define SSNFLAG_KEEP_FLOW           0x80000000

#define SSNFLAG_NONE                0x00000000 /* nothing, an MT bag of chips */

#define SSNFLAG_SEEN_BOTH (SSNFLAG_SEEN_SERVER | SSNFLAG_SEEN_CLIENT)
#define SSNFLAG_BLOCK (SSNFLAG_DROP_CLIENT|SSNFLAG_DROP_SERVER)

#define STREAM_STATE_NONE              0x0000
#define STREAM_STATE_SYN               0x0001
#define STREAM_STATE_SYN_ACK           0x0002
#define STREAM_STATE_ACK               0x0004
#define STREAM_STATE_ESTABLISHED       0x0008
#define STREAM_STATE_DROP_CLIENT       0x0010
#define STREAM_STATE_DROP_SERVER       0x0020
#define STREAM_STATE_MIDSTREAM         0x0040
#define STREAM_STATE_TIMEDOUT          0x0080
#define STREAM_STATE_UNREACH           0x0100
#define STREAM_STATE_CLOSED            0x0200
#define STREAM_STATE_BLOCK_PENDING     0x0400

class BitOp;
class Session;

namespace snort
{
class FlowHAState;
struct FlowKey;
class IpsContext;
struct Packet;

typedef void (* StreamAppDataFree)(void*);

struct FilteringState
{
    uint8_t generation_id = 0;
    bool matched = false;

    void clear()
    {
        generation_id = 0;
        matched = false;
    }

    bool was_checked(uint8_t id) const
    {
        return generation_id and (generation_id == id);
    }

    void set_matched(uint8_t id, bool match)
    {
        generation_id = id;
        matched = match;
    }
};

struct FlowStats
{
    uint64_t client_pkts;
    uint64_t server_pkts;
    uint64_t client_bytes;
    uint64_t server_bytes;
    struct timeval start_time;
};

struct LwState
{
    uint32_t session_flags;

    int16_t ipprotocol;
    SnortProtocolId snort_protocol_id;

    char direction;
    char ignore_direction;
};

// this struct is organized by member size for compactness
class SO_PUBLIC Flow
{
public:
    enum class FlowState : uint8_t
    {
        SETUP = 0,
        INSPECT,
        BLOCK,
        RESET,
        ALLOW
    };
    Flow();
    ~Flow();

    Flow(const Flow&) = delete;
    Flow& operator=(const Flow&) = delete;

    void init(PktType);
    void term();

    void flush(bool do_cleanup = true);
    void reset(bool do_cleanup = true);
    void restart(bool dump_flow_data = true);
    void clear(bool dump_flow_data = true);

    int set_flow_data(FlowData*);
    FlowData* get_flow_data(uint32_t proto) const;
    void free_flow_data(uint32_t proto);
    void free_flow_data(FlowData*);
    void free_flow_data();

    void call_handlers(Packet* p, bool eof = false);
    void markup_packet_flags(Packet*);
    void set_direction(Packet*);
    void set_expire(const Packet*, uint32_t timeout);
    bool expired(const Packet*);
    void set_ttl(Packet*, bool client);
    void set_mpls_layer_per_dir(Packet*);
    Layer get_mpls_layer_per_dir(bool);
    void swap_roles();
    void set_service(Packet* pkt, const char* new_service);
    bool get_attr(const std::string& key, int32_t& val);
    bool get_attr(const std::string& key, std::string& val);
    void set_attr(const std::string& key, const int32_t& val);
    void set_attr(const std::string& key, const std::string& val);
    // Use this API when the publisher of the attribute allocated memory for it and can give up its
    // ownership after the call.
    void set_attr(const std::string& key, std::string* val)
    {
        assert(stash);
        stash->store(key, val);
    }

    template<typename T>
    bool get_attr(const std::string& key, T& val)
    {
        assert(stash);
        return stash->get(key, val);
    }

    template<typename T>
    void set_attr(const std::string& key, const T& val)
    {
        assert(stash);
        stash->store(key, val);
    }

    uint32_t update_session_flags(uint32_t ssn_flags)
    { return ssn_state.session_flags = ssn_flags; }

    uint32_t set_session_flags(uint32_t ssn_flags)
    { return ssn_state.session_flags |= ssn_flags; }

    uint32_t get_session_flags()
    { return ssn_state.session_flags; }

    uint32_t clear_session_flags(uint32_t ssn_flags)
    { return ssn_state.session_flags &= ~ssn_flags; }

    uint32_t clear_session_state(uint32_t ssn_state)
    { return session_state &= ~ssn_state; }

    void set_to_client_detection(bool enable);
    void set_to_server_detection(bool enable);

    int get_ignore_direction()
    { return ssn_state.ignore_direction; }

    int set_ignore_direction(char ignore_direction)
    {
        ssn_state.ignore_direction = ignore_direction;
        return ssn_state.ignore_direction;
    }

    bool two_way_traffic()
    { return (ssn_state.session_flags & SSNFLAG_SEEN_BOTH) == SSNFLAG_SEEN_BOTH; }

    bool is_pdu_inorder(uint8_t dir);

    void set_proxied()
    { ssn_state.session_flags |= SSNFLAG_PROXIED; }

    bool is_proxied()
    { return (ssn_state.session_flags & SSNFLAG_PROXIED) != 0; }

    bool is_stream()
    { return pkt_type == PktType::TCP or pkt_type == PktType::PDU; }

    void block()
    { ssn_state.session_flags |= SSNFLAG_BLOCK; }

    bool was_blocked() const
    { return (ssn_state.session_flags & SSNFLAG_BLOCK) != 0; }

    bool full_inspection() const
    { return (flow_state <= FlowState::INSPECT) and !is_inspection_disabled(); }

    void set_state(FlowState fs)
    { flow_state = fs; }

    void set_client(Inspector* ins)
    {
        ssn_client = ins;
        ssn_client->add_ref();
    }

    void clear_client()
    {
        ssn_client->rem_ref();
        ssn_client = nullptr;
    }

    void set_server(Inspector* ins)
    {
        ssn_server = ins;
        ssn_server->add_ref();
    }

    void clear_server()
    {
        ssn_server->rem_ref();
        ssn_server = nullptr;
    }

    void set_clouseau(Inspector* ins)
    {
        clouseau = ins;
        clouseau->add_ref();
    }

    void clear_clouseau()
    {
        clouseau->rem_ref();
        clouseau = nullptr;
    }

    void set_gadget(Inspector* ins)
    {
        gadget = ins;
        gadget->add_ref();
    }

    void clear_gadget()
    {
        gadget->rem_ref();
        gadget = nullptr;
        if (assistant_gadget != nullptr)
            clear_assistant_gadget();
    }

    void set_assistant_gadget(Inspector* ins)
    {
        assistant_gadget = ins;
        assistant_gadget->add_ref();
    }

    void clear_assistant_gadget()
    {
        assistant_gadget->rem_ref();
        assistant_gadget = nullptr;
    }

    void set_data(Inspector* pd)
    {
        data = pd;
        data->add_ref();
    }

    void clear_data()
    {
        data->rem_ref();
        data = nullptr;
    }

    void disable_inspection()
    { flags.disable_inspect = true; }

    bool is_inspection_disabled() const
    { return flags.disable_inspect; }

    bool is_suspended() const
    { return context_chain.front(); }

    void set_default_session_timeout(uint32_t dst, bool force)
    {
        if (force || (default_session_timeout == 0))
            default_session_timeout = dst;
    }

    void set_hard_expiration()
    { ssn_state.session_flags |= SSNFLAG_HARD_EXPIRATION; }

    bool is_hard_expiration()
    { return (ssn_state.session_flags & SSNFLAG_HARD_EXPIRATION) != 0; }

    void set_deferred_trust(unsigned module_id, bool on)
    { deferred_trust.set_deferred_trust(module_id, on); }

    bool cannot_trust()
    { return deferred_trust.is_active(); }

    bool try_trust()
    { return deferred_trust.try_trust(); }

    void stop_deferring_trust()
    { deferred_trust.clear(); }

    void finalize_trust(Active& active)
    {
        deferred_trust.finalize(active);
    }

    void trust();

    bool trust_is_deferred()
    { return deferred_trust.is_deferred(); }

public:  // FIXIT-M privatize if possible
    // fields are organized by initialization and size to minimize
    // void space and allow for memset of tail end of struct

    // these fields are const after initialization
    DeferredTrust deferred_trust;

    // Anything before this comment is not zeroed during construction
    const FlowKey* key;
    BitOp* bitop;
    FlowHAState* ha_state;
    FlowStash* stash;

    uint8_t ip_proto;
    PktType pkt_type; // ^^

    // these fields are always set; not zeroed
    Flow* prev, * next;
    Session* session;
    Inspector* ssn_client;
    Inspector* ssn_server;

    long last_data_seen;
    Layer mpls_client, mpls_server;

    // everything from here down is zeroed
    IpsContextChain context_chain;
    FlowData* flow_data;
    FlowStats flowstats;

    SfIp client_ip;
    SfIp server_ip;

    LwState ssn_state;
    LwState previous_ssn_state;

    Inspector* clouseau;  // service identifier
    Inspector* gadget;    // service handler
    Inspector* assistant_gadget;
    Inspector* data;
    const char* service;

    uint64_t expire_time;

    unsigned inspection_policy_id;
    unsigned ips_policy_id;
    unsigned network_policy_id;
    unsigned reputation_id;

    uint32_t default_session_timeout;

    int32_t client_intf;
    int32_t server_intf;

    int16_t client_group;
    int16_t server_group;

    uint16_t client_port;
    uint16_t server_port;

    uint16_t ssn_policy;
    uint16_t session_state;

    uint8_t inner_client_ttl;
    uint8_t inner_server_ttl;
    uint8_t outer_client_ttl;
    uint8_t outer_server_ttl;

    uint8_t response_count;

    struct
    {
        bool client_initiated : 1;  // Set if the first packet on the flow was from the side that is
                                    // currently considered to be the client
        bool app_direction_swapped : 1; // Packet direction swapped from application perspective
        bool disable_inspect : 1;
        bool reputation_src_dest : 1;
        bool reputation_blocklist : 1;
        bool reputation_monitor : 1;
        bool reputation_allowlist : 1;
        bool trigger_detained_packet_event : 1;
        bool trigger_finalize_event : 1;
        bool use_direct_inject : 1;
        bool data_decrypted : 1;    // indicate data in current flow is decrypted TLS application
                                    //data
    } flags;

    FlowState flow_state;

    FilteringState filtering_state;

private:
    void clean();
};

inline void Flow::set_to_client_detection(bool enable)
{
    if ( enable )
        ssn_state.session_flags &= ~SSNFLAG_NO_DETECT_TO_CLIENT;
    else
        ssn_state.session_flags |= SSNFLAG_NO_DETECT_TO_CLIENT;
}

inline void Flow::set_to_server_detection(bool enable)
{
    if ( enable )
        ssn_state.session_flags &= ~SSNFLAG_NO_DETECT_TO_SERVER;
    else
        ssn_state.session_flags |= SSNFLAG_NO_DETECT_TO_SERVER;
}
}

#endif

flow_key.h

Path = src/flow/flow_key.h

#ifndef FLOW_KEY_H
#define FLOW_KEY_H

// FlowKey is used to store Flows in the caches.  the data members are
// sequenced to avoid void space.

#include <cstdint>

#include <daq_common.h>

#include "framework/decode_data.h"
#include "hash/hash_key_operations.h"
#include "utils/cpp_macros.h"

class HashKeyOperations;

namespace snort
{
struct SfIp;
struct SnortConfig;

class FlowHashKeyOps : public HashKeyOperations
{
public:
    FlowHashKeyOps(int rows)
        : HashKeyOperations(rows)
    { }

    unsigned do_hash(const unsigned char* k, int len) override;
    bool key_compare(const void* k1, const void* k2, size_t) override;
};


PADDING_GUARD_BEGIN
struct SO_PUBLIC FlowKey
{
    uint32_t   ip_l[4]; /* Low IP */
    uint32_t   ip_h[4]; /* High IP */
    uint32_t   mplsLabel;
    uint16_t   port_l;  /* Low Port - 0 if ICMP */
    uint16_t   port_h;  /* High Port - 0 if ICMP */
    int16_t    group_l;
    int16_t    group_h;
    uint16_t   addressSpaceId;
    uint16_t   vlan_tag;
    uint8_t    ip_protocol;
    PktType    pkt_type;
    uint8_t    version;
    struct {
        uint8_t group_used:1; // Is group being used to build key.
        uint8_t ubits:7;
    } flags;

    /* The init() functions return true if the key IP/port fields were actively
        normalized, reversing the source and destination addresses internally.
        The IP-only init() will always return false as we will not reorder its
        addresses at this time. */
    bool init(
        const SnortConfig*, PktType, IpProtocol,
        const snort::SfIp *srcIP, uint16_t srcPort,
        const snort::SfIp *dstIP, uint16_t dstPort,
        uint16_t vlanId, uint32_t mplsId, uint16_t addrSpaceId,
        int16_t group_h = DAQ_PKTHDR_UNKNOWN, int16_t group_l = DAQ_PKTHDR_UNKNOWN);

    bool init(
        const SnortConfig*, PktType, IpProtocol,
        const snort::SfIp *srcIP, const snort::SfIp *dstIP,
        uint32_t id, uint16_t vlanId,
        uint32_t mplsId, uint16_t addrSpaceId,
        int16_t group_h = DAQ_PKTHDR_UNKNOWN, int16_t group_l = DAQ_PKTHDR_UNKNOWN);

    bool init(
        const SnortConfig*, PktType, IpProtocol,
        const snort::SfIp *srcIP, uint16_t srcPort,
        const snort::SfIp *dstIP, uint16_t dstPort,
        uint16_t vlanId, uint32_t mplsId, const DAQ_PktHdr_t&);

    bool init(
        const SnortConfig*, PktType, IpProtocol,
        const snort::SfIp *srcIP, const snort::SfIp *dstIP,
        uint32_t id, uint16_t vlanId, uint32_t mplsId, const DAQ_PktHdr_t&);

    void init_mpls(const SnortConfig*, uint32_t);
    void init_vlan(const SnortConfig*, uint16_t);
    void init_address_space(const SnortConfig*, uint16_t);
    void init_groups(int16_t, int16_t, bool);

    // If this data structure changes size, compare must be updated!
    static bool is_equal(const void* k1, const void* k2, size_t);

private:
    bool init4(
        const SnortConfig*, IpProtocol,
        const snort::SfIp *srcIP, uint16_t srcPort,
        const snort::SfIp *dstIP, uint16_t dstPort,
        uint32_t mplsId, bool order = true);

    bool init6(
        const SnortConfig*, IpProtocol,
        const snort::SfIp *srcIP, uint16_t srcPort,
        const snort::SfIp *dstIP, uint16_t dstPort,
        uint32_t mplsId, bool order = true);
};
PADDING_GUARD_END

}

#endif

flow_stash.h

Path = src/flow/flow_stash.h

#ifndef FLOW_STASH_H
#define FLOW_STASH_H

#include <map>
#include <string>

#include "main/snort_types.h"

#include "stash_item.h"

namespace snort
{

class SO_PUBLIC FlowStash
{
public:
    ~FlowStash();
    void reset();
    bool get(const std::string& key, int32_t& val);
    bool get(const std::string& key, uint32_t& val);
    bool get(const std::string& key, std::string& val);
    bool get(const std::string& key, StashGenericObject* &val);
    void store(const std::string& key, int32_t val);
    void store(const std::string& key, uint32_t val);
    void store(const std::string& key, const std::string& val);
    void store(const std::string& key, std::string* val);
    void store(const std::string& key, StashGenericObject* val, bool publish = true);

private:
    std::map<std::string, StashItem*> container;

    template<typename T>
    bool get(const std::string& key, T& val, StashItemType type);
    template<typename T>
    void store(const std::string& key, T& val, StashItemType type);
    void store(const std::string& key, StashGenericObject* &val, StashItemType type,
        bool publish = true);
};

}

#endif

flow_uni_list.h

Path = src/flow/flow_uni_list.h

#ifndef FLOW_UNI_LIST_H
#define FLOW_UNI_LIST_H

#include "flow.h"

class FlowUniList
{
public:

    FlowUniList()
    {
        head = new snort::Flow;
        tail = new snort::Flow;
        head->next = tail;
        tail->prev = head;
        head->prev = nullptr;
        tail->next = nullptr;
    }

    ~FlowUniList()
    {
        delete head;
        delete tail;
    }

    void link_uni(snort::Flow* flow)
    {
        flow->next = head->next;
        flow->prev = head;
        head->next->prev = flow;
        head->next = flow;
        ++count;
    }

    void unlink_uni(snort::Flow* flow)
    {
        if ( !flow->next )
            return;

        flow->next->prev = flow->prev;
        flow->prev->next = flow->next;
        flow->next = flow->prev = nullptr;
        --count;
    }

    snort::Flow* get_oldest_uni()
    {
        return ( tail->prev != head ) ? tail->prev : nullptr;
    }

    snort::Flow* get_prev(snort::Flow* flow)
    {
        return ( flow->prev != head ) ? flow->prev : nullptr;
    }

    unsigned get_count() const
    { return count; }

private:
    snort::Flow* head = nullptr;
    snort::Flow* tail = nullptr;
    unsigned count = 0;

};

#endif

ha.h

Path = src/flow/ha.h

#ifndef HA_H
#define HA_H

#include <daq_common.h>

#include <cassert>

#include "framework/bits.h"
#include "main/thread.h"

//-------------------------------------------------------------------------

struct HighAvailabilityConfig;

namespace snort
{
class Flow;
struct FlowKey;
struct Packet;
struct ProfileStats;

// The FlowHAHandle is the dynamically allocated index used uniquely identify
//   the client.  Used both in the API and HA messages.
// Handle 0 is defined to be the primary session client.
// NOTE: The type, masks, and count values must be in sync,
typedef uint16_t FlowHAClientHandle;
constexpr FlowHAClientHandle ALL_CLIENTS = 0xffff;

// Each active flow will have an associated FlowHAState instance.
class SO_PUBLIC FlowHAState
{
public:
    static constexpr uint8_t CRITICAL = 0x80;
    static constexpr uint8_t MAJOR = 0x40;

    enum : uint8_t
    {
        NEW = 0x01,
        MODIFIED = 0x02,
        DELETED = 0x04,
        STANDBY = 0x08,
        NEW_SESSION = 0x10,
    };

    FlowHAState();

    void set_pending(FlowHAClientHandle);
    void clear_pending(FlowHAClientHandle);
    bool check_pending(FlowHAClientHandle);
    void set(uint8_t state);
    void add(uint8_t state);
    void clear(uint8_t state);
    bool check_any(uint8_t state);
    static void config_timers(struct timeval, struct timeval);
    bool sync_interval_elapsed();
    void init_next_update();
    void set_next_update();
    void reset();

private:
    static constexpr uint8_t INITIAL_STATE = 0x00;
    static