SRT-CTF

Zeek

10 min read

Zeek (formerly Bro) is an open-source and commercial network monitoring tool (traffic analyser).

Zeek Architecture

  • Event Engine
    • where the packets are processed; it is called the event core and is responsible for describing the event without focusing on event details. It is where the packages are divided into parts such as source and destination addresses, protocol identification, session analysis and file extraction.
  • Policy Script Interpreter
    • where the semantic analysis is conducted. It is responsible for describing the event correlations by using Zeek scripts.
ParameterDescription
-rReading option, read/process a pcap file.
-CIgnoring checksum errors.
-vVersion information.
zeekctlZeekControl module.

Zeek Logs

CategoryDescriptionLog Files
NetworkNetwork protocol logs.conn.log, dce_rpc.log, dhcp.log, dnp3.log, dns.log, ftp.log, http.log, irc.log, kerberos.log, modbus.log, modbus_register_change.log, mysql.log, ntlm.log, ntp.log, radius.log, rdp.log, rfb.log, sip.log, smb_cmd.log, smb_files.log, smb_mapping.log, smtp.log, snmp.log, socks.log, ssh.log, ssl.log, syslog.log, tunnel.log.
FilesFile analysis result logs.files.log, ocsp.log, pe.log, x509.log.
NetControlNetwork control and flow logs.netcontrol.log, netcontrol_drop.log, netcontrol_shunt.log, netcontrol_catch_release.log, openflow.log.
DetectionDetection and possible indicator logs.intel.log, notice.log, notice_alarm.log, signatures.log, traceroute.log.
Network ObservationsNetwork flow logs.known_certs.log, known_hosts.log, known_modbus.log, known_services.log, software.log.
MiscellaneousAdditional logs cover external alerts, inputs and failures.barnyard2.log, dpd.log, unified2.log, unknown_protocols.log, weird.log, weird_stats.log.
Zeek DiagnosticZeek diagnostic logs cover system messages, actions and some statistics.broker.log, capture_loss.log, cluster.log, config.log, loaded_scripts.log, packet_filter.log, print.log, prof.log, reporter.log, stats.log, stderr.log, stdout.log.

Please refer to Zeek’s official documentation and Corelight log cheat sheet for more information.

Zeek-cut - Cut specific columns from zeek logs.

Zeek Signatures

-s flag loads a signature file

structure:

signature idconditionsaction
Unique signature name.Header: Filtering the packet headers for specific source and destination addresses, protocol and port numbers.

Content: Filtering the packet payload for specific value/pattern.		Default action: Create the “signatures.log” file in case of a signature match.

Additional action: Trigger a Zeek scrip

Conditions

Condition FieldAvailable Filters
Headersrc-ip: Source IP.

dst-ip: Destination IP.

src-port: Source port.

dst-port: Destination port.

ip-proto: Target protocol. Supported protocols; TCP, UDP, ICMP, ICMP6, IP, IP6
Contentpayload: Packet payload.
http-request: Decoded HTTP requests.
http-request-header: Client-side HTTP headers.
http-request-body: Client-side HTTP request bodys.
http-reply-header: Server-side HTTP headers.
http-reply-body: Server-side HTTP request bodys.
ftp: Command line input of FTP sessions.
Contextsame-ip: Filtering the source and destination addresses for duplication.
Actionevent: Signature match message.
Comparison
Operators		==, !=, <, ⇐, >, >=
NOTE!Filters accept string, numeric and regex values.

Example | Cleartext Submission of Password

signature http-password {
     ip-proto == tcp
     dst-port == 80
     payload /.*password.*/
     event "Cleartext Password Found!"
}
 
# signature: Signature name.
# ip-proto: Filtering TCP connection.
# dst-port: Filtering destination port 80.
# payload: Filtering the "password" phrase.
# event: Signature match message.

Zeek signatures support regex. Regex “.*” matches any character zero or more times. The rule will match when a “password” phrase is detected in the packet payload. Once the match occurs, Zeek will generate an alert and create additional log files (signatures.log and notice.log).

ubuntu@ubuntu$ zeek -C -r http.pcap -s http-password.sig 
ubuntu@ubuntu$ ls
clear-logs.sh  conn.log  files.log  http-password.sig  http.log  http.pcap  notice.log  packet_filter.log  signatures.log
 
ubuntu@ubuntu$ cat notice.log  | zeek-cut id.orig_h id.resp_h msg 
10.10.57.178	44.228.249.3	10.10.57.178: Cleartext Password Found!
10.10.57.178	44.228.249.3	10.10.57.178: Cleartext Password Found!
 
ubuntu@ubuntu$ cat signatures.log | zeek-cut src_addr dest_addr sig_id event_msg 
10.10.57.178		http-password	10.10.57.178: Cleartext Password Found!
10.10.57.178		http-password	10.10.57.178: Cleartext Password Found!

Zeek Scripts

Zeek scripting is a programming language itself, and we are not covering the fundamentals of Zeek scripting. In this room, we will cover the logic of Zeek scripting and how to use Zeek scripts. You can learn and practice the Zeek scripting language by using Zeek ‘s official training platform for free.

Zeek has base scripts installed by default, and these are not intended to be modified.These scripts are located in “/opt/ zeek /share/ zeek /base”.
User-generated or modified scripts should be located in a specific path.These scripts are located in “/opt/ zeek /share/ zeek /site”.
Policy scripts are located in a specific path.These scripts are located in “/opt/ zeek /share/ zeek /policy”.
Like Snort, to automatically load/use a script in live sniffing mode, you must identify the script in the Zeek configuration file. You can also use a script for a single run, just like the signatures.The configuration file is located in “/opt/ zeek /share/ zeek /site/local.zeek “.
  • Zeek scripts use the “.zeek ” extension.
  • You can call scripts in live monitoring mode by loading them with the command load @/script/path  or  load @script-name  in local.zeek file.
  • Zeek is event-oriented, not packet-oriented! We need to use/write scripts to handle the event of interest.

running Zeek with signature

ubuntu@ubuntu$ zeek -C -r sample.pcap -s sample.sig

Now let’s see Zeek scripts in action. First, let’s look at the components of the Zeek script. Here the first, second and fourth lines are the predefined syntaxes of the scripting language. The only part we created is the third line which tells Zeek to extract DHCP hostnames. Now compare this automation ease with the rest of the methods. Obviously, this four-line script is easier to create and use. While tcpdump and tshark can provide similar results, transferring uncontrolled data through multiple pipelines is not much preferred.

Sample Script

event dhcp_message (c: connection, is_orig: bool, msg: DHCP::Msg, options: DHCP::Options)
{
print options$host_name;
}

Now let’s use the Zeek script and see the output.

extracting hostnames with tcpdump and tshark

ubuntu@ubuntu$ zeek -C -r smallFlows.pcap dhcp-hostname.zeek 
student01-PC
vinlap01

The provided outputs show that our script works fine and can extract the requested information. This should show why Zeek is helpful in data extraction and correlation. Note that

Write Basic Scripts

Scripts contain operators, types, attributes, declarations and statements, and directives. Let’s look at a simple example event called “zeek_init” and “zeek_done”. These events work once the Zeek process starts and stops. Note that these events don’t have parameters, and some events will require parameters.

Sample Script

event zeek_init()
    {
     print ("Started Zeek!");
    }
event zeek_done()
    {
    print ("Stopped Zeek!");
    }
 
# zeek_init: Do actions once Zeek starts its process.
# zeek_done: Do activities once Zeek finishes its process.
# print: Prompt a message on the terminal.

Run Zeek with a script

ubuntu@ubuntu$ zeek -C -r sample.pcap 101.zeek 
Started Zeek!
Stopped Zeek!

The above output shows how the script works and provides messages on the terminal. Zeek will create logs in the working directory separately from the scripts tasks.

Let’s print the packet data to the terminal and see the raw data. In this script, we are requesting details of a connection and extracting them without any filtering or sorting of the data. To accomplish this, we are using the “new_connection” event. This event is automatically generated for each new connection. This script provides bulk information on the terminal. We need to get familiar with Zeek ‘s data structure to reduce the amount of information and focus on the event of interest. To do so, we need to investigate the bulk data.

event new_connection(c: connection)
{
    print c;
}
ubuntu@ubuntu$ zeek -C -r sample.pcap 102.zeek 
[id=[orig_h=192.168.121.40, orig_p=123/udp, resp_h=212.227.54.68, resp_p=123/udp], orig=[size=48, state=1, num_pkts=0, num_bytes_ip=0, flow_label=0, l2_addr=00:16:47:df:e7:c1], resp=[size=0, state=0, num_pkts=0, num_bytes_ip=0, flow_label=0, l2_addr=00:00:0c:9f:f0:79], start_time=1488571365.706238, duration=0 secs, service={}, history=D, uid=CajwDY2vSUtLkztAc, tunnel=, vlan=121, inner_vlan=, dpd=, dpd_state=, removal_hooks=, conn=, extract_orig=F, extract_resp=F, thresholds=, dce_rpc=, dce_rpc_state=, dce_rpc_backing=, dhcp=, dnp3=, dns=, dns_state=, ftp=, ftp_data_reuse=F, ssl=, http=, http_state=, irc=, krb=, modbus=, mysql=, ntlm=, ntp=, radius=, rdp=, rfb=, sip=, sip_state=, snmp=, smb_state=, smtp=, smtp_state=, socks=, ssh=, syslog=]

The above terminal provides bulk data for each connection. This style is not the best usage, and in real life, we will need to filter the information for specific purposes. If you look closely at the output, you can see an ID and field value for each part.

To filter the event of interest, we will use the primary tag (in this case, it is c —comes from “c: connection”—), id value (id=), and field name. You should notice that the fields are the same as the fields in the log files.

event new_connection(c: connection)
{
    print ("###########################################################");
    print ("");
    print ("New Connection Found!");
    print ("");
    print fmt ("Source Host: %s # %s --->", c$id$orig_h, c$id$orig_p);
    print fmt ("Destination Host: resp: %s # %s <---", c$id$resp_h, c$id$resp_p);
    print ("");
}
 
# %s: Identifies string output for the source.
# c$id: Source reference field for the identifier.

Now you have a general idea of running a script and following the provided output on the console. Let’s look closer to another script that extracts specific information from packets. The script above creates logs and prompts each source and destination address for each connection.

Let’s see this script in action.

Run Zeek with a script

ubuntu@ubuntu$ zeek -C -r sample.pcap 103.zeek 
###########################################################
New Connection Found! Source Host: 192.168.121.2 # 58304/udp ---> 
Destination Host: resp: 192.168.120.22 # 53/udp <--- 
###########################################################

The above output shows that we successfully extract specific information from the events. Remember that this script extracts the event of interest (in this example, a new connection), and we still have logs in the working directory. We can always modify and optimise the scripts at any time.

Scripts 201 | Use Scripts and Signatures Together

Up to here, we covered the basics of Zeek scripts. Now it is time to use scripts collaboratively with other scripts and signatures to get one step closer to event correlation. Zeek scripts can refer to signatures and other Zeek scripts as well. This flexibility provides a massive advantage in event correlation.

Let’s demonstrate this concept with an example. We will create a script that detects if our previously created ” ftp -admin ” rule has a hit.

View Script

Sample Script

event signature_match (state: signature_state, msg: string, data: string)
{
if (state$sig_id == "ftp-admin")
    {
    print ("Signature hit! --> #FTP-Admin ");
    }
}

This basic script quickly checks if there is a signature hit and provides terminal output to notify us. We are using the ” signature_match ” event to accomplish this. You can read more about events here. Note that we are looking only for ” ftp -admin ” signature hits. The signature is shown below.

View Script

Sample Script

signature ftp-admin {
    ip-proto == tcp
    ftp /.*USER.*admin.*/
    event "FTP Username Input Found!"
}

Let’s see this script in action.

Run Zeek with signature and script

ubuntu@ubuntu$ zeek -C -r ftp.pcap -s ftp-admin.sig 201.zeek 
Signature hit! --> #FTP-Admin Signature hit! --> #FTP-Admin
Signature hit! --> #FTP-Admin Signature hit! --> #FTP-Admin

The above output shows that we successfully combined the signature and script. Zeek processed the signature and logs then the script controlled the outputs and provided a terminal output for each rule hit.

Scripts 202 | Load Local Scripts

Load all local scripts

We mentioned that Zeek has base scripts located in “/opt/ zeek /share/ zeek /base “. You can load all local scripts identified in your “local.zeek ” file. Note that base scripts cover multiple framework functionalities. You can load all base scripts by easily running the local command.

ubuntu@ubuntu$ zeek -C -r ftp.pcap local 
ubuntu@ubuntu$ ls
101.zeek  103.zeek          clear-logs.sh  ftp.pcap            packet_filter.log  stats.log
102.zeek  capture_loss.log  conn.log       loaded_scripts.log  sample.pcap        weird.log

The above output demonstrates how to run all base scripts using the “local” command. Look at the above terminal output; Zeek provided additional log files this time. Loaded scripts generated loaded_scripts.log, capture_loss.log, notice.log, stats.log files. Note that, in our instance, 465 scripts loaded and used by using the “local” command. However, Zeek doesn’t provide log files for the scripts doesn’t have hits or results.

Load Specific Scripts

Another way to load scripts is by identifying the script path. In that case, you have the opportunity of loading a specific script or framework. Let’s go back to FTP brute-forcing case. We created a script that detects multiple admin login failures in previous steps. Zeek has an FTP brute-force detection script as well. Now let’s use the default script and identify the differences.

The above output shows how to load a specific script. This script provides much more information than the one we created. It provides one single line output and a connection summary for the suspicious incident. You can find and read more on the prebuilt scripts and frameworks by visiting Zeek ‘s online book here.

Graph View