eBrain IDS Basic Instrusion Detection

A possible telnet attack from the US

A simple opensource system to analyze rejected packets from an internet connected firewall. The system can detect and identify the source of unauthorized penetration testing and vulnerability scanning otherwise known as unauthorized port scanning.

It has to be said at the outset that scanning the internet for open ports (reconnaissance) and the detection thereof is of very little interest to authorities. As can be seen from the following diagram taken from the ACSC annual report 2020-2021, this type of activity rates C5 (at most) even if some malicious cyber actor scans computers of significance to national security. The C5 classification means something like: "I'll look at your complaint after coffee and cake - it's Mary's birthday today". Furthermore, the Australian Cyber Security Centre received over 67,500 cybercrime reports in the 20-21 year among which 205 related to unauthorized penetration testing. For reasons outlined below, pretty much zero action was taken.

So even if you fully implement this system and deploy it, it's probably only going to be of interest to geeks for C6 categories and of mild interest to SecOps people in C5 organisations. Unsolicited pentration testing (port scanning) is clearly only considered a "technical" breach of the relevant legislation by the ACSC.

The information and data gleaned from scanning the internet for open ports and storing the results in an illicit database can be used to speed up malicious activity, according to the ACSC. "Rapid exploitation of security vulnerabilities: State and criminal cyber actors continued to compromise large numbers of organisations by prosecuting publicly disclosed vulnerabilities at speed and scale. Malicious actors exploited security vulnerabilities, at times within hours of public disclosure, patch release or technical write up – particularly if proof of concept (PoC) code that identified the vulnerabilities in systems was also released."

The legal aspects of port scanning have been widely canvassed by nmap.org here (a US perspective). The nmap program is widely used by professionals in the computer networking domain as a tool to check on all sorts of aspects of computer network operation. In Australia the relevant legislation is the Cybercrime Act 2001 (the act).

The act has a couple of interesting paragraphs at:
478.3  Possession or control of data with intent to commit a computer offence
and
478.4  Producing, supplying or obtaining data with intent to commit a computer offence
which essentially states that you cannot have, produce, supply data that would help in committing an offence as defined in:
Division 477—Serious computer offences

Some of the penalties that apply.....
477.1(9): serious offence means an offence that is punishable by imprisonment for life or a period of 5 or more years
477.2(1): Penalty:  10 years imprisonment.
477.3(1): Penalty:  10 years imprisonment.
478.1(1): Penalty:  2 years imprisonment.
478.2(1): Penalty:  2 years imprisonment.
478.3(1): Penalty:  3 years imprisonment.
478.4(1): Penalty:  3 years imprisonment.

Now I'm not a criminal lawyer..... however, I reckon port scans constitute such data. And it seems I'm not the only one. In THE THREAT OF THE CYBERCRIME ACT 2001 TO AUSTRALIAN IT PROFESSIONALS by Nelson Chan, Simon Coronel, & Yik Chiat Ong (from the University of Melbourne Department of Computer Science and Software Engineering and The University of Melbourne Faculty of Law) the authors say:
"programs that can be used to commit or facilitate the commission of Division 477 offences, simply because such programs also have legitimate purposes. Examples of such programs include nmap (a utility to determine the usage of each port of a system), netstumbler (program that measures signal strength of wireless networks), and even root passwords (the password of a systems administrator), all of which may be used to facilitate the commission of a serious offence."

Furthermore there is a lot of folklore around what the analytics output actually mean. Analytics is relatively new to computer security. The Cybercrimes Act 2001 almost certainly predates any wide spread deployment of what we now know as analytics. It's fairly safe to say that the act was not based on scientific measurements but rather on computer security industry fokelore. This is essentially what the above table is telling us (i.e. don't call law enforcement armed with results from this system). Authorities in Australia already know Bulgaria is scanning every computer in the country - and they don't seem to care (and/or they are sniffing the results and using the information themselves - all Bulgarian scans come on to the network used by ElectricBrain via Perth, so theoretically easy to sniff - and block). The assumption that penetration testing is a precursor to a serious offence would appear not to be the case if the ASCS statistics are to be believed. It may be the case if the information is held in a large scale database, so detecting the existence of such a database may lead to the identity of a serious criminal. It's not entirely clear what can be done if that leads to foreign govenments however.

Part of the issue for C6's (like Members of the public, Small organisations and Sole Traders, and Medium-sized Organisation(s) and Schools) is it that is it very likely impossible to commit a serious offence against you. C6's don't have anything worth stealing in the sense that the impact of the disruption is very low on the scale of things (i.e. stealing your photos compared to stealing the plans of a nuclear powered submarine for example). However at 477.1(7) of the act: "A person may be found guilty of an offence against this section even if committing the serious offence is impossible."

Other hinderances to using the this IDS output in a prosecution lie in the forensic soundness of the data collection implying a need for an expert witness to stand up. Furthermore establishing intent, which is a requirement under the act, relies on an actual successful attempt to gain unauthorized access to, modify or impair data. This aspect may be achievable with a "honeypot" trap (which I have no doubt raises even further questions). At 477.1(8): "It is not an offence to attempt to commit an offence against this section." which I take to mean: unless someone actually gets in then there is no offence. Would-be administrators are allowed to type in the wrong password. Brute force attacks, on the other hand where thousands upon thousands of attempts are made, might go some way to demonstrating an intent.

Nevertheless, for those in the C6 category (indeed all IT shops) this simple IDS does provide actual measurements and even alarms if the reader chooses to set them up. It has spotted all the usual state actors and criminals doing their reconnaissance.

Privacy, personal information and publishing IP addresses

The information discovered by the IDS contains IP addresses which could be contrued as attacking the firewall here. Depending on the reader's jurisdiction this information may be considered "personal information" and could be used to aid in the identification of an individual. As such various laws apply. In the EU case the situation is clear under GDPR regulations, it is not permitted to store or publish IP addresses. In the US the situation is somewhat more complex, however it's pretty clear that publishing IP addresses in an un-anonymized form such as was initially contemplated here would very likely land you in hot water sooner or later. This is exhaustively canvassed in Are IP addresses generated when users visit websites personal information?

In the Australian context the waters are muddy. The Office of the Information Commissioner says IP addresses might be personal information depending on the circumstances. In 2017 there was a significant case that brought the legislation in to question, possibly leading to the conclusions on the above linked page. In Privacy Commissioner v Telstra the court ruled that IP addresses were not personal information essentially because the data was not about the person. A more detailed review of the decision can be found in Australia’s privacy laws gutted in court ruling on what is ‘personal information’ where the claim is made that Australia's privacy laws are essentially a joke.

"This means certain data held by Telstra, including IP addresses, URLs (websites) visited and geolocation data, are not protected by Australian privacy law. They are not subject to any restrictions on processing or disclosure to other entities."

The extent of the resulting confusion is best illustrated by the Privacy Commission's page that attempts to explain the key concepts.

Neverthless as it is anticipated that the ElectricBrain IDS will see deployment in various jurisdictions considerable effort has gone in to safeguarding and anonymizing the personal information (IP addresses) captured by the IDS. As it turned out this is actually a common problem for analytics implementors around the globe. The plugins required to achieve good quality anonymization are both mature and supported. The downside, from a bare metal  operations point of view, is that CPU load is increased due to the high volume of hash calculations required to randomize IPs in a consistent way (i.e. it costs more to operate anonymized - something like encryption). The resulting anonymized pseudo IPs shown in the images are akin to UUIDs with the benefit that the rest of the software thinks they are valid IPv6 addresses.

Installation

The basis of this simple system is OpenSearch. From an OpenSearch perspective this is a classic deployment across multiple nodes on the eBrain PiCluster. Because it uses OpenSearch, deployment on AWS should be trivially simple. Equally it should work with ElasticSearch on their networks too.

The desktop cluster connected in to the main Swarm - 8GB ram, 4xCPUs, 512GB NVMe storage running OpenSearch. The node behind is running Logstash.

Intrusion data consists of the kernel logs which are enhanced by the inclusion of firewall packet rejection information. The logs are sent from the target machine to OpenSearch Logstash, a program that transforms the stream in to something the OpenSearch engine can digest. Once the search indices are created and updated they form the basis of analysis by OpenSearch DashBoards.

Firewall logs in Linux are produced with the aid of a kernel module named nf_log_ipv4 when iptables is in use.

The definitive reference for all possible nf_log_ipv4 kernel module output fields that can be produced, when configured as described here, can only be found in the kernel source code (shown below).
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/net/ipv4/netfilter/nf_log_ipv4.c?h=v5.11.20

nf_log_ipv4.c kernel module source

From the code above it can be seen that dump_ipv4_packet only dumps stuff after the essentials have been added to the output buffer:
[ <timestamp>] FINAL_REJECT: IN=eth0 OUT= MAC=01:02:03:04:05:06:06:05:04:03:02:01:08:00

It is important to understand what can be produced by the kernel module since it impacts the design of the "Grok" script that pulls the text line apart so its components can be inserted into the OpenSearch index. It also impacts the design of the template which forms basis of the index's mapping.

Prerequisites

It needs to be said, before proceeding, that ipv6 has been disabled on all host machines here at ElectricBrain. This is done in the belief that IPV6 is ultimately a security risk when compared to its benefits.

Firewalld Ubuntu package is also used here on all systems to setup firewalling. This is pretty much a carryover from the Fedora days.

NetConsole to ship logs to OpenSearch-Logstash

Kernel log data is captured by NetConsole.
https://www.kernel.org/doc/Documentation/networking/netconsole.txt

Install the Ubuntu NetConsole support files. Then use the netconsole-setup command to configure netconsole to send the data to the appropriate computer. (man netconsole-setup).

If preffered manually adjust /etc/defaults.netconsole to name your OpenSearch-Logstash server.

[update] Using NC to ship logs to OpenSearch-Logstash

Ultimately having stuff blurted out all over the system console (now there is a large head sharer) proved to be to intrusive. Essentially the console had become unusable requiring remote access (ssh) to control this machine. The solution has been to tail the kernel log /var/log/kernel.log and clip the appropriate parts and ship those.

Enable logging in firewalld

https://www.cyberciti.biz/faq/enable-firewalld-logging-for-denied-packets-on-linux/

For iptables firewall logging to work with ipv4 the nf_log_ipv4 kernel module needs to be loaded. This pretty much happens automatically with Ubuntu. It's presence can be confirmed by using:

Change loglevel to capture firewall events

Set the default console log level. In this case the loglevel is set to 8, which logs everything at all levels of importance. This change is made persistent with the aid of sysctl. After this your console will become pretty much unusable since every rejected packet will now be shown on screen.

It is possible to control the output from kernel logging by using dmesg.

# dmesg --console-off
# dmesg --console-on

Install and setup OpenSearch-Logstash

Data is recevied by OpenSearch-Logstash. Logstash processes the input stream and stores it in OpenSearch. It's at this point that stuff is added (data enrichment) or unwanted stuff is deleted (filtered).

The (ElectricBrain) recommended method of installing is docker. This ensures that any work done in developing your environment is portable to K8s, swarm, OpenShift etc. even on a standalone machine.

Putting it all together so far and testing things requires a configuration which will run easily from the commandline. Input to Logstash has to be via the UDP input plugin to be compatible with NetConsole.  This may require an adjustment to the local firewall on the node running Logstash receiving the internet firewall data (i.e. open port 6666udp). For now output is directed to the commandline terminal to make things extra simple.

Start by pulling the OpenSearch-Logstash container image down. Then start Logstash listening on port 6666 by using the following command, where all output is directed to the screen. Note the output is using the stdout plugin therefore things will be blurted to the screen, bypassing cluster installation for now. Just use Ctrl-C to stop Logstash once it's running.

Test run to verify data is reaching Logstash   Some example results

Part 2 in this series details the steps necessary to load the index template, the visualizations and the saved searches and the dashboards themselves.