The Dutch Honeynet chapter

HoneyNED.nl Is Now Fully IPv6 Ready!

We’re proud to announce that as of 2017 both our website and email are now fully IPv6 ready!

As the roll-out of IPv6 goes slow but steady, more and more websites add quad-A records to their DNS making them reachable over IPv6. Is your website IPv6 ready? Test your website here, and your current connection here!

DDOS Alerting Service

SIDN Fund offers financial support for DDOS alerting service

Within our HoneyNED chapter two people are working on DDOS detection techniques by using honeypot technology. The knowledge about which DDOS attacks are ‘running’ and which sites are under attack is interesting for a broader audience than our HoneyNED chapter. We’ve decided to start creating a public DDOS alerting service and applied for financial support here for by SIDN Fund.

SIDN Fund stands for ‘a strong internet for all’ and provides financial support to ideas and projects that aim to make the internet stronger or that use the internet in innovative ways. By doing so, SIDN Fund wants to help increase the social impact of the internet in the Netherlands. SIDN Fund is an independent foundation established by SIDN, the foundation for internet domain registration in the Netherlands.

SIDN Fund has decided to offer financial support for HoneyNED in order to create the DDOS alerting service. We’re very much pleased and will make sure the service will be available within 12 months. For further information please read the full SIDN Fund press release (in Dutch): https://www.sidnfonds.nl/nieuws/nieuwe-lichting-pioniers

Lennart & Rogier

Compiling the Honeyspider Network 2 Source Code

Recently HSN version 2.1 was released, see https://groups.google.com/forum/#!category-topic/honeyspider-network-2/DN2VbD9c618. Honeyspider Network is a highly-scalable system integrating multiple client honeypots to detect malicious websites. It was developed as a joint venture between CERT Polska and NCSC-NL. For more information, see http://www.honeyspider.net.

While a lot of improvements are being made in the development branch and now released in version 2.1, the binaries lack behind. Because of this I tried to compile from Git myself. This proved to be a bit difficult, mainly because of some dependencies not available via Maven anymore. In this blog I will show how to compile and configure HSN2 yourself.

The HSN2 framework is still considered experimental software. Installation and configuration still has its rough edges. The main benefit for using a framework is to have easy access to multiple plugins. If you want to write your own plugin see Niels van Eijk his Java One presentation for an introduction on writing your own plugin:

Compiling from GIT

For these instructions I assume a Debian 8 64-bit base-system. I tested this with the master branch (HSN 2.1) which was just released.

  • start with installing the following packages via apt-get:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

apt-get install \
build-essential \
devscripts \
debhelper \
openjdk-7-jdk \
maven \
python-all \
python-setuptools \
python-pika \
protobuf-compiler \
uuid-dev \
libssl-dev \
libconfig-dev \
libcurl4-openssl-dev \
libjson0-dev \
libarchive-dev \
libemu2 \
git \
libfuzzy2

  • There are two dependencies that cannot be obtained automatically anymore via Maven, so install those first in the following steps.

  • Install commons-ognl-4.0-SNAPSHOT manually (replace the tags):

1
2
3
4
5
6

apt-get install subversion

svn checkout http://svn.apache.org/repos/asf/commons/proper/ognl/trunk/ commons-ognl
cd commons-ognl
mvn package
mvn install:install-file -DgroupId=org.apache.commons -DartifactId=commons-ognl -Dversion=4.0-SNAPSHOT -Dpackaging=jar -Dfile=<REPLACE_WITH_YOUR_DIR>/commons-ognl/target/commons-ognl-4.0-SNAPSHOT.jar
1

mvn install:install-file -DgroupId=swfutils -DartifactId=swfutils -Dversion=0.3 -Dpackaging=jar -Dfile=<REPLACE_WITH_YOUR_DIR>/flexcover-0.90/sdk-modifications-3_2/lib/swfutils.jar
  • Obtain the HSN2 source via github (you can also choose to obtain the development branch):
1

git clone --recursive https://github.com/CERT-Polska/hsn2-bundle
  • Compile with Maven while in the main cloned Git repo:
1
2

cd hsn2-bundle
mvn package
  • When finished, the summary should look like this:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

[INFO] Reactor Summary:                                                                                                                              [0/1910]
[INFO]
[INFO] HSN2 Commons :: Root .............................. SUCCESS [1.144s]
[INFO] HSN2 Commons :: Protocol Buffers Impl ............. SUCCESS [15.957s]
[INFO] HSN2 Commons :: Bus ............................... SUCCESS [31.892s]
[INFO] HSN2 Commons :: Services Commons .................. SUCCESS [10.683s]
[INFO] HSN2 Commons :: Utils ............................. SUCCESS [1.545s]
[INFO] HSN2 Framework :: Root ............................ SUCCESS [0.718s]
[INFO] HSN2 Framework :: Configuration Manager ........... SUCCESS [1.890s]
[INFO] HSN2 Framework :: Core API ........................ SUCCESS [2.189s]
[INFO] HSN2 Framework :: Workflow :: Engine .............. SUCCESS [41.827s]
[INFO] HSN2 Framework :: Workflow :: HWL Parser .......... SUCCESS [4.230s]
[INFO] HSN2 Framework :: Workflow :: Git Repository ...... SUCCESS [2.938s]
[INFO] HSN2 Framework :: Main ............................ SUCCESS [3.409s]
[INFO] HSN2 Data Store ................................... SUCCESS [9.359s]
[INFO] HSN2 Object Store (MongoDB) ....................... SUCCESS [1.405s]
[INFO] HSN2 File Feeder .................................. SUCCESS [1.899s]
[INFO] HSN2 Web Client ................................... SUCCESS [3:25.166s]
[INFO] HSN2 Shellcode Analyser ........................... SUCCESS [2.737s]
[INFO] HSN2 URL Normalization Service .................... SUCCESS [2.280s]
[INFO] HSN2 JavaScript Analyzer .......................... SUCCESS [5.666s]
[INFO] HSN2 Capture HPC connector service ................ SUCCESS [16.023s]
[INFO] SWF ............................................... SUCCESS [0.001s]
[INFO] SWF CVE Plugin Commons ............................ SUCCESS [0.935s]
[INFO] SWF Tool .......................................... SUCCESS [14.308s]
[INFO] SWF server ........................................ SUCCESS [2.863s]
[INFO] cve_2007_0071 plugin .............................. SUCCESS [8.358s]
[INFO] cve_2009_1869 plugin .............................. SUCCESS [6.312s]
[INFO] SWF Tool Bundle ................................... SUCCESS [0.001s]
[INFO] SWF Service ....................................... SUCCESS [14.592s]
[INFO] HSN2 Service Reporter ............................. SUCCESS [5.716s]
[INFO] HSN2 command console .............................. SUCCESS [5.291s]
[INFO] HSN2 Cuckoo Java .................................. SUCCESS [1:33.914s]
[INFO] HSN2 MD5 To SSDeep ................................ SUCCESS [1.677s]
[INFO] HSN2 DNS Info ..................................... SUCCESS [17.111s]
[INFO] HSN2 Bundle ....................................... SUCCESS [0.004s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 8:55.242s
[INFO] Finished at: Tue May 17 16:24:39 CEST 2016
[INFO] Final Memory: 41M/113M
[INFO] ------------------------------------------------------------------------

Building Debian packages (and repo)

  • If compiling succeeds, you can also do a build and automatically generate Debian packages. You can of course also run and configure the jar files manually (which can take quite some time to figure out). Run the following script:
1

debian-build.sh
  • If you want to set up a local Debian repro, continue with the following steps:
1
2

apt-get install reprepro
debian-repo.sh yourreponame
  • Install apache2 and put the repo in /var/www/html
  • add in /etc/apt/source.list:
1

deb http://localhost/hsn2debrepo/ experimental main
  • Finally, to test the new repo, do:
1

apt-get update

Install and configure HSN2

Also see http://www.honeyspider.net/Installation.html.

  • First make sure the following packages are installed:
1

apt-get install rabbitmq-server openjdk-7-jre libmozjs185-1.0 libmozjs185-dev
1

apt-get install erlang
  • Make sure that rabbitmq and mongodb are running at startup (and start them)
1
2
3
4
5

update-rc.d rabbitmq-server defaults
update-rc.d mongodb defaults

systemctl rabbitmq-server start
systemctl mongodb start
  • Install the following subset of packages (installation will fail if rabbitmq/mongodb are not started), we skip python-hsn2-thug, hsn2-thug-docker and hsn2-capture-hpc for now.
1

apt-get install hsn2-commons-debian hsn2-cuckoo-java hsn2-data-store hsn2-dnsinfo hsn2-file-feeder hsn2-framework hsn2-js-sta hsn2-md5-to-ssdeep hsn2-norm-url hsn2-object-store-mongodb hsn2-rb-archiveinflate hsn2-rb-clamavnugget hsn2-rb-officecat hsn2-rb-pdffox hsn2-rb-swfscanner hsn2-rb-virustotal hsn2-reporter hsn2-shell-scdbg hsn2-unicorn hsn2-webclient python-hsn2-commons python-hsn2-console python-hsn2-malicious-domains python-hsn2-pcap-analyze python-hsn2-pcap-extract python-hsn2-proto python-hsn2-rb-nugget-commons python-hsn2-url-feeder python-hsn2-yara
  • Disable the following services at startup (they cause errors, start them manually in the right order):
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

systemctl disable hsn2-cuckoo-java.service
systemctl disable hsn2-data-store.service
systemctl disable hsn2-dnsinfo.service
systemctl disable hsn2-file-feeder.service
systemctl disable hsn2-framework.service
systemctl disable hsn2-js-sta.service
systemctl disable hsn2-malicious-domains.service
systemctl disable hsn2-md5-to-ssdeep.service
systemctl disable hsn2-norm-url.service
systemctl disable hsn2-object-store-mongodb.service
systemctl disable hsn2-pcap-analyze.service
systemctl disable hsn2-pcap-extract.service
systemctl disable hsn2-rb-archiveinflate.service
systemctl disable hsn2-rb-clamavnugget.service
systemctl disable hsn2-rb-officecat.service
systemctl disable hsn2-rb-pdffox.service
systemctl disable hsn2-rb-swfscanner.service
systemctl disable hsn2-rb-virustotal.service
systemctl disable hsn2-reporter.service
systemctl disable hsn2-shell-scdbg.service
systemctl disable hsn2-url-feeder.service
systemctl disable hsn2-webclient.service
  • Make and run a script to start services manually in the right order. The following uncommented services should work by default:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

#!/bin/bash
systemctl start hsn2-framework
systemctl start hsn2-object-store-mongodb
systemctl start hsn2-data-store
systemctl start hsn2-file-feeder
systemctl start hsn2-webclient
systemctl start hsn2-js-sta
systemctl start hsn2-norm-url
systemctl start hsn2-reporter
#systemctl start hsn2-rb-archiveinflate
#systemctl start hsn2-rb-clamavnugget
#systemctl start hsn2-rb-officecat
#systemctl start hsn2-rb-pdffox
#systemctl start hsn2-rb-swfscanner
#systemctl start hsn2-rb-virustotal
#systemctl start hsn2-shell-scdbg
#systemctl start hsn2-swf-cve
#systemctl start hsn2-url-feeder
  • Clone a set of initial workflows to work with:
1

git clone git://github.com/CERT-Polska/hsn2-workflows.git /etc/hsn2/workflows
  • Create file with a url on each line (at least one line) and submit a job:
1

hc j s simple feeder.uri=/home/<USERNAME>/uris.txt
  • Check whether the job was processed correctly:
1

hc j d <NUMBER OF JOB>

Now that you have the HSN2 framework running you can dive into the different plugins (or write one yourself) and make your own workflows (see /etc/hsn2/workflows). Don’t forget to look trough the logging in /var/log/hsn2 for errors/pointers. A Django based webinterface is also available, see https://github.com/CERT-Polska/hsn2-webgui and http://www.honeyspider.net/Web-Interface.html.

Gert