Python Adventures – 04

Getting a data set to play with!!!!!

The Data-Driven Security book uses free tools and sources. In Chapter 3 the fun begins by gathering some data. The authors begin with AlienVault’s open source reputation database. It’s updated hourly and I noticed sometimes not available on the hour possibly due to traffic load.

I used the following Python script, stored also on GitHub (to be linked) to get it the AlienVault data:

# -*- coding: utf-8 -*-

# Similar to Listing 3-1

import os

import sys

os.chdir(os.path.expanduser(“~”) + “/book/ch03”)

 

# Similar to Listing 3-3

# URL for the AlienVault IP Reputation Database (OSSIM format)

# storing the URL in a variable makes it easier to modify later

# if it changes. NOTE: I choose a direct URL because I don’t

# know any better J.

 

import urllib

import os.path

 

avURL = “http://reputation.alienvault.com/reputation.data”

 

# relative path for the downloaded data

avRep = “D:\Python\Data Driven Security/reputation.data”

 

To update once a day, I scheduled a Windows task. This is only the start! My version of Windows can’t update more often via the GUI and I should be able to improve this so I can script what I need.

I scheduled the task to run a script daily at 39 minutes after a random hour.

# Similar to Listing 3-3

# URL for the AlienVault IP Reputation Database (OSSIM format)

# storing the URL in a variable makes it easier to modify later

# if it changes. NOTE: I choose a direct URL because I don’t

# know any better J.

 

import urllib

import os.path

avURL = “http://reputation.alienvault.com/reputation.data”

 

# relative path for the downloaded data

avRep = “D:\Python\Data Driven Security/reputation.data”

#I’m sure there are much better ways of accomplishing this, comments warmly welcomed!

 

I am puzzled about the lack of HTTPS and/or a more secure method available to feed in trusted, verified reputation data both directly to a SIEM like OSSIM or via the website. I used a tool called EvilGrade way too much in the past (with permission) which makes me twitchy regarding insecure updates. An awesome write up on pivoting attacks with EvilGrade.

 

I’m certain there are better methods of scripting this. I made a basic variation from the DDS book. I want to improve on this script and comments and ideas are welcomed.

 

 

 

 

 

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: