GIF Creator

Shodan keeps a full history of all the information that has been gathered on an IP address. With the API, you’re able to retrieve that history and we’re going to use that to create a tool that outputs GIFs made of the screenshots that the Shodan crawlers gather.

The below code requires the following Python packages:

  • arrow
  • shodan

The arrow package is used to parse the timestamp field of the banner into a Python datetime object.

In addition to the above Python packages, you also need to have the ImageMagick software installed. If you’re working on Ubuntu or another distro using apt you can run the following command:

sudo apt-get install imagemagick

This will provide us with the convert command which is needed to merge several images into an animated GIF.

There are a few key Shodan methods/ parameters that make the script work:

  1. shodan.helpers.iterate_files() to loop through the Shodan data file
  2. history flag on the shodan.Shodan.host() method to get all the banners for an IP that Shodan has collected over the years
#!/usr/bin/env python
# gifcreator.py
#
# Dependencies:
# - arrow
# - shodan
#
# Installation:
# sudo easy_install arrow shodan
# sudo apt-get install imagemagick
#
# Usage:
# 1. Download a json.gz file using the website or the Shodan command-line tool (https://cli.shodan.io).
#    For example:
#        shodan download screenshots.json.gz has_screenshot:true
# 2. Run the tool on the file:
#        python gifcreator.py screenshots.json.gz

import arrow
import os
import shodan
import shodan.helpers as helpers
import sys


# Settings
API_KEY = ''
MIN_SCREENS = 5 # Number of screenshots that Shodan needs to have in order to make a GIF
MAX_SCREENS = 24

if len(sys.argv) != 2:
        print('Usage: {} <shodan-data.json.gz>'.format(sys.argv[0]))
        sys.exit(1)

# GIFs are stored in the local "data" directory
os.mkdir('data')

# We need to connect to the API to lookup the historical host information
api = shodan.Shodan(API_KEY)

# Use the shodan.helpers.iterate_files() method to loop over the Shodan data file
for result in helpers.iterate_files(sys.argv[1]):
        # Get the historic info
        host = api.host(result['ip_str'], history=True)

        # Count how many screenshots this host has
        screenshots = []
        for banner in host['data']:
                # Extract the image from the banner data
                if 'opts' in banner and 'screenshot' in banner['opts']:
                        # Sort the images by the time they were collected so the GIF will loop
                        # based on the local time regardless of which day the banner was taken.
                        timestamp = arrow.get(banner['timestamp']).time()
                        sort_key = timestamp.hour
                        screenshots.append((
                                sort_key,
                                banner['opts']['screenshot']['data']
                        ))

                        # Ignore any further screenshots if we already have MAX_SCREENS number of images
                        if len(screenshots) >= MAX_SCREENS:
                                break

        # Extract the screenshots and turn them into a GIF if we've got the necessary
        # amount of images.
        if len(screenshots) >= MIN_SCREENS:
                for (i, screenshot) in enumerate(sorted(screenshots, key=lambda x: x[0], reverse=True)):
                        open('/tmp/gif-image-{}.jpg'.format(i), 'w').write(screenshot[1].decode('base64'))

                # Create the actual GIF using the  ImageMagick "convert" command
                os.system('convert -layers OptimizePlus -delay 5x10 /tmp/gif-image-*.jpg -loop 0 +dither -colors 256 -depth 8 data/{}.gif'.format(result['ip_str']))

                # Clean up the temporary files
                os.system('rm -f /tmp/gif-image-*.jpg')

                # Show a progress indicator
                print result['ip_str']

The full code is also available on GitHub: https://gist.github.com/achillean/963eea552233d9550101