Feed Descriptions

Written By Marissa Fegan (Super Administrator)

Updated at April 29th, 2022

Threat IQ Server threat intelligence is organized into "feeds" that contain domain-specific data, such as lists of malicious domains, network captures from malware output, and so forth. This article contains details on the available feeds.

Threat IQ Feeds

Threat IQ feeds contain up-to-the-minute threat data from analysis of live threats found in the wild. We execute thousands of new malware samples daily in our sandbox pipeline and observe every aspect of malicious operations, from files that are dropped to network connections made to process behavior. This information is then converted into a set of consumable, downloadable feeds that you can use to drive your threat protection environment.

Delete

Warning

Malware Executable Repository contains live, dangerous malware. Use at your own risk and take care when opening and analyzing files in this repository. 


Feed Descriptions and Details

Feed Name & FTP Directory Description

Feed name: Malicious HTTP Block Report


FTP Directory: threat_track


These URL’s contain direct links to malware file downloads: exe, .dll, .ocx, and others. These URLs are generally highly malicious. The URL/IPs in the threat_track folder should be aged out between 2-5 days. These are often compromised sites that are subsequently cleaned up; permanently blocking them can lead to false positives.

Example filenames
  • pefile_urls_2019-10-22.txt
  • pefile_urls_current_daily.txt
  • pefile_urls_current_hourly.txt

Data Sample
  • htt...//pcspeedup-7ff.kxcdn.com/5/uni..._pcspeedup.exe
  • htt...//forces.proturbodom.ru/setsearchm.exe
  • htt...//forces.proloaddom.ru/setsearchm.exe
  • htt...//dl.newstatsclientcloud.com/cs...br/setup.exe_a
  • htt...//www.tip-sa.com/ivtipsa/ivUpdate.exe


Number of Records

Ranges from 140-500 new entries per day.


Frequency 

The hourly file has the current hour's worth of URLs. The daily file contains the URLs for the current day so far. The file with the date is the final roll-up of URLs for that date. The zips for each date are kept on the site for 7 days.    


Comments

Block of the full URL is okay, but not advised to block the domain. Some files may have alternative extensions like jpg, but are actually PE files.

Feed name: Malicious HTTP Phishing Report


FTP directory: threat_track

These URL’s contain characteristics of phishing techniques: transposition, misspellings, common phishkit paths, and other phishing keywords. The URL/IPs in the threat_track folder should be aged out between 2-5 days. These are often compromised sites that are subsequently cleaned up; permanently blocking them can lead to false positives.

Example filenames: 
  • phish_urls_2019-02-11.txt
  • phish_urls_current_daily.txt
  • phish_urls_current_hourly.txt


Data Samples:

  • htt...//www.zeroaccidente.ro/cache/mo...8466a513d1a7d/
    htt...://luxorchocolate.com/wp-admin/i...k.michelin.com
  • htt...//www.ramadanhd.com/~joomlana/f...ahoo/yahoo.htm
    htt...//service-admin-data2015.com/index.htm
  • htt...//www.quanti.ro/templates/syste...ocom/index.htm
    htt...//newchristianmusic.net/compone..._banners/green

Number of Records

Roughly 25,000 new entries per day.


Frequency

The hourly file has the current hour's worth of URLs. The daily file contains the URLs for the current day so far. The file with the date is the final roll-up of URLs for that date. The zips for each date are kept on the site for 7 days.

Feed name: Suspicious HTTP Alert Report


FTP directory: linkshare 

Provides raw URLs (non PE URLs) and IP addresses based on network activity logged during a threat behavior analysis. The URL/IPs in the Linkshare folder should be aged out between 2 to 5 days. These are often compromised sites that are subsequently cleaned up and permanently blocking them can lead to false positives.

Example filenames: 
  • urls_2019-02-11.txt
  • urls_current_daily.txt
  • urls_current_hourly.txt


Data Sample:

  • htt...//reprise.com.tr/welcome/shares...ngs/index.html
  • htt...//108.167.189.26/~toto/A85/8996...3678820a2f6ad/
  • htt...//cdneu.clickmeinstats.com/ofr/isicicc.cis
  • htt...//cadastramentodeclientes.esy.es/atualizacao/
  • htt...//www.daduhui999.com/joh/2013gdocs/

Number of Records

Roughly 36,000-45,000 new entries per day.


Frequency

The hourly file has the current hour's worth of URLs. The daily file contains the URLs for the current day so far. The file with the date is the final roll-up of URLs for that date. The zips for each date are kept on the site for 7 days.

Feed name: Dynamic Behavioral Analysis Report 


FTP directory: 

xml

xml5


Detailed analysis of malware samples analyzed through ThreatAnalyzer. Reports in the XML directory are generated using TA 2.6 and the reports in the xml5 directory are generated using TA 5.1.


Example filenames: 

xml-2019-11-29.xip

Notes:

  • individual files in the zip archive
  • files located in xml5 are named by the MD5 hash of the sample
  • files located in XML are named with a 9-digit number assigned by Threat Analyzer


Data Sample:

See XML output sample for format

Number of Records

50,000-75,000 XML files per day.


Frequency

There is at least one zip file generated per day and sometimes multiple zip files per day. Each zip file contains multiple individual XML files that represent each malware sample that was processed. The md5/sha1 hash of the sample is noted in each XML file. The zip files are kept on the FTP site for 7 days.    


Comments

5.1 report has more information and better analysis compared to older version

Feed name: Border Patrol Report


FTP directory: border_patrol_sites_list

The BorderPatrol Sites list is a "blacklist" consisting of a wide variety of domains and IP addresses that are associated with the distribution of potentially unwanted software (including adware, spyware, and malware) and advertising (including banner ads, pop-ups, and pop-unders), as well as the facilitation of potentially unwanted software and advertising. This list is manually updated and verified and is relatively static. Because of the manual verification and the fact it is top-level domains and sub-domains, rather than URLs of temporarily compromised sites, these domains can generally be permanently categorized according to the codes we provide. Each new list posted can replace in its entirety the previous list.

Example filenames
bp-list_15_0216.zip

Data Sample:
  • 92.240.99.70,44098
  • 62.76.188.221,44098
  • 95.163.121.188,44098
  • 95.163.121.216,44098
  • 95.163.121.217,44098
  • 1tahitian-noni.com,44098
  • 1talk.net,44098
  • 1tdsgov.co.cc,44098
  • 1text.ru,44098
  • 1to23.com,44098
  • 1touchsolutions.com,44098


Number of Records

The list contains 1.33M entries. 3,000-4,000 domains are updated per day, upwards of 10,000 entries. Domains expire after 72 hours. Domains are actively managed on a daily basis, IPs are not as active and are much more unique.    


Frequency

One zip file is generated each day around 4 am. The zip files are kept on the FTP site for 7 days.    


Comments

Domains get removed from BorderPatrol if

  1. a false positive is reported, or
  2. after a recheck where site is determined to be "dead"

Feed name: Malware Executable Repository


FTP directory: (root)

All unique PE (Portable Executable format) files by Md5hash.

sample_2019-07-21.zip

sample_2019-07-21.txt


Data Sample:

Example entries in txt file

Indexed samples shall follow:
------
0002d87ef66872d6714ca5cfe6e28492: MSIL.Adware.PullUpdate (sunbelt)
0009a72ccb5bf3653cdf5b9f6bd5c8d0: Virus.Win32.Ramnit.a (v) (sunbelt)000ac16f08b3341ba1089a0abe471b15: Trojan.Win32.Generic!BT (sunbelt)


Number of Records

30,000-40,000 files per day


Frequency

There is at least one zip file generated per day and sometimes multiple zip files per day. Each zip file contains PE files that are zipped and password protected with the password 'infected'. The txt file contains an index of each sample by MD5 and the detection name and scanner name.    


Comments

Corrupted files noted in the txt file by MD5 hash are files that don't execute properly. It may be malware that was not compiled correctly so will not run.

Feed name: Malware Traffic Report (PCAP)


FTP directory: pcaps

PCAP files that are derived from samples scanned through the internal ThreatTrack Sandbox array on a daily basis. These files are the actual network traffic generated from the analysis of the malware. The PCAP files may include analysis of samples not posted in our daily sample posting (feed #1 AVshare) since some sample- sharing arrangements with partners restrict publishing of the actual samples. 

Example filenames: 
  • pcap_2019-07-11-1.txt
  • pcap_2019-07-11-1.zip


Data Sample:

Example entries in txt file

Indexed samples shall follow:
------
0002d87ef66872d6714ca5cfe6e28492: MSIL.Adware.PullUpdate (sunbelt)
0009a72ccb5bf3653cdf5b9f6bd5c8d0: Virus.Win32.Ramnit.a (v) (sunbelt)
000ac16f08b3341ba1089a0abe471b15: Trojan.Win32.Generic!BT (sunbelt)

Whitelist Dom


Number of Records

50,000-75,000 PCAP files per day.


Frequency

There are multiple txt and zip files generated per day. The txt files provide an index of the files in the zip file. The zip file contains .pcap files that are named by the MD5 hash.

Feed name: ains


FTP directory: whitelist


This is a list of domains whose main use or function is non-malicious. Put simply, these are domains that are being used for legitimate purposes, where legitimate is defined as any purpose other than serving or facilitating malware, adware/spyware, or potentially dangerous scams.

The classification of any domain is based on the default use or purpose of the domain (malicious v. non-malicious). Any domain classified as white (non-malicious) may, of course, be compromised and used for malicious purposes. Domains classified white that are compromised will not be removed from this white list unless their compromised state persists for an extended or apparently indefinite period of time.

Example filenames:
whitelist_15_0212.zip

Data Sample:
  • burlingtoncarservice.net,1,1
    burlingtoncg.com,1,1
    burlingtoncmo.org,1,1
    burlingtoncoatfactory.com,1,1
    burlingtoncountryclub.org,1,1
    burlingtonelectric.com,1,1
    burlingtongraphics.com,1,1
    burlingtonkyfire.org,1,1
    burlingtonnj.us,1,1


Number of Records

2M domains


Frequency

One zip file is generated each day and is kept on the site for 7 days.    


Comments

Delivered as a plain-text CSV file divided into three comma separate values:

1 2 3

Domain,Type,Subdomains

  • Domain
    • The Domain column contains the base domain being classified. Please note that the third column, Subdomains, determines how subdomains of the base domain should be regarded or handled.
  • Subdomains
    • The third column indicates how subdomains of the base domain should be regarded or handled.
    • 0=not all sub-domains white: some sub-domains may be used for malicious purposes. See also the Type classification for further information on how likely it is to encounter malicious sub-domains.
    • 1=all sub-domains white: all sub-domains should be considered white (non-malicious).