3
@6^+ @ sX d dl mZmZ d dlZd dlmZmZmZ dddddd gd
d iZG d d
d
eZ dS ) ) IPAddress IPNetworkN)
SpiderFootSpiderFootPluginSpiderFootEventzCustom Threat Data _customfeedipnetblockasndomainz^{0}$)idchecksregexc @ st e Zd ZdZddddddZdddd d
Zd Zd Ze fd
dZ dd Z
dd Z dd Z dd Z
dd Zdd Zd S )sfp_customfeedzCustom Threat Feed:Investigate,Passive:Reputation Systems::Check if a host/domain, netblock, ASN or IP is malicious according to your custom feed.T r )r checkaffiliates checkcohostsurl cacheperiodzThe URL where the feed can be found. Exact matching is performed so the format must be a single line per host, ASN, domain, IP or netblock.zApply checks to affiliates?z?Apply checks to sites found to be co-hosted on the target's IP?zIMaximum age of data in hours before re-downloading. 0 to always download.)r r r r NFc C s> || _ | j | _d| _x"t|j D ]}|| | j|< q$W d S )NF)sf tempStorageresults
errorStatelistkeysopts)selfsfcuserOptsopt r 9/var/www/spiderfoot.crq.systems/modules/sfp_customfeed.pysetup5 s
zsfp_customfeed.setupc C s dddddgS )N
INTERNET_NAME
IP_ADDRESSAFFILIATE_INTERNET_NAMEAFFILIATE_IPADDRCO_HOSTED_SITEr )r r r r!
watchedEventsB s z sfp_customfeed.watchedEventsc C s dddddgS )NMALICIOUS_IPADDRMALICIOUS_INTERNET_NAMEMALICIOUS_AFFILIATE_IPADDR!MALICIOUS_AFFILIATE_INTERNET_NAMEMALICIOUS_COHOSTr )r r r r! producedEventsI s z sfp_customfeed.producedEventsc C s t |dkrDx6|D ].}tj||tjtjB r| jjd| dS qW t |dkrx6|D ].}tj||tjtjB rV| jjd| dS qVW | jjd d S )Nr z#Found to be bad against bad regex: Tz$Found to be good againt good regex: Fz Neither good nor bad, unknown.)lenrematch
IGNORECASEDOTALLr debug)r content goodregexbadregexrxr r r! contentMaliciousO s
z sfp_customfeed.contentMaliciousc C s d}|dkr | j j|| jd }xXttj D ]F}t| d }| jd }||kr0t }| j jd| | jjdd|d < |d d kr| j j || jd
| jd d }|d d kr| j j
d
| d d S | j j d| |d |dkr4t } dt| krxt| d j dd}
t
j|
t
j} | j jd| d |
xP|d jdD ].} t
j| | }
t|
dkrD| j|
d qDW n|d jd} x| D ]}t|dk s|jdrq|j }y6t|t|kr| j j|d | d | |S W n< tk
r( } z | j jdt| wW Y d d }~X nX qW d S dt| krxZ|d jdD ]H} | |kst|dkrR| |krR| j j|d | d | d |S qRW q0ytt| d j|}tt| d j|}xj|d jdD ]X} |dkrt
j || t
jst
j || t
jr| j j|d | d | d |S qW W q0 t k
rv } z | j jd t| w0W Y d d }~X q0X q0W d S ) Nr r
_internettldsr r sfmal_r r r5
_fetchtimeout
_useragent)timeout useragentzUnable to fetch Fr r z{0}z(\d+\.\d+\.\d+\.\d+)zNew regex for z:
#z found within netblock/subnet z in zError encountered parsing: /z
found in z list.z Error encountered parsing 2: ) r
hostDomainr r malchecksr dictcacheGetgetfetchUrlerrorcachePutreplacer0 compiler2 r4 splitfindallr/ append
startswithstripr r Exceptionstrformatr1
BaseException)r r target
targetType targetDomcheckcidr dataiplistr8 patlinegrpr erxDomrxTgtr r r! resourceListc sr
zsfp_customfeed.resourceListc C sl xft tj D ]V}t| d }||kr|t| d kr| jjd| d | d | | j|||S qW d S )Nr r
zChecking maliciousness of z (z) with: )r rE r r r4 rd )r
resourceIditemTyperW rZ r[ r r r!
lookupItem s zsfp_customfeed.lookupItemc C s |j }|j}|j}| jjd| d | | jr4d S | jd dkrZ| jjdd d| _d S || jkr|| jjd| d d S d| j|< |d
kr| jj d d rd S |d kr| jj d
d rd S |dkr| jj dd rd S |dkr| jj dd rd S x t
t j D ]
}t | d }| j| r|d(krFd}|dkrBd}nd}|d)krXd}d}|d*krd }|dkrtd }|d krd }|d
krd }|dkrd!}d"}|dkrd!}d#}| j
|||} | j rd S | d k r|d$ | d% d& | d' }
t||
| j|} | j| qW d S )+NzReceived event, z, from r r z:You enabled sfp_customfeed but defined no custom feed URL!FTz Skipping z, already checked.r' r r& r NETBLOCK_OWNERchecknetblocksNETBLOCK_MEMBER checksubnetsr r$ r r) r+ BGP_AS_OWNER
BGP_AS_MEMBERr
MALICIOUS_ASNr# r% r r* r, r- r MALICIOUS_NETBLOCKMALICIOUS_SUBNETz [z]
zz)r$ r& )rl rm )r# r' r% ) eventTypemoduler\ r r4 r r rJ r rH r rE r rg checkForStopr __name__notifyListeners) r event eventName
srcModuleName eventDatarZ r[ typeIdevtTyper textevtr r r! handleEvent sr
zsfp_customfeed.handleEvent)rt
__module__ __qualname____doc__r optdescsr r rF r" r( r. r9 rd rg r~ r r r r! r s&
M r )
netaddrr r r0 sflibr r r rE r r r r r! s