3
@6^-+ @ sV d dl mZmZ d dlZd dlmZmZmZ ddddgdd d
iZG d d d eZ dS )
) IPAddress IPNetworkN)
SpiderFootSpiderFootPluginSpiderFootEventZ PhishTank
_phishtanklistdomainz/http://data.phishtank.com/data/online-valid.csvz?\d+,\w+://(.*\.)?[^a-zA-Z0-9]?{0}.*,http://www.phishtank.com/.*)idtypechecksurlregexc @ st e Zd ZdZdddddZddddZd Ze fd
d Zd d
Z dd Z
dd Z dd Z dd Z
dd Zdd Zd S )
sfp_phishtankzmPhishTank:Investigate,Passive:Reputation Systems::Check if a host/domain is malicious according to PhishTank.T )r checkaffiliates checkcohosts cacheperiodzApply checks to affiliates?z?Apply checks to sites found to be co-hosted on the target's IP?z,Hours to cache list data before re-fetching.)r r r Nc C s8 || _ | j | _x"t|j D ]}|| | j|< q W d S )N)sf tempStorageresultsr keysopts)selfsfcuserOptsopt r 8/var/www/spiderfoot.crq.systems/modules/sfp_phishtank.pysetup5 s
zsfp_phishtank.setupc C s
dddgS )N
INTERNET_NAMEAFFILIATE_INTERNET_NAMECO_HOSTED_SITEr )r r r r
watchedEventsA s zsfp_phishtank.watchedEventsc C s
dddgS )NMALICIOUS_INTERNET_NAME!MALICIOUS_AFFILIATE_INTERNET_NAMEMALICIOUS_COHOSTr )r r r r producedEventsG s z sfp_phishtank.producedEventsc C s t |dkrDx6|D ].}tj||tjtjB r| jjd| dS qW t |dkrx6|D ].}tj||tjtjB rV| jjd| dS qVW | jjd d S )Nr z#Found to be bad against bad regex: Tz$Found to be good againt good regex: Fz Neither good nor bad, unknown.)lenrematch
IGNORECASEDOTALLr debug)r content goodregexbadregexrxr r r contentMaliciousL s
z sfp_phishtank.contentMaliciousc C s | j jd| d | xttj D ]}t| d }||kr&t| d dkr&tt| d }| j j|j|| jd | jd d }|d
d kr| j j d |j| d d S | j
|d
t| d
t| d r&|j|S q&W d S )Nz Querying z for maliciousness of r
r queryr
_fetchtimeout
_useragent)timeout useragentr. zUnable to fetch Fr/ r0 ) r r- r malchecksr strfetchUrlformatr errorr2 )r r
target
targetTypecheckcidr
resr r r
resourceQuery` s $
zsfp_phishtank.resourceQueryc C s d}|dkr | j j|| jd }xjttj D ]X}t| d }||koXt| d dkr0t }t| d }| j jd| | jjd d
|d < |d d kr| j j || jd | jd
d}|d d kr| j j
d| d d S | j j d| |d |dkrFt } dt| krt| d j dd}
t
j|
t
j} | j jd| d |
xP|d jdD ].} t
j| | }
t|
d
krV| j|
d
qVW n|d jd} x| D ]}t|dk s|jdrq|j }y6t|t|kr| j j|d | d | |S W n< tk
r: } z | j jd t| wW Y d d }~X nX qW d S dt| krxZ|d jdD ]H} | |ks|dkrd| |krd| j j|d | d | d |S qdW q0ytt| d j|}tt| d j|}xj|d jdD ]X} |dkrt
j || t
js"t
j || t
jr| j j|d | d | d |S qW W q0 t k
r } z | j jd t| w0W Y d d }~X q0X q0W d S )!N r
_internettldsr
r r r
sfmal_r r r. r4 r5 )r6 r7 zUnable to fetch Fnetblockr z{0}z(\d+\.\d+\.\d+\.\d+)zNew regex for z:
#z found within netblock/subnet z in zError encountered parsing: /z
found in z list.z Error encountered parsing 2: ) r
hostDomainr r r8 r dictcacheGetgetr: r< cachePutreplacer) compiler+ r- splitfindallr( append
startswithstripr r Exceptionr9 r; r*
BaseException)r r
r= r> targetDomr? r@ datar
iplistr1 patlinegrpiperxDomrxTgtr r r resourceListr sr
zsfp_phishtank.resourceListc C s xt tj D ]}t| d }||kr|t| d kr| jjd| d | d | t| d dkrt| j|||S t| d dkr| j|||S qW d S ) Nr
r zChecking maliciousness of z (z) with: r r3 r )r r8 r r r- rB rc )r
resourceIditemTyper= r? r@ r r r
lookupItem s zsfp_phishtank.lookupItemc C s~ |j }|j}|j}| jjd| d | || jkrL| jjd| d d S d| j|< |dkrr| jjdd rrd S |d kr| jjd
d rd S xtt j
D ]}t | d }| j| r|d krd
}|d krd}nd}|d krd}d}|d kr d}|dkr d}|dkrd}|dkr d}| j |||} | j r8d S | d k r|d | d d | d }
t
||
| j|} | j| qW d S )!NzReceived event, z, from z Skipping z, already checked.Tr" r FAFFILIATE_IPADDRr r
IP_ADDRESSr_ MALICIOUS_IPADDRMALICIOUS_AFFILIATE_IPADDR BGP_AS_OWNER
BGP_AS_MEMBERasn
MALICIOUS_ASNr r! r r$ r% r& z [z]
zz)rh rg )rk rl )r r" r! ) eventTypemodulerZ r r- r r rN r r8 r rf checkForStopr __name__notifyListeners) r event eventName
srcModuleName eventDatar? r@ typeIdevtTyper
textevtr r r handleEvent sR
zsfp_phishtank.handleEvent)rr
__module__ __qualname____doc__r optdescsr rL r r# r' r2 rB rc rf r| r r r r r s" Mr )
netaddrr r r) sflibr r r r8 r r r r r
s