U @6^O@sHddlZddlZddlZddlZddlmZmZmZGdddeZ dS)N) SpiderFootSpiderFootPluginSpiderFootEventc@s\eZdZdZddddgddZddd d d dZd Zefd dZddZ ddZ ddZ d S)sfp_onionsearchenginezOnionsearchengine.com:Footprint,Investigate:Search Engines::Search Tor onionsearchengine.com for mentions of the target domain. Tz .*://relate.*)timeout max_pages fetchlinks blacklist fullnameszQuery timeout, in seconds.z,Maximum number of pages of results to fetch.zQFetch the darknet pages (via TOR, if enabled) to verify they mention your target.z3Exclude results from sites matching these patterns.zSearch for human names?NcCs4||_||_t|D]}|||j|<qdS)N)sf tempStorageresultslistkeysopts)selfsfcuserOptsoptr@/var/www/spiderfoot.crq.systems/modules/sfp_onionsearchengine.pysetup*s zsfp_onionsearchengine.setupcCs dddgS)N DOMAIN_NAME HUMAN_NAME EMAILADDRrrrrr watchedEvents2sz#sfp_onionsearchengine.watchedEventscCs dddgS)NDARKNET_MENTION_URLDARKNET_MENTION_CONTENTSEARCH_ENGINE_WEB_CONTENTrrrrrproducedEvents8sz$sfp_onionsearchengine.producedEventsc Csl|j}|j}|j}|jds(|dkr(dS||jkrJ|jd|ddSd|j|<d}d}|rh|t|jdkrh|rdSd| d j d d d dd t |d}|jj dt j||jd|jdd}|dks|ds|jddS|d7}d|dkr d|dkrq\dSd|dkr2d}td|d|j|} || td|dtjtjB} | D]} |rdS| |jkrqnd|j| <d} |jdD]4} t| | tjr|jd| d| d} q| rqn|jd| |j| dsqn|jd s>td!| |j|} || qn|jj | |jd"|jdd#}|ddkr|jd$| d%qn||dkr|jd$| d&|qntd!| |j|} || z&|d|d'}|t|d(}WnBtk r.}z"|jd)|d*WYqnW5d}~XYnX|d||}td+d,|d,|j| } || qnq\dS)-Nr rzAlready did a search for z , skipping.Tr "raw_unicode_escapeasciireplace)errorsZSearch)searchsubmitpagez)https://onionsearchengine.com/search.php? _useragentr) useragentrcontentz/No results returned from onionsearchengine.com.z url.php?u=zyou didn't submit a keywordz forward >Fr!zurl\.php\?u=(.[^"']+)["']r z Skipping z as it matches blacklist zFound a darknet mention: z.onionr r _fetchtimeout)rr-z Ignoring z as no data returnedz as no mention of xzString "z" not found in content.r z...) eventTypemoduledatarrr debugint checkForStopencodedecodestrfetchUrlurllibparse urlencodegetinfor__name__notifyListenersrefindall IGNORECASEDOTALLmatchurlFQDNendswithindexlen BaseException)revent eventName srcModuleName eventData keepGoingr+paramsr4evtlinkslinkr rres startIndexendIndexerrr handleEvent;s             z!sfp_onionsearchengine.handleEvent) rA __module__ __qualname____doc__roptdescsrdictrrr"r[rrrrrs$ r) rCurllib.requestr< urllib.parse urllib.errorsflibrrrrrrrrs