3 @6^\@sHddlZddlZddlZddlZddlmZmZmZGdddeZ dS)N) SpiderFootSpiderFootPluginSpiderFootEventc@sreZdZdZddddZdddZd ZeZd Z efd d Z d d Z ddZ ddZ ddZddZddZd S) sfp_tldsearchz|TLD Search:Footprint:DNS:slow:Search all Internet TLDs for domains with the same name as the target (this can be very slow.)FT2) activeonly skipwildcards _maxthreadsz>Only report domains that have content (try to fetch the page)?z.Skip TLDs and sub-TLDs that have wildcard DNS.)rrNcCsH||_|j|_d|_tj|_x"t|jD]}|||j |<q.WdS)NDNS) sf tempStorageresults__dataSource__ threadingLocklocklistkeysopts)selfsfcuserOptsoptr8/var/www/spiderfoot.crq.systems/modules/sfp_tldsearch.pysetup+s   zsfp_tldsearch.setupcCsdgS)N INTERNET_NAMEr)rrrr watchedEvents5szsfp_tldsearch.watchedEventscCsdgS)N SIMILARDOMAINr)rrrrproducedEvents;szsfp_tldsearch.producedEventscCstjj}d|_d|_t|_|jjdddkr>|jdg|_ |jdrX|j j |rXdSyN|j j |}|s|j d|j|<WdQRXn|j d|j|<WdQRXWn>tk r}z"|j d|j|<WdQRXWYdd}~XnXdS)N _dnsserverrFT)dnsresolverResolvertimeoutlifetimersearchrget nameserversr checkDnsWildcard resolveHostr tldResults BaseException)rtargettldr$addrserrrtryTld>s$  zsfp_tldsearch.tryTldc Cst|_d}d}g}|jjdt|x\|D]T}|\}}dttjjdd} |jt j | |j ||fd||j |d7}q.Wx@|rd} x t j D]} | jjdrd} qW| sd}tjd qWx.|jD]$} |j| r| |jkr|j|| qWdS) NTrz Spawning threads to check TLDs: Zthread_sfp_tldsearch_iɚ;)namer/argsr Fg?)dictr-r infostrrandom SystemRandomrandintappendrThreadr3start enumerater4 startswithtimesleepr sendEvent) rtldList sourceEventrunningitpairdomainr0tnfoundrtresrrr tryTldWrapperUs,     zsfp_tldsearch.tryTldWrappercCs|jjd|d|j|<|jdr||jr0dS|jjd||jd|jddd}|ddk rtd ||j|}|j|ntd ||j|}|j|dS) Nz$Found a TLD with the target's name: Trzhttp:// _fetchtimeout _useragent)r& useragentnoLogcontentr) r r7r r checkForStopfetchUrlr__name__notifyListeners)rsourceresultZ pageContentevtrrrrCus     zsfp_tldsearch.sendEventc Cs`|j}|j}|j}||jkr dSd|j|<|jj||jd}|jjd|d|t}||jkrjdSd|j|<x|jdD]}t |t krt |j dd}n|j }|j dst |dkrq|j d s|j d s|j d rq|jd rq|d |}|jrdSt ||jdkr,|j||gq|j||t}qWt |dkr\|j||dS)NT _internettldszKeyword extracted from z: ignore)errorsz//r!*z..z.arpa.r ) eventTypemoduledatar r domainKeywordrdebugrtyper8stripr@lenendswithrUr<rO) revent eventName srcModuleName eventDatakeywordZ targetListr0Z tryDomainrrr handleEvents@           zsfp_tldsearch.handleEvent)rW __module__ __qualname____doc__roptdescsr r6r-rrrrr3rOrCrprrrrrs   r) r9rrA dns.resolverr#sflibrrrrrrrrs