U @6^\@sHddlZddlZddlZddlZddlmZmZmZGdddeZ dS)N) SpiderFootSpiderFootPluginSpiderFootEventc@sreZdZdZddddZdddZd ZeZd Z efd d Z d d Z ddZ ddZ ddZddZddZd S) sfp_tldsearchz|TLD Search:Footprint:DNS:slow:Search all Internet TLDs for domains with the same name as the target (this can be very slow.)FT2) activeonly skipwildcards _maxthreadsz>Only report domains that have content (try to fetch the page)?z.Skip TLDs and sub-TLDs that have wildcard DNS.)rrNcCsD||_||_d|_t|_t|D]}|||j |<q,dS)NDNS) sf tempStorageresults__dataSource__ threadingLocklocklistkeysopts)selfsfcuserOptsoptr8/var/www/spiderfoot.crq.systems/modules/sfp_tldsearch.pysetup+s   zsfp_tldsearch.setupcCsdgS)N INTERNET_NAMErrrrr watchedEvents5szsfp_tldsearch.watchedEventscCsdgS)N SIMILARDOMAINrrrrrproducedEvents;szsfp_tldsearch.producedEventsc Cstj}d|_d|_t|_|jdddkr>|jdg|_ |jdrX|j |rXdSzN|j |}|s|j d|j|<W5QRXn|j d|j|<W5QRXWn>tk r}z |j d|j|<W5QRXW5d}~XYnXdS)N _dnsserverrFT)dnsresolverResolvertimeoutlifetimersearchrget nameserversr checkDnsWildcard resolveHostr tldResults BaseException)rtargettldr%addrserrrtryTld>s$  zsfp_tldsearch.tryTldc Cst|_d}d}g}|jdt||D]T}|\}}dttdd} |t j | |j ||fd|| |d7}q,|rd} t D]} | jdrd} q| sd}td q|jD]$} |j| r| |jkr||| qdS) NTrz Spawning threads to check TLDs: Zthread_sfp_tldsearch_iɚ;)namer0argsr!Fg?)dictr.r infostrrandom SystemRandomrandintappendrThreadr4start enumerater5 startswithtimesleepr sendEvent) rtldList sourceEventrunningitpairdomainr1tnfoundrtresrrr tryTldWrapperUs,      zsfp_tldsearch.tryTldWrappercCs|jd|d|j|<|jdr||r0dS|jjd||jd|jddd}|ddk rtd ||j|}||ntd ||j|}||dS) Nz$Found a TLD with the target's name: Trzhttp:// _fetchtimeout _useragent)r' useragentnoLogcontentr) r r8r r checkForStopfetchUrlr__name__notifyListeners)rsourceresultZ pageContentevtrrrrDus     zsfp_tldsearch.sendEventc Cs^|j}|j}|j}||jkr dSd|j|<|j||jd}|jd|d|t}||jkrjdSd|j|<|jdD]}t |t krt | dd}n| }| ds~t |dkrq~| d s~| d s~| d rq~|d rq~|d |}|rdSt ||jdkr,|||gq~|||t}q~t |dkrZ|||dS)NT _internettldszKeyword extracted from z: ignore)errorsz//r!*z..z.arpa.r ) eventTypemoduledatar r domainKeywordrdebugrtyper9striprAlenendswithrVr=rP) revent eventName srcModuleName eventDatakeywordZ targetListr1Z tryDomainrrr handleEvents@          zsfp_tldsearch.handleEvent)rX __module__ __qualname____doc__roptdescsr r7r.rrrr r4rPrDrqrrrrrs$  r) r:rrB dns.resolverr$sflibrrrrrrrrs