TinesBot – sharing community threat intelligence
TinesBot is an automation story built within the Tines automation platform which shares threat intelligence generated by the infosec community. TinesBot searches for new indicators in Pastebin, URLHaus and Malshare, the Cryptolaemus feed and other sources. Using Tines and tools like IOC Parser, we refang, deduplicate, tag, enrich and share data with VirusTotal, AbuseIPDB, Netcraft, Urlscan and other threat intel platforms automatically.
Why We Care
In Tines we recognize that our success is built upon the hard work of the wider security community. We and our customers rely on the community to understand new attack vectors, build better detections, and to track the latest TTPs of different groups. Fortunately the community is strong – many companies and individuals realise that by sharing indicators one organization sees another organization may be able to detect and prevent the same attack. Groups like Cryptolaeumus, volunteer or not-for-profit organizations like urlhaus, malshare and AbuseIPDB, individuals like James_inthe_box and ps66uk and free tools like urlscan or vxvault share indicators daily. Similarly, a huge number of infosec professionals on Twitter share indicators, either one by one, or in-bulk using sites like Alienvault OTX, pastebin or github.
Sharing these indicators is vital for the community but it is time-consuming and not everyone has the resources to add a comment to VirusTotal for every hash they see, or submit every malicious IP to AbuseIPDB.
Similarly, because these indicators are shared in numerous locations, a SOC analyst investigating an IOC during an attack might need to check multiple locations (or automate the enrichment using their SOAR tool or in their threat intel platform), to find out if a URL, hash, IP or Domain is malicious. Analysts may miss detections, and therefore vital clues to understanding and preventing the attack.
This process of both retrieving and sharing threat intelligence is time-consuming, manual and error-prone – in other words, exactly the sort of process that should be automated so at Tines we have built an automation story, TinesBot, to automate the process of sharing some of those IOCs to free threat intel platforms that everyone can use.
What does TinesBot do?
Firstly, TinesBot collects IOCs from multiple sources. Then it Extracts, Analyzes, Checks Against Known-Good, Tags and Deduplicates these IOCs. Lastly, it shares those IOCs, with attribution and context, to multiple other threat intel tools.
Collects IOCs from Multiple Sources
- Pulls the latest urls from urlhaus every 10 minutes
- Retrieves the latest malware samples shared on malshare every 30 minutes
- Scrapes pastebin for new pastes by a dozen known sharers of IOCs e.g. MalwareMessiagh or Jroosenor RDPSnitch every hour
- Pulls the Cryptolaemus RSS feed for new pastes about Emotet every hour
- Collects New and Updated Pulses by members of the COVID19 Cyber Threat Coalition in Alienvault OTX every 30 minutes
- Searches for new URLs in VXVault every hour
- Scrapes Twitter for pastes related to keywords (e.g. Ursnif/Emotet/Trickbot) multiple times a day
- Analyzes data from other sources (e.g. CyberMonitor, the Cyber Threat Coalition Feeds and elsewhere Threat Intel analysts share indicators) on an ad-hoc basis
- Collects suspicious URLs explicitly shared by users of phish.ly
(please note, we have no affiliation with any of the above organizations)
Extracts, Analyzes, Gets Reputation, Tags and Deduplicates IOCs
- Refangs & Extracts IOCs either using Tines or through free online tools like IOC Parser
- IP Address
- Tags the data with relevant tags where available (e.g. Emotet, Ursnif, Hancitor, Phishing, RigEK, RDP Abuse etc.)
- Removes Known-Good IOCs shared in pastes or in text files (e.g. app.any.run or microsoft.com)
- Deduplicates on the last 1,000 seen IOCs of each type
Shares the IOCs with other Threat Intel platforms:
- Adds new URLs and Domains to Virustotal (note, you can’t add a new hash to Virustotal, you must upload a file)
- Comments with context, attribution and tags on URLs, MD5s, IPs, Domains and SHA256s in Virustotal
- Scans URLs publicly in urlscan with tags
- Adds, Comments and Tags relevant IPs to AbuseIPDB
- Shares all new URLs with Netcraft
Shortly, if you haven’t already, you should start seeing new iocs tagged by TinesBot
We want to make this service better – adding another feed or threat intel sharing site is easy in Tines – it’s as simple as adding one agent. Retrieving the VXVault Feed, or any plaintext IOC feed, is just a single http request agent:
Similarly, pulling new IOCs using the Malshare API is just one http request agent:
An entire automation story pulling IOCs from Malshare and URLHaus in Tines looks similar to this:
If you are aware of another high-fidelity feed, or are occasionally posting indicators you think should be shared more widely then simply let us know email@example.com and we’ll add the feed to TinesBot. If you’re a researcher and post to pastebin, we’re also happy to sponsor/gift pro pastebin accounts for you to do so easily for example by using the Pastebin API.
Sharing to Additional Threat Intel Feeds
Sharing to an additional threat intel feed or location to share IOCs to is also simple. For example, to send to all suspicious urls to Netcraft we use a single http request agent:
We’d love to share these indicators more widely – for example, there are thousands of hashes which we have no place to comment on if the sample does not exist in Virustotal – if you want us to share them with you, or have any suggestions let us know by emailing firstname.lastname@example.org.
Believe an IOC is a false positive, or want us to add an IOC to our known-good list?
Send us an email and we will remove the comment (if possible) and add the IOC to our known-good.
Note, this process is obviously not a substitute for your own threat intel platform, TinesBot is simply trying to make the information that’s being shared more accessible to those who need it. You can read more about how to share threat intel with threat intel platforms here.