Continuing our deep-dive into new features included in the Tines Autumn 2019 release, we’re proud to detail information about our latest feature ‘Implode’.
If you asked us how to analyze all urls and attachments from an email as one, how to process requests in batches, or how best to process time-intensive tasks in parallel then this feature is for you! Does this email contain a bad link or attachment? are any of these IPs malicious? Are any of these users VIPs? Have all these vulnerabilities been patched? Have we removed all accounts for this user? With “Implode” you can now collapse previously exploded arrays, after processing and analyzing all elements, allowing you to make complex decisions more easily.
In this blog will explore how to use the new ‘Implode’ feature to analyze multiple elements from a single event. We’ll also examine how to process tasks in parallel and only emit an event when all paths finish processing.
A popular feature in Tines is the “Explode” mode of the Event Transformation Agent which “explodes” arrays into individual elements. Tines can then perform actions on each element of the array. Common use cases for exploding arrays include extracting, exploding and then analyzing URLs in an email; processing an array of users to onboard/offboard; exploding antivirus alerts or vulnerability scan alerts to analyze each alert individually etc.
The new Tines “Implode” feature is the opposite of Explode. Implode allows users take a collection of events that have been “exploded” and collapse them back together. For example – when Tines has completed analyzing every URL in an email, Tines can implode the analysis and check if all, any, or none are malicious; Tines can process all offboarding requests in a batch and return a status when all users have been offboarded successfully; Tines can analyze all results related to an individual endpoint before closing a ticket.
In order to track and then implode events, Tines generates a guid every time an array explodes, along with an index number and size parameter for the exploded array. This guid acts as a unique correlation ID to implode events related to same source event. Let’s illustrate this with an example.
Apple Suspicious Websites Example
To demonstrate how to Implode an Exploded array we’ll create an example that pulls Apple branded suspicious websites from urlscan. These websites will be analyzed in urlscan, and then Tines will Implode the analysis. Tines can then generate a result on whether any, all or none of the scans are malicious.
(Note, to import and run this story you’ll need to create a credential, urlscan_io, using an API Key from urlscan.io.)
The first agent simply retrieves a list of scans from the last 24 hours using a potentially malicious md5 hash. Urlscan returns an array of scan objects, which the next agent, ‘explode scans’ explodes into multiple ‘individual_scan’ events. You can see in an example event, outputted below, that each exploded scan now includes a ‘guid’, ‘index’ and ‘size’.
We can now use another agent in Tines to retrieve the analysis for every individual scan in the array. Each result retrieved has a flag for whether the scan is classified malicious: ‘true’ or ‘false’.
Once we retrieve the results of each individual scan, we can implode the events to see if any, all, or none of these scans are malicious.
To implode, we simply use another event transformation agent, this time in ‘Implode’ mode. The guid path and size path are simply paths to the values from the explode_scans agent:
When an event transformation agent in Implode mode receives a single event, Tines will store the “size_path” value. Tines waits until it receives the relevant number of events before emitting all events in a single array. In the picture above the array ‘size’ is ‘4’, so Tines will wait until it receives 4 separate events, then implode them and emit them all in one event. The ‘implode events’ agent will emit an array like below.
“Any”, “All” or “None”
We can now create a “For Loop” to loop through this array to see if all, any, none of the scans are classified as malicious. The end result will look like this:
You can download this example story here.
In addition to imploding arrays, the ‘Implode’ agent can also “implode” events that have been sent down two or more parallel paths (aka Tines) at the same time.
By generating a GUID manually and sending events down two paths in Tines, time-critical or time-intensive tasks can be performed in parallel without waiting for one to complete for the other to start. Using ‘Implode’ Tines will release an event only when all tasks are completed, for example:
- Analyze email Attachments and URLs from a suspicious email at the same time
- Offboard users in multiple different systems at the same time and return a result when the user is successfully offboarded in all systems
- Prompt two different users or teams for approval to take an action
- Analyze alerts in multiple different tools at the same time
Dual Approval Example
To illustrate a Story where an event is sent down parallel paths, let’s imagine a situation where the approval of both a user and manager is required to process a request. To approve or deny the request, we’ll send them both an email. We’ll only continue the process when both have clicked a prompt with their response.
We’ll use an external service to generate the GUID so we can kick off the story easily. When you import this story in Tines, click “Run” to send an example email to a manager and user. You should also edit the “Send Prompt to User” and “Send Prompt to Manager” recipients first.
These emails each contain two prompt links generated by a “Prompt” widget. When a user clicks one of these links, Tines releases an event which will be processed by a Trigger agent. The ‘deduplicate events’ agent removes duplicate replies so the manager or user can’t click twice and trigger an implode event.
In this example we want the ‘implode events’ agent to wait until a prompt response is received from both the manager and the user. Only when the manager and user have both replied should an event be emitted.
Implode Mode, Part Deux
The implode agent configuration therefore looks slightly different to reflect that we are waiting for a static number of events, 2 (i.e. the user’s reply and the manager’s reply), rather than a dynamic number in the example above (the number of elements in the exploded array). It is now hardcoded as a value ‘2’. If we needed three approvals (e.g. the User, their manager, and IT) we would hardcode this value to ‘3’.
Once the “Implode Events” agent emits an array, Tines can use the same “Any”, “All” or “None” agent configuration to decide whether to grant the permissions to the user. You can download this story here.