There are two types of filters – user filters and pipeline filters. A user filter will give the user the chance to interact with the information in the pipe, effectively stopping the flow until the user has decided what to keep in the pipe and what to delete. A pipeline filter (or just filter) will filter the information based on certain parameters and requires no user interaction.
To stop the flow and send the output to the user for manual filtering, simple insert the command ‘userFilter()’. Let’s use it in an example. We’ll use the same pipeline as before, but this time give the user the option of editing the MX and NS records before passing them along to the sharedMX/NS transforms. The script thus looks like this:
When running this inside of Maltego it looks as follows:
In this case we’re telling Maltego that we should not pass the MarkMonitor entities onto the next transform. Once this user filter has been completed (the user clicked on Continue), the pipeline flows again.
Pipeline filters stack next to each other. Each line in a filter makes the filter more specific. Consider the following (slightly useless) script:
//filter just person entities
The pipeline for this looks as follows:
Note that the named entity recognition(NER) transform will return three types of entities – Person, Location and Phrase (which is the organization / company). To get the proper name for the entity you can look in the detail view:
If we needed to do something with the other types in the pipe (e.g. the phrases) the pipeline should look like this:
The code for this looks as follows:
//filter just person names
//filter on phrase
The following can be used to filter in pipelines:
Type. Use type()
Number of incoming links. Use incoming()
Number of outgoing links. Use outgoing()
Value of the entity. Use value()
Age of the entity – when used with perpetual machines. Use age()
Property of an entity. Use property(propertyname,value)
When working with strings – for instance in value or property value filter the following applies:
//only matches "frikkie"
value(like: "frikkie", ignoreCase:true)
//matches "frikkiesbeer", "Its me Frikkie" and "befrikkied"
//matches all internal IPs
property("ipaddress.internal", like: "true", ignoreCase:true)
//matches all internal IPs but uses a like query
When working with numbers – for incoming, outgoing, age etc. the following applies:
//matches exactly 2 outgoing links
//matches if the entity has any incoming links
//matches if the entity 3 or 4 links (in or out)
//matches nodes older than 400 seconds
In many cases you want to collect nodes from all over the graph and not just from the current location in the in the pipeline. To do this you can add the argument ‘scope:"global"’ to filter any entity on the graph. Consider the following script:
When you run this machine on any node this script will simply prune leaf nodes on a graph. Leaf nodes have no outgoing links and just one incoming link – which is precisely what the filter combination does. Another way of writing the filter would be:
Here we are simply reversing the order of the filters but it ends up to be the same thing.
Keep in mind that adding the global scope to your filter will access the ENTIRE graph everything. Therefore the second call will basically remove the first filter - this would be wrong…