The basics [Back to Top]

There are two types of filters – user filters and pipeline filters. A user filter will give the user the chance to interact with the information in the pipe, effectively stopping the flow until the user has decided what to keep in the pipe and what to delete. A pipeline filter (or just filter) will filter the information based on certain parameters and requires no user interaction.

To stop the flow and send the output to the user for manual filtering, simple insert the command ‘userFilter()’. Let’s use it in an example. We’ll use the same pipeline as before, but this time give the user the option of editing the MX and NS records before passing them along to the sharedMX/NS transforms. The script thus looks like this:

start{
        paths {
            path {
                run("paterva.v2.DomainToMXrecord_DNS")
                userFilter()
                run("paterva.v2.MXrecordToDomain_SharedMX")
            }
            path {
                run("paterva.v2.DomainToNSrecord_DNS")
                userFilter()
                run("paterva.v2.NSrecordToDomain_SharedNS")
            }
        }
    }



When running this inside of Maltego it looks as follows:



In this case we’re telling Maltego that we should not pass the MarkMonitor entities onto the next transform. Once this user filter has been completed (the user clicked on Continue), the pipeline flows again.
 

User filter options [Back to Top]

There are many ways to customize the user filter. The filter in the screenshot above was created as follows:

userFilter(title:"Remove hosted NSes",heading:"NS records",description:"Please remove the NS records that are hosted. We will see what's shared on the selected ones.",proceedButtonText:"Next>")


The following options are available:
  • title (string) – the title of the dialog
  • heading (string) – the heading of the first column
  • proceedButtonText (string) – the text on the proceed button
  • icon (string) – the name of an icon to display
  • removePromptText (string) – the text on the "remove unselected entities" checkbox
  • removePromptChecked (boolean) – the default value of said checkbox
  • showIncomingLinks (boolean) – display the incoming links column
  • showOutgoingLinks (boolean) – display the outgoing links column
  • selectEntities (boolean) – default selection state of entities
Consider the following screen shot:



The user filter for above is as follows:

userFilter(title:"TITLE",heading:"HEADING",description:"DESCRIBE",proceedButtonText:"PROCEED", removePromptText:"RPT", removePromptChecked:true, showIncomingLinks:true)

Remember that strings should be enclosed in quotes (") while Booleans are either true or false and are not enclosed in quotes.

 

Pipeline filter [Back to Top]

Pipeline filters stack next to each other. Each line in a filter makes the filter more specific. Consider the following (slightly useless) script:

start{
        run("paterva.v2.URLToPerson_NLP")
        
        //filter just person entities
        type("maltego.Person")
        run("paterva.v2.PersonToEmailAddress_SamePGP")
    }

The pipeline for this looks as follows:



Note that the named entity recognition(NER) transform will return three types of entities – Person, Location and Phrase (which is the organization / company). To get the proper name for the entity you can look in the detail view:



When we feed this transform the BlackHat Europe 2010 speaker lineup (http://www.blackhat.com/html/bh-eu-10/bh-eu-10-speakerbios.html) the result is as follows:



If we needed to do something with the other types in the pipe (e.g. the phrases) the pipeline should look like this:



The code for this looks as follows:

    start{
        run("paterva.v2.URLToPerson_NLP")
        
        paths{
            path{
                //filter just person names
                type("maltego.Person")
                run("paterva.v2.PersonToEmailAddress_SamePGP")
            }
            path {
                //filter on phrase
                type("maltego.Phrase")
                run("paterva.v2.PhraseToWebsite_SE")
            }
        }
    }



The following can be used to filter in pipelines:
  • Type. Use type()
  • Number of incoming links. Use incoming()
  • Number of outgoing links. Use outgoing()
  • Value of the entity. Use value()
  • Age of the entity – when used with perpetual machines. Use age()
  • Property of an entity. Use property(propertyname,value)

When working with strings – for instance in value or property value filter the following applies:


value("frikkie") 
//only matches "frikkie"

value(like: "frikkie", ignoreCase:true) 
//matches "frikkiesbeer", "Its me Frikkie" and "befrikkied"

property("ipaddress.internal", equalTo:true)
//matches all internal IPs

property("ipaddress.internal", like: "true", ignoreCase:true) 
//matches all internal IPs but uses a like query



When working with numbers – for incoming, outgoing, age etc. the following applies:


outgoing(2) 
//matches exactly 2 outgoing links

incoming(moreThan:0) 
//matches if the entity has any incoming links

outgoing(lessThan:5, moreThan2) 
//matches if the entity 3 or 4 links (in or out)

age(moreThan:400) 
//matches nodes older than 400 seconds

 

Making filters global [Back to Top]

In many cases you want to collect nodes from all over the graph and not just from the current location in the in the pipeline. To do this you can add the argument ‘scope:"global"’ to filter any entity on the graph. Consider the following script:


//prune leaf nodes
    start{
	incoming(1,scope:"global")
	outgoing(0)
	delete()
    }

When you run this machine on any node this script will simply prune leaf nodes on a graph. Leaf nodes have no outgoing links and just one incoming link – which is precisely what the filter combination does. Another way of writing the filter would be:


outgoing(0,scope:"global")
incoming(1)
delete()



Here we are simply reversing the order of the filters but it ends up to be the same thing.

Keep in mind that adding the global scope to your filter will access the ENTIRE graph everything. Therefore the second call will basically remove the first filter - this would be wrong…

outgoing(0,scope:"global")
incoming(1,scope:"global")
delete()

…as the second filter line also contains the global scope and negates the first filter.

Actions »