Maltego

TRX
Writing Python transforms (for use with the TDS)
RT

2013/04

TableofContents
Introduction why use the TDS? ...................................................................................................................................... 4
What is TRX? ......................................................................................................................................................................... 5
Architecture of the TDS ......................................................................................................................................................... 5
Developing transforms for the TDS ................................................................................................................................. 6
Publically available TDS ................................................................................................................................................... 6
Internal/private TDS ......................................................................................................................................................... 6
The future of the TDS ............................................................................................................................................................. 6
Functionality of the NTDS ............................................................................................................................................... 6
Pricing model of the NTDS .............................................................................................................................................. 7
Development environments ............................................................................................................................................... 7
PHP ........................................................................................................................................................................................... 7
Python ..................................................................................................................................................................................... 7
Prepping a server for use with the TDS (Python) ...................................................................................................... 7
Install Apache2 .................................................................................................................................................................... 7
Install mod_wsgi .................................................................................................................................................................. 7
Install bottle .......................................................................................................................................................................... 7
Copy & extract TRX files ................................................................................................................................................... 8
Edit Apache configuration ............................................................................................................................................... 8
Install TRX files .................................................................................................................................................................... 8
Restart Apache2 .................................................................................................................................................................. 8
Setting up the TDS to use the example transform (DNS2IP) ................................................................................. 9
Setting up a seed ................................................................................................................................................................. 9
Setting up the transform ................................................................................................................................................ 10
Setting up Maltego to use the sample transform ................................................................................................. 11
Under the hood ....................................................................................................................................................................... 15
Debugging transforms ......................................................................................................................................................... 16
Changing to the bottle server .................................................................................................................................. 16
Deploying transforms with Apache ...................................................................................................................... 17
Multiple seeds ............................................................................................................................................................... 17
Components of TRX .............................................................................................................................................................. 17
debugTRX_Server.py ....................................................................................................................................................... 17
Transform library (e.g. DNSTRANSFORMS.py) .................................................................................................... 18
TRX

Page2

Maltego library (maltego.py) ....................................................................................................................................... 19

Using TRX .................................................................................................................................................................................. 19
Basic use ............................................................................................................................................................................... 19
Using Display Info ............................................................................................................................................................. 22
Link properties, bookmarks and notes .................................................................................................................... 25
Entity properties ............................................................................................................................................................... 27
Static vs. dynamic properties .................................................................................................................................. 27
A quick note on entity design .................................................................................................................................. 27
Calculated properties ................................................................................................................................................. 28
Property types ............................................................................................................................................................... 28
Reading properties ........................................................................................................................................................... 28
Adding dynamic properties .......................................................................................................................................... 29
Using transform settings ............................................................................................................................................... 32
Introduction ................................................................................................................................................................... 32
Transform setting types ............................................................................................................................................ 35
Using transform settings in TRX ............................................................................................................................ 35
Conclusion ........................................................................................................................................................................... 39
Entity reference ...................................................................................................................................................................... 40
V2/V3 Confusion ............................................................................................................................................................... 40
Calculated properties ...................................................................................................................................................... 40
Inheritance .......................................................................................................................................................................... 41
Why are we stuck with V2 properties? .................................................................................................................... 41
Entity reference guide .................................................................................................................................................... 42
The TRX Maltego.py API................................................................................................................................................. 47
MaltegoMsg class .............................................................................................................................................................. 47
MaltegoTransform class ................................................................................................................................................. 47
MaltegoEntity class .......................................................................................................................................................... 48

TRX

Page3

IntroductionwhyusetheTDS?
There are several ways to add custom transforms to Maltego. One option is to use local transforms.
Local transforms are an easy way to get going but has several drawbacks in the long term:
1. Code base needs to be installed on local machine.
a. In many cases this means installing the development environment on local machine
(think Python, PHP, PERL).
b. This makes it difficult to install on the client PC and means more moving parts.
c. Every transform needs to be manually registered on the Maltego GUI or Maltego
configuration files needs to be patched. While this is feasible it might not be
backward compatible with future releases of Maltego.
d. The code needs to be maintained for every installation of Maltego. A change in the
transform code means it has to be reinstalled on each Maltego client..
2. Limited functionality / access to Maltego features. The following features are not exposed to
local transforms:
a. Slider value the user cannot limit how many results are returned.
b. Transform settings cannot access any transform settings. Transform settings are
the popups asking the user to configure a transform prior to running it.
c. Notes / link labels / bookmarking. Access to these fields will likely be removed from
local transforms in future.
d. Meta data such as disclaimers etc.
3. When transforms depend on 3rd party software these need to be installed and maintained
across all clients. It is not always feasible to install this type of software on each type of
platform (e.g. if their Operating Systems are different).
4. No access control or auditing. Future releases of the TDS will include access control and
auditing.
5. Transforms such as SocialNet (from Packetninjas) are provided as a commercial product
as pay per transform. With local transforms it's more difficult for transform writers to offer
their transforms as a service.
6. Hard to control intellectual property. As transforms run locally on the client it becomes
difficult to protect the code against reverse engineering.
7. Paterva currently has no plans to provide future enhancements.
Most of these limitations can be somehow bypassed but these shortcuts are clunky, hardly
maintainable and not elegant at all.
Having said that local transforms have the advantage that:
1. They are simple to write in any language.
2. They can interact with software that requires the use of a GUI. Think here of Skype thats
difficult to use headless.
3. Is infrastructureless it can run straight off your workstation.
4. Data does not travel over the wire everything is contained in a single workstation.

TRX

Page4

Local transforms are a good start for proof of concept code, rapid development or where
interaction with GUI based software is required but in enterprise models or more serious
deployments it does not scale well or provide the necessary control mechanisms. These limitations
of local transforms are well known to Paterva. The more elegant solution to the problem would be
to centralize the transforms on a server and provide access to run them remotely, similar to
running the builtin transforms. Such a solution has been implemented using the Transform
Distribution Server (TDS).

WhatisTRX?
TRX is a framework (although this perhaps a bit presumptuous) that facilitates coding Python
transforms for Maltego using the TDS. It assumes the transform writer has access to of a web facing
Ubuntu server. It uses Apache with mod_WSGI, Bottle and Python.

ArchitectureoftheTDS
The following diagram shows how the Maltego clients, the transforms themselves and the TDS fit
together:

TRX

Page5

The TDS acts like a proxy for transforms. It hides several of the complexities of seed management,
entity metadata, transform settings, transform discovery etc. from the transform writer and allows
him/her to concentrate on developing the actual transform. The transform code is hosted on the
end users infrastructure and is served from a web server.
Configuration of transforms, seeds, transform settings and metadata is performed via a web
interface on the TDS.

DevelopingtransformsfortheTDS
There are currently two ways to develop transforms for the TDS.

PublicallyavailableTDS
This is a TDS that is located on the Internet and is free for all to use. Its a convenient way to
immediately start writing transforms. Since this server is located on Patervas infrastructure data
will be flowing from the Maltego GUI to this server and finally to your transform code hosted on a
web server of your choice. Note that your transforms need to be hosted on an Internet facing
server. This is not the preferred way to deploy transforms in a production environment (especially
if the transform content is sensitive) but it allows developers to get a feel for how the system works.
The public TDS requires registration and transforms and seeds are defined per user. The server
interface lives at https://cetas.paterva.com/TDS/

Internal/privateTDS
The internal TDS functions in the same way as the public TDS but can be hosted on the endusers
private infrastructure. It is delivered as Virtual Machine (VMWare image) and is commercial.
Several high profile companies use the TDS in its current format.
Clients with a valid license for the TDS will automatically be upgraded to the NTDS (see next
section).

ThefutureoftheTDS
The TDS was first released in 2010 and has seen very little changes since then. Initially Paterva was
reluctant to sell the TDS to clients but since mid 2012 have lifted this restriction. Several high
profile clients now use the TDS to centralize and distribute their custom enterprise wide
transforms. This renewed attention to the TDS caused us to rethink the future of this server.

FunctionalityoftheNTDS
The following key areas will be developed for the NTDS (New TDS):
1. Distribution of elements including:
a. Transforms (currently the only functionality of the TDS)
b. Entity definitions
c. Transform sets
TRX

Page6

d. Machine scripts
e. Viewlets
2. Access control and auditing
3. Transform stats, system health
4. Future proof for implementation of protocol 3.

PricingmodeloftheNTDS
Currently the TDS (when used as a private server) is sold at a fixed price but with unlimited amount
of transforms and users. This all or nothing model has proved problematic for small enterprises
that wish to share a handful of transforms amongst a small number of users. In future the NTDS
pricing model will be based on users and transforms and should be priced more attractively for
smaller enterprises.
The public TDS will also be upgraded to the new NTDS and will remain free for use.

Developmentenvironments
PHP
An extensive PHP library is provided for the TDS. The library and documentation can be found here:
https://www.paterva.com/web6/documentation/developertds.php#7

Python
The rest of this document focuses around the Python development environment.
In the past Paterva only provided a CGI based Python library. This is easy to work with but has
obvious limitations for every transform that runs an instance of Python was started. This can
easily drain resources on the host and is not very efficient. The more efficient method is using
WSGI.

PreppingaserverforusewiththeTDS(Python)
We recommend using Ubuntu Server with Apache2, Python 2.6, mod_wsgi and bottle. Assuming a
stock standard Ubuntu server with Python installed the following is a recipe for quickly building
your environment:

InstallApache2
sudo apt-get update
sudo apt-get install apache2

Installmod_wsgi
sudo apt-get install libapache2-mod-wsgi

Installbottle
sudo apt-get install python-setuptools
TRX

Page7

sudo easy_install bottle

Copy&extractTRXfiles
cd /tmp
wget http://www.paterva.com/TRX_Ubuntu.tgz
tar -xvzf TRX_Ubuntu.tgz

EditApacheconfiguration
Edit the file /etc/apache2/ports.conf and add the line:
Listen 9001
Add it just below the line that reads Listen 80 so the file looks like so:

NameVirtualHost *:80
Listen 80
Listen 9001

Copy the TRX Apache configuration file:

sudo cp /tmp/etc.apache2.sites-available/TRX /etc/apache2/sitesavailable/
Now enable the site:
sudo a2ensite TRX

InstallTRXfiles
sudo mkdir /var/www/TRX
cd /var/www/TRX
sudo cp /tmp/var.www.TRX.tgz .
sudo tar -xvzf var.www.TRX.tgz
An example transform DNS2IP is provided (its a function defined in DNSTRANSFORMS.py, but
more about that later). In the next section we will see how to configure the TDS to use this
transform.

RestartApache2
sudo /etc/init.d/apache2 restart
Feel free to inspect the configuration of the TRX site defined in /etc/apache2/sitesavailable and customize this to your liking. The configuration will route all traffic on port 9001
to the WSGI script TRX.wsgi located in /var/www/TRX.

TRX

Page8

SettinguptheTDStousetheexampletransform(DNS2IP)
Next we are going to configure the TDS to use the sample DNS2IP transform. Start by browsing to
the TDS. If you are using an internal/private TDS it will ask you for your SSL certificate provided by
Paterva. On the server portal youll find instructions on how to install the supplied client certificate
and use it.
If you are using the public TDS server you need to first log in with the user/password used at
registration (and complete that horrible reCAPTCHA).
Once authenticated to the TDS youll see three sections in the interface:

Settingupaseed
A seed is really just a URL that points to a group of transforms. Its this URL that Maltego will use to
load all of the transforms contained in the seed into its internal configuration. Thus, the first step
we want to do is create a new seed (container). Once weve done that well create a new transform
and put the transform inside that seed.
Click on Seeds. Youll see a list of seeds (if you have any). Click on the Add Seed button. You need
to give a seed name, a seed URL and select which transforms you want to add into the seed. You can
complete the form as follows:

When youre done, click on the Add Seed button.

TRX

Page9

Now our seed is configured. Its empty there are no transforms in the seed, but thats OK well
populate it in a bit.

Settingupthetransform
You can easily navigate around with the bread > crumb at the top of the screen:

Click on the very first item it will navigate to the root of the TDS.
Click on Transforms on the internal/private TDS you see a screen with a couple of PHP sample
transforms. On the public TDS there wont be any transforms configured (unless youve added them
there of course!)

We want to add a transform, so click on Add Transform. You are now going to configure the
transform. Here is a section of that webpage:

TRX

Page10

Change the fields as follows:

Transform name:

DNS2IP

Transform URL:

http://<Your_DevServer_Here>:9001/DNS2IP

Input Entity:

maltego.DNSName (from the dropdown)

Disclaimer:

Leave empty unless you feel like putting something there

Description:

My first sample transform (or get creative)

Version:

Debug:

Checked

Transform settings:

Leave as is well get to this later

Seeds:

Click on MyTDSSeed to highlight.

If everything worked you should see Successfully Inserted 'DNS2IP' at the top of the screen:

We now successfully added a transform to the seed and can use this seed in Maltego.

SettingupMaltegotousethesampletransform
We now want to tell Maltego to discover transforms from our seed. Dont worry most of this is a
onetime configuration and once its all set up we never have to touch it again.
Were lazy and so we are going to copy and paste the seed from the TDS. Go to Seeds and right click
on the URL of the seed. Copy it to your clipboard:

TRX

Page11

Open Maltego. Inside of Maltego go to the Manage tab and click on the bottom of Discover
Transforms. Youll see a dropdown. Select Advanced:

The following screen should appear:

TRX

Page12


Hopefully yours would not be that blurry . In the Name section type MyTDS (it could be anything
you want) and in the URL section paste the URL from the clipboard. Click on the Add button and
youll see it appears in the list of Seeds below:

Now all thats left is to discover the seed. You may choose to uncheck your other seeds it wont do
you any harm, but it will ensure that you just discover from the TDS seed. Follow the wizard (next
>next).
Finally, youll see:
TRX

Page13


Dont worry about the 23 new entities were installed. Thats a legacy bug (actually its not really,
but thats an entire other chapter). Now you are ready to use the transform.
Open a new graph, drag a DNSName from the palette to the graph and right click. Voila your
transform shows up (complete with description):

The transform is fully functional and will resolve the DNS name to an IP address:

TRX

Page14


If you want to share this transform with anyone you can simply give them the URL and guide them
through the discovery process in the same way as which you did. And if you change the code on
the server the change will transparently happen on all clients (as the transform runs on the server).
You can now add more transforms to the seed. Maltego periodically checks for new transforms in
the seeds which mean you never need to touch the client side anymore. Joy!

Underthehood
Lets see what really happened here. These are the steps:
1) After you discovered from the TDS server Maltego knows that theres an additional
transform called DNS2IP available on the entity type Maltego.DNSName.
2) When you right click on a DNS Name it makes this transform visible
3) It sends the transform request to the TDS server
4) The TDS server knows that the transform really lives on
http://<Your_DevServer_Here>:9001/DNS2IP
5) The TDS makes a connection to port 9001 on the dev server and sends some XML in a POST
6) The Apache WSGI configuration on the dev server send all requests on port 9001 to a file
located in /var/www/TRX/TRX.wsgi
7) TRX.wsgi sees that its destined for /DNS2IP and routes it to the right piece of code
8) The Maltego Python library interprets the XML to something thats easy to work with in
Python and passes it along
9) A DNS lookup is done
10) The reply is wrapped back into XML by the library and the data flows back the way it came
in.
Seems like a mouthful. The actual transform code is quite simple and looks like this:

TRX

Page15

def trx_DNS2IP(m):
TRX = MaltegoTransform()
DNSName=None
try:
DNSName = socket.gethostbyname(m.Value)
TRX.addEntity("maltego.IPv4Address",DNSName)
except socket.error as msg:
TRX.addUIMessage("Problem:"+str(msg),UIM_PARTIAL)
return TRX.returnOutput()

Hardly rocket science.

Debuggingtransforms
Changingtothebottleserver
One of the painful realities of working with WSGI is that you need to restart the Apache server
every time you make a change to the code. There are some scripts on the Internet available to make
this easier but this method of coding is altogether unintuitive for most coders. Luckily theres a way
around it.
Bottle has its own web server built in and this server can be set to reload whenever a change in
code happens this goes for changes to the stub as well as any modules/libraries that it depends
on. Lets look a little closer.
Go to /var/www/TRX. In here youll see a file called debugTRX_Server.py. This file is a near
exact copy of TRX.wsgi but it can run as a standalone server. Youll see that weve uncommented
the bottle HTTP server and commented the WSGI server (at the end of the script). In this case its
listening on the IP 10.77.0.106 and port 9001. Before you can run this script you need to disable
the Apache server (because it is grabbing port 9001):
/etc/init.d/apache2 stop
Now you can start the (sub optimal, but great for debugging) server:
sudo python debugTRX_Server.py
It starts up as follows:
Bottle v0.11.6 server starting up (using WSGIRefServer())...
Listening on http://0.0.0.0:9001/
Hit Ctrl-C to quit.
Whats great about this server is that it automatically reloads every time you make a change to the
code. Better still, it verbosely complains about any mistake you might have made in your code.
Heres a screenshot of that happening:
TRX

Page16


In the example above we made a silly mistake in a transform by using m.slider and not
m.Slider (note the difference in case). The bottle server complained bitterly about it. We fixed
the code and as soon as we saved the file the server reloaded with the changes. (If you wondering
we had our server listen on 10.77.0.106 and not on 0.0.0.0.)
DeployingtransformswithApache
The main reason why we dont want to run the bottle server in production is because its not
optimized for heavy load and we dont want to have to struggle with startup scripts.
Once you are happy with your transforms you should stop the bottle server, simply uncomment the
first couple of lines, change the server to run as a WSGI and start Apache again:
/etc/init.d/apache2 start
Multipleseeds
Another way to do this is to run the bottle server on a different port. You might want to add another
seed (DEV?) and register the transforms on this port. Once you want to move it into production
you can simple change the port and insert it into the production seed.

ComponentsofTRX
There are basically three files that are of interest. All of them are located in /var/www/TRX/.

debugTRX_Server.py
This is the dispatcher (TRX.wsgi and debugTRXServer.py is essentially the same script the
former used with Apache, the latter when debugging). Lets look at the file in a little bit more detail:

TRX

Page17


Each transform has a small router stub in here that defines the URL of the transform and to which
function it routes to. The request.body.getvalue() reads the entire POST from the message.
This XML is sent to the function MaltegoMsg which is defined in the Maltego library. The
function returns an object which can be used to easily extract all the information from the XML.
This object is passed along to the actual transform. The transform does its work and returns XML
this is returned back to the TDS. Well look into the transform code in more detail shortly.
In the example weve included the library DNSTRANSFORMS. This is the transform code library
e.g. where the transform itself is defined. In the router for transform 1 (DNS2IP) case its called
trx_DNS2IP. Of course, as you go along you may choose to add more libraries.

Transformlibrary(e.g.DNSTRANSFORMS.py)
You can include as many libraries as you want. For our example we used DNSTRANSFORMS.py.
Lets take a look at whats really happening:

TRX

Page18

This is the code that actually performs the work of the transform. As input it gets an object that
contains all the details about the request this is called m in this case. Next, it creates a vessel that
will eventually contain the response here its called TRX. It then does the DNS lookup and adds
an entity of type maltego.IPv4Address with the value of the lookup to the vessel. It also adds
some UIMessages. Finally the transform returns the vessels XML to the router which in turn
sends it back to the TDS.

Maltegolibrary(maltego.py)
This is the library that shuffles XML to object and back. You are more than welcome to optimize,
tinker with this library should you feel inclined to do so. But keep in mind that should we change
the library youll need to redo any changes youve made to it.

UsingTRX
Basicuse
Lets look at a couple of examples. You might want to refer to the TRX reference guide at the end of
this document every now and again to follow we are using the library.
Lets say we want to create a transform that runs on a Phrase entity. It will assume that the phrase
is a number and it will return as many entities and just as a test well return these as AS number
entities.
Lets begin really simple:
Our stub looks like this (in TRX.wsgi or debugTRX_Server.py):
@route('/EnumAS', method='ANY')
def EnumAS():
if request.body.len>0:
return(trx_EnumAS(MaltegoMsg(request.body.getvalue())))

Our transform looks like this (in DNSTRANSFORMS.py too lazy to put in another library):
def trx_EnumAS(m):
TRX = MaltegoTransform()
howmany = int(m.Value)
for i in range(1,howmany+1):
TRX.addEntity('maltego.AS', str(i))
return TRX.returnOutput()

We call it EnumAS on the TDS and point it to a URL http://server:9001/EnumAS .
Remember that if you used Apache as a server you will need to restart the server to register the
changes (/etc/init.d/apache2 reload). If you used the bottle server and it is running you
shouldnt have to do a thing.
TRX

Page19

Here is the output:

This code is really nasty. What happens if we put the value 250000 in the graph? What if its not
really a number but a phrase like test? So, as a start, lets make sure we dont go over the limit set
by the user using the result slider:

We change the code to read the slider:
def trx_EnumAS(m):
TRX = MaltegoTransform()
howmany = int(m.Value)
if (howmany > m.Slider):
howmany=m.Slider
for i in range(1,howmany+1):
TRX.addEntity('maltego.AS', str(i))
return TRX.returnOutput()

Now we at least honor the users wishes on how many entities to return. Next, we want to make
sure its a number and if its not, return a nice message to the user saying he/she is not
understanding the transform. The code now changes to this:
def trx_EnumAS(m):
TRX = MaltegoTransform()

TRX

Page20

if (not m.Value.isdigit()):
TRX.addUIMessage('Sorry but ['+m.Value+'] is not a whole
number',UIM_PARTIAL)
return TRX.returnOutput()
#here we know we're good to go.
howmany = int(m.Value)
if (howmany > m.Slider):
howmany=m.Slider
for i in range(1,howmany+1):
TRX.addEntity('maltego.AS', str(i))
return TRX.returnOutput()

Right! When we now run our transform on an entity thats not a number we get the following:

It seems that everything is mostly sorted out, but theres one more thing. Youll notice that the
entities seem to be laid out on the graph in a random fashion. Maltego actually lays out entities
according to their weight (top left to bottom right). Since we didnt set the weight all the entities
have the same weight the default of 100. Lets see if we can fix this:
def trx_EnumAS(m):
TRX = MaltegoTransform()
if (not m.Value.isdigit()):
TRX.addUIMessage('Sorry but ['+m.Value+'] is not a whole
number', UIM_PARTIAL)
return TRX.returnOutput()
#here we know we're good to go.
howmany = int(m.Value)
if (howmany > m.Slider):
howmany=m.Slider
for i in range(1,howmany+1):
Ent = TRX.addEntity('maltego.AS', str(i))
Ent.setWeight(howmany-i)
return TRX.returnOutput()

TRX

Page21

Here you see that the addEntity method actually returns a MaltegoEntity object which we can
modify. In this case we use it to set the weight and now the graph looks as follows:

The first node (with value 1) has a weight of 19 and the last node (20) has a weight of 0. You can
see this in the Property View. The following screenshot shows the property view when the AS node
with value 1 was selected:

UsingDisplayInfo
Lets next assume we want to create some fancy HTML in the Detail View to go with our AS entities.
The addDisplayInformation method is used to do this. The content could be plain text or
HTML. Lets start with something really simple. We want to display the words This is number X on
each entity returned. We want this to be rendered in a label called AS Number. Here is how the
script looks now:
TRX

Page22

def trx_EnumAS(m):
TRX = MaltegoTransform()
if (not m.Value.isdigit()):
TRX.addUIMessage('Sorry but ['+m.Value+'] is not a whole
number',UIM_PARTIAL)
return TRX.returnOutput()
#here we know we're good to go.
howmany = int(m.Value)
if (howmany > m.Slider):
howmany=m.Slider
for i in range(1,howmany+1):
Ent = TRX.addEntity('maltego.AS', str(i))
Ent.setWeight(howmany-i)
Ent.addDisplayInformation("This is number "+str(i),"AS
Number")
return TRX.returnOutput()

When you run the transform youll see that each node rendered the label and content:

You can easily add more labels. Display information is merged when other transforms creates the
same entity instance. Maltego will expand the label by default, but Detail View labels can be
collapsed and expanded by the user and the client will remember the choice for future operations.

TRX

Page23

Lets see how it works with HTML. Lets assume we want to make a clickable link in HTML. We will
create the link so that it is dynamic and based on the node number. Consider the following code:
def trx_EnumAS(m):
TRX = MaltegoTransform()
if (not m.Value.isdigit()):
TRX.addUIMessage('Sorry but ['+m.Value+'] is not a whole
number',UIM_PARTIAL)
return TRX.returnOutput()
#here we know we're good to go.
howmany = int(m.Value)
if (howmany > m.Slider):
howmany=m.Slider
for i in range(1,howmany+1):
Ent = TRX.addEntity('maltego.AS', str(i))
Ent.setWeight(howmany-i)
Ent.addDisplayInformation("This is number "+str(i),"AS
Number")
Ent.addDisplayInformation('Click <a
href="https://www.ultratools.com/tools/asnInfoResult?domainName=AS'+st
r(i)+'"> here </a> to get more information about this AS','AS Info
Link')
return TRX.returnOutput()
Now each node has a link that will open the browser to a specific webpage:

TRX

Page24

Clicking on the links gives us:

It important to note that display information is never sent to the server (when you run subsequent
transforms on the node) its read only info and is kept in the client.

Linkproperties,bookmarksandnotes
You can set the link color, label, thickness and style really easily. Consider this code snippet (used in
the same transform). We are not going to discuss each method in detail because its rather obvious
from the code and the resultant graph:
if (i%2==0):
Ent.setLinkColor('0x00FF00')
TRX

Page25

Ent.setNote('Even')
Ent.setLinkLabel('Even link')
Ent.setLinkStyle(LINK_STYLE_NORMAL)
Ent.setLinkThickness(1)
Ent.setBookmark(BOOKMARK_COLOR_GREEN)
else:
Ent.setLinkColor('0xFF0000')
Ent.setNote('Odd')
Ent.setLinkLabel('Odd link')
Ent.setLinkStyle(LINK_STYLE_DASHED)
Ent.setLinkThickness(2)
Ent.setBookmark(BOOKMARK_COLOR_RED)

Although we recommend you keep it a little simpler this code illustrates the concept. The resultant
graph (ugly) looks like this (layout in organic mode else its REALLY messy):

The only thing to keep in mind is that entity notes stack on top of each other that means that
when a subsequent transform writes to a note it will append to the note rather than deleting the
previous note.

TRX

Page26

Entityproperties
Entities can have additional properties. For example a person might have a property called SSN.
Staticvs.dynamicproperties
When you design an entity youll see that you can add additional properties to the entity. These are
called static properties. It means that should a user drag an entity from the palette to the graph
these properties will be available for editing. As an example, here is the property view for Person:

Transforms can create new properties for entities they return. The transform simply adds a
property to the entity when its returned. These are called dynamic properties. If a transform
returns a property that had been defined as static in the entity definition it will store the value as a
static property, if its not defined in the entity definition it then becomes dynamic.
The only real difference between static and dynamic properties is that users cannot edit dynamic
properties on a fresh node from the palettesimply because the property does not exist yet a
transform needed to create it.
Aquicknoteonentitydesign
One of the most important questions to ask yourself when doing entity design is if data should be
stored in a property of an entity or if it should become a new entity type. Similarly you should
consider if some information should only be display information or stored as a property within an
entity.
Usually you can answer these questions as follows. If you can think of a transform that will
regularly use a property of an entity it would be better to make it a separate entity type. For
instance it would be better to have SSN as an entity type and have a Person to SSN transform than
to only have SSN as a property in the Person entity (you might still store the SSN as a property
TRX

Page27

within the Person entity, but it makes sense to have it as a separate entity too). Similarly if you
cannot decide if something should be a property or simply display info consider if another
transform would need to read that property to work properly or if its only shown to the user for
information (because display info is never passed back to the server). If its just useful as display
info dont bloat the entity by adding it as a property.
Calculatedproperties
In some cases properties are calculated from other properties. In the case of a person the property
FullName is created from FirstName and LastName. This does not mean you cannot set the
property, but you dont need to. For a full list of entity names, properties and calculated properties
refer to the Entity definition reference at the end of this document.
Propertytypes
The current version of the Python library only supports string types but subsequent releases might
extend this to include all the other types of properties that Maltego support. When dealing with
entities that have property types other than strings, make sure you format the string value properly
so that the Maltego client can parse the correct type from the string value.

Readingproperties
This is fairly straightforward. Consider the Website entity. It has a static property called ports with
a default of 80 (see Entity definition reference for a list of static entity properties). Inside of Maltego
this is really defined as a list of integers but well see how we deal with it in TRX.

Lets start a new transforms. Well call it Mangle, register it on the TDS (running on a website
entity) and in our TRX.wsgi stub.
Our transform code looks like this:
def trx_Mangle(m):
#define response
TRX = MaltegoTransform()
TRX.addUIMessage("Property value is:"+m.getProperty("ports"))
TRX

Page28

return TRX.returnOutput()
We use the getProperty method to get the propertys value. You will see that we do not return
any entities (thats 100% OK), we simply write a message to the output with the Output screen. On
a default website entity the output looks like this:

If we add a port to the list

and run the transform again we get:

In other words the list is returned as a comma delimited list.

Addingdynamicproperties
Lets see how we can write properties using TRX. Each property has

TRX

Displayname
Name
Value
Matching rule
merged

:
:
:
:

how its rendered in Maltego

the actual name used in code
he value of the property
if entities with the same value, type and property value should be

Page29

Lets assume we want to create a transform that runs on an IP address and generates a netblock. It
will always assume a class C netblock (255 IPs) but it will add a dynamic property to the netblock to
indicate if its a private netblock or not. Well do this with a dynamic property called private and it
will be set to yes or no.
We start by building a stub (in TRX.wsgi for Apache deployment or debugTRX_Server.py
when debugging) and registering our transform on the TDS. We begin our transform (without input
validation its not the point) like this:
def trx_NetblocksRUs(m):
TRX=MaltegoTransform()
start=m.Value[0:m.Value.rfind('.')]
netblock=start+".0-"+start+".255"
Ent = TRX.addEntity('maltego.Netblock')
Ent.setValue(netblock)
return TRX.returnOutput()
Note that at this point we havent added any dynamic properties to the netblock. We need to first
have something that checks if an IP address is private or not. A quick search on the Internet finds
this useful piece of code:
>>> from IPy import IP
>>> ip = IP('10.0.0.1')
>>> ip.iptype()
'PRIVATE'

This function relies on a Python module called IPy. On Ubuntu installing this is as simple as:
sudo apt-get install python-ipy
This seems perfect for our transform. We add a line importing this library to our
DNSTRANSFORMS.py:
from IPy import IP
And the transform now looks like this:
def trx_NetblocksRUs(m):
TRX=MaltegoTransform()
start=m.Value[0:m.Value.rfind('.')]
netblock=start+".0-"+start+".255"
Ent = TRX.addEntity('maltego.Netblock')
ip = IP(m.Value)
TRX

Page30

if (ip.iptype()=='PRIVATE'):
Ent.addProperty('private','Private network','strict','yes')
else:
Ent.addProperty('private','Private network','strict','no')
Ent.setValue(netblock)
return TRX.returnOutput()
Now running the transform on two different IP addresses we get:

Perfect! Of course we can create a transform that uses netblocks as input and simply outputs a
phrase public or private (this is not very useful, but hopefully will show how to use additional
properties).
def trx_PublicPrivate(m):
TRX=MaltegoTransform()
Ent = TRX.addEntity('maltego.Phrase')
if m.getProperty('private')=='yes':
Ent.setValue('Private')
else:
Ent.setValue('Public')
return TRX.returnOutput()

TRX

Page31

The resultant graph (when populated with some IPs>Netblock>Public/Private and graph set to
Bubble view block layout):

Usingtransformsettings
Introduction
Sometimes you want the user to be able to send you information about how you should run your
transform. These could be settings that could change every time the transform is executed, or could
be settings that the user wants to customize and have Maltego remember. We call these transform
inputs or settings.
Almost all of the standard transforms have settings most of the time users dont really bother to
change them because the defaults are working as expected. A good example of this can be seen in
the reverse DNS transform where the HTTP timeout to Serversniff and Robtex can be configured:

TRX

Page32


To get to the transform manager click on the Manage ribbon and go to Manage transforms. Any
transform setting can be configured to popup. This is useful when the user would want to change
the setting prior to the transform running. Such is the case with IP to Netblock (natural
boundaries). The user might choose to select class C (256) or sub class Cs (16,32,64) or perhaps
even class Bs (65535).
This transform setting looks as follows in the manager:

TRX

Page33


The small dialog icon next to the setting indicates that this transform will pop up a setting prior to
executing the transform:

The user might choose to fix the value for subsequent transforms by checking the Remember
these settings checkbox. In order to get the transform to pop up the setting dialog again the Popup
checkbox needs to be selected in the transform manager.

TRX

Page34

To quickly get to each transforms settings you can click on the small gear in the context menu:

Transformsettingtypes
The Maltego GUI client supports many different popup types (integer, dates, lists of strings and
integers) but the TDS currently only support the use of strings. This will be extended in the NTDS.
UsingtransformsettingsinTRX
Now that we know what transform settings are, lets see how to use them with custom transforms.
Because transform settings are prompted for prior to the transform running their definitions are
stored in the transform itself they are loaded when the transforms are discovered. This means that,
should you change transform settings, you need to rediscover from the seed so that their definitions
are updated. Luckily background discovery makes this really painless:

TRX

Page35

Transform settings are defined on the TDS and can be reused across any number of transforms.
The TDS only supports string based transform settings, but this might change in the near future.
Basically a transform setting requires a
Name

As used in the code

Display Dialog :

How the user is asked for it

Default value :

Initially setting will have this value (choose carefully)

Popup state :

Will it pop up?

Optional

Is the setting optional or mandatory?

The current specification says that a transform setting will only pop up if its mandatory and there
is no default value. This is something Paterva is aware of and will fix in the NTDS.
Lets go back to our initial transform (the one with the phrase as a number and ASes as results).
We want to add a transform setting that will obtain a number prior to the transform running and
only return AS numbers that are divisible by that setting (if that sounds silly, it is, but again
hopefully is shows the concept). First off we need to define a transform setting on the TDS. We go to
the root, click on Settings and click Add setting. We fill out the form as such:

Our variable will be called ISDIV. Once youve added the transform setting click on Add Transform
Setting. Navigate to transforms, click on the EnumAS transform (which we created at the start of
this document) and scroll down. At Transform Settings highlight the ISDIV item (you can shift
click to select multiple settings):

TRX

Page36


This tells the TDS that you wish to use the ISDIV transform setting in your transform. Transforms
could have multiple settings and the same settings can be used on multiple transforms.
Next lets look whats needed in the code. Our previous EnumAS ended making a graph that
looked like pea soup so well start with fresh code:
def trx_EnumAS(m):
TRX = MaltegoTransform()
#read the value
if (not m.Value.isdigit()):
TRX.addUIMessage('Sorry but ['+m.Value+'] is not a whole
number',UIM_PARTIAL)
return TRX.returnOutput()
#read the setting
isdiv = m.getTransformSetting('ISDIV')
if (not isdiv.isdigit()):
TRX.addUIMessage('Silly! We need a number',UIM_FATAL)
return TRX.returnOutput()

###here we know we're good to go.

howmany = int(m.Value);
# how many have accumulated?
accum=0;
for i in range(1,howmany+1):
if (i % int(isdiv) == 0):
Ent = TRX.addEntity('maltego.AS', str(i))
Ent.setWeight(howmany-i)
accum=accum+1;
if accum>=m.Slider:
break

TRX

Page37

return TRX.returnOutput()

Lets see what happens when we run the transform. First up we get a popup asking us some
questions:

In the code we do:
#read the setting
isdiv = m.getTransformSetting('ISDIV')
if (not isdiv.isdigit()):
TRX.addUIMessage('Silly! We need a number',UIM_FATAL)
return TRX.returnOutput()
This checks if the value is indeed a digit if its not we complain bitterly (with UIM_FATAL):

Its important to remember that the TDS only supports strings. As such we need to convert our
string to an integer (after checking that it really is one!) and make sure we give the right amount of
nodes back to the user.
TRX

Page38

The output of the transform is as expected:

Conclusion
The combination of all of these components makes Maltego very flexible and powerful in the hands
of a crafty developer. TRX makes it very easy to wield this type of power and the developer is
properly shielded from any of the Maltego internal complexities.

TRX

Page39

Entityreference
V2/V3Confusion
With the release of Maltego version 3 (around 2010) a new data model was developed. This was
done to ensure scalability of entities. Where Maltego v2 just knew about a Person we started to
understand that other users would want to perhaps create their own Person entity. Thus we
moved to maltego.Person. The same principle was applied to properties. Where we had
firstname in V2 we now understood that we should have person.firstname. Furthermore
the model was built in such a way that entities could have any number of properties but that there
would be a mapping between any of these properties and whats displayed underneath the entity in
the graph (the display value). A good example of this is the maltego.Document entity it makes
more sense to display the title of the document rather than the URL where the document is located.
As such the display value for a Document entity is the title. This can be easily seen on the Display
Settings when editing an entity within the Maltego client. Here the maltego.Document entitys
Display Settings is shown:

In the same way the value thats edited when the user clicks on the entity and the IconURL can be
mapped to any property of the entity.

Calculatedproperties
Another concept that was introduced in version 3 was the use of calculated properties. A persons
fullname for instance is calculated by the concatenation of the firstnames and the lastname.
This is exposed in the Maltego client:

TRX

Page40


The only entities that use calculated properties are

maltego.Person
maltego.Location
maltego.PhoneNumber

Inheritance
CaseFile offered many more entities than Maltego. In CaseFile you can have a Judge, Criminal and
Officer that are essentially all Persons. When importing a graph made in CaseFile into Maltego you
would want to be able to run the Person transforms on all of these but the early data model did not
support it.
We added the concept of inheritance for the standard Maltego installation this meant that the
MXRecord, NSRecord and Website entities were really just specialized DNSNames. The upside of it
is that one transform (DNSName 2 IPAddress) worked on all of them this saved a lot of transform
configuration. For example if you specify on the TDS that a transform will run on a DNSName it
will also run on all entities down the family tree MXRecord, Website and NSRecord.
At the top of the tree is maltego.Unknown. This means that if you configure a transform to run
on this base entity type it will be available when you right click on any entity.

WhyarewestuckwithV2properties?
The trouble with this new data model was that the server was still expecting the old property
names for incoming values and was still supplying the old property names as results. With many
different versions of Maltego in the wild changing the server to use the new properties would mean
that many clients would have a nonfunctional Maltego client.
TRX

Page41

As such a conversion layer was implemented that mapped the old property names internally to the
new names. It meant that when setting the value of a property it would internally be setting a
specific property of the entity. It also meant that when the GUI supplied entities to the server it
would convert the properties to the old names.
This worked fine until other developers started to code transforms. The Maltego GUI exposed the
internal property names and there was no way to know what these properties translated to when it
was supplied to the transforms. That is unless you followed the specification that was last
updated in 2009.
As such we provide here both the internal (V3) property names as well as their V2 equivalents.
When coding transforms it is best to stick with the V2 equivalents. It would be trivial to change the
server (CTAS) to use the new properties but it will almost certainly result in some clients stuck with
a nonfunctional GUI. It seems that these legacy properties are here to stay.
When working with your own custom entities this legacy problem is a nonissue of course. It only
comes into play when you want to leverage the transforms that are located on the CTAS then you
need to make sure you are mapping to the right V2 equivalent properties.

Entityreferenceguide

maltego.

Domain
PropertyName
fqdn
whois-info

Inheritsfrom:
maltego.Unknown
Type

Displayname

string
string

Domainname
WHOISinfo

V2equivalent
Value
whois

maltego.

DNSName
PropertyName
fqdn

Inheritsfrom:
maltego.Unknown
Type

Displayname

string

DNSName

V2equivalent
Value

maltego.

MXRecord
PropertyName
fqdn
TRX

Inheritsfrom:
Maltego.DNSName
Type

Displayname

string

MXRecord

V2equivalent
Value
Page42

mxrecord.priority

integer

Priority

mxrecord.priority

maltego.

Inheritsfrom:
Maltego.DNSName
Type

Displayname

string

MXRecord

V2equivalent
Value

NSRecord
PropertyName
fqdn

maltego.

IPv4Address
PropertyName
ipv4-address
ipaddress.internal

Inheritsfrom:
maltego.Unknown
Type

Displayname

string
boolean

IPAddress
Internal

V2equivalent
Value
ipaddress.internal

maltego.

Netblock
PropertyName
ipv4-range

Inheritsfrom:
maltego.Unknown
Type

Displayname

string

IPRange

V2equivalent
Value

maltego.

AS
PropertyName
as.number

Inheritsfrom:
maltego.Unknown
Type

Displayname

integer

ASNumber

V2equivalent
Value

maltego.

Website
PropertyName
fqdn
TRX

Inheritsfrom:
maltego.DNSName
Type

Displayname

string

Website

V2equivalent
Value
Page43

website.ssl-enabled
ports

boolean
int[]

SSLEnabled
Ports

website.ssl-enabled
ports

maltego.

Inheritsfrom:
maltego.Unknown
Type

Displayname

string
URL
string

ShortTitle
Value
Title

V2equivalent
Value
theurl
fulltitle

URL
PropertyName
short-title
url
title

maltego.

Phrase
PropertyName
text

Inheritsfrom:
maltego.Unknown
Type

Displayname

string

Text

V2equivalent
Value

maltego.

Document
PropertyName
url
Title
document.meta-data

Inheritsfrom:
maltego.Unknown
Type

Displayname

string
String
String

URL
Title
MetaData

V2equivalent
link
Value
metainfo

maltego.

Person
PropertyName
*
person.fullname
+
person.firstnames
+
person.lastname

TRX

Inheritsfrom:
maltego.Unknown
Type

Displayname

string
string
string

FullName
FirstNames
Surname

V2equivalent
Value
firstname
lastname

Page44


maltego.

EMailAddress
PropertyName
email

Inheritsfrom:
maltego.Unknown
Type

Displayname

string

EmailAddress

V2equivalent
Value

maltego.

Location
PropertyName
*
location.name
+
country
+
city
location.area
countrycode
longitude
latitude

Inheritsfrom:
maltego.Unknown
Type

Displayname

string
string
string
string
string
float
float

Name
Country
City
Area
CountryCode
Longitude
Latitude

V2equivalent
Value
country
city
area
countrysc
long
lat

maltego.

PhoneNumber
PropertyName
*
Phonenumber
+
phonenumber.countrycode
+
phonenumber.citycode
+
phonenumber.areacode
+
phonenumber.lastnumbers

Inheritsfrom:
maltego.Unknown
Type

Displayname

string
string
string
string
string

PhoneNumber
CountryCode
CityCode
AreaCode
LastDigits

V2equivalent
Value
countrycode
citycode
areacode
lastnumbers

maltego.

Alias
PropertyName
alias

Inheritsfrom:
maltego.Unknown
Type

Displayname

string

Alias

V2equivalent
Value

TRX

Page45

maltego.

Inheritsfrom:
maltego.Unknown

Image
PropertyName
description
url

Type

Displayname

string
URL

Description
URL

V2equivalent
Value
url

maltego.

Inheritsfrom:
maltego.Unknown

Twit
PropertyName
twit.name
id
author
author_uri
content
img_link
pubdate
title

Type

Displayname

string
string
string
string
string
string
string
string

Twit
TwitID
Author
AuthorURI
Content
ImageLink
Datepublished
Title

V2equivalent
value
id
author
author_uri
content
img_link
pubdate
title

maltego.affiliation.

Twitter
PropertyName
person.name
affiliation.network
affiliation.uid
affiliation.profile-url
twitter.number
twitter.screen-name
twitter.friendcount
person.fullname

Inheritsfrom:
maltego.Unknown
Type

Displayname

string
string
string
string
int
string
int
string

Name
Network
UID
ProfileURL
TwitterNumber
ScreenName
FriendCount
RealName

V2equivalent
value
network
uid
affiliation.profile-url
twitter.number
twitter.screen-name
twitter.friendcount
person.fullname

* Calculated properties
+ Used to calculate property

TRX

Page46

TheTRXMaltego.pyAPI
MaltegoMsgclass
This is used to read the Maltego request. It is passed along to each transform.
Member name
Value (String)
Weight (Integer)
Slider (Integer)
Type (String)
Properties (List)
TransformSettings (List)

Description
The value of the node as displayed on the graph. Note
that this is not necessarily the value you want work with
(see URL entity).
The weight of the node.
The sliders value e.g. how many results should be
returned.
The type of the input node. See entity definitions for
possible values.
A list of properties of the node. Name, value pairs as
strings
The settings for the transform. A list of name, value pairs
as strings.

The following methods are defined to read entity properties and transform settings. You can read it
straight out of the Property and TransformSettings list, but its nicer to use these functions
as they do some error checking:
Method name
getProperty
(String key)
getTransformSetting
(String key)

Description
Returns
Returns the value of the key, Value
None if not defined
Returns the value of the key, Value
None if not defined

MaltegoTransformclass
This is used to construct the reply to the TDS. All values are strings.
Method name
addEntity
([String Type,
String Value])
addUIMessage
(String msg,
Const type)

addException
(String msg)
TRX

Description
Adds an entity to the return vessel with type
Type and value Value. Note that these can be
set using functions in MaltegoEntity library.
Shows a message msg in the Maltego GUI.
Types could be
UIM_FATAL (pop up window)
UIM_PARTIAL (yellow)
UIM_INFORM (default)
UIM_DEBUG (light gray)
Throws a transform exception

Returns
MaltegoEntity
none

none

Page47

returnOutput()

Returns the XML of the vessel

none

MaltegoEntityclass
This is the object that defines a single entity within Maltego. An entity can be created using the
addEntity method in the MaltegoTransform class.
Method name
setType

(String Type)
setValue
(String Value)
setWeight
(Integer weight)
addDisplayInformation
(String Value,
[String Label])
addProperty
(String propertyname,
String displayname,
String matchingrule
String value)
setIconURL
(String URL)
setLinkColor
(String HexColor)
setLinkStyle
(Const Style)

setLinkThickness
(Integer thickness)
setLinkLabel
(String value)
setBookmark
(Const Color)

setNote
(String note)

TRX

Description
Sets the type of the entity see list of Entity definitions for possible
values
Sets the value of the entity
Sets the weight of the entity
Adds display information for label named Label. This field is
rendered as HTML within Maltego. Default label is Info.
Adds a property to the entity. Each property has a name, value and a
display name. The display name is how it will be represented within
Maltego. The matching rule determines how entities will be matched
and could be strict (default) or loose
If set it will change the appearance of the icon. The URL should point
to a PNG or JPG file. Maltego will size to fit but lots of large files will
drain resources.
Sets the color of the link to the node. Colors are in hex for example
0xff00ff
Sets the style of the link to the node. The following constants should
be used:
LINK_STYLE_NORMAL
LINK_STYLE_DASHED
LINK_STYLE_DOTTED
LINK_STYLE_DASHDOT
Sets the thickness of the link to the node. Value is in pixels.
Sets the label of the link to the node.
Sets the bookmark color of the node. Keep in mind that these are
chosen from a set number of colors. Use the following constants:
BOOKMARK_COLOR_NONE
BOOKMARK_COLOR_BLUE
BOOKMARK_COLOR_GREEN
BOOKMARK_COLOR_YELLOW
BOOKMARK_COLOR_ORANGE
BOOKMARK_COLOR_RED

Creates an annotation to the node with value note. If a subsequent
transform sets an annotation on the node it will appended to the
note.
Page48

TRX

Page49