At first part of the post, i wrote that some tools use
“whois” client to query RIR databases.
So it is possible to use a “whois cli tool” to gather the necessary data of
your interest.
Whois
You can find whois flags for RIPE
on http://www.ripe.net/data-tools/support/documentation/queries-ref-card to elaborate the whois queries. Lets pull routes of an AS34984 in Linux
command Line:
root@test:~# whois -h whois.ripe.net -i or as34984 | grep route: route: 151.250.0.0/16 …output omitted
In order to get route data as a variable, to play with, you
still need to need to parse the related part of the data from the output. A better approach, which i
generally prefer , is to use a native library, so that your script won’t depend
on a tool, as a result it would be executable both on Windows, or or Linux/Mac. For a native client library check https://pypi.python.org/pypi/WhoisClient
Restful Web Services API
Even with a native library, you will still need to parse the output data. There is a better way, which is getting the data in a format like XML, YAML, JSON etc so that it is easier to get related parts systematically. Fortunately RIPE/ARIN has a RESTFul Web Services API which is a REST interface to their WhoIS Database which are invoked via http requests. Here are the documentation links:
- RIPE: https://labs.ripe.net/ripe-database/database-api/api-documentation https://labs.ripe.net/ripe-database/database-api/api-introduction
For ARIN documentation check:
- ARIN: https://www.arin.net/resources/whoisrws/whois_api.html#whoisrws https://www.nanog.org/meetings/nanog48/presentations/Wednesday/Kosters_Update_N48.pdf
Querying RIPE Web API and Processing XML Input With Python
By looking at the documentation, it can be seen that for AS5400,
which is BritishTelecom, url
http://rest.db.ripe.net/search.xml?query-string=as5400&inverse-attribute=origin
could be used to get all routes that has
an origin of AS5400. The ony thing to get prefixes for another AS is to change
the “as5400” value in the url itelf. If you open the url in your web browser
you will see an xml output like below:
So if we could open this url in python, and get the output, then parse it as xml, everything will be ok. To do so, lets examine the code i wrote, part by part:
We determined how to create a URL to pull prefixes that is originated from an AS. Creating an AS variable is logical so that in further usages you can change the AS number or get the value as an input.
As url is ready we need to open the url and assign the output to a string variable:
Then we need to parse XML from string into an element:
Last part is to get the prefix values from the xml output, which we turned into an element by using xml.etree.ElementTree. First we need to determine the hierarchy so that we can write a path argument. Having a look at the xml output, the hierarchy should be:
To write such a path argument, you may wanted to have a look at https://docs.python.org/2/library/xml.etree.elementtree.html#elementtree-xpath. Here is the code:
Our job is still not done yet, cause inside the tree we created, which is "interested" variable, we need to take the values of "value": Here is the last piece of the code:
To download the script, click here, or copy-paste the full code below:
Examining the code:
At first part of the code we need to import libraries, which are: urllib, urllib2 and xml.etree.ElementTree (you may also use lxml instead of xml.etree.ElementTree)import urllib,urllib2
try:
import xml.etree.cElementTree as ET
except ImportError:
import xml.etree.ElementTree as ET
We determined how to create a URL to pull prefixes that is originated from an AS. Creating an AS variable is logical so that in further usages you can change the AS number or get the value as an input.
as_to_pull_prefixes = "AS5400" url = "http://rest.db.ripe.net/search.xml?query-string=%s&inverse-attribute=origin" % as_to_pull_prefixes
As url is ready we need to open the url and assign the output to a string variable:
fp = urllib2.urlopen (url) response = fp.read()
Then we need to parse XML from string into an element:
tree = ET.fromstring(response) fp.close()
Last part is to get the prefix values from the xml output, which we turned into an element by using xml.etree.ElementTree. First we need to determine the hierarchy so that we can write a path argument. Having a look at the xml output, the hierarchy should be:
- inside <objects>
- inside <object> that has a type value of "route" (for ipv4 prefixes)
- inside <primary-key>
- every <attribute> which has a name value of "route"
To write such a path argument, you may wanted to have a look at https://docs.python.org/2/library/xml.etree.elementtree.html#elementtree-xpath. Here is the code:
interested = tree.findall("./objects/object[@type='route']/primary-key/attribute[@name='route']")
Our job is still not done yet, cause inside the tree we created, which is "interested" variable, we need to take the values of "value": Here is the last piece of the code:
for child in interested:
print child.get('value')
Trying It Out
Here is the output when i executed the script:To download the script, click here, or copy-paste the full code below:
import urllib,urllib2
try:
import xml.etree.cElementTree as ET
except ImportError:
import xml.etree.ElementTree as ET
###
###Variables which changes per request
as_to_pull_prefixes = "AS5400"
url = "http://rest.db.ripe.net/search.xml?query-string=%s&inverse-attribute=origin" % as_to_pull_prefixes
###
###Pull Info From IRR(RIPE)and assign it to a variable as string
fp = urllib2.urlopen (url)
response = fp.read()
###
###parse xml from from string into an element
tree = ET.fromstring(response)
fp.close()
###
###get interested data from element
interested = tree.findall("./objects/object[@type='route']/primary-key/attribute[@name='route']")
for child in interested:
print child.get('value')
###############
##for more info on how to select interested data from xml
##https://docs.python.org/2/library/xml.etree.elementtree.html#elementtree-xpath
###############
Posts
