This is an example about how to use aws boto3 with yUML to visualize your aws cloud resources. Please feel free to 👉📱message my twilio bot +447479275693.I will come back to you shortly 😃.

In my last post, I have talked about how to play with aws boto3 api. During a usual dev-ish-ops day, sometimes I need to draw an aws network diagram. I have to log in to the console, click click click, and then copy-paste…hmmmm…It’s a little tiny bit boring and obviously, I want to automate it. Then I came across this site talking about UML. WOW! This could be fun 😃😃😃!!!

_“The Unified Modeling Language is a general-purpose, developmental, modeling language in the field of software engineering that is intended to provide a standard way to visualize the design of a system”_

Click here to view the SVG version of the UML graph.

Let’s start with a simple use case - aws network diagram

Let’s start with the simplest AWS VPC setup. One VPC with 2 public subnets and 2 private subnets across 2 availability zones. Let’s think about the key entities of this diagram. There should be vpc, subnet and route table. Once we got the key entities, time to think about the relationships between them. vpc_id link to subnet_id link to a route table . Okie dokie, time to write some code.

'''
first of all, let's get the details of the vpc
'''
import boto3
ec2_client = boto3.client('ec2')

def describevpcs():    
    response = ec2_client.describe_vpcs()
    result = []
    for re in response['Vpcs']:
        vpc = [re['VpcId'], re['CidrBlock'], re['IsDefault']]
        result.append(vpc)
    return result
Copy

Here you go! There is the vpc details:

>>> print(result)
[['vpc-0db8d6908430ced1d', '10.163.32.0/20', False], ['vpc-eb64708d', '172.31.0.0/16', True]]
Copy
'''
now let's get the subnets details
'''
def get_routes(vpc_id):
    response = response = ec2_client.describe_route_tables(
        Filters=[
            {
                'Name': 'vpc-id',
                'Values': [
                    vpc_id
                ]
            }
        ]
    )
    result = []
    for res in response['RouteTables']:
        for rel in res['Associations']:
            if rel['Main'] == False :
                rule = [[rel['SubnetId'], rel['RouteTableId']]]
                result.append(rule)
    return result
Copy

Here you go. Now you have the subnets details.

>>> print(result)
[['subnet-04cc7aa1106b95a87', '10.163.40.0/22', 'eu-west-1a', 'clojure-subnet-private-eu-west-1a'], ['subnet-0ef0383796b11b817', '10.163.36.0/22', 'eu-west-1b', 'clojure-subnet-public-eu-west-1b'], ['subnet-0c19b616d6e5468d6', '10.163.32.0/22', 'eu-west-1a', 'clojure-subnet-public-eu-west-1a'], ['subnet-032bba5091e3d0755', '10.163.44.0/22', 'eu-west-1b', 'clojure-subnet-private-eu-west-1b']]
>>>
Copy
'''
now let's get the route tables and route table association details
'''
def get_routes(vpc_id):
    response = response = ec2_client.describe_route_tables(
        Filters=[
            {
                'Name': 'vpc-id',
                'Values': [
                    vpc_id
                ]
            }
        ]
    )
    result = []
    for res in response['RouteTables']:
        for rel in res['Associations']:
            if rel['Main'] == False :
                rule = [[rel['SubnetId'], rel['RouteTableId']]]
                result.append(rule)
    return result
''''
returned results: 
[[['subnet-04cc7aa1106b95a87', 'rtb-0d16ec0e10307b3b9']], [['subnet-032bba5091e3d0755', 'rtb-0078be8c529367e44']], [['subnet-0ef0383796b11b817', 'rtb-0cf65b428d51c5032']], [['subnet-0c19b616d6e5468d6', 'rtb-0cf65b428d51c5032']]]
''''

def get_route_table(id):
    response = ec2_client.describe_route_tables(
        RouteTableIds=[
            id,
            ],
        )
    result = []
    for r in response['RouteTables']:
        route = r['Routes']
        result.append(route)
    return result
''''
returned result:
[[{'DestinationCidrBlock': '10.163.32.0/20', 'GatewayId': 'local', 'Origin': 'CreateRouteTable', 'State': 'active'}, {'DestinationCidrBlock': '0.0.0.0/0', 'NatGatewayId': 'nat-06f9b490c02cc9e13', 'Origin': 'CreateRoute', 'State': 'active'}]]
[[{'DestinationCidrBlock': '10.163.32.0/20', 'GatewayId': 'local', 'Origin': 'CreateRouteTable', 'State': 'active'}, {'DestinationCidrBlock': '0.0.0.0/0', 'NatGatewayId': 'nat-0f171865592318f10', 'Origin': 'CreateRoute', 'State': 'active'}]]
[[{'DestinationCidrBlock': '10.163.32.0/20', 'GatewayId': 'local', 'Origin': 'CreateRouteTable', 'State': 'active'}, {'DestinationCidrBlock': '0.0.0.0/0', 'GatewayId': 'igw-08671cfe01495863a', 'Origin': 'CreateRoute', 'State': 'active'}]]
[[{'DestinationCidrBlock': '10.163.32.0/20', 'GatewayId': 'local', 'Origin': 'CreateRouteTable', 'State': 'active'}, {'DestinationCidrBlock': '0.0.0.0/0', 'GatewayId': 'igw-08671cfe01495863a', 'Origin': 'CreateRoute', 'State': 'active'}]]
''''
Copy

Now we have all the data we needed.

# vpc
[['vpc-0db8d6908430ced1d', '10.163.32.0/20', False], ['vpc-eb64708d', '172.31.0.0/16', True]]

# subnets with description
[['subnet-04cc7aa1106b95a87', '10.163.40.0/22', 'eu-west-1a', 'clojure-subnet-private-eu-west-1a'], ['subnet-0ef0383796b11b817', '10.163.36.0/22', 'eu-west-1b', 'clojure-subnet-public-eu-west-1b'], ['subnet-0c19b616d6e5468d6', '10.163.32.0/22', 'eu-west-1a', 'clojure-subnet-public-eu-west-1a'], ['subnet-032bba5091e3d0755', '10.163.44.0/22', 'eu-west-1b', 'clojure-subnet-private-eu-west-1b']]

# subnets to route tables
[[['subnet-04cc7aa1106b95a87', 'rtb-0d16ec0e10307b3b9']], [['subnet-032bba5091e3d0755', 'rtb-0078be8c529367e44']], [['subnet-0ef0383796b11b817', 'rtb-0cf65b428d51c5032']], [['subnet-0c19b616d6e5468d6', 'rtb-0cf65b428d51c5032']]]

# route table id to route tables description
[[{'DestinationCidrBlock': '10.163.32.0/20', 'GatewayId': 'local', 'Origin': 'CreateRouteTable', 'State': 'active'}, {'DestinationCidrBlock': '0.0.0.0/0', 'NatGatewayId': 'nat-06f9b490c02cc9e13', 'Origin': 'CreateRoute', 'State': 'active'}]]
[[{'DestinationCidrBlock': '10.163.32.0/20', 'GatewayId': 'local', 'Origin': 'CreateRouteTable', 'State': 'active'}, {'DestinationCidrBlock': '0.0.0.0/0', 'NatGatewayId': 'nat-0f171865592318f10', 'Origin': 'CreateRoute', 'State': 'active'}]]
[[{'DestinationCidrBlock': '10.163.32.0/20', 'GatewayId': 'local', 'Origin': 'CreateRouteTable', 'State': 'active'}, {'DestinationCidrBlock': '0.0.0.0/0', 'GatewayId': 'igw-08671cfe01495863a', 'Origin': 'CreateRoute', 'State': 'active'}]]
[[{'DestinationCidrBlock': '10.163.32.0/20', 'GatewayId': 'local', 'Origin': 'CreateRouteTable', 'State': 'active'}, {'DestinationCidrBlock': '0.0.0.0/0', 'GatewayId': 'igw-08671cfe01495863a', 'Origin': 'CreateRoute', 'State': 'active'}]]
Copy

Now Let’s map all the data together and convert it to UML. something like this:

...
if 'NatGatewayId' in rt[0][1]:
                rt_node = '[{0}|{1}:{2};{3}:{4}]'.format(rr[1], rt[0][0]['DestinationCidrBlock'], rt[0][0]['GatewayId'],rt[0][1]['DestinationCidrBlock'], rt[0][1]['NatGatewayId'])
            else:
                rt_node = '[{0}|{1}:{2};{3}:{4}]'.format(rr[1], rt[0][0]['DestinationCidrBlock'], rt[0][0]['GatewayId'],rt[0][1]['DestinationCidrBlock'], rt[0][1]['GatewayId'])
            rule = '{0}-{1}'.format(sub_node, rt_node)
            result = rule+',' + result
    
    result_note = result+'[{0}{{bg:cornsilk}}]'.format(note)
Copy

Oh! Yeah! We have converted AWS Boto3 dictionary data type into UML Diagrams

There is the converted UML:

[subnet-0c19b616d6e5468d6|10.163.32.0/22;eu-west-1a;clojure-subnet-public-eu-west-1a]-[rtb-0cf65b428d51c5032|10.163.32.0/20:local;0.0.0.0/0:igw-08671cfe01495863a],[subnet-0ef0383796b11b817|10.163.36.0/22;eu-west-1b;clojure-subnet-public-eu-west-1b]-[rtb-0cf65b428d51c5032|10.163.32.0/20:local;0.0.0.0/0:igw-08671cfe01495863a],[subnet-032bba5091e3d0755|10.163.44.0/22;eu-west-1b;clojure-subnet-private-eu-west-1b]-[rtb-0078be8c529367e44|10.163.32.0/20:local;0.0.0.0/0:nat-0f171865592318f10],[subnet-04cc7aa1106b95a87|10.163.40.0/22;eu-west-1a;clojure-subnet-private-eu-west-1a]-[rtb-0d16ec0e10307b3b9|10.163.32.0/20:local;0.0.0.0/0:nat-06f9b490c02cc9e13],[vpc-0db8d6908430ced1d;10.163.32.0/20{bg:cornsilk}]

Some more thoughts…

The difficult/fun/interesting bit is the structure of the data. What is the use case, what are the key entities and what is the right link between each entities. I have made a draft of some data structure might work to some extend.

uml = {
    'notes': "This is an example for you",
    'relationships': [{
        'entry':[
            {
                'start_node':'cloudwatch logstream{bg:skyblue}', 
                'rel': 'trigger',
                'end_node':'lambda function{bg:skyblue}'
            },
            {
                'start_node':'lambda function{bg:skyblue}', 
                'rel': 'forward to',
                'end_node':'tcp server endpoint {bg:skyblue}'
            }

        ]
    }
    ]
}
Copy

It is indeed very similar to the idea of graphDB and makes me want to write another post on NEO4j. Please feel free to leave some comments if you have any questions or have any other interesting use cases. If you have enjoyed reading this post, please feel free to buy me a (r'virtual|physical', 'coffee').

Buy me a coffeeBuy me a coffee