Tag Archives: vcenter

Cisco ASAv firewall HA Pair


In previous post I have introduced and demonstrated the ASA in virtual form factor. This post will built on top of previous that one and will show you how to setup redundant high available pair of these firewalls.

The HA setup is ideal in situations where you are building virtual infrastructure hosted on private or public cloud that needs to be available all the time, surviving failure of one of the firewalls that run as virtual machines. Standard vSphere VM placement best practices should be also considered, such as anti-affinity rules and resource allocation. Consult these with you VMware administrators.

The following diagram outlines the final setup. My home lab only has one ESXi hosts, so everything is running over there.

ASAv HA Setup

Active Firewall rollout

Start with the deployment of first firewall which will be the active one. After downloading the OVA package from Cisco go to vSphere\Virtual Machines\Right Click on Cluster\Deploy OVF Template.


After browsing and selecting the OVA package, I am using asav931.ova, you will be ask to accept the extra configuration options and estimated disk size requirements.


Next, accept the license agreement and and click next. You will have an option to select the name of the new primary ASA which will be ASAv03-Active and the location of the VM.


You will now have a choice to select deployment size, from 1vCPU Standalone all the way up to 4vCPU. If you are deploying in production, you should consult the deployment guide for the right flavor. I am deploying in lab, therefore I have selected 1vCPU HA Primary.


After selecting the deployment size, we need to specify the resource that ASAv will consume. If you select a cluster DRS can automatically place the VM to least utilized hypervisor. You can also specify certain ESXi hosts. I have just one ESXi host, so the choice will be obvious.


To same some storage space, change the default virtual disk format from Thick Provisioned to Thin Provision.

On the next page, we need to configure network mapping. ASAv has by default 10 vNIC adapters which first of them is the Management and the last Gig0/8 is used by HA Heartbeat. Remaining interfaces can be used for production traffic. We leave unused interface in DMZ for now.


Now you have an option to Customize the template by typing configuration parameters such as Management Interface Settings, Device Manager IP Settings and HA Connection Settings.

The Management address of Primary Unit will be and Secondary will be The default gateway will not be required at this moment, our management station sits on the same virtual switch. To allow remote access from day 1, specify that the allowed management subnet is

The Primary Unit HA address will be and the Secondary will be


In Ready to Complete page, you get the summary of all configuration options, and you will have a choice to Power VM after deployment. Congratulations you just deployed your first HA Pair Firewall. After first power on, ASAv will perform initial configuration and reboot.

If everything worked as expected, you should be able to log in from management station via SSH.


By default, however, not username was configured in template therefore it is still needed to jump to virtual console and create one and point the AAA authentication for SSH to local user database.

It is also worth to mention that although ASDM https access was enable, it would still need to have an account and aaa authentication configured properly to work.

ciscoasa(config)# username admin password cisco privilege 15
ciscoasa(config)# enable password cisco
ciscoasa(config)# aaa authentication ssh console LOCAL

Now you can actually connect to the ASA remotely. If you examine the configuration a little bit you find that it has setup the failover interface and peer and currently shows that its mate is down/unknown.

ciscoasa# sh failover
Failover On
Failover unit Primary
Failover LAN Interface: fover GigabitEthernet0/8 (up)
Unit Poll frequency 1 seconds, holdtime 15 seconds
Interface Poll frequency 5 seconds, holdtime 25 seconds
Interface Policy 1
Monitored Interfaces 1 of 61 maximum
MAC Address Move Notification Interval not set
Version: Ours 9.3(1), Mate Unknown
Last Failover at: 10:16:56 UTC Feb 20 2015
        This host: Primary – Active
                Active time: 873 (sec)
                slot 0: empty
                  Interface management ( Unknown (Waiting)
        Other host: Secondary – Failed
                Active time: 0 (sec)
                  Interface management ( Unknown (Waiting)

ciscoasa# show failover state

State          Last Failure Reason      Date/Time
This host  –   Primary
Active         None
Other host –   Secondary
Failed         Comm Failure             10:17:13 UTC Feb 20 2015

====Configuration State===
====Communication State===

Standby Firewall rollout

Before we are going to configure the other interfaces, lets set up the secondary unit. Again we are going to start by deploying a template from .OVA. To save some space, I will only show difference from deploying the primary unit.

The VM name will be set to ASAv03-Standby, and the deployment configuration will be set to 1vCPU HA Secondary. Selecting the right resource in production should be also considered to minimize that a single hypervisor failure will cause both firewalls go down. Therefore I recommended anti-affinity rules so those two VMs will never run on same machine.

The interface mapping will be exactly same as with primary unit. See the reference above. In the customization template page, enter the Management IP address and HA IP address settings for this unit.


Deployment of this small flavor took no longer than 30 seconds, and after the initial reboot the Standby firewall is up an running.

While still setting remotely on primary unit we are going to check the failover pair state again.

Beginning configuration replication: Sending to mate.
ciscoasa# show failover
Failover On
Failover unit Primary
Failover LAN Interface: fover GigabitEthernet0/8 (up)
Unit Poll frequency 1 seconds, holdtime 15 seconds
Interface Poll frequency 5 seconds, holdtime 25 seconds
Interface Policy 1
Monitored Interfaces 1 of 61 maximum
MAC Address Move Notification Interval not set
Version: Ours 9.3(1), Mate 9.3(1)
Last Failover at: 10:16:56 UTC Feb 20 2015
        This host: Primary – Active
                Active time: 2111 (sec)
                slot 0: empty
                  Interface management ( Normal (Monitored)
        Other host: Secondary – Standby Ready
                Active time: 0 (sec)
                  Interface management ( Normal (Waiting)

ciscoasa# sh failover state

               State          Last Failure Reason      Date/Time
This host  –   Primary
               Active         None
Other host –   Secondary
               Standby Ready  Comm Failure             10:17:13 UTC Feb 20 2015

====Configuration State===
        Sync Done
====Communication State===
        Mac set

====VM Properties Compatibility===
vCPUs – This host:  1
        Other host: 1
Memory – This host:  2048 Mhz
         Other host: 2048 Mhz
Interfaces – This host:  9
             Other host: 9

Looking much better now. To see failover in action, lets first complete the configuration of other interfaces to get some traffic flowing through the firewall

prompt hostname priority state
interface GigabitEthernet0/0
 description OUTSIDE
 nameif outside
 security-level 0
 ip address standby
interface GigabitEthernet0/1
 description INSIDE
 nameif inside
 security-level 100
 ip address standby
interface GigabitEthernet0/2
 description DMZ
 nameif dmz
 security-level 50
 ip address standby
monitor-interface outside
monitor-interface inside
monitor-interface dmz
route outside 0 0
policy-map global_policy
 class inspection_default
  inspect icmp

My outside router does not have static routes to these private network behind ASA so a object NAT will help to mitigate that.

object network inside_net
nat (inside,outside) source dynamic inside_net interface

Now we should have connectivity from inside host to Internet. We get verify that by pulling google web page from client and also checking the NAT or connection table.

ciscoasa# sh nat detail
Manual NAT Policies (Section 1)
1 (inside) to (outside) source dynamic inside_net interface
    translate_hits = 12, untranslate_hits = 0
    Source – Origin:, Translated:

Now start continuous ping from inside machine and power off the primary firewall.

root@deb01:~# ping -c 100
100 packets transmitted, 85 received, 15% packet loss, time 99250ms
rtt min/avg/max/mdev = 10.771/11.988/21.956/2.426

From the output, I examined that from 100 packets, 15 was lost during switchover. With default pool and hold times it was not amazingly fast. We can do better.

failover polltime unit 1 holdtime 3
failover polltime interface 1 holdtime 5

Repeat the 100 ping test again and examine the results.

Now with adjusted timers, from 100 packets we lost 4 during switchover. Getting better :-). Lets see how this HA pair will cope with TCP sessions.

We are going to initiate a large file download from inside host and then shutdown the secondary (now active) ASA and examine the result on traffic flow.

The primary ASA took back the role of Active Firewall, but the TCP session has died. There is one another parameter that we need to tweak for HTTP sessions and to explicitly enable them.

ciscoasa/pri/act(config)# failover replication http
Now initiate the download again, and check that it will be re-established after failover.


After tweaking some default settings you have a highly available firewall cluster running in fully visualized environment.


VCP-DCV Series Part 2: What is vSphere

I do not know why but each time someone mentions the word vSphere I remember on old movie with Dustin Hoffman called Sphere. Unfortunately for you I do not have a space craft that crashed to be examined, but still the VMware’s vSphere is also interesting.

vSphere is a collection of products from VMware that help you manage data center resources more efficiently. It aggregates common physical resources such as CPU, RAM and storage and presents tham as virtual resoures for applications to consume.


vSphere architecture

vSphere Editions

There are three different vSphere offerings, a higher edition contains all lower edition features and something on top.

  • vSphere Standard
  • vSphere Enterprise
  • vSphere Enterprise Plus

More information about editions and their pricing can be found here. For our lab purposes we are going to use 60-Day trial Enterprise Plus license for vCenter and ESXi hosts.

Addional Resources

If you are looking for good visio stencils that cover VMware products, head up to technodrone.

Lost objects in vCenter

To realized how important is DNS for vSphere installation, try shutting down your DNS server, wait for cache to expire and log into vCenter again. You may be presented with this error which on the first sight might not look like that is related to DNS.


Could not connect to one or more vCenter Server Systems

And, you won’t have access to any objects:


You can find more evidence that this is related to DNS in vCenter main log which is located at /var/log/vmware/vpx/vpxd.log

Fixing vCenter dirty shutdown

From time to time when I am hurry I tend to break the best practices in my home lab. Recently one of them was shutting down couple of vcenter servers and ESXi hosts unlugging the virtual power cords :-). Usually they boot and work well again, but this time I received a lovely error saying something went nuts with vcenter server, and I was unable to boot any VMs.

The operation is not allwed in the current connection state of the host.The solution was restarting the vCenter Services in servces.msc console.